What is Threat Hunting?

It is a focused and iterative approach used to detect and remove cyber threats that may have evaded traditional security tools. These threats include attacks or malware that infiltrate a business or organization’s network, leading to stolen intellectual property or personal information.

As network complexity has increased, the sophistication of “bad actors” and their cyber-crimes has followed suit. Threat hunting is now an important and fast-growing element of the cybersecurity landscape.

To qualify as a threat, a bad actor must have malicious intent, capability, and the opportunity to carry out their attacks. The field of cyber threat hunting has been established to counteract the most advanced malicious activity. Many such threats can slip under the radar of existing security tools.

Resource: Threat Hunting Podcast

Listen to the below podcast from Enterprise Security Weekly to learn threat hunting techniques and understand the important role it plays in identifying subtle attacks that can disrupt common AI and machine learning tools.

Effective threat hunting begins with a foundation of planning, baselining, and hypothesis testing performed by experienced cybersecurity professionals. Beyond these basics, there is a growing need for advanced threat hunting tools and practices. Automated security tools can be used to boost the horsepower behind your analytics for more intelligent hypothesis generation. Analytics tools can perform data-mining activities on a large scale and identify suspicious patterns and relationships. Many threat detection tools also provide intuitive displays and graphs. These interfaces aid cyber threat hunters in their investigations and analysis.

Forensic capability is another essential arrow in the threat hunter’s quiver. Security professionals should have quick access to enriched-flow records including traffic and application types, volumes, IP addresses, and device type history on the network. Observer GigaFlow can support threat hunting activities through forensic recall capability of unstructured flow-based data sets from hybrid environments. This enables easy identification of unauthorized devices or malicious activity. Indicators of compromise IOC such as odd-hour traffic and exposure to known bad actor IP addresses can then be flagged for further analysis at the packet level.

Powerful packet capture appliances such as Observer GigaStor provide outstanding packet-level storage and recall with over a petabyte of capacity. Security professionals and threat hunters benefit from the long-term retention and analysis of granular packet level data sets. This captured data can be used to reconstruct specific file and URL details from the most critical times or traffic sequences identified at the flow level. GigaStor can also assist in validating the root cause of compromise.

Network performance monitoring tools such as Observer Apex combine access to packet level data and enriched flow records in a single  interface. This creates a powerful threat detection, investigation, and analysis tool. Post-event security forensics are supported by long-term data retention and advanced analytics capabilities.

Observer Apex and GigaFlow working together can be used to profile traffic for every host and device. Access to packet level data via GigaStor allows for a seamless transition into deep packet analysis. For example, if a known malicious IP has touched internal IP addresses, details of the transactions can be quickly reviewed in-depth at the packet level with the click of the mouse.

Do you have network performance or security issues in your queue? Learn how comprehensive network visibility from Observer can help you with your business goals at viavisolutions.com/observerdemo

Basic threat detection tools, such as firewalls and antivirus software, can usually weed out most cybersecurity threats, particularly the less sophisticated ones. The rare attacker who manages to go undetected requires a more advanced cybersecurity approach that utilizes automated security tools. This includes external attackers as well as malicious insiders who carry out IT sabotage or fraud.

Insider threats can be very difficult to detect. Since access to sensitive data is often part of the job description, malicious activity can be difficult to separate from normal activity. Whether they originate from within or outside the organization, advanced cyber threats can often go weeks or months evading detection. All the while, sensitive data is exposed to corruption or theft.

The 80/20 rule is based on the “pareto” principle that 80% of all effects are the result of 20% of the causes. This can also be applied to network security, since roughly 80% of the problems are induced by 20% of the threats. This means a higher level of focus, technology and manpower should be directed towards the most dangerous attackers. 

APT

With regards to threat hunting, an advanced persistent threat (APT) is defined as a cyber attack through which an unauthorized party gains access to a network and steals or corrupts data unabated over an extended period of time. The cyber attacker typically utilizes multiple entry points to evade detection and elimination.

Initial access can be obtained through emails, files, applications or other benign pathways which lead to a compromised condition. Once established, malicious software will create additional avenues of compromise through which more instructions and/or code may be delivered. After this access foothold has been constituted, data such as account numbers, passwords and other sensitive information is often targeted by the attackers. Once this data leaves the network and reaches the hands of the infiltrators, the network is considered breached, rather than simply compromised. This breached condition leaves an open door for the cyber criminals to freely enter.

The common denominator linking APTs to threat hunting is the omnipresent human factor. A foiled APT strategy may quickly evolve to exploit newly discovered vulnerabilities or security weaknesses. Threat hunters can stay one step ahead of the bad actors by using deception technologies, such as luring attackers into fake servers, where the methods of the unknowing perpetrators can be observed while an effective counterstrategy is developed. This ongoing game of “cat and mouse” means these well-coordinated and adaptive cyber-attacks require an equally dynamic and malleable response. Advanced security and forensics tools combined with the elite knowledge and ingenuity of the threat hunter might be the best recipe for proactively preventing, detecting and defeating APTs.

Increased Vulnerability

The “Internet of Things” (IoT) continues to advance technology and applications throughout the cyberscape but has also introduced new attack surfaces that create increased vulnerability. By nature, most IoT devices do not include a human interface, making them appear immune to attack by default. If not protected by end-point monitoring solutions, IoT devices can leave endpoints exposed to security threats, such as botnets, that once relied on networks of PCs for their point of entry. The proliferation of millions of additional IoT devices will continually raise the likelihood of compromise through similar methods.

Artificial intelligence (AI) has gained increased acceptance in recent years and the massive expansion of networks and data centers has made progressive adoption unavoidable. AI and machine learning (ML) can automate and improve many processes, including cybersecurity using AI-driven algorithms. Unfortunately, AI and ML have also proven to be beneficial for the attackers, enabling them to adapt threats quickly, create additional layers of insulation between the bad actors and their targets, and automate infiltration tactics such as phishing.

Each organization uses their own unique blend of cyber security tools and talent to develop effective threat hunting practices. Once this process is established, it can be used as a blueprint for future hunting activities.

• Who will do the hunting? The answer may seem obvious, but many organizations lack the bandwidth or expertise within their existing IT departments. If additional resources are required to lead or perform threat hunting activities, this must be addressed.
• Baselining of network operations and traffic patterns should be performed so that anomalous conditions can be recognized. Full implementation of network security architecture and passive defense systems should also be completed. This includes any existing network security monitoring or threat detection tools like firewalls or antivirus software that actively collect security data.
• The planning phase of threat hunting is extremely important. Successful planning combines a review of past incidences and hunts with industry trends and any other security threat intelligence relevant to the current situation. This includes all anomalies observed in traffic flows. 
• Tactics, techniques, and procedures (TTPs) are the three pillars around which the bad actors conduct their attacks. Common methods and patterns can be used to establish profiles for individuals or groups of cyber criminals. It is important to have TTP cybersecurity defense measures in place.

• A hypothesis can be developed based on the observed malicious activity and known vulnerabilities. A hypothesis is simply a data-driven theory based on what compromised conditions may currently exist. Experiments are designed to test the validity of the hypothesis. Once the indicators of compromise (IOC) and data sets to support the hypothesis testing have been collected, they can be analyzed to determine whether the hypothesis was valid.
• An incident response team with pre-defined roles and responsibilities goes into action once the breach is confirmed and the issue is escalated. Compromised conditions or new vulnerabilities that have not yet led to a full breach may also be detected.
• Data collection and analysis is a cyclical, rather than linear process, known as the intelligence cycle. Using this threat intelligence, all information and evidence gathered throughout data collection and experimentation can be used to improve future threat hunting efforts. 

Cyber Threat Hunters

Cyber threat hunting is now a full-time job for many security professionals. With a growing number of organizations now performing continuous threat hunting, the value of this specialty is being recognized. Along with extensive experience in the cyber security field, the best threat hunters also share common abilities and talents, including pattern recognition, data analytics, forensics, and communication skills. Effective communication by professional cyber threat hunters helps them to navigate the dynamic, human element of this specialty.

The talents and abilities that make a threat hunter successful must be backed up by solid technical skills. Coding experience in multiple languages, operating system expertise, and advanced knowledge of TTPs are other preferred traits of threat hunters.

The challenges for threat hunters will intensify as the tactics, techniques and procedures of the bad actors evolve. The landscape has shifted from random malware attacks towards more customized, professional, and focused attacks. Mining data, developing hypotheses, and performing investigations all consume precious time in a field where the adversary is moving quickly.

Having the right data sets readily available in the right format can make the threat hunter’s task faster and more effective. Moving rapidly between metadata, enriched flow records, and packet level data is often necessary to reach conclusions. The right combination of automated security tools can beat the attackers to the punch. 

Another challenge faced by threat hunters is the lack of standardization and infrastructure built around this process. Since threat hunting practices have developed independently at different organizations, there are few standard guidelines and protocols available for would-be threat hunters. Unfortunately, creating more threat hunting standardization could unintentionally provide intelligence to the bad actors themselves.

Despite these challenges, organizations that have employed effective threat hunting practices have improved their response times and accuracy, reduced dwell (undetected infection) and containment times, decreased frequency of breaches, and optimized resource allocation. The best threat hunters can stay one step ahead through their skills and talents, along with a proactive mode of operation that continually seeks rather than simply monitors and waits. As security threat intelligence and forensics tools develop and improve, threat hunting can tip the balance of power in cybersecurity.

Why Investing in Threat Hunting is Essential

Enterprises dedicate 50x more budget to prevention than investigation, but it is not working. Learn why.