You are on page 1of 37

2012 McAfee, Inc. All Rights Reserved.

Policy Tuning

Upon successful completion of this module you will be able to:


Summarize tuning and its purpose
Discuss what is noise in the network
Give example of how to identify and prevent false positives
Explain why you should look at high-volume attacks
Configure NSP to prevent false positives

2012 McAfee, Inc. All Rights Reserved.

Policy Tuning

2012 McAfee, Inc. All Rights Reserved.

Policy Tuning

The policies applied to the Sensors determine what traffic analysis they will perform. Policy tuning is
renowned to be an involved task. However, because networks and attacks constantly evolve, the
policy tuning process is never truly complete. The ultimate goal of policy tuning is to eliminate false
positives and noise, and avoid overwhelming quantities of legitimate, but anticipated alerts.
Some things to remember when tuning policies:
When initially deployed, McAfee Network Security frequently exposes unexpected conditions in
the existing network and application configuration. What may at first seem like a false positive
might actually be the manifestation of a misconfigured router or Web application.
Take steps to reduce false positives and noise from the start. If you allow a large number of
noisy alerts to continue to sound on a very busy network, parsing and pruning the database
can quickly become cumbersome tasks. It is preferable to all parties involved to put energy into
preventing false positives than into working around them. One method may be is to disable all
alerts that are obviously not applicable to the hosts that are being protected. For example, if
only using Apache Web servers, its preferable to disable IIS-related attacks.

2012 McAfee, Inc. All Rights Reserved.

Policy Tuning

There are great advantages for effective policy tuning. An appropriately tuned NSP deployment will
not only reduce false positives, but will also:
Improve Security
Speed (Want Maximum IPS performance)
Fuel efficiency (Want to reduce Analyst expenses)
No accidents (Want to harden the policy for maximum protection)
Maintenance (Monitor Constantly)
Improve Scaling
McAfee Network Security Manager (NSM) can handle up to 150 alerts per second. Each alert is
~650 bytes. If sustaining alerts at that rate, then it equates to a bandwidth rate of 780 Kbps
resulting in 67.392 Gigabytes (GB) of alerts space per day!
A tuned NSP deployment can now manage more sensors across an organization, and it also
saves precious network bandwidth and storage.
Help Engine Performance
The bulk of IPS engine resources are consumed in protocol parsing. This entails doing a deep
packet inspection, protocol anomaly detection, parsing protocol content against respective
vulnerability signatures etc. Therefore the greater the number of unwanted attack signatures
that are included, the more bulky the IPS engine becomes. It also increases the possibility of
false positives.
A customized and relevant IPS policy is crucial to providing best protection.
2012 McAfee, Inc. All Rights Reserved.

Policy Tuning

The initial tuning of Network Security Platform is very important in establishing the solutions
credibility and usability. Though each network environment will have its own characteristics, there
are some best practice steps that will make tuning more efficient and effective.

Setting expectations about tuning is very important. A proper McAfee Network Security
implementation includes multiple tuning phases. False positives and excess noise are generally
routine for the first 3 to 4 weeks. Once properly tuned, however, they can be reduced to a rare
occurrence.
Attack definitions are written to accommodate many different types of customers. Therefore, they
will not be perfect in every environment. Before beginning, be aware of the network topology and
the hosts in the network, so you can enable the policy to detect the correct set of attacks for the
environment.
There is a difference between evaluation tuning and tuning for the production environment. An
initial reaction is to install the Network Security Platform into the busiest part of the network and just
see what it is detecting. However, you can minimize the need for tuning by focusing the IPS policy on
a hot spot, for example, Web Server attacks. Then determine sources of devices in the network that
would be known false positive offendersSNMP Managers, Vulnerability Scanners, and so forth. You
can further tune by having Wireshark (formerly Ethereal) installed and ready on a client PC.

2012 McAfee, Inc. All Rights Reserved.

Policy Tuning

There are 2 phases to IPS tuning. Generally, one is owned by the security architecture team, and the
other is owned by security operations team.
The first phase is relevant from security design, where you make sure that the IPS sensors are placed
in the right locations, and are protecting with the correct configured policies. If there are Linux
servers behind a VIPS and you have not enabled the Linux OS attacks in the policy, then IPS is not
protecting the Linux servers. Similarly, if you have placed the IPS with just one VIPS and all default
policy whereas the network behind is far more diverse, then the IPS will not be able to do its job
efficiently or effectively. It is important that customers review their IPS security architecture before
going to the second phase.
The second phase involves the day-to-day operations of IPS alerts. If receiving 100,000 alerts per day,
of course it is going to take a bit of time of the security analysts to review those. And if receiving
those repetitively, then there is something needing to be reviewed. If those 100,000 are all attacks,
then of course you need to block the source of attacks and find a way to reduce the attack reporting.
But if a bulk of those 100,000 are not attacks, then you need to ensure that they do not show up
every day. From experience, 70% or more of those repeating alerts can be eliminated, without
impacting security.

2012 McAfee, Inc. All Rights Reserved.

Policy Tuning

The mere mention of false positives always causes concern in the mind of any security analyst. However, false
positives may mean quite different things to different people.
Incorrect identification
These alerts typically result from overly aggressive signature design, special characteristics of the user
environment, or system bugs. For example, typical users will never use nested file folders with a path more
than 256 characters long; however, a particular user may push the Windows' free-style naming to the extreme
and create files with path names more than 1024 characters. Issues in this category are rare. They can be fixed
by signature modifications or software bug fixes.
Correct identification; significance subject to usage policy
Events of this type include those alerting on activities associated with Instant Messaging (IM), Internet Relay
chat (IRC), and Peer to Peer programs (P2P). Some security policies forbid such traffic; for example, within a
corporate common operation environment (COE); others may allow them to various degrees, such as
Universities.
McAfee Network Security provides two means by which to tune out such events:
customized policy in which these events are disabled. In doing so, the sensor will not even look for these
events.
create attack filters to suppress alerts when a few hosts are involved.
Correct identification; significance subject to user sensitivity (also known as noise)
McAfee Network Security will detect a UDP-based host sweep when a given host sends UDP packets to a
certain number of distinct destinations within a given time interval. Some users will consider alerts from this
type of event as noise; however, others will take notice because it indicates possible reconnaissance activity.

2012 McAfee, Inc. All Rights Reserved.

Policy Tuning

False positives tend to result in a high attack count. This is because the traffic creating the false
positive is usually commonplace on the particular network. The approach to identifying and removing
false positives therefore starts with identifying the attacks with the highest attack counts.

Look for a pattern within each attack. For example, if you see that the attack is originating from the
same group of sources and is taking place at a regular interval, there is a pretty good chance the alert
is a false alarm. If those source hosts happen to be internal, you can often figure out what service
they have in common by having a closer look at a subset of the group. You can then temporarily shut
down services on those hosts until the culprit service is isolated.
Be sure to leverage the knowledge of someone who is very familiar with the network. He or she may
recognize a source or destination address as belonging to a host with a special function. Also consult
attack descriptions for clues; the attack in question might just have a high Benign Trigger Probability.
If there is no obvious pattern or convincing attack information, take advantage of McAfee Network
Security's options to capture additional packets. Specifically, temporarily increase the quantity of
traffic captured for the alert in question. The customer can use the capture to analyze the attack. If
still not able to isolate the pattern, the customer should save the capture file as part of an evidence
report.

2012 McAfee, Inc. All Rights Reserved.

Policy Tuning

Enterprise HTTP caching servers, DNS servers, and even outbound mail relays, are susceptible to false
positives. These types of hosts are constantly receiving requests from many different clients and, in
turn, making requests to many different hosts to service those clients. Vulnerability assessment
software will also generate false positives as it scans the network. Check the source and destination
addresses within the attack details for these known internal hosts. If you find such a false positive,
create an exception or a firewall rules for that host.
Once it is determined that an alert is indeed a false positive, take measures to reduce the noise.
The most secure way is to create an exception for the hosts and attacks in question. Of course, this
assumes the hosts are consistent and few in number. If there are too many hosts or the range of IP
addresses involved is too unpredictable, the options are to live with the false positives or disable the
alert (possibly until McAfee has produced an updated signature set). Most folks disable the alert.
One of the options in Manager is to create Firewall Rules. Customers can configure an firewall rules
to process specific traffic without sending it through the IDS engine. So if there is a need to not
process traffic to or from a given host or range of hosts, Firewall rules make it easy to do so.

2012 McAfee, Inc. All Rights Reserved.

10

Policy Tuning

The Rule Set Editor action enables the use of a powerful tool for defining the exact environment
resources needing protection.
A Rule Set consists of select attacks specific to a network environment, such as the operating
system(s) employed, the installed applications (email, chat), and the transport and application
protocols (HTTP, FTP) used for data delivery.
Within a Rule Set are Include/Exclude rules that in turn contain one or multiple rules. The rules are
based upon selected attacks, and they are filtered through seven different views: Categories,
Protocols, OS, Applications, Severity, Benign Trigger probability and Attack.
McAfee Network Security Manager provides pre-defined rule sets to enable customers jump start
with the right policy.
Each rule set may have include rules, exclude rules or both. The Include/Exclude rules contain one or
multiple rules and the rules can be based upon attacks NSP will analyze for in the data traversing the
network.

2012 McAfee, Inc. All Rights Reserved.

11

Policy Tuning

The single most effective way to prevent false positives is to apply a scanning policy that is specific to
the network at hand. It is therefore imperative to understand the details of the protocols and
applications used on the network needing protection. For example, if a sensor port is connected to a
network containing only Windows workstations, there is no reason to use a policy that looks for
attacks against UNIX hosts, Web servers, and so on. Once the network configuration has been
identified, use an applicable default policy or create a custom one.
Also consider virtualization of the IPS to refine scanning by VLAN tag or CIDR address. If a DMZ
contains isolated networks of Web and mail servers, for example, you can prevent false positives by
creating sub-interfaces and applying policies specific to Web and mail servers, respectively.
The noise-to-incorrect-identification ratio can be fairly high, particularly in the following conditions:
The configured policy includes a lot of Informational alerts, or scan alerts which are based on
request activities (such as the All Inclusive policy)
Deployment links where there is a lot of hostile traffic, such as in front of a firewall
Overly coarse traffic virtual interface definitions that contains very disparate applications, e.g. a
highly aggregated link in dedicated interface mode
Users can effectively manage the noise level by defining appropriate virtual interfaces and customize
the policy accordingly.

2012 McAfee, Inc. All Rights Reserved.

12

Policy Tuning

Its best to begin the tuning process by identifying high-volume attacks. To do this, use the Top10
table in the Consolidated View within Threat Analyzer to view the attacks that are generating the
most alerts within the network. You will want to investigate the legitimacy of these alerts.

The investigation will involve use of packet captures to decode these attacks. The Wireshark decoding
function is used at this stage to discover the packet level characteristics of a data flow that triggered a
particular alert. At this point it usually becomes evident whether the alert is generated by an attack
or by a false positive.
Many of the top alerts that are seen on the initial deployment of sensors will be common false
positives seen in many environments. As a result, this initial tuning of high volume alerts will move
along quickly. Typically, at the beginning of the tuning process, it will be evident which network or
security policy will affect the overall level of alerts. If, for instance, IM is allowed traffic on the
network then there might not be a need to alert on IM set-up flows.

2012 McAfee, Inc. All Rights Reserved.

13

Policy Tuning

As in the case of a 1 to Many, if we are seeing one alert, then an exception should be made. If we are
seeing a multitude of alerts then its advised to create a firewall rule.
NOTE: The create exception wizard can create both a firewall rule or an exception.

2012 McAfee, Inc. All Rights Reserved.

14

Policy Tuning

Once its determine that an alert is indeed a false positive, its important to take measures to "reduce
the noise. When a particular alert is declared a false positive, the next decision is whether to disable
this alert altogether or apply a particular exception to that attack that will disable alerting for a
particular IP address or range of IP addresses.
In almost all cases it is best practice to implement the latter. For instance, an SMS server may be
generating the alert Netbios: Copy Executable file attempt during the legitimate transfer of login
scripts. Rather than disable the alert altogether, and cancel the possibility of finding a real attack of
this nature, it is recommended that an exception be created for the SMS server and applied to this
attack. Every exception created is globally stored so that the exception can be applied to any Exploit
or Reconnaissance attack chosen.
It is also best practice to document all tuning activities. Configuration reports for exception objects
can be used to assist the documentation process. These reports can be used to list Exceptions that
have been applied and attacks that have been otherwise customized.

2012 McAfee, Inc. All Rights Reserved.

15

Policy Tuning

By choosing to create an exception for an alert, the user is presented with Create Exception wizard.
Here you can create an exception for a specific attack that will be applied to all devices in the admin
domain, a particular Sensor, or interface.

2012 McAfee, Inc. All Rights Reserved.

16

Policy Tuning

However, the wizard can also disable the attack by setting the exception for any combination of
source and destination. By doing so, the selected attack definition will therefore be disabled.

2012 McAfee, Inc. All Rights Reserved.

17

Policy Tuning

The wizard can also create a statefull firewall rule. Selecting Attack Name as Any, will create an
ignore rule that will be added to the top of the firewall policy currently assigned to the device
identified.

2012 McAfee, Inc. All Rights Reserved.

18

Policy Tuning

The scope of the exception can be restricted to an device, an interface or to the admin domain as a
whole.

2012 McAfee, Inc. All Rights Reserved.

19

Policy Tuning

You can also, optionally, select to acknowledge or delete the alerts that match the settings specified.

2012 McAfee, Inc. All Rights Reserved.

20

Policy Tuning

Another method of preventing false positives is to disable attacks. For example, you can edit the
policy to disable all attacks that have a severity level of low or informational (lower than 4 ). To do
this, you can sort the attacks by severity and then bulk disable all Low and Informational attacks. For
these alerts, you can choose to uncheck the option Enable Attack You can do this for both Inbound
and Outbound attacks.
Its best practice to disable all alerts that are obviously not applicable to the hosts being protected.
For example, if only using Apache Web servers, you can safely disable IIS-related attacks.
Adding low severity attacks
Once you have eliminated the false positives with only medium and high severity attacks enabled,
you can come back and re-enable low and informational attack alerts.
Low and informational severity attacks tend to generate more alerts than higher severity attacks, so
expect another round of false positives after enabling low and informational severity attacks.

2012 McAfee, Inc. All Rights Reserved.

21

Policy Tuning

Causes of Excessive Alerts


The large set of alerts can be categorized into 3 categories.

Valid Alerts: In this case, the alerts are pertinent to your network and taken as attacks. The
impact of the attack may be critical, or moderate, or none. In this scenario, youve a challenge if
you see 1000s of alerts being generated for the same attack, yet analyzing one alert is same as
analyzing all those 1000s alerts. Then it becomes important how to design the configuration so
that it reduces the number of alerts, without impacting security. Alert Throttling feature can help
in this situation.
Irrelevant Alerts: By default NSP will raise alerts when it detects P2P traffics. Now that may be
relevant for some administrators, and may not be relevant for some. It depends on the company
policy of specific customer. Another scenario is where you see 100s of UDP sweeps or probes.
NSP, by default, has certain thresholds defined for detecting such reconnaissance attacks. Those
thresholds may need to be changed for your network scenario. Alternately, a massive set of
alerts are being generated from a IT approved scanner, then you need to white-list the scanner
or use granular set of attack filters.
False Positives: These refer to incorrectly identified events by the IPS. An example is a Vendor
signature alerts an attack, where the attack was not there at all. This signifies an error in vendor
signature. This is actually false positive. False Positives are error on part of a Vendor IPS
inspection. In contrast, False negative is when a vendor protection delivered for an attack is
completely missing to detect the attack.

2012 McAfee, Inc. All Rights Reserved.

22

Policy Tuning

A high level bottoms up approach to tuning can be summed up as follows:

1. Access Threat Analyzer


2.
3.
4.
5.
6.
7.

Identify the noisiest event


Investigate the event and correlate with network resources
Understand the characteristics of the event
Determine the criticality of the event and of the components involved
Decide on an action to remediate/tune the event
Depending on action to be taken, proceed with the proper response procedures

8. Repeat

2012 McAfee, Inc. All Rights Reserved.

23

Policy Tuning

Analyzing events can be done in two ways, either by starting on the Threat Explorer and using the
filtering capabilities, or by jumping straight into the Threat Analyzer and sorting events from there. As
shown in the image, the Threat Analyzer IPS tab gives a quick glimpse of attacks over time, the top
hosts under attack, and it allows you to drill down using various alert properties.

2012 McAfee, Inc. All Rights Reserved.

24

Policy Tuning

Using the Threat Analyzer Alert page, customers can display alerts sorted by Attack name.
As you can see, in this image, there are a large number of HTTP: Gomplayer ActiveX Control OpenURL
Overflow alerts. This is a good starting point to investigating the relevancy and applicability of the
highest volume alerts.

2012 McAfee, Inc. All Rights Reserved.

25

Policy Tuning

In this next section we will look at some examples and use cases of typical high volume alerts. This
example is dealing with a high volume of ICMP unsolicited Echo Reply alerts.

2012 McAfee, Inc. All Rights Reserved.

26

Policy Tuning

This example is dealing with a high volume of DNS: Microsoft DNS Services Resolver Overflow
alerts.

2012 McAfee, Inc. All Rights Reserved.

27

Policy Tuning

This example is dealing with a high volume of ICMP: Nachi Like Ping alerts.

2012 McAfee, Inc. All Rights Reserved.

28

Policy Tuning

This example is dealing with a high volume of Backdoor: Back Orifice Trojan alerts.

2012 McAfee, Inc. All Rights Reserved.

29

Policy Tuning

This example is dealing with a high volume of HTTP: HTTP Login Bruteforce Detected alerts.

2012 McAfee, Inc. All Rights Reserved.

30

Policy Tuning

This example is dealing with a high volume of ARP: ARP Spoofing Detected alerts.

2012 McAfee, Inc. All Rights Reserved.

31

Policy Tuning

This example is dealing with a high volume of ARP: MAC Address Flip-Flop alerts.

2012 McAfee, Inc. All Rights Reserved.

32

Policy Tuning

This example is dealing with a high volume of P2P: BitTorrent Meta-Info Retrieving alerts.

2012 McAfee, Inc. All Rights Reserved.

33

Policy Tuning

This example is dealing with a high volume of IM: Yahoo Messenger Server Lookup alerts.

2012 McAfee, Inc. All Rights Reserved.

34

Policy Tuning

This example is dealing with a high volume of UDP: Host Sweep alerts.

2012 McAfee, Inc. All Rights Reserved.

35

Policy Tuning

2012 McAfee, Inc. All Rights Reserved.

36

Policy Tuning

2012 McAfee, Inc. All Rights Reserved.

37

Policy Tuning

You might also like