Deep Packet Inspection Using Layer 7 Traffic Shaping

Deep packet inspection

The Layer 7 tab in the Traffic Shaper in pfSense 2.1.

For quite a while, traffic shaping has been considered an integral part of any good firewall. This necessitates some means of classifying the traffic so the traffic can then be policed. The traditional method of traffic shaping centered on classifying traffic based on network and transport data fields, not using deep packet inspection. This usually centered around the following elements:

  • Service class marks
  • Source and/or destination IP addresses
  • Ports

However, these methods are not always effective in traffic classification. This is especially the case with P2P traffic, which often uses random, non-default ports. An HTTP server utilizing port hopping and encrypted traffic may also defy level 3 (network) and level 4 (transport) classification.


Enter Layer 7 Deep Packet Inspection

One possible solution to the shortcomings of network and transport level classification is layer 7 (L7) classification, which involves deep packet inspection. In L7 classification, user traffic can be identified based on an application pattern, which is a sort of signature used by an application during its communications. All applications either use a specific application pattern or may share the pattern with other applications.

Deep packet inspection

Configuring P2P options in the wizard.

IPCop is a good example of utilizing L7 deep packet inspection and classification. IPCop is a Linux-based firewall that was originally a fork of the SmoothWall firewall. Although not an official part of IPCop, an advanced QoS (Quality of Service) add-on is available. But while IPCop can support classification by application protocol, it does not allow the definition of shaping policies. Rather, it can only block such traffic, which greatly limits the use of this feature.

pfSense, however, has fully incorporated L7 deep packet inspection and classification into its traffic shaper. Traffic shaping is achieved in pfSense through AltQ, which makes available Class Based Queueing (CBQ), Priority Queueing (PRIQ) and Hierarchical Fair Service Curve (HFSC). All of these can be configured automatically through the use of a wizard. Beginning with pfSense 2.0, an additional shaping mechanism called Dummynet became available. Dummynet was originally designed for the ipfw firewall, and has a related application called ipfw-classifyd. This application is able to produce blocking rules for incoming traffic or perform traffic shaping by assigning IP packets to an AltQ queue or a Dummynet pipe or queue. It was modified to work with the pf firewall and is the component responsible for L7 classification. It also allows different types of operations to be applied to an identified application protocol, usually either blocking it or assigning it to a limiter or queue.

In order to invoke ipfw-classifyd, pf uses divert sockets. Essentially, it interrupts the normal flow of packets and sends them to a listening socket (ipfw-classifyd). Overhead is kept to a minimum by teaching pf about the actions to be taken ahead of time and by limitng the number of packets that are diverted from the kernel to the application. All of this is controlled via a graphical interface, in which the user must specify at least one protocol (but may specify more than one). The user can create L7 rules groups containing one or more L7 rules. The user can take any one of the created rules groups and assign it a firewall rule.

But with pfSense, the user does not have to explicitly create L7 rules groups. This is because the Traffic Shaper Wizard in versions 2.0 and newer invokes L7 classification in the Peer-to-Peer and Network Games sections. In both sections, the select box on the top of the page can be enabled, and the related protocols or applications can be blocked one by one. Finally, the user can extend the functionality of L7 packet inspection by uploading new application patterns to the system. This feature is important when the user wants to block an application that uses a protocol pattern that is not defined in the system. If such a pattern is uploaded to the system, it only appears in the list of protocols when a container is created or modified. It does not affect the Traffic Shaper Wizard, which remains unchanged.


Other Articles in This Series:

Traffic Shaping in pfSense: What it Does
Traffic Shaping Wizard: Introduction
Queue Configuration in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Layer 7 Rules Groups in pfSense 2.1
Bandwidth Limiting with the pfSense Limiter

External Links:

L7 Classification and Policing in the pfSense Platform – a scholarly paper about the addition of layer 7 deep packet inspection to pfSense 2.0.

Ad Links:

QoS Management Using the Traffic Shaper Wizard

In this article, we will go through the pfSense traffic shaper wizard to achieve quality of service (QoS) goals and cover some of the options which are configurable through the wizard.

QoS Management: Queueing Disciplines and Bandwidth

QoS

Specifying the number of WAN connections in the wizard.

In the wizard, you first have to specify the number of WAN connections, and if you selected multi LAN, the number of LAN connections. On the next screen, there are several more options. At “Download Scheduler“, there are three options for queueing discipline: HFSC (Hierarchical Fair Sharing Curve), which is designed to ensure that link delay is low while bandwidth is not over-reserved. CBQ (Class-Based Queueing) allows for bandwidth to be shared equally among different classes, while PRIQ (Priority Queueing) allows for different priority levels to be assigned to classes. Under “Setup connection speed and scheduler information for WAN #n“, at “Interface” you select a valid interface. At “Upload Scheduler“, you chose a queueing discipline (again, the options are HFSC, CBQ and PRIQ). Finally, the “Connection Upload” and “Connection Download” speed must be entered.

QoS Management: VoIP, P2P, and Network Games

QoS

Configuring VoIP traffic with the wizard in pfSense 2.1.

On the next screen, there are various QoS options for VoIP traffic. The first checkbox, “Prioritize Voice over IP traffic“, is self-explanatory. The “Provider” drop-down box allows you to specify your VoIP provider. There are a few well-known providers, including Vonage, Voicepulse, and PanasonicTDA, and there’s Asterisk as well, in case you connect to an Asterisk server. If you have a different provider, you can choose “Generic“, or override this setting with the “Upstream SIP Server” field by entering the IP of your VoIP phone or an Alias containing the IPs of all your phones. With the next two fields, “Upload bandwidth for each WAN” and “Download bandwidth (speed) for Voice over IP phones“, you can choose the amount of bandwidth to guarantee to your VoIP phones. The amount of bandwidth you actually use will vary based on how many phones you have and how much each session will use.


The next screen contains options for the “Penalty Box“. The penalty box is a place where you can relegate misbehaving users or devices that would otherwise consume more bandwidth than desired. Click on the “Enable” checkbox to enable this feature, and enter the IP address of the computer to penalize at “Address“. At the “Bandwidth” field, enter the limit you wish to apply.

QoS

Configuring P2P options in the wizard.

The next screen covers “Peer to Peer networking“. Click on the “Enable” checkbox to lower the priority of P2P traffic below all other traffic. By design, P2P protocols and software will utilize all available bandwidth unless limits are put in place. If you expect P2P traffic on your network, it is a good idea to ensure that other traffic will not suffer degradation of QoS due to its use.

Many P2P technologies will deliberately try to avoid detection; Bittorrent is a good example of this. It often utilizes non-standard or random ports, or ports associated with other protocols. You can check the “p2pCatchAll” checkbox which will cause any unrecognized traffic to be assumed as P2P traffic and its priority lowered accordingly. You can also set hard bandwidth limits for this traffic in the “Bandwidth” field underneath the “p2pCatchAll” checkbox. Below this is the “Enable/Disable specific P2P protocols” section. Here you can enable or disable specific services; there are about 20 listed, including BitTorrect, DCC, Gnutella, and others.

The next page covers network games. Many games require on low latency to deliver a good online gaming experience and good QoS. Other traffic, such as downloading large files, can easily swallow up the packets associated with the game itself and cause lag or disconnections. By checking the “Enable” checkbox at the top of the page, you can raise the priority of game traffic so that it will be transferred first and given a guaranteed chunk of bandwidth. Beneath that is a section called “Enable/Disable specific games“. There are many games listed here, including Call of Duty, Doom 3, Halo 2, Quake 3 and 4, and World of Warcraft. Even if your game is not listed, you may want to check a similar game so that you have a reference rule you can modify later.


QoS Management: Everything Else

Next is the “Raise or lower other Applications” page, which lists many other commonly available applications and protocols. How these protocols should be handled will depend on the environment that your pfSense box will be protecting. Applications such as VNC, PCAnywhere (both popular remote access programs), IRC, Teamspeak (popular messenger programs) are can be raised or lowered in priority (or kept at the default level), as well as protocols such as PPTP, IPsec, HTTP, SMTP, POP3 and IMAP. If you enabled p2pCatchAll, you will want to use these options to ensure that these other protocols are recognized and treated normally, rather than being penalized by the default p2pCatchAll rule.

Once you finish configuration on the “Raise of lower other Applications” screen and press the “Next” button, all the rules and queues will now be created, but not yet in use. By pressing the “Finish” button on the final screen, the rules will be loaded and active. Shaping will now be activated for all new connections. Due to the stateful nature of the shaper, however, only new connections will have the new rules applied. In order for the new configuration to be fully active on all connections, you must clear the states. To do this, navigate to Diagnostics -> States, click the “Reset States” tab, check, Firewall state table, then press the “Reset” button.

Now that you have enabled the traffic shaper, you can view the rules and queues defined when you invoked the wizard by navigating to Firewall -> Traffic Shaper and clicking on the different tabs. There should be a tree on the left side of the page; clicking on different parts of the tree should show different relevant QoS settings. For example, clicking on “qVoIP” will show the settings for the VoIP queue. But there will be more about this in a future blog posting.

Other Articles in This Series:

Traffic Shaping in pfSense: What it Does
Traffic Shaping Wizard: Introduction
Queue Configuration in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Layer 7 Groups in pfSense 2.1
Bandwidth Limiting with the pfSense Limiter
Deep Packet Inspection Using Layer 7 Traffic Shaping

External Links:

Traffic Shaping Guide at doc.pfsense.org

Link Ads:


Traffic Shaping in pfSense: What it Does

traffic shapingTraffic shaping is a computer network traffic management technique designed to delay some or all datagrams to bring them into compliance with a traffic profile. Without traffic shaping, packets are processed on a first in/first out basis by the firewall. Traffic shaping, or Quality of Service (QoS) offers a means of prioritizing different types of traffic. This ensures that higher priority services receive the bandwidth they need before lesser priority services. This helps to optimize or guarantee performance, improve latency, and/or increase usable bandwidth for some kinds of packets by delaying other kinds.

Another way of managing computer traffic is traffic policing. The difference between policing and shaping is that traffic policing propagates bursts. When the traffic rate reaches the configured maximum rate, excess traffic is dropped or remarked. The result is an output rate that appears on a graph as a saw-tooth with crests and troughs. In contrast to policing, traffic shaping retains excess packets in a queue and then schedules the excess for later transmission over increments of time. The result of traffic shaping is a smoothed packet output rate.


For purposes of this discussion, we are concerned mainly with traffic shaping in pf (and therefore pfSense). The way traffic shaping is accomplished in pf is that incoming traffic from the Internet going to a host on the LAN is actually shaped coming out of the LAN interface from the pfSense system. In the same manner, traffic going from the LAN to the Internet is shaped when leaving the WAN. This is because traffic has to be limited in a place where pf/pfSense can actually control the flow of data.

There are two means by which traffic shaping is accomplished: traffic shaping queues and traffic shaping rules. The queues are where bandwidth and priorities are actually allocated, while traffic shaping rules control how traffic is assigned into those queues. If a packet matches a shaper rule, it will be assigned into the queues specified by that rule. In that manner, traffic shaping rules are similar to firewall rules, with matching criteria and with outcomes dictated based on whether a packet matches the criteria.

Traffic Shaping: Reasons

The primary reasons you would use traffic shaping are to control access to available bandwidth, to ensure that traffic conforms to the policies established for it, and to regulate the flow of traffic in order to avoid congestion that can occur when the sent traffic exceeds the access speed of its target (remote) interface. Here are some examples why you might want to use traffic shaping:

  • Control access to bandwidth when policy dictates that the rate of a given interface should not on the average exceed a certain rate even though the access rate exceeds the speed.
  • If the network has differing access rates. Suppose that one end of the link in a Frame Relay network runs at 256 kbps and the other end of the link runs at 128 kbps. Sending packets at 256 kbps could cause failure of the applications using the link.
  • If you offer a subrate service. In this case, traffic shaping enables you to use the router to partition your T1 or T3 links into smaller channels.
  • Smoothing out asymmetric links, where the download speed differs from the upload speed (such as DSL connections). Some links are so out of balance that the maximum download speed is unattainable because it is difficult to send out enough ACK packets to keep traffic flowing. By using the traffic shaper to prioritize ACK packets you can achieve faster and more stable download speeds on asymmetric links.
  • Prioritizing VoIP calls. If your VoIP calls use the same circuit as data, then uploads and downloads may degrade your call quality. pf/pfSense can prioritize the call traffic above other protocols and ensure the calls make it through without breaking up.
  • Network gaming. There are also options to give priority to the traffic associating with networking gaming, even if you are downloading while playing.
  • P2P applications. By lowering the priority of traffic associated with known peer-to-peer ports, pf/pfSense ensures that P2P applications will not interfere with other traffic on your network.


Other Articles in This Series:

Traffic Shaping Wizard: An Introduction
QoS Management Using the Traffic Shaper Wizard
Queue Configuration in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Layer 7 Groups in pfSense 2.1
Bandwidth Limiting with the pfSense Limiter
Deep Packet Inspection Using Layer 7 Traffic Shaping

External Links:

Traffic shaping at Wikipedia

Comparing Traffic Policing and Traffic Shaping for Bandwidth Limiting [QoS Policing] at www.cisco.com

Ad Links:


© 2013 David Zientara. All rights reserved. Privacy Policy