Traffic Shaping in pfSense: Part Seven

Editing traffic shaping settings in pfSense.

Editing traffic shaping settings in pfSense.

After using the shaper wizard, you might find that the rules it generates do not fit your requirements. Fortunately, once the basic rules have been created by the wizard, it should be relatively easy to edit or copy those rules and create custom ones of your own.
The queues are where bandwidth and priorities are actually allocated. Each queue is assigned a priority from 0 to 7. When there is an overload of traffic, the higher-numbered queues are preferred over the lower-numbered queues. Each queue is assigned either a hard bandwidth limit, or a percantage of the total link speed. The queues can also be assigned other attributes that control how they behave. For example, they can be set up so they have low latency or they might have certain congestion avoidance algorithms applied. Queues may be changed by navigating to Firewall -> Traffic Shaper and clicking on the By Queues tab. A list of rules will apeear.

Editing queues can be a complex tast with powerful results. Still, without a thorough understanding of the settings involved, it is probably best to stick with the queues generated by the wizard and alter their settings.

The queue listings have changed somewhat in pfSense 2.2. Each queue is listed on the left side of the tab. Clicking on one of the queues will bring up a listing for each of that queues subordinate queues (one for each interface). Clicking on any of these subordinate queues will allow you to edit the settings for it. The screen capture at the top of this article shows the settings for one such queue. At the top of the page, there’s a check box which allows you to enable/disable the queue and its children. There are settings for the queue name, the queue priority (0-7), the queue limit in packets, and various scheduler options. There is also a field in which you can enter an optional description. At the bottom of the page, there are two buttons: a “Save“ button to save the queue and a “Delete this queue“ button to delete it. You should not attempt to delete a queue if it is being referenced by a rule.

External Links:

PF: Packet Queueing and Prioritization at openbsd.org

Traffic Shaping in pfSense: Part Four

Traffic shaping in pfSense

Configuring VoIP settings in pfSense 2.2.4. Note that you can guarantee upload and download bandwidth with the traffic shaper wizard.

Once you enter the queuing disciples and connection speeds in the traffic shaper wizard, there are a number of other options to configure. The next is Voice over IP, and there are several options available for handing VoIP traffic. The first choice, the Prioritize Voice over IP traffic check box, is self-explanatory. It will enable the prioritization of VoIP traffic, and this behavior can be fine-tuned by the other settings on the same page. First, you can chose your VoIP provider:

 

    • VoicePulse: A U.S.-based VoIP provider founded in 2003. VoicePulse provides not only home phone services, but also business PBX services and enterprise-level SIP trunking.

 

  • Vonage: Another U.S.-based VoIP provider founded in 2001. Their most popular plan, Vonage World, offers unlimited international calling to over 60 countries for a flat monthly rate. Vonage supplies an analog telephone adapter with which the customer can connect standard analog telephones to the Internet.

 

 

  • Panasonic TDA: Panasonic’s VoIP PBX solution, done via a T1 or E1, and which provides mobile phone integration and BRI or PRI ISDN capability.

 

 

  • Asterisk: Open-source VoIP software which includes many features available in proprietary PBX systems: voice mail, conference calling, interactive voice response, and automatic call distribution. Although initially developed in the United States, it has become popular worldwide because it is freely available under open-source licensing and has a modular, extensible design.

 

 

If you have a different provider, you can choose Generic, or override this setting with the Address field by entering the IP of your VoIP phone or an alias containing the IPs of all your phones.

There is also an edit box in which you can enter the IP address of the upstream SIP server. If you do, the information in the Provider field will be overridden. You can also use a firewall alias in this field.

You may also choose the amount of upload and download bandwidth to guarantee for your VoIP phones. This will vary based on how many phones you have, and how much bandwidth each session will utilize. When you have finished entering the provider information and upload/download bandwidth, you can press the Next button.

The next page allows you to configure settings for the penalty box. This is a place to which you can relegate misbehaving users or devices that would otherwise consume more bandwith than desired. These users are assigned a hard bandwidth cap which they cannot exceed. Check the check box at the top of the page to enable this feature, enter an IP or alias in the address box, and then enter upload and download limits in kilobits per second in the appropriate edit boxes. It does not appear that you can type multiple IP addresses in the Address edit box, so if you want to penalize multiple hosts, you will have to create an alias.

Once you are finished configuring penalty box settings, you can press the Next button and move on to configuring settings for peer-to-peer networking, which will be covered in the next article.

External Links:

Traffic Shaping at Wikipedia
Voice over IP at Wikipedia

Traffic Shaping in pfSense: Part One

Traffic Shaping with pfSense

Using the traffic shaping wizard in pfSense 2.2.4.

Traffic shaping, otherwise known as network Quality of Service (QoS), is a means of prioritizing the network traffic crossing your firewall. Without traffic shaping, all packets are processed on a first in/first out basis by your firewall. QoS offers a means of prioritizing different types of traffic, ensuring that high priority services receive the bandwidth they need before lesser piroity services. The traffic shaper wizard in pfSense gives you the ability to quickly configure QoS for common scenarios, and custom rules may also be created for more complex tasks.

Traffic shaping is essentially like a gatekeeper in which important packets are prioritized, while regular packets have to wait, and low-priority packets are kept out until there is not enough higher-priority traffic to use up the bandwidth.

There are traffic shaping queues and traffic shaping rules. The queues are where bandwidth and priorities are actually allocated. Traffic shaping rules control how traffic is assigned into those queues. Rules for the shaper work in a similar way to firewall rules, and allow similar matching characteristics. If a packet matches a shaper rule, it will be assigned into the queues specified by that rule.

The idea of raising or lowering the priority of packets is a simple one, but one which has many possible applications. Here are a few ways in which traffic shaping can be used.

Traffic Shaping in pfSense: Prioritizing ACK Packets

Asymmetric Internet connections (where the download speed differs from the upload speed, usually in such a way that download speed > upload speed) are commonplace, especially with DSL. Some links are so out of balance that the maximum download speed is almost unattainable because it is difficult for the client to send back enough ACK packets to keep traffic flowing. ACK packets are transmitted back to the sender by the receiver to indicate that data has been successfully received, and to signal that it is OK to send more. If the sender does not receive ACKs in a timely manner, TCP’s congestion control will be invoked and it will slow down the connection.

This can happen if you are uploading and downloading simultaneously over an asymmetric connection. The uploading part of the circuit is full from the file upload, and there is little room to send ACK packets which allow downloads to keep flowing. By using the shaper to prioritize ACK packets, you can achieve faster, more stable download speeds on asymmetic links. [This is not as important on symmetric links, but it may still be desirable if the available outgoing bandwidth is heavily utilized.]

Traffic Shaping in pfSense: VoIP, Online Gaming and Peer-to-Peer Traffic

If your VoIP calls use the same circuit as data, then uploads and downloads may degrade your call quality. pfSense can prioritize the call traffic above other protocols and ensure that the calls make it through clearly without breaking up. If there are other transfers occurring simultaneously when the VoIP call is in progress, the speed of the other transfers will be reduced to leave room for the calls.

There are also options in pfSense to give priority to the traffic associated with network gaming. Similar to prioritizing VoIP calls, the effect is that even if you are downloading while playing, the response time of the game should be nearly as fast as if the rest of your connection were idle.

In addition, by lowering the priority of traffic associated with known peer-to-peer ports, you will have the assurance that even if these programs are in use, they won’t hinder other traffic on your network. Due to peer-to-peer traffic’s lower priority, other protocols will be favored over P2P traffic, which will be limited when any other services need the bandwidth.

In the next article, we will discuss some of the limitations of pfSense’s traffic shaper.

External Links:

Traffic Shaping at Wikipedia

Deep Packet Inspection Using Layer 7 Traffic Shaping

Deep packet inspection

The Layer 7 tab in the Traffic Shaper in pfSense 2.1.

For quite a while, traffic shaping has been considered an integral part of any good firewall. This necessitates some means of classifying the traffic so the traffic can then be policed. The traditional method of traffic shaping centered on classifying traffic based on network and transport data fields, not using deep packet inspection. This usually centered around the following elements:

  • Service class marks
  • Source and/or destination IP addresses
  • Ports

However, these methods are not always effective in traffic classification. This is especially the case with P2P traffic, which often uses random, non-default ports. An HTTP server utilizing port hopping and encrypted traffic may also defy level 3 (network) and level 4 (transport) classification.


Enter Layer 7 Deep Packet Inspection

One possible solution to the shortcomings of network and transport level classification is layer 7 (L7) classification, which involves deep packet inspection. In L7 classification, user traffic can be identified based on an application pattern, which is a sort of signature used by an application during its communications. All applications either use a specific application pattern or may share the pattern with other applications.

Deep packet inspection

Configuring P2P options in the wizard.

IPCop is a good example of utilizing L7 deep packet inspection and classification. IPCop is a Linux-based firewall that was originally a fork of the SmoothWall firewall. Although not an official part of IPCop, an advanced QoS (Quality of Service) add-on is available. But while IPCop can support classification by application protocol, it does not allow the definition of shaping policies. Rather, it can only block such traffic, which greatly limits the use of this feature.

pfSense, however, has fully incorporated L7 deep packet inspection and classification into its traffic shaper. Traffic shaping is achieved in pfSense through AltQ, which makes available Class Based Queueing (CBQ), Priority Queueing (PRIQ) and Hierarchical Fair Service Curve (HFSC). All of these can be configured automatically through the use of a wizard. Beginning with pfSense 2.0, an additional shaping mechanism called Dummynet became available. Dummynet was originally designed for the ipfw firewall, and has a related application called ipfw-classifyd. This application is able to produce blocking rules for incoming traffic or perform traffic shaping by assigning IP packets to an AltQ queue or a Dummynet pipe or queue. It was modified to work with the pf firewall and is the component responsible for L7 classification. It also allows different types of operations to be applied to an identified application protocol, usually either blocking it or assigning it to a limiter or queue.

In order to invoke ipfw-classifyd, pf uses divert sockets. Essentially, it interrupts the normal flow of packets and sends them to a listening socket (ipfw-classifyd). Overhead is kept to a minimum by teaching pf about the actions to be taken ahead of time and by limitng the number of packets that are diverted from the kernel to the application. All of this is controlled via a graphical interface, in which the user must specify at least one protocol (but may specify more than one). The user can create L7 rules groups containing one or more L7 rules. The user can take any one of the created rules groups and assign it a firewall rule.

But with pfSense, the user does not have to explicitly create L7 rules groups. This is because the Traffic Shaper Wizard in versions 2.0 and newer invokes L7 classification in the Peer-to-Peer and Network Games sections. In both sections, the select box on the top of the page can be enabled, and the related protocols or applications can be blocked one by one. Finally, the user can extend the functionality of L7 packet inspection by uploading new application patterns to the system. This feature is important when the user wants to block an application that uses a protocol pattern that is not defined in the system. If such a pattern is uploaded to the system, it only appears in the list of protocols when a container is created or modified. It does not affect the Traffic Shaper Wizard, which remains unchanged.


Other Articles in This Series:

Traffic Shaping in pfSense: What it Does
Traffic Shaping Wizard: Introduction
Queue Configuration in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Layer 7 Rules Groups in pfSense 2.1
Bandwidth Limiting with the pfSense Limiter

External Links:

L7 Classification and Policing in the pfSense Platform – a scholarly paper about the addition of layer 7 deep packet inspection to pfSense 2.0.

Ad Links:

Layer 7 Rules Groups in pfSense 2.1

Layer 7

Adding a layer 7 rules group in pfSense 2.1.

In the previous article, I described how to create a traffic shaping rule to place BitTorrent traffic into the P2P queue. Another way of directing traffic into queues is to create a layer 7 rules group. In this article, I will describe how to do this.

Traditionally, network traffic is identified by looking at IP packet fields or by referring to which port is being used. In the OSI network model, this method is limited to looking at layers 3 and 4. This is highly constricting, but fortunately there is a better way. We can inspect packets at the application layer (also known as deep packet inspection), which provides us with a powerful solution for controlling traffic based on application patterns. Since this functionality is built into pfSense 2.0 and later, we can easily create rules for layer 7 inspection.


Creating an Layer 7 Rules Group

As an illustration, I will again turn to the example of limiting bandwidth used by BitTorrent traffic by placing it in the P2P queue. First, navigate to Firewall -> Traffic Shaper, and click on the Layer 7 tab. Once there, click on the “plus” button to add a new Layer 7 rule. At “Enable/Disable“, check the checkbox to enable this layer 7 container. At “Name“, you can enter a name, and at “Description“, you can enter a description that will not be parsed. At “Rule(s)“, press the “plus” button to add one or more rules. There are three dropdown boxes: “Protocol“, “Structure“, and “Behaviour“. For “Protocol“, you can select any one of dozens of protocols; I won’t list all of them here, but some of the more significant ones are:

  • DHCP: Dynamic Host Configuration Protocol, an application level netwprk protocol used to configure devices that are connected to a network so they can communicate on that network using the Internet Protocol (IP).
  • Finger: The Finger user information protocol, which provides basic user information on some systems.
  • HTTP: Hypertext Transfer Protocol, the main application protocol for the World Wide Web.
  • UUCP: Unix-to-Unix Copy, a suite of computer programs and protocols allowing remote execution of commands and transfer of files, email, and netnews between computers.

In our case, we’ll choose “bittorrent” as the protocol. Under “Structure“, we can choose either “action” or “queue”. “action” seems to have one option under “Behaviour“: “block”. Since we don’t want to block bittorrent traffic, but instead want to put it in the P2P queue, we select “queue”. For “Behavior“, we select “qP2P” (the P2P queue). We could add another rule, but instead we will press the “Save” button to save the rules group, and “Apply changes” on the next page.

This covers how to add an layer 7 rules group. But there is an alternative way of adding a layer 7 rules group: when you first click on the “Layer 7” tab, there should be a hyperlink to add new layer 7 protocol patterns. Click on this link, then on the “Add layer7 pattern” page, press the “Choose” button and select a file with the file dialog box. When you are done, press the “Upload Pattern file” button to upload the file.

This article should be enough to get you started with using layer 7 rules groups, but if you want a more in-depth explanation of Layer 7 traffic control and how it was implemented in pfSense, you may want to read this scholarly paper on L7 in the pfSense platform (also linked to in the external links section).


Other Articles in This Series:

Traffic Shaping in pfSense: What it Does
Traffic Shaping Wizard: Introduction
Queue Configuration in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Bandwidth Limiting with the pfSense Limiter
Deep Packet Inspection Using Layer 7 Traffic Shaping

External Links:

L7 Classification and Policing in the pfSense Platform – a more comprehensive explanation of layer 7 rules and their integration into pfSense.

Traffic Shaping Guide at doc.pfsense.org

Ad Links:


Queue Configuration in pfSense 2.1

traffic shaping queue

Configuring the qVoIP traffic shaping queue under pfSense 2.1.

In the previous articles, I covered the basics of the traffic shaper and how to configure it using the wizard. In this article, we’ll examine how to monitor traffic shaping and some advanced configuration techniques.

In order to make sure the traffic shaper is working as it should, you can monitor it. In order to do so, navigate to Status -> Queues. This screen will show each queue listed by name, its current usage, and some other statistics. The graphical bar on this screen shows you how full a queue is. The rate of data in the queue is shown as both packets per second (or pps) and bits per second (or b/s). Borrows happen when a neighboring queue is not yet full and capacity is borrowed from there when needed. Drops happen when traffic in a queue is dropped in favor of higher priority traffic. It is normal to see drops, and this does not mean that a full connection is dropped, just a packet. Usually, one side of the connection will see that a packet was missed and then resend it, often slowing down to avoid future drops. The suspends counter indicates when a delay action happens.

If you find that the rules generated by the traffic shaper do not quite fit your requirements, you may want to edit the shaper queues manually. You may have a service that is not handled by the wizard, a game that uses a different port, or other services not covered by the wizard. It should be relatively easy to cover these cases.

The queues are where bandwidth and priorities are actually allocated. In pf, each queue is assigned a priority, from 0 to 7. When there is an overload of traffic, the higher-numbered queues are preferred over the lower-numbered ones. Each queue is assigned either a hard bandwidth limit or a percentage of the total link speed. The queues can be assigned other attributes that control how they behave. Queues may be changed by navigating to Firewall -> Traffic Shaper and clicking on the Interfaces tab. You will see a list of rules here, sorted by interface. Clicking on one of the queues will enable you to edit the settings for it.


Queue Configuration Options

Editing queues is not easy, and if you do not have a thorough understanding of the settings involved, it is best to leave the settings alone. That having been said, here are the options. “Enable/Disable queue and its children” and “Queue Name” are self-explanatory. “Priority” assigns a priority to the queue from 0 to 7 (7 is highest). The “Queue limit” field specifies the queue limit in packets per second. Next is “Scheduler options“; there are five different options here:

  • Default Queue: Selects this queue as the default, the one which will handle all unmatched packets. Each interface should have one and only one default queue.
  • Random Early Detection (RED): This is a method to avoid congestion on a link; it will actively attempt to ensure that the queue does not get full. If the bandwidth is above the maximum given for the queue, drops will occur. Also, drops may occur if the average queue size approaches the maximum. Dropped packets are chosen at random, so the more bandwidth in use by a given connection, the more likely it is to see drops. The net effect is that the bandwidth is limited in a fair way, encouraging a balance. RED should only be used with TCP connections; TCP is capable of handling lost packets and can resend when needed.
  • Random Early Detection In and Out (RIO): Enables RED with in/out, which will result in having queue averages being maintained and checked against for incoming and outgoing packets.
  • Explicit Congestion Notification (ECN): Along with RED, it allows the sending of control messages that will throttle connections if both ends support ECN. Instead of dropping the packets as RED will normally do, it will set a flag in the packet indicating network congestion. If the other side sees and obeys the flag, the speed of the ongoing transfer will be reduced.
  • CoDel Active Queue: A newish form of AQM (Active Queue Management) designed to solve at least in part the problem of bufferbloat (persistently full buffers). The three primary innovations of CoDel are: [1] using the local minimum queue as the queue measure; [2] simplified single-state variable tracking of how long the minimum has been above or below the target value for standing queue delay rather than keeping a window of values to compute the minimum; and [3] use of queue-sojourn time rather than measuring queue size in bytes or packets. This simple description does not do justice to the CoDel AQM, but you can read more about it at the ACM white paper on CoDel: (HTML PDF).

The next two fields are fairly self-explanatory: at “Description“, you can enter a brief description, and at “Bandwidth“, you can specify the total bandwidth allocated to this queue. The options for “Service Curve” are interesting because they enable you to configure a Hierarchical Fair Service Curve (HFSC) for the queue. I covered HFSC in somewhat greater depth in a previous article, but here are the options in a nutshell:

  • m1: Burstable bandwidth limit
  • d: Time limit for the bandwidth burst, specified in milliseconds
  • m2: Normal bandwidth limit

For example, you need m1 bandwidth within d time, but with a normal maximum of m2. Within the time set by d, m1 is the maximum. After d milliseconds has expired, if the traffic is still above m2, it will be shaped. If you read my article about HFSC or any of the other material relating to it, you will see that m1 and m2 are two different slopes for the curve.


Each of the values m1, d and m2 can be set for the following uses:

  • Upper Limit: Maximum bandwidth allowed for the queue. Will do hard bandwidth limiting.
  • Real Time: Minimum bandwidth guarantee for the queue. This is valid only for child queues.
  • Link Share: The bandwidth share of a backlogged queue. It will share the bandwidth between classes if the Real Time guarantees have been satisfied.

Finally, at the bottom of the page, there is a “Save” button to save the changes to a queue, “Add new queue” if you want to add a queue, and “Delete queue” to delete it. This covers the queue options in pfSense 2.1. In writing this article, it has occurred to me that I have not yet addressed some of the configuration and troubleshooting issues involved with traffic shaping, so these subjects will likely be topics for future blog postings.

Other Articles in This Series:

Traffic Shaping in pfSense: What it Does
Traffic Shaping Wizard: Introduction
QoS Management Using the Traffic Shaper Wizard
Traffic Shaping Rules in pfSense 2.1
Layer 7 Groups in pfSense 2.1
Bandwidth Limiting with the pfSense Limiter
Deep Packet Inspection Using Layer 7 Traffic Shaping

External Links:

Traffic Shaper in pfSense 2.0 – doesn’t cover 2.1, but it’s still a good article.

Controlling Queue Delay at queue.acm.org – this is the white paper on the CoDel AQM.

Link Ads:


QoS Management Using the Traffic Shaper Wizard

In this article, we will go through the pfSense traffic shaper wizard to achieve quality of service (QoS) goals and cover some of the options which are configurable through the wizard.

QoS Management: Queueing Disciplines and Bandwidth

QoS

Specifying the number of WAN connections in the wizard.

In the wizard, you first have to specify the number of WAN connections, and if you selected multi LAN, the number of LAN connections. On the next screen, there are several more options. At “Download Scheduler“, there are three options for queueing discipline: HFSC (Hierarchical Fair Sharing Curve), which is designed to ensure that link delay is low while bandwidth is not over-reserved. CBQ (Class-Based Queueing) allows for bandwidth to be shared equally among different classes, while PRIQ (Priority Queueing) allows for different priority levels to be assigned to classes. Under “Setup connection speed and scheduler information for WAN #n“, at “Interface” you select a valid interface. At “Upload Scheduler“, you chose a queueing discipline (again, the options are HFSC, CBQ and PRIQ). Finally, the “Connection Upload” and “Connection Download” speed must be entered.

QoS Management: VoIP, P2P, and Network Games

QoS

Configuring VoIP traffic with the wizard in pfSense 2.1.

On the next screen, there are various QoS options for VoIP traffic. The first checkbox, “Prioritize Voice over IP traffic“, is self-explanatory. The “Provider” drop-down box allows you to specify your VoIP provider. There are a few well-known providers, including Vonage, Voicepulse, and PanasonicTDA, and there’s Asterisk as well, in case you connect to an Asterisk server. If you have a different provider, you can choose “Generic“, or override this setting with the “Upstream SIP Server” field by entering the IP of your VoIP phone or an Alias containing the IPs of all your phones. With the next two fields, “Upload bandwidth for each WAN” and “Download bandwidth (speed) for Voice over IP phones“, you can choose the amount of bandwidth to guarantee to your VoIP phones. The amount of bandwidth you actually use will vary based on how many phones you have and how much each session will use.


The next screen contains options for the “Penalty Box“. The penalty box is a place where you can relegate misbehaving users or devices that would otherwise consume more bandwidth than desired. Click on the “Enable” checkbox to enable this feature, and enter the IP address of the computer to penalize at “Address“. At the “Bandwidth” field, enter the limit you wish to apply.

QoS

Configuring P2P options in the wizard.

The next screen covers “Peer to Peer networking“. Click on the “Enable” checkbox to lower the priority of P2P traffic below all other traffic. By design, P2P protocols and software will utilize all available bandwidth unless limits are put in place. If you expect P2P traffic on your network, it is a good idea to ensure that other traffic will not suffer degradation of QoS due to its use.

Many P2P technologies will deliberately try to avoid detection; Bittorrent is a good example of this. It often utilizes non-standard or random ports, or ports associated with other protocols. You can check the “p2pCatchAll” checkbox which will cause any unrecognized traffic to be assumed as P2P traffic and its priority lowered accordingly. You can also set hard bandwidth limits for this traffic in the “Bandwidth” field underneath the “p2pCatchAll” checkbox. Below this is the “Enable/Disable specific P2P protocols” section. Here you can enable or disable specific services; there are about 20 listed, including BitTorrect, DCC, Gnutella, and others.

The next page covers network games. Many games require on low latency to deliver a good online gaming experience and good QoS. Other traffic, such as downloading large files, can easily swallow up the packets associated with the game itself and cause lag or disconnections. By checking the “Enable” checkbox at the top of the page, you can raise the priority of game traffic so that it will be transferred first and given a guaranteed chunk of bandwidth. Beneath that is a section called “Enable/Disable specific games“. There are many games listed here, including Call of Duty, Doom 3, Halo 2, Quake 3 and 4, and World of Warcraft. Even if your game is not listed, you may want to check a similar game so that you have a reference rule you can modify later.


QoS Management: Everything Else

Next is the “Raise or lower other Applications” page, which lists many other commonly available applications and protocols. How these protocols should be handled will depend on the environment that your pfSense box will be protecting. Applications such as VNC, PCAnywhere (both popular remote access programs), IRC, Teamspeak (popular messenger programs) are can be raised or lowered in priority (or kept at the default level), as well as protocols such as PPTP, IPsec, HTTP, SMTP, POP3 and IMAP. If you enabled p2pCatchAll, you will want to use these options to ensure that these other protocols are recognized and treated normally, rather than being penalized by the default p2pCatchAll rule.

Once you finish configuration on the “Raise of lower other Applications” screen and press the “Next” button, all the rules and queues will now be created, but not yet in use. By pressing the “Finish” button on the final screen, the rules will be loaded and active. Shaping will now be activated for all new connections. Due to the stateful nature of the shaper, however, only new connections will have the new rules applied. In order for the new configuration to be fully active on all connections, you must clear the states. To do this, navigate to Diagnostics -> States, click the “Reset States” tab, check, Firewall state table, then press the “Reset” button.

Now that you have enabled the traffic shaper, you can view the rules and queues defined when you invoked the wizard by navigating to Firewall -> Traffic Shaper and clicking on the different tabs. There should be a tree on the left side of the page; clicking on different parts of the tree should show different relevant QoS settings. For example, clicking on “qVoIP” will show the settings for the VoIP queue. But there will be more about this in a future blog posting.

Other Articles in This Series:

Traffic Shaping in pfSense: What it Does
Traffic Shaping Wizard: Introduction
Queue Configuration in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Layer 7 Groups in pfSense 2.1
Bandwidth Limiting with the pfSense Limiter
Deep Packet Inspection Using Layer 7 Traffic Shaping

External Links:

Traffic Shaping Guide at doc.pfsense.org

Link Ads:


Traffic Shaping Wizard: Introduction

traffic shaping wizard

These are the options seen under the Wizards tab in the Traffic Shaper in pfSense 2.1.

Traffic Shaping Wizard Introduced

You can configure the traffic shaper through one of the traffic shaping wizards, manually through the pfSense web GUI, or even at by editing pf.conf in FreeBSD at the command line. It is recommended, however, that when you are using the traffic shaper for the first time, you use the traffic shaping wizard, which will guide you through the process. Due to the complexity of the shaper queues and rules, it is not a good idea to attempt starting out configuring the traffic shaper manually. If you need custom rules, step through the wizard and approximate what you will need, then make the custom rules afterward. Each screen will set up unique queues and rules that will control what traffic is assigned to the queues. If you want to configured everything manually, simply specify your WAN speed at the first screen, then click Next through all the remaining screens without actually configuring anything.


To start the traffic shaping wizard under pfSense 2.1, navigate to Firewall -> Traffic Shaper, and click on the “Wizards” tab. There are several different wizards, each one pertaining to a different pfSense network interface setup:

  • Single LAN multi WAN: There is one local network, but there are multiple interfaces for the WAN (this would be common in a load balancing scenario, in which outgoing and incoming traffic is distributed among two or more WAN interfaces). NOTE: You would also use this in the case where you only have one WAN and one LAN. Just specify 1 for the number of WAN links when going through the wizard
  • Single WAN multi LAN: There is only a single WAN interface, but there are multiple local networks.
  • Multiple LAN/WAN: This will set up traffic shaping in cases where you have both multiple local networks and multiple WAN networks.
  • Dedicated Links: This is for when you want to manage more than one link which gets routerd to a separate, different LAN in the same box. For example, assume you were providing services to 4 different customers in one building and they each had their own separate internet connections. You could run all 4 internet connections through a single pfSense box and provide different traffic shaping configurations for each. Thus, you could have several “virtual” links through a single interface.

In the next article, we will begin working our way through the traffic shaping wizard and look at the different options available within it.


Other Articles in This Series:

Traffic Shaping in pfSense: What it Does
QoS Management Using the Traffic Shaper Wizard
Queue Configuration in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Layer 7 Groups in pfSense 2.1
Bandwidth Limiting with the pfSense Limiter
Deep Packet Inspection Using Layer 7 Traffic Shaping

External Links:

Traffic shaping at Wikipedia

External Links:

Traffic Shaping Guide at doc.pfsense.org

Link Ads:


Traffic Shaping in pfSense: What it Does

traffic shapingTraffic shaping is a computer network traffic management technique designed to delay some or all datagrams to bring them into compliance with a traffic profile. Without traffic shaping, packets are processed on a first in/first out basis by the firewall. Traffic shaping, or Quality of Service (QoS) offers a means of prioritizing different types of traffic. This ensures that higher priority services receive the bandwidth they need before lesser priority services. This helps to optimize or guarantee performance, improve latency, and/or increase usable bandwidth for some kinds of packets by delaying other kinds.

Another way of managing computer traffic is traffic policing. The difference between policing and shaping is that traffic policing propagates bursts. When the traffic rate reaches the configured maximum rate, excess traffic is dropped or remarked. The result is an output rate that appears on a graph as a saw-tooth with crests and troughs. In contrast to policing, traffic shaping retains excess packets in a queue and then schedules the excess for later transmission over increments of time. The result of traffic shaping is a smoothed packet output rate.


For purposes of this discussion, we are concerned mainly with traffic shaping in pf (and therefore pfSense). The way traffic shaping is accomplished in pf is that incoming traffic from the Internet going to a host on the LAN is actually shaped coming out of the LAN interface from the pfSense system. In the same manner, traffic going from the LAN to the Internet is shaped when leaving the WAN. This is because traffic has to be limited in a place where pf/pfSense can actually control the flow of data.

There are two means by which traffic shaping is accomplished: traffic shaping queues and traffic shaping rules. The queues are where bandwidth and priorities are actually allocated, while traffic shaping rules control how traffic is assigned into those queues. If a packet matches a shaper rule, it will be assigned into the queues specified by that rule. In that manner, traffic shaping rules are similar to firewall rules, with matching criteria and with outcomes dictated based on whether a packet matches the criteria.

Traffic Shaping: Reasons

The primary reasons you would use traffic shaping are to control access to available bandwidth, to ensure that traffic conforms to the policies established for it, and to regulate the flow of traffic in order to avoid congestion that can occur when the sent traffic exceeds the access speed of its target (remote) interface. Here are some examples why you might want to use traffic shaping:

  • Control access to bandwidth when policy dictates that the rate of a given interface should not on the average exceed a certain rate even though the access rate exceeds the speed.
  • If the network has differing access rates. Suppose that one end of the link in a Frame Relay network runs at 256 kbps and the other end of the link runs at 128 kbps. Sending packets at 256 kbps could cause failure of the applications using the link.
  • If you offer a subrate service. In this case, traffic shaping enables you to use the router to partition your T1 or T3 links into smaller channels.
  • Smoothing out asymmetric links, where the download speed differs from the upload speed (such as DSL connections). Some links are so out of balance that the maximum download speed is unattainable because it is difficult to send out enough ACK packets to keep traffic flowing. By using the traffic shaper to prioritize ACK packets you can achieve faster and more stable download speeds on asymmetric links.
  • Prioritizing VoIP calls. If your VoIP calls use the same circuit as data, then uploads and downloads may degrade your call quality. pf/pfSense can prioritize the call traffic above other protocols and ensure the calls make it through without breaking up.
  • Network gaming. There are also options to give priority to the traffic associating with networking gaming, even if you are downloading while playing.
  • P2P applications. By lowering the priority of traffic associated with known peer-to-peer ports, pf/pfSense ensures that P2P applications will not interfere with other traffic on your network.


Other Articles in This Series:

Traffic Shaping Wizard: An Introduction
QoS Management Using the Traffic Shaper Wizard
Queue Configuration in pfSense 2.1
Traffic Shaping Rules in pfSense 2.1
Layer 7 Groups in pfSense 2.1
Bandwidth Limiting with the pfSense Limiter
Deep Packet Inspection Using Layer 7 Traffic Shaping

External Links:

Traffic shaping at Wikipedia

Comparing Traffic Policing and Traffic Shaping for Bandwidth Limiting [QoS Policing] at www.cisco.com

Ad Links:


pfSense Traffic Shaping: Part Three (Class Based Queuing and Priority Queuing)

Class-Based QueuingIn the last article, I covered traffic shaping with Hierarchical Fair Service Curve (HFSC). In this article, I cover the other two algorithms implemented in pfSense 2.0: Class Based Queuing (CBQ) and Priority Queuing (PRIQ).

Class Based Queuing (CBQ)

Class based queuing is a network router queuing method that allows traffic to share bandwidth equally, after being grouped by classes. It was developed by the Network Research Group at Lawrence Berkeley National Laboratory as an alternative to traditional router-based technology. Class based queuing divides user traffic into a hierarchy of queues, or classes, based on any combination of IP addresses, protocols and application types. It provides a means of implementing guaranteed service on a network.

Class based queuing enables you to generate several classes, and even classes within classes. As an example, you may have a 10 Gbps connection to the internet which is to be shared by your customers and for your company’s needs. You cannot allow a few people at the office to steal away large amounts of bandwidth which you should sell to your customers. On the other hand, your customers should not be able to drown out the traffic from your field offices to the customer database.


One way of solving this problem was to either use frame relay/ATM and create virtual circuits. This works, but frame relays are not very fine-grained and ATM is inefficient at carrying IP traffic. Moreover, neither have standardized ways to segregate different types of traffic into different virtual circuits. If you use ATM with Linux, then Linux can do traffic classification, which would help. Another way is to order separate connections, but this is not very practical and does not solve all your problems.

One possible solution is class based queuing. Clearly, you have two main classes, which we can call “ISP” and “Office”. We could further subdivide these classes, but for the moment we won’t. You decide the customers should always be guaranteed 8 Gbps of downstream traffic, and your office 2 Gbps. With CBQ, you can simply set up a root class and two subclasses, ISP and Office. The upper bound on ISP’s bandwidth is 8 Gbps, and the upper bound on Office’s is 2 Gbps. This is not the most sophisticated form of bandwidth management, but it will work.

In this example, you may find that there are times when ISP customers are mostly offline, but your office only gets 2 Gbps. You could make the Office class unbounded, so the office class can borrow bandwidth from the ISP class.

You can also go further than this, introducing subclasses. If employees at the office start using peer-to-peer software, for example, the database may run out of bandwidth. Therefore, you create two subclasses: “Human” and “Database”. In this scenario, you allocate 500 Mbps to Database and allocate the rest (1.5 Gbps) to Human.

Class based queuing assigns each queue a priority level. Queues with a higher priority are preferred during congestion over queues with a lower priority as long as both queues share the same parent (in other words, as long as both queues are on the same branch in the hierarchy). Queues with the same priority are processed in a round-robin fashion. For example, we could assign priorities in the above example like this:

  • Office (priority 1)
    • Human (priority 2)
    • Database (priority 3)
  • ISP (priority 1)

In this example, Office and ISP traffic will be processed in a round-robin fashion – they have the same priority level. But within the Office class/queue, Database gets preferential treatment because it has a higher priority level than Human. Notice that Database is not compared to Office or ISP for priority levels because Office and ISP are not on the same level of the hierarchy.


Priority Queuing (PRIQ)

Priority Queuing assigns multiple queues to a network interface with each queue being given a priority level. A queue with a higher priority is always processed ahead of a queue with a lower priority. If two or more queues are assigned the same priority then those queues are processed in a round-robin fashion.

Unlike class based queuing, which has a hierarchical queue structure, the queuing structure in PRIQ is flat – you cannot define queues within queues. The root queue is defined, which sets the total amount of bandwidth that is available, and then sub queues are defined under the root. Consider the following example:

Root Queue (2 Mbps)

  • Queue A (priority 1)
  • Queue B (priority 2)
  • Queue C (priority 3)

The root queue is defined as having 2 Mbps of bandwidth available to it and three subqueues are defined. The queue with the highest priority (the highest priority number) is served first. Once all the packets in that queue are processed, or if the queue is found to be empty, PRIQ moves onto the next highest priority. Within a given queue, packets are processes in a first in first out (FIFO) manner.

One attribute of PRIQ that should be noted is that PRIQ always processes a higher priority queue before a lower priority one. Consequently, it is possible for a high priority queue to cause packets in a lower priority queue to be delayed or dropped if the high priority queue is receiving a constant stream of packets. Therefore, if you use PRIQ, you want to plan your queues carefully.

External Links:

Class based queuing on Wikipedia

Packet Queuing and Prioritization

© 2013 David Zientara. All rights reserved. Privacy Policy