Log File Analysis with LightSquid

log file

The “Settings” tab of LightSquid in pfSense 2.1.3.

In the previous article, we covered installation and configuration of Squid under pfSense. In this article, we will cover how to monitor Internet usage with Squid and by installing and using the Squid log file analyzer LightSquid.

LightSquid Installation

Like other packages, LightSquid can easily be installed through the pfSense package manager. To access the package manager, navigate to System -> Packages. Scroll down to LightSquid and click on the plus symbol on the right side of the package to start the installation. Click the “Confirm” button on the next screen to confirm the installation, and LightSquid will be installed within a few minutes. When the installation is complete, there will be a new entry on the “Status” menu called “Proxy Report“.

LightSquid Settings for Log File Analysis

Here are some of the settings available (configurable by navigating to Status -> Proxy report and clicking on the “Settings” tab):

Language: Here you can select the language in which LightSquid reports are displayed in.

Bar color: This setting lets you control the color of the parts in the reports.

Report scheme: Lets you choose the theme for the appearance of the reports. Unless you have a preference for one of the alternate themes, you can leave it as the default of “Base”.

IP resolve method: When parsing the log files, LightSquid attempts to resolve the IP address into domain names. You can change the method it uses to resolve the IPs with this setting. The choices are: IP (just return the IP address); Demo (return AUTHNAME; if AUTHNAME not available return DNSNAME; if DNSNAME not available, return IP address); DNS (return DNSNAME); Simple (return AUTHNAME; if not available return IP); SMB (return smb name of PC); and Squidauth (return AUTHNAME; if not available, return IP address, and allow cryllic names).

log file

A LightSquid user access report.

Refresh sheduler: This setting affects how often the Squid log files are analyzed. Decreasing the value will make the reports stay more up to date but will consume more system resources. Be careful not to set the refresh cycle to occur to frequently. If the system cannot finish one update before another one is requested, you will eventually crash the system.

Skip url: If there are any URLs you do not want to show up in the reports, you can list them here.

To view the reports, click on the “Lightsquid Report” tab. If you get an error message when you click on this tab, you may have to run lightparser.pl (to parse the log files) from the command line on your pfSense box. [To do this, SSH into your pfSense box or access the console directly, drop to the shell, and type in /usr/local/www/lightsquid/lightparser.pl.] Once reports have been generated, you should be able to navigate them. After you select a day you will see a list of clients that accessed the proxy on that day. Once you select a host from this list, you will see a list of all the URLs accessed by that client. Clicking the clock icon at the top of the page will show you the time of the day that each URL was accessed.

External Links:

LightSquid official site

Monitoring Internet Usage with LightSquid and pfSense at hubpages.com

pfSense Load Balancing: Part One

pfSense Load Balancing

Configuring OPT1 as WAN2 so we can set up a gateway group later on.

In computer networking, load balancing is a method for distributing workloads across multiple computers or a computer cluster, network links, CPUs, storage devices, or other resources. When load balancing is employed, we are looking not just to distribute workloads but to optimize resource use, maximize throughput, minimize response time, and avoid overhead. Using multiple components with load balancing instead of a single company can also increase reliability through redundancy. Load balancing has implicit failover capabilities, since load balancing software is capable of detecting when a resource (e.g. network interface, hard drive) is down and excludes it from the group. Load balancing is usually provided by dedicated software or hardware, such as a multilayer switch or a Domain Name System process, or, as we shall soon see, through pfSense. In this article, I will begin our look at pfSense load balancing.

pfSense Load Balancing: Gateway Configuration

As an example, let’s assume we want to set up multiple WAN interfaces and use load balancing on the group. A default WAN gateway was already created when pfSense was set up. In this example, we will use OPT1 as an additional gateway, and then add both the default interface and OPT1 to a newly-created gateway group, which will employ pfSense load balancing to distribute the workload in round-robin fashion.

The first part of our configuration follows the steps outlined in my <a href=”http://pfsensesetup.com/pfsense-gateways/”>article on gateways</a>. In order to set up our second gateway, first browse to System -> Routing. Click on the “Gateway” tab, if it is not already selected. Click on the “plus” button to add a new gateway. At “Interface”, select OPT1 in the drop-down box. At “Name”, type a name, such as “WAN2”. At “Gateway”, type in the IP address of the network interface (in this case, Check “Default Gateway”, and at “Description”, add a description. Then press the “Save” button to save changes, and, if necessary, press the “Apply changes” button on the next screen.

Next, we will make some changes to the WAN interface (the one described as “Interface WAN Dynamic Gateway”). From the Gateways tab, click on the “edit” button. We can leave “Interface and Name” unchanged, but at “Gateway” we will type an IP address (in this case, Click on “Default Gateway” and change the description to something appropriate (e.g. “WAN gateway). Then press the “Save” button to save the changes, and press the “Apply Changes” button if necessary.

Now we have the two interfaces configured correctly. In part two of this series on pfSense load balancing, we will take our newly-configured WAN interfaces and add them to a gateway group, and configure load balancing for the group.

Erratum: The Original Instructions I Posted Contained an Error, and Here’s Why

It occurred to me when composing Part Two of this article that I made a mistake. I set the WAN gateway to originally; however, since WAN2 is on the subnet, and both WAN gateways will likely be connecting to the same network, they should be on the same subnet. Therefore, I amended the instructions for Part One so that WAN is set to I apologize for any confusion I may have caused.

Other Articles in This Series

pfSense Load Balancing: Part Two

pfSense Load Balancing: Part Three (Web Server Failover)

External Links:

Load Balancing at Wikipedia

Setup Incoming pfSense Load Balancing at doc.pfsense.org

Multi-WAN Load Balancing at pfsensesolution.blogspot.com

pfSense Gateways Explained

pfSense Gateways

Adding and configuring a gateway in pfSense 2.0.

pfSense gateways are relatively easy to add and configure, and pfSense also supports gateway groups, which I will briefly discuss in this article (a more detailed explanation, however, will be the subject of a future article). A gateway is a router interface connected to the local network that sends packets out of the local network. It has both a physical and a logical address. Since it is involved in sending packets to other networks, it operates at the network layer of the OSI Model. When packets are sent over a network, the destination IP address is examined. If the destination IP is within the network, the router can use the Address Resolution Protocol (ARP) table to find the MAC address of the target host and send the packets.

If the destination IP is outside of the network, however, then will not be able to find the MAC address of the target host in its ARP table. The packet will go to the gateway for transmission outside of the network. In this case, the frame header will add the gateway’s MAC address (the gateway operates on the data link layer of the OSI model as well). The gateway is on the same network as host devices and must have the same subnet mask as host devices. Each host on the network uses the same gateway.

Adding pfSense Gateways

pfSense Gateways

Now that we have added our gateway, it shows up on the list.

Unless you are configuring a gateway group, pfSense gateways should not take long to set up. To add a gateway, navigate to System -> Routing. Click the “Gateways” tab if it is not already selected and click the “plus” button to add a new gateway. At “Interface“, select a network interface for the new gateway. At “Name“, specify a name for the gateway (no spaces). At “Gateway“, specify the IP address for the gateway (it must be a valid IP address on the interface). Check the “Default Gateway” checkbox to make this the default gateway. The next checkbox is “Disable Gateway Monitoring“; check this if you want to disable monitoring so pfSense will consider this gateway as always being up. At “Monitor IP“, you can assign an an alternative address to be used to monitor the link. It will be used for the quality Round Robin Database (RRD) graphs as well as the load balancer entries. Leave it blank to use the gateway’s IP address by default. At “Description“, add a description if desired. Finally, press “Save” to save the changes and “Apply Changes” to apply the changes if necessary. Now the new gateway should appear on the list of pfSense gateways at the “Gateways” tab.

There are a number of advanced options for pfSense gateways you can view by clicking the “Advanced” button just below the “Alternative monitor IP” edit box. The “Weight” drop-down box allows you to assign a weight for the gateway when used in a gateway group. Gateway groups are just what their name implies. They group together gateways to act in a coordinated fashion. Increasing the weight of the gateway increases the likelihood it will be used. “Latency thresholds” defines the low and high water marks for latency in milliseconds. Once latency exceeds the high water mark, the gateway will go down. The default latency thresholds are 10 ms and 50 ms. “Packet Loss Thresholds” define the low and high water mark for packet loss in percentage. Again, once packet loss exceeds the high water mark, the gateway goes down. The defaults are 1% and 5%. “Frequency Probe” defines in seconds how often an ICMP probe will be sent. The default is 1 second. “Down” defines the number of bad probes before the alarm will be sent. The default is 10.

Now that the OPT1 is configured as the gateway, packets whose destination is outside of the network will be forwarded to OPT1. There, the frame will be stripped off the packets, leaving the IP packets with the IP address of the destination host. The gateway interface will then wrap the IP packets in whatever type of frame the outgoing connection needs, and sends them toward the target host.

External Links:

Settings for pfSense Gateways at doc.pfsense.org

How to set up a pfSense firewall when the default gateway is on a different subnet

pfSense Gateway Grouping

Port Forwarding with NAT in pfSense

Firewall Configuration: NAT port forwarding

Firewall -> NAT configuration page in the pfSense web GUI.

In computer networking, Network Address Translation (NAT) is the process of modifying IP address information in IPv4 headers while in transit across a traffic routing device. In most cases, it involves translating from the WAN IP address to the 192.168.x.x addresses of your local network. In this article, I will describe how to set up NAT port forwarding.

NAT and firewall rules are distinct and separate. NAT rules forward traffic, while firewall rules block or allow traffic. In the next article, I will cover firewall rules, but for now keep in mind that just because a NAT rule is forwarding traffic does not mean the firewall rules will allow it.

NAT Port Forwarding

NAT port forwarding rules can differ in complexity, but in this example, let’s assume we set up an Apache server at on the local network, and we want to direct all HTTP traffic (port 80) to that address. First, browse to Firewall -> NAT. The options are “Port Forward“, “1:1” and “Outbound“. Select the “Port Forward” tab. Click the “plus” button in order to create a new NAT port forward rule. “Disable the rule” and “No RDR” can be left unchanged. For “Interface” you can choose WAN and LAN; we are concerned about incoming requests from the Internet, so you can keep it as WAN.

For “Protocol”, there are five choices: TCP, UDP, TCP/UDP, GRE, and ESP. TCP stands for Transmission Control Protocol, and is the transport level protocol of the Internet protocol suite. This is usually what we want to use. Next is UDP, or User Datagram Protocol, another transport level protocol which is also part of the Internet protocol suite. It is suitable for purposes where error checking and correction are either not necessary or are performed in the application. GRE stands for Generic Routing Encapsulation, a tunneling protocol that can encapsulate a wide variety of network layer protocols inside virtual point-to-point links. It can be used, among other things, in conjunction with PPTP to create VPNs. ESP stands for Encapsulating Security Payload, a member of the IPsec protocol suite which provides authenticity, integrity and confidentiality protection of packets. In this port forwarding scenario, you can leave the protocol unchanged (TCP).

Firewall Configuration: NAT

Adding a NAT port forwarding rule.

For “Source“, you can specify the allowed client source. Typically you can leave it as “any”, but there are several choices: “Single host or alias“, “Network“, “PPTP clients“, “PPPoE clients“, “L2TP clients“, “WAN subnet“, “WAN address“, “LAN subnet“, and “LAN address“. In this case, you can leave the default (any) unchanged.

For “Source port range“, we want to redirect HTTP traffic (port 80), so choose HTTP for the from and to drop-down boxes. “Destination” offers the same choices as “Source” and can be left unchanged. “Destination port range” should be changed to HTTP for the from and to drop-down boxes. For “Redirect target IP“, specify the web server the traffic to be forwarded to (in our case, For “Redirect target Port“, choose HTTP. Next is “No XMLRPC Sync“; enable this option to prevent this rule from being applied to any redundant firewalls using CARP. This option can be left unchecked now. “NAT Reflection” can be enabled or disabled, usually it is disabled. “Filter Rule association” will automatically create a firewall rule and associate it to this NAT rule. Check this box to avoid having to create a separate firewall rule. Add a description if you wish, and press “Save” to save the changes. The port forwarding rule set up should now be in effect.

NAT Port Redirection

In this case, we passed traffic from port 80 on the source to port 80 on the destination, which is the classic port forwarding scenario. But there’s no reason you can’t redirect traffic to a different port. There are two reasons you might want to do this:

[1] Security: A good way to thwart hackers is to put services on non-standard ports. For example, everyone knows the standard port for FTP is 21, but an outsider is unlikely to find your FTP server if you place it on port 69, or better yet, an even higher port number (e.g. 51782). The same can be said of SSH. Users will have to know the port in order to access it.

[2] Single Public IP Address, more than one computer with the same services: Smaller networks with only a single public IP address may be stuck if the want to expose a lot of public services. For example, imagine that we want to have two separate FTP servers, but on two separate computers. With port redirection, we create two different NAT rules: the first rule will redirect port 51782 to port 21 on FTPServer1, and the second will redirect port 51783 to port 21 on FTPServer2. We can then remote into two separate FTP servers on two different computers using the same IP address.

External Links:

Port Forwarding Troubleshooting at doc.pfsense.org

Firewall Configuration: Aliases

One of the main functions of any firewall is to carry out port forwarding and firewall security rules, and pfSense, like any firewall, is capable of performing these functions, which can be found on the “Firewall” menu of the pfSense web interface. In this article, the first in a series covering pfSense firewall configuration, I cover creating an alias in pfSense.

Firewall Configuration: Aliases

Firewall configuration

Firewall -> Aliases page in the pfSense web GUI.

A good description of aliases can be found from the pfSense web GUI page for Firewall -> Aliases:

Aliases act as placeholders for real hosts, networks or ports. They can be used to minimize the number of changes that have to be made if a host, network or port changes. You can enter the name of an alias instead of the host, network or port in all fields that have a red background. The alias will be resolved according to the list above. If an alias cannot be resolved (e.g. because you deleted it), the corresponding element (e.g. filter/NAT/shaper rule) will be considered invalid and skipped.

Firewall configuration

Here, I create a sub-alias called “allhosts”.

With this in mind, here is how you can set up an alias in pfSense. First, browse to Firewall -> Aliases. Click the “plus” button to add a new alias. The first field is “Name“. Here, you should type in a name for the alias. At “Description“, you can add an optional description. Next, select an alias type at “Type“. Depending on which type you choose (Host, Network, Ports, URL, or URL Table), you will have different fields which must be filled out to complete the configuration. Selecting “Host(s)” as an a type allows you to create an alias that holds one or more IP addresses. Selecting “Network” allows you to create an alias that holds one or more networks (i.e. ranges of IP addresses). Selecting “Ports” allows you to create an alias that holds one or more ports. Selecting “OpenVPN Users” allows you to create an alias that holds one or more OpenVPN usernames. Selecting “URL” allows you to create an alias that holds one or more URLs. And selecting “URL Table” allows you to create an alias that holds a single URL pointing to a large list of addresses. This can come in handy if you need to import a large list of IP addresses and/or subnets. When you are done entering the configuration data for whichever type you selected, press “Save” to save the changes, and if necessary, press “Apply changes” to apply the changes.

Firewall configuration

An example of using an alias in adding a NAT port forwarding rule.

It is also possible to set up sub-aliases, which potentially make firewall management even easier. For example, if we have three hosts – host1, host2, and host3 – all of which must connect to our FTP server. We could set up a sub-alias called allhosts composed of host1, host2, and host3.

Once you have added an alias, you can use it wherever there is a red text box in the pfSense GUI. Just type the name of the alias and it can be invoked.

That covers firewall configuration of aliases under pfSense. In a future installation, I will cover NAT and firewall rules.

External Links:

Aliases from the pfSense wiki at doc.pfsense.org

Static DHCP Mapping in pfSense

In the previous posting, I covered how to configure basic settings for the DHCP server. In this part, I cover static DHCP mappings. A static DHCP mapping ensures a client is always given the same IP address.

Static DHCP Mapping: First Method

Static DHCP Mapping

Edit static mapping page in the pfSense web GUI.

In order to add static DHCP mappings, browse to Status -> DHCP Leases to view the list of clients who have been issued DHCP requests. Click the “plus” button to add a new static DHCP mapping. The MAC address field will be pre-filled; enter an IP address, which must be outside of the range of dynamically assigned DHCP addresses. Finally, enter a “Hostname” and “Description” if desired. Now press “Save” to save the changes, and “Apply” to apply changes if necessary.

Static DHCP Mapping: Second Method

If no DHCP leases have been issued yet, you may not be able to add static DHCP mappings from Status -> DHCP Leases. Fortunately, there is a second method for adding static DHCP mappings. Browse to Services -> DHCP Server -> Interface (if you followed along with my previous DHCP setup scenario, the interface will be “LAN“). Scroll to the bottom of the page, and you will find “DHCP Static Mappings for this interface.” Click on the Add button to the right. From the Services ->¬† DHCP -> Edit static mapping¬†page, you can type in “IP Address“, “Hostname” and “Description“, as described above.

Now, when a client connects to your DHCP server, the firewall will first check for a mapping in the “DHCP Static Mappings” table. If the client’s MAC address matches a mapping you specified, then the DHCP server uses the IP address specified in the mapping. If no mappings exists for your client’s MAC address, your DHCP server uses an IP address from its available range. Alternatively, you could have selected “Deny Unknown Clients” under Services -> DHCP Server -> Interface, in which case the client will not get a DHCP lease unless the client is defined in the static mappings table.

Static mappings can always be viewed at the bottom of the DHCP Server configuration page for each interface. All static mappings for a given interface can be managed here. Existing mappings can be modified or removed, and new static mappings can be created (but you will have to enter the MAC addresses manually).

External Links:

Configuring DHCP Server and Dynamic DNS Services


pfSense Setup: Part Four (Setting up a DMZ)


The optional interface configuration page in the pfSense web GUI (which is similar to the WAN and LAN config pages).

In the first three parts, I covered booting and installing pfSense, general configuration options in the pfSense web GUI, and configuring WAN and LAN interfaces (also with the web GUI). In this part, I cover using an optional interface to create a DMZ.

In networking, a DMZ (de-militarized zone) is a place where some traffic is allowed to pass and some traffic is not. The area is separate from the LAN and WAN. In simple terms, a DMZ looks like this in relation to the rest of the network:

Internet traffic | <- DMZ <- LAN

Unsafe Internet traffic is allowed to enter the DMZ, but not the LAN. To configure it, we will need an optional interface.

Configuring the DMZ

From the web GUI, browse to Interfaces -> OPT1. If “Enable Interfaces” isn’t checked, check it. Set “Description” to DMZ. Under “Type”, choose “Static” as the address configuration method. For “IP address”, enter an IP address and the subnet mask (the subnet should be different than the subnet for your LAN). For example, if your subnet for the LAN is 192.168.1.x, it could be 192.168.2.x for the optional interface.

For “Gateway”, leave this option set to “None”. The last two options are “Block private networks” and “Block bogon networks”. Ensure that these two options are unchecked; we don’t want the system to block access from the Internet to the DMZ. Finally save changes by pressing the “Save” button.

Now that the DMZ is configured, your DMZ will allow WAN access. Your DMZ will also allow access from the LAN, but it won’t be permitted to send traffic to the LAN. This will allow devices on the Internet to access DMZ resources without being able to access any of your LAN. This could be useful, for example, for setting up an e-mail or FTP server.

You could now attach a switch to your DMZ interface. This would enable you to connect multiple machines to the DMZ.

External Links:

Setting Up a DMZ in pfSense

The Rest of the Guide:

Part One (installation from LiveCD)

Part Two (configuration using the web GUI)

Part Three (WAN and LAN settings)

Ad Links:

© 2013 David Zientara. All rights reserved. Privacy Policy