Failover with CARP in PF: Part Three (Checking for CARP & pfsync)

In the previous two articles in this series, I mentioned some of the advantages of using CARP and outlined a hypothetical CARP implementation involving two redundant firewalls. In this article, I will take it one step further and discuss how to configure the firewalls, starting by making sure your flavor of BSD has both CARP and pfsync enabled.

Making Sure CARP and pfsync Are Enabled

pfsync

Checking the value of the net.inet.carp variables at the command prompt in FreeBSD.

If you are setting this up on a pfSense box, then both the carp and pfsync devices should be compiled into the kernel. But if it isn’t, and you are running FreeBSD, then you need to check that the kernel has both these devices are compiled in. You can check for CARP with the following command:

sysctl net.inet.carp.allow

If you see:

net.inet.carp.allow=1

then your system comes equipped with CARP. If, however, you see something like:

sysctl: unknown old ‘net.inet.carp.allow’

then your kernel is not configured for CARP.

If you are running OpenBSD or NetBSD, you should heed the following regarding CARP and pfsync:

  • OpenBSD: Both carp and pfsync devices are in the default GENERIC and GENERIC.mp kernel configurations, so unless you are running a custom kernel, you should be OK.
  • NetBSD: NetBSD’s default generic kernel does not have CARP compiled in, and NetBSD does not yet support pfsync.

There are actually four CARP-related variables: net.inet.carp.allow, net.inet.carp.preempt, net.inet.carp.suppress_preempt, net.inet.carp.log, and net.inet.carp.arpbalance. Setting net.inet.carp.log to 1 gives you debug information about the CARP traffic you logged, but this is turned off by default. The inet.carp.arpbalance command load balances local network traffic using ARP. net.inet.carp.suppress_preempt is a read-only variable showing the status of preemption suppression. Preemption can be suppressed if the link on an interface is down. A value of 0 means that preemption is not suppressed. Every problem increments this variable.

The important variables are net.inet.carp.allow and net.inet.carp.preempt. Setting net.inet.carp.allow to 1 allows the system to accept incoming CARP packets (this option is enabled by default). For graceful failover between the gateways in our hypothetical network, we need to set the net.inet.carp.preempt variable to 1. Setting the net.inet.carp.preempt variable means that on hosts with more than one network interface, all CARP interfaces will set their advskew to the extremely high value of 240 in order to prod other hosts in the CARP group to start failover when one of the interfaces goes down. This setting needs to be identical on all hosts in the CARP group, so you need to repeat the setting on all hosts.


Creating the CARP Interfaces

Once we’re sure both CARP and pfsync are enabled, we want to set up the network interfaces. In part two, I specified that the external CARP interface (carp0) would have an IP of 50.87.147.42, and the internal CARP interface (carp1) would have an IP address of 192.168.10.1.

We first create the interfaces using ifconfig:

ifconfig carp0 50.87.147.42 vhid 1
ifconfig carp1 192.168.10.1 vhid 2

Here we do not need to set the physical interface explicitly The cp0 and cp1 virtual interfaces here will bind themselves to the physical interfaces that are already configured with addresses in the same subnets as the assigned CARP address. with ifconfig you should be able to check that each CARP interface is properly configured.

On the backup firewall, the ifconfig command is similar, except that you add the advskew parameter:

ifconfig carp0 50.87.147.42 vhid 1 advskew 100
ifconfig carp1 192.168.10.1 vhid 2 advskew 100

The advskew parameter indicates how much less preferred it is for the specified machine to take over for the current master. In this case, the master will announce every second (1 + 0/256), while the backup will wait for 1 + 100/256 seconds. Since we set net.inet.carp.preempt to 1, when the master stops announcing or announces it is not available, the backups will take over. Smaller advskew values mean shorter announcement intervals and higher likelihood for becoming the new master.

In the next article in this series, I will continue with configuration of our hypothetical CARP setup.

Correction

In Part Two of this series, I erroneously referred to the CARP interfaces as cp0 and cp1. They need to be carp0 and carp1. This has been corrected. I apologize for any inconvenience.


External Links:

Common Address Redundancy Protocol (CARP) at freebsd.org – a section from the FreeBSD handbook.

Firewall Failover with CARP

pfSense CARP Firewall Failover

Configuring CARP settings on the primary firewall in pfSense 2.0.

In the past few articles, I have explained some of the typical load balancing and failover scenarios for which pfSense can be used. In this article, I will demonstrate how to set up a CARP redundant firewall using pfSense.

The Common Address Redundancy Protocol (CARP) is a protocol which allows multiple hosts on the same network to share a set of IP addresses. Its primary purpose is to provide failover redundancy, although it can also provide load balancing functionality. If there are two or more computers running CARP, if the primary server fails, then one of the other servers will take over, and pfsyncd will be used to synchronize packet filter states.

A group of hosts using CARP is called a “group of redundancy”; this group allocates itself an IP address which is shared or divided among the members of this group. Within this group, a host is designated as a “Master” and the other members are designated as slaves. The main host is the one that takes the IP address; this host answers any traffic or ARP request brought to the attention of this address. Each host has, in addition to the shared IP address, a second unique IP address is required. For example, if you want to have 2 cluster members, you will need 2 IP addresses for the real interfaces and then an IP for each virtual IP address. So in this case it would amount to 3.


One use of CARP is the creation of redundant firewalls. The virtual IP address allotted to the group of redundancy is indicated as the address of the default router on the computers behind the group of firewalls. If the main firewall breaks down or is disconnected from the network, the virtual IP address will be taken by one of the firewall slaves and the firewall will continue to be available. Setting up a redundant CARP firewall requires two separate an identical pfSense machines. We want each machine to have at least 3 interfaces: a WAN interface, LAN interface, and an interface dedicated to the syncing process (pfsync).

Firewall Failover with CARP, Step One:  Interface Settings

First, we need to set up the WAN, LAN and SYNC interfaces on both machines. On the first system, designated as the primary system, the settings are as follows:

  • WAN: 192.168.4.1
  • LAN: 192.168.1.30
  • SYNC: 192.168.5.1

For the backup system, the settings are as follows:

  • WAN: 192.168.4.2
  • LAN: 192.168.1.31
  • SYNC: 192.168.5.2


Firewall Failover with CARP, Step Two:  Adding Rules and Enabling CARP Synchronization

On both machines, we need to add a firewall rule to allow traffic on the SYNC interface. Navigate to Firewall -> Rules and click on the SYNC interface tab. Click the “plus” button to add a new firewall rule. At “Protocol“, set the protocol to “any“. At “Description“, add an appropriate description. Then press the “Save” button to save the changes and press the “Apply changes” button on the next page if necessary.

Next, we need to go to the backup pfSense machine and enable CARP synchronization. Navigate to Firewall -> Virtual IPs and click the “CARP Settings” tab. In the “State Synchronization Settings (pfsync)” section, check the “Synchronize States” check box. At “Synchronize Interface“, choose SYNC as the interface. Then press the “Save” button to save the changes.

Returning to the primary pfSense machine, we also need to enable CARP synchronization. Again we will navigate to Firewall -> Virtual IPs and click the “CARP Settings” tab. We will again click the “Synchronize States” check box and choose SYNC as the “Synchronize Interface“. In addition, we will check the following:

  • Synchronize Rules
  • Synchronize nat
  • Synchronize Virtual IPs

At “Synchronize Config to IP“, enter the IP address of the backup pfSense system. Also set the “Remote System Password” to the password of the backup pfSense system. Then press the “Save” button and save the changes.

Firewall Failover with CARP, Step Three: Adding Virtual IPs

Finally, we must configure a virtual IP address for the WAN and LAN interfaces on the primary pfSense machine. Navigate to Firewall -> Virtual IPs and click on the “Virtual IPs” tab. Click the “plus” button to add a new virtual IP. At “Type“, choose the CARP radio button. At “Interface”, set the interface to LAN. At “IP Address“, set the address as the single WAN address that will be used for all clients regardless of whether the primary or backup firewall is active. At “Virtual IP Password“, set a password. You can leave “VHID Group” set to 1 and “Advertising Frequency” set to 0. At “Description“, add an appropriate description. Press the “Save” button to save the changes and press the “Apply changes” button on the next page to apply the changes if necessary.

In order to create the virtual IP address for the LAN interface, we can repeat the above steps, with the following modifications:

  • At “Interface“, set the interface to LAN
  • At “IP Address“, set the address to the single LAN address that will be used as the default gateway for all clients regardless of whether the primary or backup firewall is active
  • The default “VHID Group” setting will be 2; leave this unchanged

Now that we have added the virtual IP addresses, configuration is done. The two firewalls will constantly sync their rules, NAT, and virtual IP settings so that if the primary dies, the backup will immediately take its place.

External Links:

Configuring pfSense Hardware Redundancy (CARP) at doc.pfsense.org

How to Configure a pfSense 2.0 Cluster Using CARP

© 2013 David Zientara. All rights reserved. Privacy Policy