Failover with CARP in PF: Part Four (Configuration)

pfSense 2.1In this article, I continue with configuration of a hypothetical CARP setup with two redundant firewalls.

Configuring the Interfaces

In the previous article, we used ifconfig to configure the CARP interfaces. In production use, you probably want to configure these interfaces with a passphrase; e.g.:

ifconfig carp0 pass mypassphrase 50.87.147.42 vhid 1
ifconfig carp1 pass mypassphrase 192.168.10.1 vhid 2

Configurations for the network cards is placed in /etc/rc.conf, and for carp0 and carp1, we can put the following:

#External CARP Interface
ifconfig_carp0=”up 50.87.147.42/24 vhid 1 pass mypassphrase”

#Internal CARP interface
ifconfig_carp1=”up 192.168.10.1/24 vhid 2 pass mypassphrase”

We also need to configure the external and internal public interfaces:

#External Public Interface
ifconfig_if0=”inet 50.87.147.50 netmask 255.255.255.0″

#Internal Public Interface
ifconfig_if1=”inet 192.168.10.10/24 netmask 255.255.255.0″


Once the two CARP interfaces and the internal and external interfaces are configured, it is time to consider state table synchronization between the hosts in the redundancy group. With synchronized state tables on the redundant firewalls, in almost all cases the traffic will see no disruption during failover. For state table synchronization, you will need a set of properly configured pfsync interfaces.

Configuring pfsync is fairly straightforward, but it does require some planning. As I mentioned in part two of this series, it is generally a better idea to set up a separate network for the synchronization. In our hypothetical configuration, I have set aside 192.168.11.224/28 for pfsync. In this setup, a crossover cable would be good enough to connect the two Ethernet interfaces, but in setups with more than two hosts, you may want to set up a separate switch, hub, or VLAN.

In this configuration, the interfaces we are planning to use for the synchronization have been assigned IP addresses 192.168.11.225 and 192.168.11.226, respectively. With the basic TCP/IP configuration done already, we add the following lines to rc.conf:

#pfSync Interface:
ifconfig_if2=”inet 192.168.11.225 netmask 255.255.255.224″
ifconfig_pfsync0=”up syncif if2 syncpeer 192.168.11.226″

Here we have configured the if2 interface, and we configured the pfsync, enabling pfsync on the if2 interface. We could have just typed:

ifconfig_pfsync0=”up syncif if2″

But adding the syncpeer command adds an additional layer of security. By default, pfsync updates are multicast on the local network. The syncpeer options overrides that behavior and instead unicasts the update to 192.168.11.226.

Putting it All Together: A Sample rc.conf

This takes us to the end of the basic network configuration for CARP-based failover. Our hypothetical rc.conf for firewall one should look like this:

cloned_interfaces=”carp0 carp1″
network_interfaces=”lo0 if0 if1 if2 pfsync0″

gateway_enable=”YES”
 if_config_loo=”inet 127.0.0.1″

#External Public Interface
ifconfig_if0=”inet 50.87.147.50 netmask 255.255.255.0″

#Internal Public Interface
ifconfig_if1=”inet 192.168.10.10/24 netmask 255.255.255.0″

#External CARP Interface
ifconfig_carp0=”up 50.87.147.42/24 vhid 1 pass mypassphrase”

#Internal CARP interface
ifconfig_carp1=”up 192.168.10.1/24 vhid 2 pass mypassphrase”

#pfSync Interface:
ifconfig_if2=”inet 192.168.11.225 netmask 255.255.255.224″
ifconfig_pfsync0=”up syncif if2 syncpeer 192.168.11.226″

icmp_drop_redirects=”YES”

pf_enable=”YES”
pf_rules=”/etc/pf.conf”
pf_flags=””
pflog_enable=”YES”
pflog_logfile=”/var/log/pflog”
pflog_flags=””


The rc.conf for the second firewall is similar, but the IPs for if0, if1 and if2 are different:
cloned_interfaces=”carp0 carp1″
network_interfaces=”lo0 if0 if1 if2 pfsync0″

gateway_enable=”YES”
if_config_lo0=”inet 127.0.0.1″

#External Public Interface
ifconfig_if0=”inet 50.87.147.60 netmask 255.255.255.0″

#Internal Public Interface
ifconfig_if1=”inet 192.168.10.11/24 netmask 255.255.255.0″

#External CARP Interface
ifconfig_carp0=”up 50.87.147.42/24 vhid 1 pass mypassphrase”

#Internal CARP interface
ifconfig_carp1=”up 192.168.10.1/24 vhid 2 pass mypassphrase”

#pfSync Interface:
ifconfig_if2=”inet 192.168.11.226 netmask 255.255.255.224″
ifconfig_pfsync0=”up syncif if2 syncpeer 192.168.11.225″

icmp_drop_redirects=”YES”

pf_enable=”YES”
pf_rules=”/etc/pf.conf”
pf_flags=””
pflog_enable=”YES”
pflog_logfile=”/var/log/pflog”
pflog_flags=””

This covers the basic configuration of our hypothetical dual firewall CARP setup. In the next article, I will construct a rule set for it.

External Links:

PF: Firewall Redundancy with CARP and pfsync at openbsd.org

Redundant Failover Firewall with pf, pfsync and CARP under FreeBSD at familywilson.collectingrarecoins.com

pfSense 2.1 Released

pfSense 2.1pfSense 2.1 was released on 9-15-2013. This release adds several new features, the biggest change being IPv6 support in almost every part of the system. Some of the other features added to pfSense include the following:

    • PBI (push button installer) package support, which keeps all of a packages files and dependencies in an isolated location so packages cannot interfere with one another
    • Update Atheros drivers
    • Improved navigation and service status in the GUI
    • Multiple language support, a mostly-complete translation for Brazilian Portuguese is included
    • Multi instance captive portal; multiple captive portal RADIUS authentication sources; many optimizations for captive portal processing
    • AES-NI support (Cryptographic accelerator feature on new Intel/AMD CPUs)

There are many other new features, but you can read about all of them at the pfSense Digest blog. You can download the pfSense 2.1 at qubenet’s pfSense mirror.

Failover with CARP in PF: Part Three (Checking for CARP & pfsync)

In the previous two articles in this series, I mentioned some of the advantages of using CARP and outlined a hypothetical CARP implementation involving two redundant firewalls. In this article, I will take it one step further and discuss how to configure the firewalls, starting by making sure your flavor of BSD has both CARP and pfsync enabled.

Making Sure CARP and pfsync Are Enabled

pfsync

Checking the value of the net.inet.carp variables at the command prompt in FreeBSD.

If you are setting this up on a pfSense box, then both the carp and pfsync devices should be compiled into the kernel. But if it isn’t, and you are running FreeBSD, then you need to check that the kernel has both these devices are compiled in. You can check for CARP with the following command:

sysctl net.inet.carp.allow

If you see:

net.inet.carp.allow=1

then your system comes equipped with CARP. If, however, you see something like:

sysctl: unknown old ‘net.inet.carp.allow’

then your kernel is not configured for CARP.

If you are running OpenBSD or NetBSD, you should heed the following regarding CARP and pfsync:

  • OpenBSD: Both carp and pfsync devices are in the default GENERIC and GENERIC.mp kernel configurations, so unless you are running a custom kernel, you should be OK.
  • NetBSD: NetBSD’s default generic kernel does not have CARP compiled in, and NetBSD does not yet support pfsync.

There are actually four CARP-related variables: net.inet.carp.allow, net.inet.carp.preempt, net.inet.carp.suppress_preempt, net.inet.carp.log, and net.inet.carp.arpbalance. Setting net.inet.carp.log to 1 gives you debug information about the CARP traffic you logged, but this is turned off by default. The inet.carp.arpbalance command load balances local network traffic using ARP. net.inet.carp.suppress_preempt is a read-only variable showing the status of preemption suppression. Preemption can be suppressed if the link on an interface is down. A value of 0 means that preemption is not suppressed. Every problem increments this variable.

The important variables are net.inet.carp.allow and net.inet.carp.preempt. Setting net.inet.carp.allow to 1 allows the system to accept incoming CARP packets (this option is enabled by default). For graceful failover between the gateways in our hypothetical network, we need to set the net.inet.carp.preempt variable to 1. Setting the net.inet.carp.preempt variable means that on hosts with more than one network interface, all CARP interfaces will set their advskew to the extremely high value of 240 in order to prod other hosts in the CARP group to start failover when one of the interfaces goes down. This setting needs to be identical on all hosts in the CARP group, so you need to repeat the setting on all hosts.


Creating the CARP Interfaces

Once we’re sure both CARP and pfsync are enabled, we want to set up the network interfaces. In part two, I specified that the external CARP interface (carp0) would have an IP of 50.87.147.42, and the internal CARP interface (carp1) would have an IP address of 192.168.10.1.

We first create the interfaces using ifconfig:

ifconfig carp0 50.87.147.42 vhid 1
ifconfig carp1 192.168.10.1 vhid 2

Here we do not need to set the physical interface explicitly The cp0 and cp1 virtual interfaces here will bind themselves to the physical interfaces that are already configured with addresses in the same subnets as the assigned CARP address. with ifconfig you should be able to check that each CARP interface is properly configured.

On the backup firewall, the ifconfig command is similar, except that you add the advskew parameter:

ifconfig carp0 50.87.147.42 vhid 1 advskew 100
ifconfig carp1 192.168.10.1 vhid 2 advskew 100

The advskew parameter indicates how much less preferred it is for the specified machine to take over for the current master. In this case, the master will announce every second (1 + 0/256), while the backup will wait for 1 + 100/256 seconds. Since we set net.inet.carp.preempt to 1, when the master stops announcing or announces it is not available, the backups will take over. Smaller advskew values mean shorter announcement intervals and higher likelihood for becoming the new master.

In the next article in this series, I will continue with configuration of our hypothetical CARP setup.

Correction

In Part Two of this series, I erroneously referred to the CARP interfaces as cp0 and cp1. They need to be carp0 and carp1. This has been corrected. I apologize for any inconvenience.


External Links:

Common Address Redundancy Protocol (CARP) at freebsd.org – a section from the FreeBSD handbook.

Failover with CARP in PF: Part Two

FailoverIn the previous article, I briefly discussed the advantages of using CARP with PF, as well as the difference between CARP and other protocols commonly used for redundancy. In this article, I will outline a hypothetical CARP implementation involving two redundant firewalls.

Keep in mind CARP addresses are virtual addresses. Unless you always have console access to all machines in your CARP group, you will almost always want to assign an IP address to the physical interfaces in order to be able to communicate with the host and be absolutely sure with which machine you are interacting. The convention is for the IP addresses assigned to the physical interface to belong to the same subnet as the virtual, shared IP address. In fact, the kernel will by default try to assign the CARP address to a physical interface that is already configured with an address in the same subnet as the CARP address. You can override this by specifying a different interface in the carpdev option in the ifconfig command string you use to set up the CARP interface.


A Hypothetical Two-Firewall CARP Setup

In this example, fw0 and fw1 will be the two firewalls. fw0 will be the master and will be used exclusively as long as it is up and running properly. fw1 is to be used only if fw0 goes down. They update and exchange state information over the CARP interface, if2. Both firewalls have a physical external WAN interface (if0), an internal LAN interface (if1), and two virtual interfaces for CARP: carp0, bound to if0, and carp1, bound to if1. Here is a brief overview of the network interfaces:

  • if0 is the external internet interface, and will only be sending CARPv2 broadcast packets to the other CARP firewalls. It will not be accepting any connections for any services.
  • if1 is the internal LAN interface. This will be used to broadcast CARPv2 packets to the other firewalls on the internal LAN.
  • if2 is the CARP synchronization interface. It is used to transfer the state information between the master (fw0) firewall to the backup (fw1) firewall. The link between the two interfaces can be a crossover cable or a secured switch. If you only have two firewalls and they are close to each other, a crossover cable is a good idea. As I noted in the previous article, keeping the CARP sync traffic on its own network is a good idea so no one can snoop on it.
  • carp0 is the external virtual CARP interface which will be bound to the physical network interface if0. It will be the public IP that will be moved from the master to the backup when the firewalls fail over. The carp0 IP in this example will be 50.87.147.42. This is the IP that will be answering any public services the network hosts.
  • carp1 is the internal CARP interface which will be bound to the physical network interface if1. It will be the internal IP that is moved from the master to the backup when the firewalls fail over. The carp1 IP address in this example will be 192.168.10.1.

This covers our outline of the hypothetical network. In the next part, I will cover configuration of our CARP firewall pool.


External Links:

Common Address Redundancy Protocol at Wikipedia

CARP Pf Firewall Failover at Calomel.org

Failover with CARP in PF: Part One

Failover with CARP in PF

FailoverCommon Address Redundancy Protocol (CARP) is a protocol which allows multiple hosts on the same local network to share a set of IP addresses. Its primary purpose is to provide failover redundancy. It was developed as a non-patent-encumbered alternative to Virtual Router Redundancy Protocol (VRRP), which is defined in RFC 2281 and 3768 and was quite far along towards becoming an IETF-sanctioned standard. It is also an alternative to the Hot Standby Router Protocol (HSRP). It manages failover at the intersection of the link layer and IP layer of the OSI model. Each CARP group has a virtual MAC address, and the CARP advertisements are sent out with this as the source address, which helps switches determine which port the virtual MAC address is currently at.

One of the main purposes of CARP is to ensure that the network will keep functioning even when a firewall goes down because of errors or planned maintenance activities such as upgrades (failover). CARP works by allowing a group of hosts on the same network segment to share an IP address. This group of hosts is referred to as a “redundancy group”. The redundancy group is assigned an IP address that is shared among the group members. Within the group, one host is designated as the master and the other as backups. The master host is the one that currently holds the shared IP; it responds to any traffic or ARP requests directed at it. If the master goes down, one of the backups will inherit the IP address. The handover from one CARP host to another may be authenticated, essentially by setting a shared secret, much like a password. The master then sends out CARP advertisement messages via multicast using the CARP protocol (IP Protocol 112) on a regular basis, and the backup hosts listen for this advertisement. If the advertisements stop, the backup hosts will begin advertising. The advertisement frequency is configurable, and the host which advertises the most frequently is the one most likely to become master in the event of a failure.


CARP is rather similar to VRRP, with a few significant differences:

  • The CARP protocol is address family independent, with the OpenBSD implementation supporting both IPv4 and IPv6.
  • CARP has an “arpbalance” feature that allows multiple hosts to share a single IP address simultaneously (in this configuration, there is a virtual MAC address for each host, but only one IP address).
  • CARP uses a cryptographically strong SHA-1 keyed-hash message authentication code (HMAC) to protect each advertisement.

Some synchronization is required, an at least in the case of PF firewalls, pfsync can handle it. If it is done properly, active connections will be handed over without noticeable interruption. The purpose of pfsync in this context is to act as a virtual network interface specially designed to synchronize state information between PF firewalls. In order to ensure that pfsync meets the packet volume and latency requirements, the initial implementation has no built-in authentication. An attacker who has local link layer access to the subnet used for pfsync traffic can add, change, or remove states from the firewalls. Therefore, it is strongly recommended that a dedicated, trusted network be used for pfsync. This can be as simple as a crossover cable between interfaces on two firewalls.


Failover with CARP: Configuration

Here is a simple example CARP configuration:

sysctl -w net.inet.card.allow=1
ifconfig carp1 create
ifconfig carp1 vhid 1 pass password carpdev em0 \
advskew 100 10.0.0.1 netmask 255.255.255.0

This does the following:

  • Enales receipt of CARP packets (the default setting)
  • Creates a carp(4) interface (carp1)
  • Configures carp1 for virtual host #1, enables a password, sets em0 as the interface belonging to the group, and makes the host a backup die to the advskew of 100 (assuming the master has an advskew less than 100). The shared IP assigned to this group is 10.0.0.1/255.255.255.0.

In the next article, I will cover failover CARP configuration in BSD with some real world scenarios.

External Links:

Firewall Redundancy with CARP and pfsync at openbsd.org

Firewall Failover with pfsync and CARP at countersiege.com

Spam Blocking in BSD: Part Five (Trap Lists)

trap listIn the previous articles in this series, I discussed spam blocking in BSD, how to configure the spamd daemon, greylisting, and how to use spamlogd and spamdb. In the final article of this series, I will cover some additional details involved in setting up a trap list.

Configuring Trap Lists

The purpose of the trap list is to provide a list of e-mail addresses in domains your servers handle e-mail for but will never receive any legitimate e-mails. There is no upper limit on the number of e-mail addresses on the trap list, but you need to have at least one. When spamd is run in greylisting mode, if a client tries to send an e-mail to an address on your traplist, the client will simply greylist the client, just like any other client we have not exchanged e-mail with before.

If the same machine tries again later, either trying to deliver to the same bogus address or to a different bogus address on your trap list, the greytrap is triggered. The offender is then put into a temporary blacklist, and for the next 24 hours, any SMTP traffic from the greytrapped host will be stuttered at with one-byte replies. Generally, clients that continue spamming after 24 hours will spam again, and return to the tarpit.


To implement greytrapping, you need a trap list, and one example is Bob Beck’s ghosts of usenet postings past trap list, which rarely contains less than 20,000 addresses. His trap list can be downloaded here. Peter Hansteen also posted his own trap list at bsdly.net.

To set up a trap list, use spamdb. The following command adds an entry:

sudo spamdb -T -a <e-mail-address>

For example, to add 46a745f7@foo.no to the list, we would type:

sudo spamdb -T -a 46a745f7@foo.no

Per the spamdb man page, -T specifies that the action being undertaken involves the trap list, and -a adds or updates the entry for 46a745f7@foo.no.

Additional Trap List Options

There are a few more noteworthy spamdb options. The -T option combined with -d lets you delete trap list entries, while the -t (lowercase) option combined with -a or -d lets you add or delete trapped IP address entries from the database.


Starting with OpenBSD 4.1, spamd can keep the greylisting databases in sync across any number of cooperating greylisting gateways. It is implemented via a set of spamd command-line options. The -Y option specifies a sync target that is the IP address(es) or other spamd-running gateways you want to inform of updates to your greylisting information. On the receiving end, the y-option specifies a sync listener, which is the address or interface where the spamd instance is prepared to receive greylisting updates from other hosts.

Thus the spamd options will have to be modified as follows:

-Y -y <sync-target> <sync-listener>

spamd also supports shared-secret authentication between the synchronization partners. If you create the file /etc/mail/spamd.key and distribute copies of that file to all synchronization partners, the contents of that file will be used to calculate the necessary checksums for authentication. The file itself can be any kind of data, such as random data harvested from /dev/arandom.

Another feature introduced with OpenBSD 4.1 was spamd’s ability to detect out-of-order MX use. Contacting a secondary e-mail exchanger first instead of trying the main one is a common tactic of spammers and it is a practice that runs contrary to the behavior we expect from ordinary e-mail clients. If someone tries the e-mail exchangers in the wrong order, we can be pretty sure that they are trying to deliver spam. For example, if our main mail server has the IP address 192.168.1.100 and the backup has the address 192.168.1.101, adding -M 192.168.1.101 to spamd’s startup options would mean that any host that tries to contact 192.166.1.101 before contacting the main mail server at 192.168.1.100 would be added to the local spamd-greytrap list for the next 24 hours.

External Links:

spamdb man page at freebsdspamd.berlios.de

 

Spam Blocking in BSD: Part Four (spamlogd and spamdb)

spamlogdIn parts one, two and three, I discussed the spamd daemon, how to configure it, and how to set it up for greylisting. In this part, I will discuss spamlogd and spamdb.

Using spamlogd

Spamlogd is a whitelist updater, and it works quietly in the background, recording logged connections to and from your mail servers to keep your whitelist updated. It thus manipulates the spamd database in /var/db/spamd used for greylisting. Spamlogd helps ensure valid e-mail to and from hosts you communicate with regularly goes through with a minimum of fuss.

In order to perform its job properly, spamlogd needs you to log SMTP connections to and from your mail servers (this will enable it to add IP addresses that receive e-mail from you to the whitelist). On OpenBSD 4.1 and later, you can create several pflog interfaces and specify which interface a rule should log to. If you want to separate the data spamlogd needs to read from the rest of your PF logs, create a separate pflog1 interface using this command:

ifconfig pflog1 create

Alternatively, you can create a hostname.pflog1 file that contains only the line “up”. We want to add this to the rules, or if you already have rules for logging, change them accordingly:

pass log (to pflog1) proto tcp from any to $emailserver port $email

pass log (to pflog1) proto tcp from $emailserver to any port smtp


Where $emailserver is the IP address of the e-mail server and $email is the port the server is listening on. Next you need to add:

-l pflog1

to spamlogd’s setup parameters. Now you have separated the spamd-related logging from the rest of the logging. Now, spamlogd will add the IP addresses that receive e-mail you send to the whitelist.

Using spamdb

If you need to view or change the contents of your blacklists, whitelists or greylists, then spamdb should be your main interface to managing the lists. From the earliest times, spamdb offered the capability of adding whitelist entries to the database (spamdb -a x.x.x.x) or deleting entries (spamdb -d x.x.x.x). Now, spamdb has even more capabilities, such as the capability of adding a client to a traplist for greytrapping. A traplist is a key component in greylisting; it contains a list of invalid e-mail addresses on servers for which we handle e-mail. When the spammer connects to our server for the first time, he will be greylisted, just like any client who has not connected to the server before. The second time, if the spammer tries to resend the same e-mail without waiting long enough or if he tries to send an e-mail to an address on the traplist, he will be caught in our tarpit and the SMTP header will be sent 1 byte at a time. The command:

sudo spamdb -T -a <e-mail address>

will add the e-mail address specified to the traplist.


External Links:

spamlogd at openbsd.org

spamdb at openbsd.org

Spam Blocking in BSD with spamd: Part Three (Greylisting)

greylistingIn part one and part two of this series, I discussed the advantages of using spamd and some basic configuration steps. In this article, I will cover greylisting in more detail.

The Problem

Spammers try to use other people’s equipment to send their messages. In addition, the software they install without the legal owner’s permission needs to be lightweight in order to run undetected. Moreover, spammers do not typically consider any individual message they send to be important. From this we can conjecture that the typical spam sender is probably not set up to interpret SMTP status codes correctly, if at all.

The current standard for Internet e-mail transmission is defined in RFC 2821. Section 4.5.4 covers retry strategies, and subsection 4.5.4.1 covers sending strategies. This section specifies the following:

  1. Mail that cannon be transmitted MUST be queued and periodically retried by the sender.
  2. The sender MUST delay retrying a particular destination after one attempt has failed. In general, the retry interval SHOULD be at least 30 minutes; however, more sophisticated and variable strategies will be beneficial when the SMTP clients can determine the reason for non-delivery.
  3. Retries continue until the message is transmitted or the sender gives up; the give-up time generally needs to be at least 4-5 days. The parameters to the retry algorithm MUST be configurable.


The Solution

We can use these requirements to our advantage. When a compromised machine is used to send spam, the sender application tends to try delivery only once, without checking for any results or return codes. Real SMTP implementations interpret SMTP return codes and act on them, as explained above. E-mail, like most other Internet services, are described as best-effort services, with a significant amount of design and development being put into make services such as SMTP fault tolerant. Thus, real mail servers retry if the initial attempt fails with any kind of temporary error.
This is what is so elegant about greylisting. spamd reports a temporary local problem to the client. Well-behaved senders with legitimate messages will try to resend the message later, but spammers have no interest in waiting for the chance to retry, since it will increase their cost of delivering the messages. Thus, the spammers will keep hammering spamd, and will get greylisted.


Configuring spamd for Greylisting

Setting up spamd for greylisting is fairly easy, although under FreeBSD, you need to have a file descriptor filesystem mounted on /dev/fd. You can do this by adding this line to your /etc/fstab file (fstab lists all the available static disks and disk partitions and where they are to be mounted):

fdescfs /dev/fd fdescfs rw 0 0

You also want to place the relevant lines in /etc/rc.conf or /etc/rc.conf.local; e.g.:

spamd_flags = “-v -G 2:4:864”
spamd_grey = YES

This tells spamd to enable vpopmail configuration (-v); -G sets the passtime for greylisting to 2 minutes, greyexp to 4 hours and whiteexp to 864 hours (36 days). This means that the client must wait 2 minutes before resending an e-mail to avoid getting greylisted. A client will remain on the the greylist for 4 hours after violating this standard. If the client waits 2 minutes before resending, it will get whitelisted, and remain on the whitelist for 864 hours.

Why It Might Not Work

The reason why greylisting should work is because any standards compliant SMTP client is required to retry delivery after some reasonable amount of time. But there are circumstances in which it may not work. First of all, the first e-mail message sent from any site which has not contacted you for as long as the greylister keeps its data aroun will be delayed for some random amount of time which depends mainly on the sender’s retry interval. There are some circumstances where avoiding even a minimal delay is desirable. In addition, you are bound to encounter misconfigured mail servers which either do not retry at all or retry at all or retry too quickly, perhaps stopping delivery retries after a few attempts. Finally, there are some sites which are large enough to have several outgoing SMTP servers (e.g. GMail); these sites do not work well with greylisting since they are not guaranteed to retry delivery of any given message from the same IP address as the last delivery attempt for that message. The RFCs do not state that they new delivery attempts have to come from the same IP address, so these sites can legitimately claim to comply with the RFCs.

If you need to make allowances for such situations in your setup, it is fairly easy. One approach is to define a table for a local whitelist; the list should be fed from a text file in case of reboots:

table <my_localwhite> file “/etc/mail/local_whitelist.txt”

To make sure SMTP traffic from the addresses in the table is not fed to spamd, you add a no rdr rule at the top of your redirection block:

no rdr proto tcp from <my_localwhite> to $mailservers port smtp

Once you have added these changes to your rule set, enter the addresses you need to protect from redirection into the local_whitelist.txt file, then reload your rule set using pfctl -f.

External Links:

Greylisting on Wikipedia

RFC 2821 – the RFC with the specifications for Simple Mail Transfer Protocol (SMTP).

Greylisting.org – a site devoted to articles and tools related to greylisting.

Greylisting: The Next Step in the Spam Control War – Greylisting site created by Evan Harris, the guy who originally came up with the idea of greylisting. Also contains whitepapers and a fully functional example of greylisting called relaydelay.

 

 

Spam Blocking in BSD with spamd: Part Two

spamdIn part one of this series, I discussed some of the advantages of spamd and how it works. In this article, I will cover spamd configuration.

spamd Configuration

Configuring the spamd daemon is pretty straightforward. First, you have to edit your rule set in /etc/pf.conf to include the following:

table persist
table persist
 pass on $ext_if inet proto tcp from <spamd> to \
( $ext_if, $localnet ) smtp rdr-to 127.0.0.1 port 8025
pass on $ext_if inet proto tcp from \

!<spamd-white> to ( $ext_if, $localnet ) port smtp rdr-to 127.0.0.1 port 8025

These rules set up two tables, (a table of known spammers) and (a list of whitelisted hosts which we assume are not spammers). SMTP traffic from the addresses in the first table () plus the ones not in the second table () is redirected to a daemon listening at port 8025 of the machine. The application which uses these tables, of course, is spamd, which is designed to waste spammers’ time and keep their traffic off our network.


You also need to edit your spamd.conf file (under FreeBSD. it should be located at /usr/local/etc/spamd/spamd.conf). First, you need to define which lists you will use:

all:\
:my_blacklist:my_whitelist:

This is where you add all the blacklists you want to use, separated by colons (:). If you want to use whitelists to subtract addresses from your blacklist, you add the name of the whitelist immediately after the name of each blacklist.

Next, you need to define your blacklist:

my_blacklist:\
:black:\
:msg=”SPAM. Your address %A has sent pam with the last 24 hours”:\
:method=http:\
:file=www.openbsd.org/spamd/traplist.gz

After the name, the first data field specifies the list type (black). The msg field contains the message to display to blacklisted senders during the SMTP dialogue. The method field specifies how spamd-setup fetches the list data. In this example, the type is http, but there are other options: you can fetch the list data via ftp, from a file in a mounted file system or via exec of an exernal program. The file filed specifies the name of the file spamd expects to receive.


You also need to define a whitelist:

my_whitelist:\
:white:\
:method=file:\
:file=/etc/mail/whitelist.txt

The whitelist in this example is similar to the blacklist, only the message parameter is omitted since it is not needed, and the source is a file on the local system.

You should be aware that enabling the suggested blacklists in the default spamd.conf could lead to blacklisting of large blocks of the internet, including several countries such as South Korea. Using other lists than the default ones is possible, and its within your discretion to use other ones or to make your own copy of the default blacklist and edit it as needed.

Next, put the lines for spamd and the startup parameters you want in /etc/rc.conf or /etc/rc.conf.local. In FreeBSD, spamd_grey has to be set to YES to enable greylisting (on OpenBSD 4.1 and later, spamd runs in greylisting mode by default and you need to set spamd_black to YES to run spamd in pure blacklisting mode. There are several parameters that can be used; see the spamd man page for a complete list of these options.

You can then view the table contents using pfctl or other applications. If you want to change or delete entries, you should use the spamdb utility, not pfctl. Also, note that if the redirection (rdr) rules you use do not include a “pass” part, you need to set up pass rules to let through traffic to your redirection, and also set up rules to let legitimate e-mail through.

External Links:

spamd man page at openbsd.org

The Apache SpamAssassin Project – download SpamAssassin here; you can also find SpamAssassin’s documentation.

Spam Blocking in BSD with spamd: Part One

SpamKeeping spam out of the network is one of the more conspicuous responsibilities of the typical network tech. Fortunately, not only is FreeBSD (the OS under which pfSense runs) secure and solid, but tools are available for this platform which should make your life easier.

Spam Blocking: Enter spamd

The main tool for combating spam under BSD is spamd. Spamd is a BSD-licensed lightweight spam-deferral daemon written for OpenBSD. It is included with OpenBSD, and is designed to work in conjunction with pf, and should be fully functional on any POSIX system where pf is available, including NetBSD, FreeBSD, and DragonFly BSD. This daemon’s design is based on the following underlying assumptions:

  • Spammers send a large number of messages, and the probability that you are the first person receiving a particular message is small.
  • Spam is sent mainly by a few spammer-friendly networks and a large number of hijacked machines.


Both the messages and the machines will be reported to blacklists quickly, and you can use these blacklists as the basis of your own blacklist. When a blacklisted computer makes an SMTP connection to your e-mail server, spamd presents its banner and immediately switches to a mode where it answers SMTP traffic 1 byte at a time. If the sender is patient enough to complete the SMTP dialogue, which will take ten minutes or more, spamd will return a “temporary error” code (4xx), which indicates that the message could not be delivered successfully and that the sender should keep the mail in his queue and retry later. If he does, the same procedure repeats, until he gives up, which should waste both his queue space and socket handles for several days. Such a process is called tarpitting, and it has the potential to tie up the spammer’s resources to a significant degree at a minimal cost to the system defending against such an attack.


spamd can be used quite effectively as a traditional spam filter, with a blacklist and a whitelist. But it also supports greylisting, which works by rejecting messages from unknown hosts temporarily with 4xx reply codes (temporary errors). Fully capable SMTP implementations are expected to maintain queues for retrying message transmissions in such cases (this behavior is required if an SMTP sender is to comply with the relevant RFCs). When a sender has proven itself able to properly retry delivery, it will be whitelisted for a longer period of time, so that future delivery attempts will be unimpeded. The viability of greylisting as a tactic for blocking unwanted e-mail was first discussed in a 2003 paper by Evan Harris. It is a simple technique, and amazingly enough, it still works, as spammers and malware writers have been slow to adapt.

External Links:

spamd at Wikipedia

RFC 1123 and RFC 2821 at IETF – the key RFCs concerning SMTP’s fault tolerance capabilities

Giving spammers a hard time – a tutorial

Annoying spammers with pf and spamd

spamd tarpit/greylisting “how to”

© 2013 David Zientara. All rights reserved. Privacy Policy