spamd: Part One

spamd

The spamd settings page in pfSense.

spamd is a ISC-licensed lightweight spam-deferral daemon which is part of the OpenBSD project. It works directly with SMTP connections and supports such features as greylisting and minimizing false positives. It should be fully functional on any system where pf is available; conveniently, there is a package available for pfSense.

spamd can be used to prevent inbound spam from reaching mail servers. It can also be used as an application-level proxy to ensure that external mail servers connecting to internal mail servers behave legitimately, and it can prevent outgoing spam from systems that may be compromised or under the control of spammers. It can be used to block spammers with the following features:

  • Blacklisting, such as the SPEWS database or other lists of IPs
  • Tarpitting, which can slow down spam reception considerably and hold connections open for a significant amount of time.
  • Greylisting, which forces e-mail to be delayed for a configurable period, requiring the remote end to resend mail at least once in order for it to be delivered.
  • SpamTrapping/GreyTrapping, which is the seeding of an e-mail address so that spammers can find it, but normal users can’t. If the e-mail address is used, then the sender must be a spammer, and they are blacklisted.

spamd Installation and Configuration

Installation of spamd in pfSense is easy. Navigate to System -> Packages, and scroll down to “spamd” in the package list. Click on the “plus” button to select the spamd package, and press the “Confirm” button on the subsequent page to confirm installation, which should complete in about three minutes.

Once spamd is installed, there will be a new item on the “Services” menu called “SpamD“. Navigate to Services -> SpamD to begin spamd configuration. There are four tabs available: “External Sources“, which allows spamd to connect to an external database, “SpamD Whitelist“, which enables you to specify hosts that will be exempt for spamd, “SpamD Settings“, which allows you to configure some general settings, and “SpamD Database“, which produces a listing of items in spamd’s database. We will begin with “SpamD Settings“.


The first setting is “Identifier“. In this edit box, you can specify the SMTP version banner that is reported upon initial connection. At the spamd command line, this would be done with: spamd -n name (where name is the SMTP version banner). The next setting is “Maximum blacklisted connections“. This is the maximum number of concurrent blacklisted connections to allow in greylisting mode. This value may not be greater than the “Max concurrent connections“, which is the next setting. The default is maxcon-100. At the command line, this would be specified with: spamd -B maxblack. You can specify the maximum number of concurrent connections at the command line with: spamd -c maxcon.

The next setting is the “Grey listing” check box. If this box is checked, connections from addresses not blacklisted on the black lists tab will be considered for greylisting. Such connections will receive an SMTP temporary failure message. If the host returns after a certain interval (the default is 25 minutes), the host will be whitelisted. If the host does not wait, however, they will be blacklisted. This is based on the assumption that non-spammers will abide by SMTP protocol practices and wait to re-send a message, while spammers will not play nice, and will likely keep hammering the e-mail server. The next setting, “Passtime“, allows you to adjust the pasttime parameter. After passtime minutes, if spamd sees a retried attempt to deliver mail for the same tuple, spamd will whitelist the connecting address by adding it as a whitelist entry. “Grey Expiration” allows you to set the interval after which spamd removes connection entries from the database if delivery has not been retired within greyexp hours from the initial time a connection is seen. Finally, “White Exp” is the interval after which spamd removes whitelist entries from the database if no mail delivery activity has been seen from the whitelisted address within whitexp hours from the initial time an address is whitelisted.

These four settings (Grey listing, Pasttime, Grey Expiration and White Exp) can be controlled from the spamd command line with a single parameter: spamd -G passtime:greyexp:whitexp. passtime defaults to 25 minutes, greyexp to 4 hours, and white exp to 864 hours (36 days).

In the next article, we will cover the remaining spamd settings, including configuring an external database and whitelisting hosts.


External Links:

OpenBSD’s spamd man page.

Hitting back at spammers with OpenBSD and spamd at www.hungryhacker.com

Spam Blocking in BSD with spamd: Part Three (Greylisting)

greylistingIn part one and part two of this series, I discussed the advantages of using spamd and some basic configuration steps. In this article, I will cover greylisting in more detail.

The Problem

Spammers try to use other people’s equipment to send their messages. In addition, the software they install without the legal owner’s permission needs to be lightweight in order to run undetected. Moreover, spammers do not typically consider any individual message they send to be important. From this we can conjecture that the typical spam sender is probably not set up to interpret SMTP status codes correctly, if at all.

The current standard for Internet e-mail transmission is defined in RFC 2821. Section 4.5.4 covers retry strategies, and subsection 4.5.4.1 covers sending strategies. This section specifies the following:

  1. Mail that cannon be transmitted MUST be queued and periodically retried by the sender.
  2. The sender MUST delay retrying a particular destination after one attempt has failed. In general, the retry interval SHOULD be at least 30 minutes; however, more sophisticated and variable strategies will be beneficial when the SMTP clients can determine the reason for non-delivery.
  3. Retries continue until the message is transmitted or the sender gives up; the give-up time generally needs to be at least 4-5 days. The parameters to the retry algorithm MUST be configurable.


The Solution

We can use these requirements to our advantage. When a compromised machine is used to send spam, the sender application tends to try delivery only once, without checking for any results or return codes. Real SMTP implementations interpret SMTP return codes and act on them, as explained above. E-mail, like most other Internet services, are described as best-effort services, with a significant amount of design and development being put into make services such as SMTP fault tolerant. Thus, real mail servers retry if the initial attempt fails with any kind of temporary error.
This is what is so elegant about greylisting. spamd reports a temporary local problem to the client. Well-behaved senders with legitimate messages will try to resend the message later, but spammers have no interest in waiting for the chance to retry, since it will increase their cost of delivering the messages. Thus, the spammers will keep hammering spamd, and will get greylisted.


Configuring spamd for Greylisting

Setting up spamd for greylisting is fairly easy, although under FreeBSD, you need to have a file descriptor filesystem mounted on /dev/fd. You can do this by adding this line to your /etc/fstab file (fstab lists all the available static disks and disk partitions and where they are to be mounted):

fdescfs /dev/fd fdescfs rw 0 0

You also want to place the relevant lines in /etc/rc.conf or /etc/rc.conf.local; e.g.:

spamd_flags = “-v -G 2:4:864”
spamd_grey = YES

This tells spamd to enable vpopmail configuration (-v); -G sets the passtime for greylisting to 2 minutes, greyexp to 4 hours and whiteexp to 864 hours (36 days). This means that the client must wait 2 minutes before resending an e-mail to avoid getting greylisted. A client will remain on the the greylist for 4 hours after violating this standard. If the client waits 2 minutes before resending, it will get whitelisted, and remain on the whitelist for 864 hours.

Why It Might Not Work

The reason why greylisting should work is because any standards compliant SMTP client is required to retry delivery after some reasonable amount of time. But there are circumstances in which it may not work. First of all, the first e-mail message sent from any site which has not contacted you for as long as the greylister keeps its data aroun will be delayed for some random amount of time which depends mainly on the sender’s retry interval. There are some circumstances where avoiding even a minimal delay is desirable. In addition, you are bound to encounter misconfigured mail servers which either do not retry at all or retry at all or retry too quickly, perhaps stopping delivery retries after a few attempts. Finally, there are some sites which are large enough to have several outgoing SMTP servers (e.g. GMail); these sites do not work well with greylisting since they are not guaranteed to retry delivery of any given message from the same IP address as the last delivery attempt for that message. The RFCs do not state that they new delivery attempts have to come from the same IP address, so these sites can legitimately claim to comply with the RFCs.

If you need to make allowances for such situations in your setup, it is fairly easy. One approach is to define a table for a local whitelist; the list should be fed from a text file in case of reboots:

table <my_localwhite> file “/etc/mail/local_whitelist.txt”

To make sure SMTP traffic from the addresses in the table is not fed to spamd, you add a no rdr rule at the top of your redirection block:

no rdr proto tcp from <my_localwhite> to $mailservers port smtp

Once you have added these changes to your rule set, enter the addresses you need to protect from redirection into the local_whitelist.txt file, then reload your rule set using pfctl -f.

External Links:

Greylisting on Wikipedia

RFC 2821 – the RFC with the specifications for Simple Mail Transfer Protocol (SMTP).

Greylisting.org – a site devoted to articles and tools related to greylisting.

Greylisting: The Next Step in the Spam Control War – Greylisting site created by Evan Harris, the guy who originally came up with the idea of greylisting. Also contains whitepapers and a fully functional example of greylisting called relaydelay.

 

 

Spam Blocking in BSD with spamd: Part Two

spamdIn part one of this series, I discussed some of the advantages of spamd and how it works. In this article, I will cover spamd configuration.

spamd Configuration

Configuring the spamd daemon is pretty straightforward. First, you have to edit your rule set in /etc/pf.conf to include the following:

table persist
table persist
 pass on $ext_if inet proto tcp from <spamd> to \
( $ext_if, $localnet ) smtp rdr-to 127.0.0.1 port 8025
pass on $ext_if inet proto tcp from \

!<spamd-white> to ( $ext_if, $localnet ) port smtp rdr-to 127.0.0.1 port 8025

These rules set up two tables, (a table of known spammers) and (a list of whitelisted hosts which we assume are not spammers). SMTP traffic from the addresses in the first table () plus the ones not in the second table () is redirected to a daemon listening at port 8025 of the machine. The application which uses these tables, of course, is spamd, which is designed to waste spammers’ time and keep their traffic off our network.


You also need to edit your spamd.conf file (under FreeBSD. it should be located at /usr/local/etc/spamd/spamd.conf). First, you need to define which lists you will use:

all:\
:my_blacklist:my_whitelist:

This is where you add all the blacklists you want to use, separated by colons (:). If you want to use whitelists to subtract addresses from your blacklist, you add the name of the whitelist immediately after the name of each blacklist.

Next, you need to define your blacklist:

my_blacklist:\
:black:\
:msg=”SPAM. Your address %A has sent pam with the last 24 hours”:\
:method=http:\
:file=www.openbsd.org/spamd/traplist.gz

After the name, the first data field specifies the list type (black). The msg field contains the message to display to blacklisted senders during the SMTP dialogue. The method field specifies how spamd-setup fetches the list data. In this example, the type is http, but there are other options: you can fetch the list data via ftp, from a file in a mounted file system or via exec of an exernal program. The file filed specifies the name of the file spamd expects to receive.


You also need to define a whitelist:

my_whitelist:\
:white:\
:method=file:\
:file=/etc/mail/whitelist.txt

The whitelist in this example is similar to the blacklist, only the message parameter is omitted since it is not needed, and the source is a file on the local system.

You should be aware that enabling the suggested blacklists in the default spamd.conf could lead to blacklisting of large blocks of the internet, including several countries such as South Korea. Using other lists than the default ones is possible, and its within your discretion to use other ones or to make your own copy of the default blacklist and edit it as needed.

Next, put the lines for spamd and the startup parameters you want in /etc/rc.conf or /etc/rc.conf.local. In FreeBSD, spamd_grey has to be set to YES to enable greylisting (on OpenBSD 4.1 and later, spamd runs in greylisting mode by default and you need to set spamd_black to YES to run spamd in pure blacklisting mode. There are several parameters that can be used; see the spamd man page for a complete list of these options.

You can then view the table contents using pfctl or other applications. If you want to change or delete entries, you should use the spamdb utility, not pfctl. Also, note that if the redirection (rdr) rules you use do not include a “pass” part, you need to set up pass rules to let through traffic to your redirection, and also set up rules to let legitimate e-mail through.

External Links:

spamd man page at openbsd.org

The Apache SpamAssassin Project – download SpamAssassin here; you can also find SpamAssassin’s documentation.

Spam Blocking in BSD with spamd: Part One

SpamKeeping spam out of the network is one of the more conspicuous responsibilities of the typical network tech. Fortunately, not only is FreeBSD (the OS under which pfSense runs) secure and solid, but tools are available for this platform which should make your life easier.

Spam Blocking: Enter spamd

The main tool for combating spam under BSD is spamd. Spamd is a BSD-licensed lightweight spam-deferral daemon written for OpenBSD. It is included with OpenBSD, and is designed to work in conjunction with pf, and should be fully functional on any POSIX system where pf is available, including NetBSD, FreeBSD, and DragonFly BSD. This daemon’s design is based on the following underlying assumptions:

  • Spammers send a large number of messages, and the probability that you are the first person receiving a particular message is small.
  • Spam is sent mainly by a few spammer-friendly networks and a large number of hijacked machines.


Both the messages and the machines will be reported to blacklists quickly, and you can use these blacklists as the basis of your own blacklist. When a blacklisted computer makes an SMTP connection to your e-mail server, spamd presents its banner and immediately switches to a mode where it answers SMTP traffic 1 byte at a time. If the sender is patient enough to complete the SMTP dialogue, which will take ten minutes or more, spamd will return a “temporary error” code (4xx), which indicates that the message could not be delivered successfully and that the sender should keep the mail in his queue and retry later. If he does, the same procedure repeats, until he gives up, which should waste both his queue space and socket handles for several days. Such a process is called tarpitting, and it has the potential to tie up the spammer’s resources to a significant degree at a minimal cost to the system defending against such an attack.


spamd can be used quite effectively as a traditional spam filter, with a blacklist and a whitelist. But it also supports greylisting, which works by rejecting messages from unknown hosts temporarily with 4xx reply codes (temporary errors). Fully capable SMTP implementations are expected to maintain queues for retrying message transmissions in such cases (this behavior is required if an SMTP sender is to comply with the relevant RFCs). When a sender has proven itself able to properly retry delivery, it will be whitelisted for a longer period of time, so that future delivery attempts will be unimpeded. The viability of greylisting as a tactic for blocking unwanted e-mail was first discussed in a 2003 paper by Evan Harris. It is a simple technique, and amazingly enough, it still works, as spammers and malware writers have been slow to adapt.

External Links:

spamd at Wikipedia

RFC 1123 and RFC 2821 at IETF – the key RFCs concerning SMTP’s fault tolerance capabilities

Giving spammers a hard time – a tutorial

Annoying spammers with pf and spamd

spamd tarpit/greylisting “how to”

© 2013 David Zientara. All rights reserved. Privacy Policy