SPAM-Defense: a simple Recipe

In times of increasing SPAM like a flood, there is a big question to answer for every email-user: how to deal with that threat ?. A simple recipe is to try to keep the personal email-address secret, to avoid becoming a target for SPAMMERS - and if that doesn't work out, frequently changing email-addresses and/or ISP's.
Both strategies have several drawbacks:
They do also affect communication with friends and business partners, and they usually do only work for a limited time.
Only beeing able to keep the personal address absolutely secret, (and, of course, ALL people knowing it do the same!), never writing someting in the internet etc. will eventually help.
But, to be honest, for what will you need email at all in this case ?
Even though there is an increasing will to fight SPAM in governments, ther is no hope for a fast solution. There may still years go by, until something substantial happens here and SPAMMERS get sentenced effectively.
So what: do something on your own, ACTIVELY and not by hiding or running away like mentioned above!
In the following, I try to present some practical advice on how to fight SPAM, which works well here for me, although beeing far from perfect ;-).
Best of course, is running a dedicated mailserver ('root-server') on the Internet, where everything can be done to avoid most SPAM from reaching any user-inbox at all.
But who does have one ? - I do not...
So, next step is an ISP, who provides some SPAM-Filters - ideally user-configurable (e.g via a web-interface) - examples are, gmx and others (even AOL, which sucks, btw., has got something alike recently...).
Unfortunaltely, efficiency of these filters is often limited, due to limitatins in computation-ressources or policy (big providers cannot afford to use too stringent filters, they would loose customers, it they did).
What turns out is that a personal SPAM-fighting system is really best, or just use one of the dedicated commercial offers like spamcop, which I will not take into account in the following.
Ok, now I will describe some of the options in building a personal system for SPAM-defense:
One importance especially for users with limited bandwidth (modem or ISDN) are filters which work already before actually downloading mails from the ISP - obvious SPAM which is already deleted there saves download-time and costs !
Drawback is here that these programs usually don't have full access to the mails on the server, but only the so called headers, but anyway, many SPAM's can be identified and deleted. Examples for programs working this way are:Mailfilter or Swendeleter.
Of course, programs which have access to the entire mail, can work more effectively, at the cost of having to download the mails first.With this done, you can now use 'big bore weapons'. For example Spamassassin or Bogofilter.Having used Spamassassin quite successful in the past, I've recently switched to bogofilter, which is, compared to the former, lightning fast (well, it's written in C, not Perl ;-), in addition, self-learning is far superior - see below.
In order to have all that running properly, best recipe is to use a combination of the mentioned methods, which gives more efficiency, and comfort.
Here is what I've done so far:
procmail works with so called 'recipes', which can be setup in a manner to directly incorporate bogofilter, this way everything is properly sorted and SPAM is put into Trash or directly deleted.
Examples for configuration of the mentioned programs can be found in the Appendix A. Ideally, all programs are run directly on a server, which provides the already filtered and sorted mails to the clients, e.g. via IMAP, transparent and independent from the client-OS.
The User doesn't have to fiddle with configuration-options - the only thing left is to put gone-through SPAMS in the folder 'SPAM', where bogofilter learns and deletes them.
Those who have only one workstation, can of course setup procmail to deliver the mails directly into the local folders, IMAP would be some sort of overkill in this case.
Of course, a mail client capable to read maildir/mailbox format is required in this case (note: OE can't do this!).
Harnessed this way, you're free again to mail (nearly) as in the past times, when the SPAM-FLOOD was not yet present... so: Happy Mailing !
Anyone interested in such a setup, may freely contact me, I will help if possible (except for Windoze, as I don't do support for M$...).

Instead of a long Link-List on the topic, I just want to refer to:
Appendix A - Configuration-Files: - shellscript:
ping -c 1 $host	# first, 1 ping, evtl. 'awake' router
# here bogofilter learns new SPAM and deletes it immediately
find /mnt/pub/werner/imap/.SPAM/cur -type f | bogofilter -s -b
find /mnt/pub/werner/imap/.SPAM/new -type f | bogofilter -s -b
find /mnt/pub/werner/imap/.SPAM/ -type f -exec rm -f {} \;
# please copy some 'good' mails here from time to time, so that bogofilter keeps in mind how these look like
find /mnt/pub/werner/imap/.HAM/cur -type f | bogofilter -n -b
find /mnt/pub/werner/imap/.HAM/new -type f | bogofilter -n -b
find /mnt/pub/werner/imap/.HAM/ -type f -exec rm -f {} \;
if (ping -c 1 $host)
        /usr/bin/perl /usr/bin/ -s $host -l werner -p xxxx -n
        fetchmail -s -a

crontab - controlfile for cron so the obove script executes automatically at scheduled times:
5,20,35,50 * * * *      /home/werner/bin/ 1>/dev/null
# This file was written by KCron. Copyright (c) 1999, Gary Meyer
# Although KCron supports most crontab formats, use care when editing.
# Note: Lines beginning with "#" indicates a disabled task.

~.fetchmailrc (Steuerdatei f. fetchmail):
# server + protocol
proto pop3

#user + password
user werner
pass xxxx

# forward all to procmail
mda /usr/bin/procmail

~.procmailrc (Controlfile f. procmail):

# filtered junk from bogofilter to Trash
* ? bogofilter -l
# /dev/null	# as soon as bogofilter is trained well enough, go here

# gentoo-user-de example for sorting mails into mailinglist-folder
* ^List-Id:.*
And last but not least my practical experience with the above configuration:
As I'm often on kde/linux mailinglists, also writing there, I'm sent about 50 SPAM's per Day, from Viagra-advertisement to Investment-Fonds in Uganda...
Since my bogofilter has been trained with about 100 sample-SPAMS, there is about one in 2 days still coming through.
This one is then dropped in the SPAM-LEARN-BOX, and that's it...
Best of all: bogofilter ist lightning fast - whereas formerly used Spamassassin worked for minutes, this is now done in 10 seconds !
Appendix B - Setting up kmail to use bogofilter: For those who do not want to fiddle with fetchmail/procmail/imap... and use kmail as their email-client to fetch mail from pop-accounts, here is a short description how to setup kmail to directly use bogofilter (nearly the same applies for spamassassin, you will just have to adjust the filter rules to use spamassassin's output).
Anyone interested in more detailed descriptions as well as tips for other spamfilters with kmail may take a look here:
Kmail Tips site
Kmail w. bogofilter.

Screenshots for setting up the filters are as follows:
Be aware, that this will only work with bogofilter beeing 'trained' with 'good' and 'bad' sample mails (in folder HAM/SPAM) - for instructions on how to do this, see the script '' mentioned above, which can be invoked by cron or 'by hand'. Also note that kmail will have to be setup to store mails in maildir-format, with mailbox-format, 'bogofilter training' will not work !
In addition to the screenshots, I also attach the relevant part of the ~/.kde/share/config/kmailrc file:
[Filter #0]
action-args-0=bogofilter -ep
action-name-0=filter app

[Filter #1]
action-name-1=set status
action-name-2=set status

Appendix C - setting up the Courier-Imap Mailserver:

In general, the documentation for Courier Imap is very good, so everyone is encouraged to read/use it.
Installation itself is not covered here, as this depends very much on the actual distro used, for debian, e.g., this can be done by just issuing 'apt-get install courier-imap' from the commandline. For gentoo, use 'emerge courier-imap'. Others may use different procedures, so please refer to the appropriate documentation.
However, there are so many configuration options that some people may feel lost about the decision which one('s) to try.
So I just want to explain in short, which of them I used and why. Maybe this is helpful for some to get started. Here goes:

As each user in my home lan has a regular login on the server running Imap, I decided to use the USERDB authentification module, where basically the already existing authentication information for 'normal' login - namely /etc/passwd and /etc/shadow - are used as a template to generate the Imap-Login information from. There are helpful scripts to do this task (makeuserdb, userdb, pw2userdb) which come with Courier Imap - they are well documented, so I will not explain them further here.
Next step is to create the necessary directories where the mails are to be stored in. These must conform to the so-called 'maildir-format' - to achieve this, there is a program called 'maildirmake' which will do the job for you. maildirmake should be used for each user-account with access to Courier-Imap. Having accompished that, /etc/userdb has to be updated to reflect the changes made, so far.
The next step is to compile /etc/userdb into binary format, makeuserdb is used to do this. If all goes well, one should end up having 2 binary files, /etc/userdb.dat and /etc/userdbshadow.dat which hold the necessary information for users to login to their Imap-accounts.
Before starting the Imap-authdaemon, you will have to adjust /etc/courier/authdaemonrc to tell the daemon it should use the userdb authentication method:
#NAME: authmodulelist:0
# The authentication modules that are linked into authdaemond.  The
# default list is installed.  You may selectively disable modules simply
# by removing them from the following list.  The available modules you
# can use are: authcustom authcram authuserdb authldap authmysql authpam

Now we are ready to fire up the authdaemond server:
/etc/init.d/courier-imap start
/etc/init.d/courier-authdaemon start
(these init-scripts are usually created by the courier-imap installation).
Having done this, we are ready to configure the mail client(s) to use the accounts on the Imap-server. The following applies for kmail.
Go to 'settings-configure kmail-network-receive'.
Choose 'add-(disconnected) Imap'. Fill in the form as follows: Imap-account. Note that for 'Server', you should use 'localhost', if the Imap-server is running on the same machine as kmail.
Having accomplished this, the rest is easy - just create new folders at will from within kmail, the result should look like this: Kmail Imap Folders. Of course, the actual mails will have to be fetched from your ISP's (pop3-) server and delivered into the mentioned folders, this can be done by using fetchmail/procmail as described further above, preferably by a periodic cron-job. If anything goes wrong/doesn't work as expected, the first place to look at is your server's /var/log/syslog file, successful logins show up there like this:
Dec 28 10:02:10 dsrv imaplogin: authdaemon: starting client module
Dec 28 10:02:10 dsrv imaplogin: authdaemon: ACCEPT, username josswern
Dec 28 10:02:10 dsrv imaplogin: LOGIN, user=josswern, ip=[::ffff:], protocol=IMAP
Another good hint in case of Problems with Login is always, first to connect to the Imap-Server via telnet on port 143:
telnet 143
then, e.g.
a001 USER username
a002 PASS password
Sorry, I can't give much more Informations on the Imap4-Command set, as I have insufficient experience with this, anyways, a google-Search for 'Imap telnet 143' should get you further here...
Please note that this description is just about how I got it working for me - as mentioned, there are nearly unlimited possibilities to set this up in another way.
As always, notes and comments/corrections are welcome - have fun .
back to Overview