Raspberry Pi Email Server Part 4: Spam Detection with Spamassassin

Powered by Drupal
Submitted by Sam Hobbs on

Spamassassin Logo This is the fourth part of a five part tutorial that will show you how to install a full featured email server on your Raspberry Pi. This tutorial covers how to mark emails as spam with Spamassassin.

The parts are:

The Introduction & Contents Page (read first)

Raspberry Pi Email Server Part 1: Postfix

Raspberry Pi Email Server Part 2: Dovecot

Raspberry Pi Email Server Part 3: Squirrelmail

Raspberry Pi Email Server Part 4: Spam Detection with Spamassassin

Raspberry Pi Email Server Part 5: Spam Sorting with LMTP & Sieve

Intro

I don’t actually get very many spam emails (famous last words, right?) but the occasional email gets past my helo access restrictions list (discussed in Raspberry Pi Email Server Part 1: Postfix). So, I decided to set up Spamassassin, a program that will check incoming emails and mark them as spam if they look suspicious. Spamassassin is pretty clever, it uses bayesian filtering to decide what’s spam and what’s not, and it will learn based on previous results, so it gets more accurate over time if you correct it when it gets things wrong. Spamassassin will only mark emails as spam, it will not sort them into folders for you as well. We’ll be doing the sorting with Dovecot’s Local Mail Transfer Protocol (LMTP) and the Sieve plugin, in the next tutorial: Raspberry Pi Email Server Part 5: Spam Sorting with LMTP & Sieve. Let’s get started:

Installing & Configuring Spamassassin

First, install Spamassassin:

sudo apt-get update
sudo apt-get install spamassassin

Now we need to edit values in the file /etc/spamassassin/local.cf. Some of these may already be set, in which case you can leave them as they are; add or amend the others as necessary: This one will add the spam score to the subject line of emails that Spamassassin considers to be spam:

rewrite_header Subject [***** SPAM _SCORE_ *****]

Spamassassin will also flag spam emails with “X-Spam-Flag: YES” in the headers. This flag is what we will eventually use to sort emails with; the rewritten subject line is purely to make the score easier to see. This next setting will tell Spamassassin to modify headers only, without making any changes to the body of the email:

report_safe 0

This one lowers the threshold for mail to be considered spam from 5 to 2. You can change this later if you get lots of false positives, but it’s nice to have some emails set off the rules to begin with, just so you know it’s working:

required_score 2.0

This tells Spamassassin to use Bayesian filtering:

use_bayes 1

This turns on automatic learning:

bayes_auto_learn 1

Now edit /etc/default/spamassassin and set:

ENABLED=1

You can now start the spamassassin daemon:

sudo service spamassassin start

If you are using a modern Debian derivative (Jessie or later), the init system has changed to systemd. You need to run this additional command to enable spamassassin, which will cause it to automatically start when you boot:

sudo systemctl enable spamassassin

Instructing Postfix to use Spamassassin

At this stage, the Spamassassin daemon is running but none of your incoming emails are being passed through it. We need to edit this line in /etc/postfix/master.cf (just under the headers):

smtp      inet  n       -       -       -       -       smtpd
        -o content_filter=spamassassin

And append this to the bottom of that same file, which will pipe the output back to Postfix using the Postfix’s Sendmail compatibility interface:

spamassassin    unix  -       n       n       -       -       pipe user=debian-spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Note: this is all one line, even if it appears wrapped in your browser. Now restart postfix:

sudo service postfix restart

If you get an error like this:

[....] Stopping Postfix Mail Transport Agent: postfix/usr/sbin/postconf: fatal: file /etc/postfix/master.cf: line 22: bad field count
postfix/postfix-script: fatal: cannot execute /usr/sbin/postconf!
 failed!

…then check the whitespace before the -o in content_filter=spamassassin. I can’t quite remember what I did but I think I changed tabs to spaces or the other way round, and then restarted postfix. Now watch the mail log with this command:

tail -f /var/log/mail.log

…and send a test email. You should see testing something like this:

Jan  8 22:21:18 samhobbs postfix/smtpd[952]: connect from blu0-omc2-s3.blu0.hotmail.com[65.55.111.78]
Jan  8 22:21:19 samhobbs postfix/smtpd[952]: 542E83F519: client=blu0-omc2-s3.blu0.hotmail.com[65.55.111.78]
Jan  8 22:21:19 samhobbs postfix/cleanup[957]: 542E83F519: message-id=
Jan  8 22:21:19 samhobbs postfix/qmgr[941]: 542E83F519: from=, size=1579, nrcpt=1 (queue active)
Jan  8 22:21:19 samhobbs spamd[445]: spamd: connection from localhost [127.0.0.1] at port 35680
Jan  8 22:21:19 samhobbs postfix/smtpd[952]: disconnect from blu0-omc2-s3.blu0.hotmail.com[65.55.111.78]
Jan  8 22:21:19 samhobbs spamd[445]: spamd: setuid to debian-spamd succeeded
Jan  8 22:21:19 samhobbs spamd[445]: spamd: creating default_prefs: /var/lib/spamassassin/.spamassassin/user_prefs
Jan  8 22:21:19 samhobbs spamd[445]: config: created user preferences file: /var/lib/spamassassin/.spamassassin/user_prefs
Jan  8 22:21:19 samhobbs spamd[445]: spamd: processing message  for debian-spamd:111
Jan  8 22:21:24 samhobbs spamd[445]: spamd: clean message (0.0/2.0) for debian-spamd:111 in 5.0 seconds, 1541 bytes.
Jan  8 22:21:24 samhobbs spamd[445]: spamd: result: . 0 - HTML_MESSAGE,MSGID_FROM_MTA_HEADER scantime=5.0,size=1541,user=debian-spamd,uid=111,required_score=2.0,rhost=localhost,raddr=127.0.0.1,rport=35680,mid=,autolearn=ham
Jan  8 22:21:24 samhobbs postfix/pickup[940]: D83DE3F521: uid=111 from=
Jan  8 22:21:24 samhobbs postfix/pipe[958]: 542E83F519: to=, relay=spamassassin, delay=5.7, delays=0.44/0.05/0/5.2, dsn=2.0.0, status=sent (delivered via spamassassin service)
Jan  8 22:21:24 samhobbs postfix/qmgr[941]: 542E83F519: removed
Jan  8 22:21:24 samhobbs postfix/cleanup[957]: D83DE3F521: message-id=
Jan  8 22:21:24 samhobbs postfix/qmgr[941]: D83DE3F521: from=, size=1890, nrcpt=1 (queue active)
Jan  8 22:21:25 samhobbs postfix/local[964]: D83DE3F521: to=, relay=local, delay=0.2, delays=0.06/0.1/0/0.03, dsn=2.0.0, status=sent (delivered to maildir)
Jan  8 22:21:25 samhobbs postfix/qmgr[941]: D83DE3F521: removed
Jan  8 22:21:25 samhobbs spamd[439]: prefork: child states: II

So the steps you can see here are:

  1. Outlook server connects to RasPi/Postfix on port 25
  2. Postfix accepts the message and hands it to Spamassassin to process
  3. Spamassassin decides the message is clean and marks it as HAM
  4. The email is passed back from Spamassassin to Postfix and delivered to the inbox

Training Spamassassin

We’ve deliberately set the score limit for spam to a low value. This inevitably means we’ll get some false positives, but we can use these to train Spamassassin and make it better. First, some things to understand about the Maildir format we’re using. Here’s what my structure looks like:

admin@samhobbs ~ $ sudo ls -al /home/sam/Maildir/
total 604
drwx------ 12 sam sam   4096 Mar  6 14:55 .
drwxr-xr-x  3 sam sam   4096 Mar  5 23:07 ..
drwx------  2 sam sam  36864 Mar  6 12:59 cur
-rw-------  1 sam sam  11920 Mar  6 04:14 dovecot.index
-rw-------  1 sam sam 415744 Mar  6 14:50 dovecot.index.cache
-rw-------  1 sam sam  10332 Mar  6 13:08 dovecot.index.log
-rw-------  1 sam sam  32784 Mar  5 16:22 dovecot.index.log.2
-rw-------  1 sam sam     30 Jan 13 22:30 dovecot-keywords
-rw-------  1 sam sam    144 Mar  3 17:49 dovecot.mailbox.log
-rw-------  1 sam sam  27138 Mar  6 09:27 dovecot-uidlist
-rw-------  1 sam sam      8 Mar  5 23:07 dovecot-uidvalidity
-r--r--r--  1 sam sam      0 Nov 23 22:55 dovecot-uidvalidity.52913278
drwx------  5 sam sam   4096 Mar  5 22:36 .Drafts
drwx------  5 sam sam   4096 Mar  4 21:53 .foo
drwx------  5 sam sam   4096 Mar  3 17:49 .INBOX.foo
drwx------  2 sam sam   4096 Mar  6 09:37 new
drwx------  5 sam sam   4096 Mar  5 22:36 .Sent
drwx------  5 sam sam   4096 Mar  6 14:37 .Spam
-rw-------  1 sam sam     37 Mar  3 17:49 subscriptions
drwx------  5 sam sam   4096 Nov 27 19:00 .Templates
drwx------  2 sam sam   4096 Mar  6 09:27 tmp
drwx------  5 sam sam   4096 Mar  6 04:08 .Trash

You can see I’ve created a couple of test folders here: one top level folder called “foo” and another subfolder in the inbox also called “foo” (.INBOX.foo). Each folder has three subdirectories: new for new (unread) emails, cur for emails that have been read, and tmp for temporary storage during delivery. You can read more about this on the Dovecot Wiki if you’d like to know more. So, the important thing to take away from this is that HAM emails are stored here: /home/username/Maildir/cur …and SPAM emails will be stored here (after sieve has been configured): /home/username/Maildir/.Spam/cur Spamassassin has a commandline training tool that is invoked like this:

sa-learn --no-sync [--spam or --ham] [folder/{cur,new}]

Each user has its own spamassassin database, which is located in the user's home directory in a hidden folder (.spamassassin). By default, the sa-learn command trains the database in the home directory of the user running the command, and since the spamassassin pipe we set up processes email as the user debian-spamd, we need to make sure we train the database in debian-spamd's home directory (which is /var/lib/spamassassin - you can check by looking in /etc/passwd). Unfortunately, if you run the command as debian-spamd using sudo -u debian-spamd command, you won't have read permissions for your emails. Here’s the plan: move any false positives back into the inbox with your email client, and move any missed spam into the spam folder. Then run these three commands using sudo, so you have permission to read your emails and write to the spamassassin database, and use the --dbpath option to specify which database to write to:

# Scan HAM
sudo sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --ham /home/username/Maildir/{cur,new}
# Scan SPAM
sudo sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --spam /home/username/Maildir/.Spam/{cur,new}
# sync the journal and databases
sudo sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --sync

On my Pi, running the HAM command took about 5mins to process ~500 messages, with WordPress running at the same time. If you’re sure you will always move emails into the correct folders, you could add these two commands to a cron job so that they run regularly and keep everything up to date. Alternatively, you can just run the commands when you notice a few false positives or missed spam emails. Over time, your spam filter will get better and better.

Automated learning using a script

If you don't want to run the commands manually all the time, you can use this simple cron job I wrote. The cron job runs as root, so you don't need the sudo part we used earlier. Create the script like this:

sudo nano /etc/cron.daily/spamassassin-learn

Now copy and paste this into the file (ctrl + shift + v to paste in nano):

#!/bin/bash

# Script by Sam Hobbs, see the following URL for updates:
# https://samhobbs.co.uk/2014/03/raspberry-pi-email-server-part-4-spam-detection-with-spamassassin

# redirect errors and output to logfile
exec 2>&1 >> /var/log/spamassassin.log

NOW=$(date +"%Y-%m-%d")

# Headers for log
echo ""
echo "#================================ $NOW ================================#"
echo ""

# learn HAM
echo "Learning HAM from Inbox"
sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --ham /home/sam/Maildir/{cur,new}

# learn SPAM
echo "Learning SPAM from Spam folder"
sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --spam /home/sam/Maildir/.Spam/{cur,new}

# Synchronize the journal and databases.
echo "Syncing"
sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --sync

Important: edit the paths so that they match your username! If you want to scan ham and spam for all users (this only works if you trust all users to be sensible and move ham/spam to the right folder) then replace the username "sam" with a glob ("*"), i.e:

sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --ham /home/*/Maildir/{cur,new}

Now make the script executable:

sudo chmod +x /etc/cron.daily/spamassassin-learn

The script will learn from ham/spam daily, and write a log file at /var/log/spamassassin.log. Make sure you move any spam you find into your spam folder, and any false positives back into your inbox. Don't worry if ham is accidentally marked as spam one day and gets "learned", if you move the messages to their correct locations then the next time the script runs spamassassin will correct itself.

What’s next?

We’re now done with Spamassassin. The only thing left to do is find a way to sort spam emails directly into the spam folder, which is covered in the next tutorial: Raspberry Pi Email Server Part 5: Spam Sorting with LMTP & Sieve. Feel free to leave a comment to let me know how you get on!

Comments

Don't forget to install spamc, otherwise the relay will not work:

sudo apt-get install spamc

Thanks for the comment, I found that spamc was auto installed for me:
admin@samhobbs ~ $ sudo apt-get install spamassassin
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libdigest-hmac-perl libencode-locale-perl libfile-listing-perl libfont-afm-perl
  libhtml-form-perl libhtml-format-perl libhtml-parser-perl libhtml-tagset-perl
  libhtml-tree-perl libhttp-cookies-perl libhttp-daemon-perl libhttp-date-perl
  libhttp-message-perl libhttp-negotiate-perl libio-socket-inet6-perl libio-socket-ip-perl
  libio-socket-ssl-perl liblwp-mediatypes-perl liblwp-protocol-https-perl libmail-spf-perl
  libmailtools-perl libnet-dns-perl libnet-http-perl libnet-ip-perl libnet-ssleay-perl
  libnetaddr-ip-perl libsocket-perl libsocket6-perl libsys-hostname-long-perl liburi-perl
  libwww-perl libwww-robotrules-perl re2c spamc
Suggested packages:
  libdata-dump-perl libcrypt-ssleay-perl libauthen-ntlm-perl razor libnet-ident-perl pyzor
  libmail-dkim-perl
The following NEW packages will be installed:
  libdigest-hmac-perl libencode-locale-perl libfile-listing-perl libfont-afm-perl
  libhtml-form-perl libhtml-format-perl libhtml-parser-perl libhtml-tagset-perl
  libhtml-tree-perl libhttp-cookies-perl libhttp-daemon-perl libhttp-date-perl
  libhttp-message-perl libhttp-negotiate-perl libio-socket-inet6-perl libio-socket-ip-perl
  libio-socket-ssl-perl liblwp-mediatypes-perl liblwp-protocol-https-perl libmail-spf-perl
  libmailtools-perl libnet-dns-perl libnet-http-perl libnet-ip-perl libnet-ssleay-perl
  libnetaddr-ip-perl libsocket-perl libsocket6-perl libsys-hostname-long-perl liburi-perl
  libwww-perl libwww-robotrules-perl re2c spamassassin spamc
0 upgraded, 35 newly installed, 0 to remove and 6 not upgraded.
Need to get 3,708 kB of archives.
After this operation, 10.2 MB of additional disk space will be used.
Do you want to continue [Y/n]? y
...
Adding system user `debian-spamd' (UID 111) ...
Adding new group `debian-spamd' (GID 115) ...
Adding new user `debian-spamd' (UID 111) with group `debian-spamd' ...
Creating home directory `/var/lib/spamassassin' ...
SpamAssassin Mail Filter Daemon: disabled, see /etc/default/spamassassin
Did it not auto install for you? Sam

Hello Sam,

In the section "Instructing Postfix to use Spamassassin" you suggest setting in /etc/postfix/master.cf the following:

smtp inet n - - - - smtpd
-o content_filter=spamassassin

However, In part 2 of this tutorial you suggest adding smtps to master.cf. So shouldn't the content filter flag be set for the smtps service too, and not only smtp?

In any case, I'd like to use this comment to thank you for writing this tutorial. I've had some trouble setting up a mail server and your tutorial made my life easier.

Thanks
Ariel

Hi Ariel, thanks for your comment. Good question! I recently added a bit to the tutorial just before the "testing IMAP" heading because of this, so it depends when you last read that tutorial. The bit I added was this (for smtps):
# line below added 17/05/2014 to stop spammers sending incoming mail on port 465
# and bypassing spamassassin
-o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
Which overrides the SMTP recipient restrictions list for port 465 so that it can only be used to send authenticated email... so there's no need to scan email on this port because you know it's HAM. You're right that we would have needed it if we were accepting incoming unauthenticated email on this port, though! Sam

alessandro

Mon, 06/23/2014 - 21:39

hi,
great tutorial!
I set up everything and it's 99% working. The only problem I have is that I cannot receive emails on my client.
I'm using clawsmail, and the 'get mail' button seems just unresponsive.
I do get the emails, and theget stored in ~/Maildir/new folder. Is this correct and my clawsmail settings are wrong?
Or incoming mails are supposed to land somewhere else?

In case you're familiar with clawsmail, these are my settings:
'Basic' tab:
Name of account: myName@mySite.com
Mail address: myName@mySite.com
Protocol: IMAP4
Server for receiving: mySite.com
User ID: myName
password: myPassword

'Receive' tab:
Authentication method: login
IMAP server directory: empty

'SSL' tab:
Use SSL for IMAP4 connection
Use non-blocking SSL

Thanks again for this magnificent tutorial!

alessandro

Mon, 06/23/2014 - 22:00

hi,
sorry for spamming you.
I managed to fix my clawsmail, so no need to answer me.
Thanks anyway and all my best,
AC

Just run through this excellent tutorial and am pleased with how well it replicates my existing setup (In addition I have a getmail running to collect mail from my various freemail accounts). However, spamassassin is SLLOOOWWW and maxing out my Pi CPU alarmingly. Is that just par for the course or have I mucked something up? The only change I made in my spamassassin install was to import the old 'dictionary' of learnt spam/ham from my existing box...

Hi Al! My Pi slowed down noticeably when I received emails, but that's not necessarily a problem unless it's doing lots of other things as well - it'll just chug along until it's done. When I had a WordPress site running on the same Pi the website was slower when SA was processing an email, so I was considering moving the email server to a different Pi, i.e. one Pi for WP + Squirrelmail, another for Postfix + Dovecot + Spamassassin. Maybe having a large database of learnt spam/ham makes it slower, I'm not sure... it would make sense though, as there are more things to check against! I actually upgraded myself to an Intel NUC so I don't use my Pi for email any more (they're still being put to good use though!). If you're just using yours for email, SA shouldn't be a problem! Sam

Hi, thank you for the tutorial. While I can send email to and from k9 and thunderbird, I'm having trouble sending from openssl. I'm using the command provided:


# openssl s_client -connect localhost:465 -quiet

But, whenever I try

rcpt to: user@domain.com

I get the message "554 5.7.1 : Recipient address rejected: Access denied

Any clue whats happening here?

Glyn Phillips

Mon, 09/08/2014 - 03:48

Hi, you have used 'user=debian-spamd' in the example above, should this have been created for me? Or if I create it, or use another user, what privs should it have? I tried root but that seems to create another error.

Thanks for your work,

Glyn

No, user does not exist, tried removing and re-installing but still no such user... Any ideas of creating by hand? My first trawl through Google has not given me a quick fix. Don't worry if not, I'll keep trawling until I get a good idea.

Thanks,
Glyn

Sam Hobbs

Mon, 09/08/2014 - 09:16

In reply to by Glyn Phillips

I'm still skeptical that the user hasn't been created... I kept a record of the installation process on my Pi:
admin@samhobbs ~ $ sudo apt-get install spamassassin
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libdigest-hmac-perl libencode-locale-perl libfile-listing-perl libfont-afm-perl
  libhtml-form-perl libhtml-format-perl libhtml-parser-perl libhtml-tagset-perl
  libhtml-tree-perl libhttp-cookies-perl libhttp-daemon-perl libhttp-date-perl
  libhttp-message-perl libhttp-negotiate-perl libio-socket-inet6-perl libio-socket-ip-perl
  libio-socket-ssl-perl liblwp-mediatypes-perl liblwp-protocol-https-perl libmail-spf-perl
  libmailtools-perl libnet-dns-perl libnet-http-perl libnet-ip-perl libnet-ssleay-perl
  libnetaddr-ip-perl libsocket-perl libsocket6-perl libsys-hostname-long-perl liburi-perl
  libwww-perl libwww-robotrules-perl re2c spamc
Suggested packages:
  libdata-dump-perl libcrypt-ssleay-perl libauthen-ntlm-perl razor libnet-ident-perl pyzor
  libmail-dkim-perl
The following NEW packages will be installed:
  libdigest-hmac-perl libencode-locale-perl libfile-listing-perl libfont-afm-perl
  libhtml-form-perl libhtml-format-perl libhtml-parser-perl libhtml-tagset-perl
  libhtml-tree-perl libhttp-cookies-perl libhttp-daemon-perl libhttp-date-perl
  libhttp-message-perl libhttp-negotiate-perl libio-socket-inet6-perl libio-socket-ip-perl
  libio-socket-ssl-perl liblwp-mediatypes-perl liblwp-protocol-https-perl libmail-spf-perl
  libmailtools-perl libnet-dns-perl libnet-http-perl libnet-ip-perl libnet-ssleay-perl
  libnetaddr-ip-perl libsocket-perl libsocket6-perl libsys-hostname-long-perl liburi-perl
  libwww-perl libwww-robotrules-perl re2c spamassassin spamc
0 upgraded, 35 newly installed, 0 to remove and 6 not upgraded.
Need to get 3,708 kB of archives.
After this operation, 10.2 MB of additional disk space will be used.
Do you want to continue [Y/n]? y
...
Adding system user `debian-spamd' (UID 111) ...
Adding new group `debian-spamd' (GID 115) ...
Adding new user `debian-spamd' (UID 111) with group `debian-spamd' ...
Creating home directory `/var/lib/spamassassin' ...
SpamAssassin Mail Filter Daemon: disabled, see /etc/default/spamassassin
Note the creation of new system user and group debian-spamd. I definitely didn't have to manually create a user on ubuntu either. Sam

Glyn Phillips

Mon, 09/08/2014 - 09:34

The problem could be that the system is still on debian linux 6, running Linux 2.6.37 on armv5tel. It's a sheevaplug. The second install I did showed :
SqueezePlug:~# apt-get install spamassassin
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
razor libnet-ident-perl libio-socket-ssl-perl pyzor libmail-dkim-perl
The following NEW packages will be installed:
spamassassin
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/1236 kB of archives.
After this operation, 3977 kB of additional disk space will be used.
Selecting previously deselected package spamassassin.
(Reading database ... 59411 files and directories currently installed.)
Unpacking spamassassin (from .../spamassassin_3.3.1-1.1_all.deb) ...
Processing triggers for man-db ...
Setting up spamassassin (3.3.1-1.1) ...
Starting SpamAssassin Mail Filter Daemon: spamd.

The directory wasn't created either.

I may have found enough hints to have a go myself - but I'll wait until I've slept on it! Not something to try this tired...

Many thanks again.

Glyn

I wonder though, you say that was the second time you installed... it's possible the user was created the first time, in which case it wouldn't be created again the second time. I'm assuming the installation script does something like "if debian-spamd does not exist then create it, if it does exist then do nothing". Of course it's entirely possible that your spamassassin packaging is different and the installation script doesn't create the user for you... although that would be strange. The example output I posted earlier is from Raspbian, which is basically Debian Squeeze (6, the same as you). Any developments? Sam

I have added the user, and group, that created the dir, which I chowned. I'll get back to you after I have received some spam with the details.

The sheevaplug looks like a wallwart but has arm +linux, it was one of the first to be so simple and under £100. If I was starting again there are others I would choose.

If you are ever after a project how about documenting LDAP etc needed to create a Windows domain controller? Sometime when I have time free I really want to have a bash at this...

Thanks again! Will report on success or failure.

Glyn

Sounds like you're making good progress... keep me posted! As for the LDAP thing, all of the devices in my house run free software so I don't have any need for it or even a way to test it! Sounds like an interesting project though, maybe you should start a blog and I can read all about it! :) Sam

Glyn Phillips

Wed, 09/10/2014 - 06:38

Well, been watching the log all night and have recently received some spam, and it's marked as spam! On to the next lesson soon.

Aha, lucky fellow, the dreaded gpaid werk demands Windows but I'm getting (minutely) better at linux every week.

Yes, If I go for an ADalike I will do my best to document it. Thanks, yet again. Now, back to Telnet client on an Arduino.

Glyn

I am still getting some spam coming thru and therefore I have copied it to a folder called NewSpam and have attempted using sa-learn to train spamassassin but without success. The command I tried was:-
/usr/bin/sa-learn --spam --mbox --showdots /home/alf/Maildir/.NewSpam
Please tell me what I got wrong.

Hi Alf, You don't need to copy the spam into a new folder, just run the commands on your whole spam folder! Spamassassin just skips messages it has already learned. There's definitely one thing wrong with that command: we're not using mbox - the tutorial sets you up to use the Maildir format. However, if you haven't "learned" lots of spam, then it's probably just that spamassassin hasn't been trained enough yet, it needs you to tell it what spam looks like before it can make a good decision. For example, if you get a spam message trying to sell viagra, and you tell spamassassin it's spam, then the next email with the word "viagra" in it will have a higher spam score. If you view the message source of the spam email you can see how spamassassin scored it. Here's the headers (slightly edited) from a message I got recently that slipped through (look at the X-Spam-Status part):
Return-Path: 
Delivered-To: 
Received: from samhobbs.co.uk
	by samhobbs (Dovecot) with LMTP id 75HHFzqrVlQ8EAAA+i6E6g
	for ; Sun, 02 Nov 2014 22:07:54 +0000
Received: by samhobbs.co.uk (Postfix, from userid 119)
	id 5E0CC140E63; Sun,  2 Nov 2014 22:07:54 +0000 (GMT)
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on samhobbs
X-Spam-Level: ***
X-Spam-Status: No, score=3.9 required=5.0 tests=HTML_MESSAGE,RCVD_IN_SBL,
	RDNS_NONE,T_DKIM_INVALID autolearn=no autolearn_force=no version=3.4.0
Received: from mail2.yeast solutiontips.club (unknown [141.105.71.202])
	by samhobbs.co.uk (Postfix) with ESMTP id E568D140887
	for ; Sun,  2 Nov 2014 22:07:53 +0000 (GMT)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=dkim; d=yeast solutiontips.club;
 h=To:Subject:Message-ID:Date:From:Reply-To:MIME-Version:List-Unsubscribe:Content-Type:Content-Transfer-Encoding; i=barbara.maurer at yeast solutiontips.club;
 bh=FypH8kHGKG/SpFU4tnQvMN+9M3A=;
 b=2euePhMQliBvUX6l2by08Wn5JSpr91G8LnwCOpoDQhF1qDde39RmJnjByVRLlmgGj0JJFl14luoE
   89CVaott4DH/BmA+xO4cgBo4+i+ruvqb95+ADTz1QA7vg5yBT0o2NrkyAFMTOgikt9fwWpDLjhCT
   yMk62nVj5esFPW419lY=
Received: from mail2.yeast solutiontips.club (141.105.71.152) by mail2.yeast solutiontips.club for ; Mon, 3 Nov 2014 00:04:45 +0200 (envelope-from )
To: root at samhobbs.co.uk
Subject: Are these fruits making you fat?
BTW, I added some new bits at the bottom of the tutorial that you might like to use (including a simple script so that learning is automatic) Sam

Sorry that it took so long to get back to you. Caused by your email ending up in my Spam folder.....:-( Should not happen again thanks to your showing how to automate spam handling.
Again Thank you.

Hey Sam,

Fist off I would like to say that this is a great tutorial! Everything up until spamassassin has worked great! So the installation and configuration of spam assassin appeared to go smoothly, but when I send an email with spamassassin enabled the email doesn't make it into my inbox. I have been going through my logs and researching online, but haven't as of yet been able to sort out the problem. The only clue I have, is in my logs I get the following error:

Jan 14 06:12:01 raspberrypi spamd[9057]: spamd: processing message <46D1CF71-D35E-400E-AD43-D7571F5C01CD@mac.com> for debian-spamd:110
Jan 14 06:12:03 raspberrypi spamd[9057]: spamd: clean message (0.0/2.0) for debian-spamd:110 in 2.0 seconds, 1266 bytes.
Jan 14 06:12:03 raspberrypi spamd[9057]: spamd: result: . 0 - FREEMAIL_FROM,T_TO_NO_BRKTS_FREEMAIL scantime=2.0,size=1266,user=debian-spamd,uid=110,required_score=2.0,rhost=localhost,raddr=127.0.0.1,rport=59582,mid=<46D1CF71-D35E-400E-AD43-D7571F5C01CD@mac.com>,autolearn=ham
Jan 14 06:12:03 raspberrypi spamc[9230]: exec failed: No such file or directory
Jan 14 06:12:03 raspberrypi postfix/pipe[9229]: DC06140A72: to=, relay=spamassassin, delay=2.5, delays=0.2/0.05/0/2.2, dsn=4.3.0, status=deferred (system resource problem)
Jan 14 06:12:04 raspberrypi spamd[9054]: prefork: child states: II

I have verified that spamc is located in "/usr/bin/spamc" which is something some people have seemed to have an issue with. I have also double checked my master.cf settings which seem correct? Here are the key parts;

smtp inet n - - - - smtpd -o content_filter=spamassassin
spamassassin unix - n n - - pipe user=debian-spamd argv=/usr/bin/spamc -f -e /user/sbin/sendmail -oi -f ${sender} ${recipient}

So I'm not really sure where the problem is? Is it possible that there is a permission issue with a certain file or folder? or that something else could be missing? Any help would be greatly appreciated, Thanks!

Ok, SpamAssasin just put [***** SPAM 3.4 *****] in the subject line of an email that a friend sent to my pi email server. So I did a little tuning on it. Can you point me to any good source on how to do the tuning?

I use git for all my code and other settings that I use for various software - to keep track of changes. I'd like to use git to keep track of the changes to the email server settings, postfix, dovecot and spamassassin. I realize that there are not a lot of settings to play with and keep track of in such a formal way. However, do you have any suggestions on the best way use git for this?

I also added the spamassassin configuration folder to my backup script. Here are the folders that I backup from the email server:
/home/pi/ (this one has a git repository already - maybe I just add the email settings files to this repository?)
/home/pi/Maildir
/etc/postfix
/etc/dovecot
/var/log/spamassassin.log
/etc/spamassassin

I'm working on putting Squirrelmail on the ownCloud pi, so the settings for it will be backed up from there.

Now that you know spamassassin is working, reset the required score to 5 - that message is spammy for some reason but not so much that it would normally be marked as spam. If you want to know why it's spammy, show the full headers for the email, you'll get a breakdown of the tests and scores. It might be a problem for your friend to fix, not you. No suggestions for git, I'm just learning it myself. Sam

Here is the reason:

* 3.4 RCVD_ILLEGAL_IP Received: contains illegal IP address

Failure of this test suggests that the mailout software added extra message headers to disguise the real source of the message, or else is not using a professionally run network.

I did a whois of the IP's in "Receved:" section (listed in the header) and really didn't see anything suspicious there. All from yahoo and inktomi.

I asked my friend about what device and software he used, and where he sent the reply to my message (when SpamAssassin put in the 3.4) - if this matters? Waiting for his reply.

Add new comment

The content of this field is kept private and will not be shown publicly.

Filtered HTML

  • Web page addresses and email addresses turn into links automatically.
  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.