This is the fourth part of a five part tutorial that will show you how to install a full featured email server on your Raspberry Pi. This tutorial covers how to mark emails as spam with Spamassassin.
The parts are:
The Introduction & Contents Page (read first)
Raspberry Pi Email Server Part 1: Postfix
Raspberry Pi Email Server Part 2: Dovecot
Raspberry Pi Email Server Part 3: Squirrelmail
Raspberry Pi Email Server Part 4: Spam Detection with Spamassassin
Raspberry Pi Email Server Part 5: Spam Sorting with LMTP & Sieve
Intro
I don’t actually get very many spam emails (famous last words, right?) but the occasional email gets past my helo access restrictions list (discussed in Raspberry Pi Email Server Part 1: Postfix). So, I decided to set up Spamassassin, a program that will check incoming emails and mark them as spam if they look suspicious. Spamassassin is pretty clever, it uses bayesian filtering to decide what’s spam and what’s not, and it will learn based on previous results, so it gets more accurate over time if you correct it when it gets things wrong. Spamassassin will only mark emails as spam, it will not sort them into folders for you as well. We’ll be doing the sorting with Dovecot’s Local Mail Transfer Protocol (LMTP) and the Sieve plugin, in the next tutorial: Raspberry Pi Email Server Part 5: Spam Sorting with LMTP & Sieve. Let’s get started:
Installing & Configuring Spamassassin
First, install Spamassassin:
sudo apt-get update sudo apt-get install spamassassin
Now we need to edit values in the file /etc/spamassassin/local.cf
. Some of these may already be set, in which case you can leave them as they are; add or amend the others as necessary: This one will add the spam score to the subject line of emails that Spamassassin considers to be spam:
rewrite_header Subject [***** SPAM _SCORE_ *****]
Spamassassin will also flag spam emails with “X-Spam-Flag: YES” in the headers. This flag is what we will eventually use to sort emails with; the rewritten subject line is purely to make the score easier to see. This next setting will tell Spamassassin to modify headers only, without making any changes to the body of the email:
report_safe 0
This one lowers the threshold for mail to be considered spam from 5 to 2. You can change this later if you get lots of false positives, but it’s nice to have some emails set off the rules to begin with, just so you know it’s working:
required_score 2.0
This tells Spamassassin to use Bayesian filtering:
use_bayes 1
This turns on automatic learning:
bayes_auto_learn 1
Now edit /etc/default/spamassassin and set:
ENABLED=1
You can now start the spamassassin daemon:
sudo service spamassassin start
If you are using a modern Debian derivative (Jessie or later), the init system has changed to systemd. You need to run this additional command to enable spamassassin, which will cause it to automatically start when you boot:
sudo systemctl enable spamassassin
Instructing Postfix to use Spamassassin
At this stage, the Spamassassin daemon is running but none of your incoming emails are being passed through it. We need to edit this line in /etc/postfix/master.cf
(just under the headers):
smtp inet n - - - - smtpd -o content_filter=spamassassin
And append this to the bottom of that same file, which will pipe the output back to Postfix using the Postfix’s Sendmail compatibility interface:
spamassassin unix - n n - - pipe user=debian-spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}
Note: this is all one line, even if it appears wrapped in your browser. Now restart postfix:
sudo service postfix restart
If you get an error like this:
[....] Stopping Postfix Mail Transport Agent: postfix/usr/sbin/postconf: fatal: file /etc/postfix/master.cf: line 22: bad field count postfix/postfix-script: fatal: cannot execute /usr/sbin/postconf! failed!
…then check the whitespace before the -o in content_filter=spamassassin. I can’t quite remember what I did but I think I changed tabs to spaces or the other way round, and then restarted postfix. Now watch the mail log with this command:
tail -f /var/log/mail.log
…and send a test email. You should see testing something like this:
Jan 8 22:21:18 samhobbs postfix/smtpd[952]: connect from blu0-omc2-s3.blu0.hotmail.com[65.55.111.78] Jan 8 22:21:19 samhobbs postfix/smtpd[952]: 542E83F519: client=blu0-omc2-s3.blu0.hotmail.com[65.55.111.78] Jan 8 22:21:19 samhobbs postfix/cleanup[957]: 542E83F519: message-id=Jan 8 22:21:19 samhobbs postfix/qmgr[941]: 542E83F519: from= , size=1579, nrcpt=1 (queue active) Jan 8 22:21:19 samhobbs spamd[445]: spamd: connection from localhost [127.0.0.1] at port 35680 Jan 8 22:21:19 samhobbs postfix/smtpd[952]: disconnect from blu0-omc2-s3.blu0.hotmail.com[65.55.111.78] Jan 8 22:21:19 samhobbs spamd[445]: spamd: setuid to debian-spamd succeeded Jan 8 22:21:19 samhobbs spamd[445]: spamd: creating default_prefs: /var/lib/spamassassin/.spamassassin/user_prefs Jan 8 22:21:19 samhobbs spamd[445]: config: created user preferences file: /var/lib/spamassassin/.spamassassin/user_prefs Jan 8 22:21:19 samhobbs spamd[445]: spamd: processing message for debian-spamd:111 Jan 8 22:21:24 samhobbs spamd[445]: spamd: clean message (0.0/2.0) for debian-spamd:111 in 5.0 seconds, 1541 bytes. Jan 8 22:21:24 samhobbs spamd[445]: spamd: result: . 0 - HTML_MESSAGE,MSGID_FROM_MTA_HEADER scantime=5.0,size=1541,user=debian-spamd,uid=111,required_score=2.0,rhost=localhost,raddr=127.0.0.1,rport=35680,mid= ,autolearn=ham Jan 8 22:21:24 samhobbs postfix/pickup[940]: D83DE3F521: uid=111 from= Jan 8 22:21:24 samhobbs postfix/pipe[958]: 542E83F519: to= , relay=spamassassin, delay=5.7, delays=0.44/0.05/0/5.2, dsn=2.0.0, status=sent (delivered via spamassassin service) Jan 8 22:21:24 samhobbs postfix/qmgr[941]: 542E83F519: removed Jan 8 22:21:24 samhobbs postfix/cleanup[957]: D83DE3F521: message-id= Jan 8 22:21:24 samhobbs postfix/qmgr[941]: D83DE3F521: from= , size=1890, nrcpt=1 (queue active) Jan 8 22:21:25 samhobbs postfix/local[964]: D83DE3F521: to= , relay=local, delay=0.2, delays=0.06/0.1/0/0.03, dsn=2.0.0, status=sent (delivered to maildir) Jan 8 22:21:25 samhobbs postfix/qmgr[941]: D83DE3F521: removed Jan 8 22:21:25 samhobbs spamd[439]: prefork: child states: II
So the steps you can see here are:
- Outlook server connects to RasPi/Postfix on port 25
- Postfix accepts the message and hands it to Spamassassin to process
- Spamassassin decides the message is clean and marks it as HAM
- The email is passed back from Spamassassin to Postfix and delivered to the inbox
Training Spamassassin
We’ve deliberately set the score limit for spam to a low value. This inevitably means we’ll get some false positives, but we can use these to train Spamassassin and make it better. First, some things to understand about the Maildir format we’re using. Here’s what my structure looks like:
admin@samhobbs ~ $ sudo ls -al /home/sam/Maildir/ total 604 drwx------ 12 sam sam 4096 Mar 6 14:55 . drwxr-xr-x 3 sam sam 4096 Mar 5 23:07 .. drwx------ 2 sam sam 36864 Mar 6 12:59 cur -rw------- 1 sam sam 11920 Mar 6 04:14 dovecot.index -rw------- 1 sam sam 415744 Mar 6 14:50 dovecot.index.cache -rw------- 1 sam sam 10332 Mar 6 13:08 dovecot.index.log -rw------- 1 sam sam 32784 Mar 5 16:22 dovecot.index.log.2 -rw------- 1 sam sam 30 Jan 13 22:30 dovecot-keywords -rw------- 1 sam sam 144 Mar 3 17:49 dovecot.mailbox.log -rw------- 1 sam sam 27138 Mar 6 09:27 dovecot-uidlist -rw------- 1 sam sam 8 Mar 5 23:07 dovecot-uidvalidity -r--r--r-- 1 sam sam 0 Nov 23 22:55 dovecot-uidvalidity.52913278 drwx------ 5 sam sam 4096 Mar 5 22:36 .Drafts drwx------ 5 sam sam 4096 Mar 4 21:53 .foo drwx------ 5 sam sam 4096 Mar 3 17:49 .INBOX.foo drwx------ 2 sam sam 4096 Mar 6 09:37 new drwx------ 5 sam sam 4096 Mar 5 22:36 .Sent drwx------ 5 sam sam 4096 Mar 6 14:37 .Spam -rw------- 1 sam sam 37 Mar 3 17:49 subscriptions drwx------ 5 sam sam 4096 Nov 27 19:00 .Templates drwx------ 2 sam sam 4096 Mar 6 09:27 tmp drwx------ 5 sam sam 4096 Mar 6 04:08 .Trash
You can see I’ve created a couple of test folders here: one top level folder called “foo” and another subfolder in the inbox also called “foo” (.INBOX.foo). Each folder has three subdirectories: new for new (unread) emails, cur for emails that have been read, and tmp for temporary storage during delivery. You can read more about this on the Dovecot Wiki if you’d like to know more. So, the important thing to take away from this is that HAM emails are stored here: /home/username/Maildir/cur
…and SPAM emails will be stored here (after sieve has been configured): /home/username/Maildir/.Spam/cur
Spamassassin has a commandline training tool that is invoked like this:
sa-learn --no-sync [--spam or --ham] [folder/{cur,new}]
Each user has its own spamassassin database, which is located in the user's home directory in a hidden folder (.spamassassin
). By default, the sa-learn
command trains the database in the home directory of the user running the command, and since the spamassassin pipe we set up processes email as the user debian-spamd
, we need to make sure we train the database in debian-spamd
's home directory (which is /var/lib/spamassassin
- you can check by looking in /etc/passwd
). Unfortunately, if you run the command as debian-spamd
using sudo -u debian-spamd command
, you won't have read permissions for your emails. Here’s the plan: move any false positives back into the inbox with your email client, and move any missed spam into the spam folder. Then run these three commands using sudo, so you have permission to read your emails and write to the spamassassin database, and use the --dbpath
option to specify which database to write to:
# Scan HAM sudo sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --ham /home/username/Maildir/{cur,new} # Scan SPAM sudo sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --spam /home/username/Maildir/.Spam/{cur,new} # sync the journal and databases sudo sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --sync
On my Pi, running the HAM command took about 5mins to process ~500 messages, with WordPress running at the same time. If you’re sure you will always move emails into the correct folders, you could add these two commands to a cron job so that they run regularly and keep everything up to date. Alternatively, you can just run the commands when you notice a few false positives or missed spam emails. Over time, your spam filter will get better and better.
Automated learning using a script
If you don't want to run the commands manually all the time, you can use this simple cron job I wrote. The cron job runs as root, so you don't need the sudo
part we used earlier. Create the script like this:
sudo nano /etc/cron.daily/spamassassin-learn
Now copy and paste this into the file (ctrl + shift + v to paste in nano):
#!/bin/bash # Script by Sam Hobbs, see the following URL for updates: # https://samhobbs.co.uk/2014/03/raspberry-pi-email-server-part-4-spam-detection-with-spamassassin # redirect errors and output to logfile exec 2>&1 >> /var/log/spamassassin.log NOW=$(date +"%Y-%m-%d") # Headers for log echo "" echo "#================================ $NOW ================================#" echo "" # learn HAM echo "Learning HAM from Inbox" sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --ham /home/sam/Maildir/{cur,new} # learn SPAM echo "Learning SPAM from Spam folder" sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --spam /home/sam/Maildir/.Spam/{cur,new} # Synchronize the journal and databases. echo "Syncing" sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --sync
Important: edit the paths so that they match your username! If you want to scan ham and spam for all users (this only works if you trust all users to be sensible and move ham/spam to the right folder) then replace the username "sam" with a glob ("*"), i.e:
sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --ham /home/*/Maildir/{cur,new}
Now make the script executable:
sudo chmod +x /etc/cron.daily/spamassassin-learn
The script will learn from ham/spam daily, and write a log file at /var/log/spamassassin.log
. Make sure you move any spam you find into your spam folder, and any false positives back into your inbox. Don't worry if ham is accidentally marked as spam one day and gets "learned", if you move the messages to their correct locations then the next time the script runs spamassassin will correct itself.
What’s next?
We’re now done with Spamassassin. The only thing left to do is find a way to sort spam emails directly into the spam folder, which is covered in the next tutorial: Raspberry Pi Email Server Part 5: Spam Sorting with LMTP & Sieve. Feel free to leave a comment to let me know how you get on!
Comments
Bayes data base issue
Bonjour Sam, many thanks for you tutorial, the email server does work as a charm else an error I'm getting in the mail log on Bayes, Jan 13 07:09:36 specialpierson spamd[1240]: bayes: cannot open bayes databases /var/lib/spamassassin/.spamassassin/bayes_* R/O: tie failed: Permission non accordée
Jan 13 07:09:36 specialpierson spamd[1240]: spamd: processing message <023501d26d63$9b76f4e0$d264dea0$@free.fr> for debian-spamd:116
Jan 13 07:09:36 specialpierson spamd[1240]: bayes: cannot open bayes databases /var/lib/spamassassin/.spamassassin/bayes_* R/O: tie failed: Permission denied
Jan 13 07:09:38 specialpierson spamd[1240]: bayes: cannot open bayes databases /var/lib/spamassassin/.spamassassin/bayes_* R/W: tie failed: Permission denied
Many thanks for your help, Cheers Eric
See if ownership of databases is correct
debian-spamd
, if not you can chown it: See if that solves your problem. SamBayes database issue
Dear Sam, well spotted, bayes_seen and bayes_tolk had a root ownership, fix now !! many thanks for your help and your very quick turn round, cheers Eric
Fix for issue of large spam getting past spamassassin
In the tutorial about SA, there is a step about appending a last line to Postfix' master.cf file. As mentioned above, adding the -s (size) option fixed my problem of large spam getting past SA without being scanned. I think the default is 500k, but I had some spams with an image that was around 700k that were not being scanned at all by SA. I initially change the -s to 1000000, but yesterday I had a 1M spam get through, so today I changed the last line of master.cf to:
spamassassin unix - n n -
- pipe user=debian-spamd argv=/usr/bin/spamc -s 2000000-f -e
/usr/sbin/sendmail -oi -f ${sender} ${recipient}
Note the -s 2000000 is added to the line as printed in the tutorial. I recommend offering this option/discussion in the tutorial.
Thanks!
Thanks for reporting back
Spamassassin failing every day or two
Hi Sam,
I hope your move went well and you're settled in your new home.
For the last week or two my spamassassin service seems to be failing (stops) every day or two. I'm not sure what the issue is...it might have started after updating apt-get update/upgrade etc...
systemctl shows the following:
admin@pi-box:~ $ systemctl status -l spamassassin.service
● spamassassin.service - Perl-based spam filter using text analysis
Loaded: loaded (/lib/systemd/system/spamassassin.service; enabled)
Active: failed (Result: signal) since Sat 2017-04-22 07:00:12 UTC; 2h 57min ago
Process: 602 ExecStart=/usr/sbin/spamd -d --pidfile=/var/run/spamassassin.pid $OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 1350 (code=killed, signal=SEGV)
journalctl shows the following (extract from the relevant time):
Apr 22 06:39:01 pi-box CRON[4196]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 22 06:39:01 pi-box CRON[4200]: (root) CMD ( [ -x /usr/lib/php5/sessionclean ] && /usr/lib/php5/sessionclean)
Apr 22 06:39:01 pi-box CRON[4196]: pam_unix(cron:session): session closed for user root
Apr 22 06:39:52 pi-box postfix/anvil[4177]: statistics: max connection rate 1/60s for (smtp:108.179.227.115) at Apr 22 06:33:33
Apr 22 06:39:52 pi-box postfix/anvil[4177]: statistics: max connection count 1 for (smtp:108.179.227.115) at Apr 22 06:33:33
Apr 22 06:39:52 pi-box postfix/anvil[4177]: statistics: max cache size 1 at Apr 22 06:33:33
Apr 22 06:39:53 pi-box dovecot[1515]: imap-login: Login: user=, method=PLAIN, rip=213.205.251.187, lip=192.168.1.100, mpid=4247, TLS, session=, method=PLAIN, rip=213.205.251.187, lip=192.168.1.100, mpid=4253, TLS, session=
Apr 22 07:00:12 pi-box postfix/cleanup[4279]: A4F303EA1E: message-id=<20170422070012.A4F303EA1E@pi-box.co.uk>
Apr 22 07:00:12 pi-box postfix/qmgr[1703]: A4F303EA1E: from=, size=1621, nrcpt=1 (queue active)
Apr 22 07:00:12 pi-box CRON[3935]: pam_unix(cron:session): session closed for user root
Apr 22 07:00:13 pi-box dovecot[1515]: lmtp(4286): Connect from local
Apr 22 07:00:13 pi-box dovecot[1515]: lmtp(4286, admin): m6/vAX3/+li+EAAA6C2HcQ: sieve: msgid=<20170422070012.A4F303EA1E@pi-box.co.uk>: stored mail into mailb
Apr 22 07:00:13 pi-box postfix/lmtp[4285]: A4F303EA1E: to=, orig_to=, relay=pi-box.co.uk[private/dovecot-lmtp], delay=0.68, delays=0
Apr 22 07:00:13 pi-box dovecot[1515]: lmtp(4286): Disconnect from local: Successful quit
Apr 22 07:00:13 pi-box postfix/qmgr[1703]: A4F303EA1E: removed
Apr 22 07:07:02 pi-box postfix/smtpd[4290]: warning: database /etc/postfix/helo_access.db is older than source file /etc/postfix/helo_access
Apr 22 07:07:03 pi-box postfix/smtpd[4290]: connect from out083.topica-gold-y.com[66.227.61.3]
Apr 22 07:07:11 pi-box postfix/smtpd[4290]: 075253EA1E: client=out083.topica-gold-y.com[66.227.61.3]
Apr 22 07:07:19 pi-box postfix/cleanup[4294]: 075253EA1E: message-id=<1351457768-1463747838-1492843864@cloud.b.topica.com>
Apr 22 07:07:24 pi-box postfix/qmgr[1703]: 075253EA1E: from=, size=20224, nrcpt=1 (queue ac
Apr 22 07:07:24 pi-box spamc[4297]: connect to spamd on ::1 failed, retrying (#1 of 3): Connection refused
Apr 22 07:07:24 pi-box spamc[4297]: connect to spamd on 127.0.0.1 failed, retrying (#1 of 3): Connection refused
Apr 22 07:07:25 pi-box spamc[4297]: connect to spamd on ::1 failed, retrying (#2 of 3): Connection refused
Apr 22 07:07:25 pi-box spamc[4297]: connect to spamd on 127.0.0.1 failed, retrying (#2 of 3): Connection refused
Apr 22 07:07:26 pi-box spamc[4297]: connect to spamd on ::1 failed, retrying (#3 of 3): Connection refused
Apr 22 07:07:26 pi-box spamc[4297]: connect to spamd on 127.0.0.1 failed, retrying (#3 of 3): Connection refused
Apr 22 07:07:26 pi-box spamc[4297]: connection attempt to spamd aborted after 3 retries
Apr 22 07:07:26 pi-box postfix/pickup[3863]: CDD9D3EA20: uid=114 from=
Apr 22 07:07:26 pi-box postfix/cleanup[4294]: CDD9D3EA20: message-id=<1351457768-1463747838-1492843864@cloud.b.topica.com>
Apr 22 07:07:26 pi-box postfix/pipe[4296]: 075253EA1E: to=, relay=spamassassin, delay=19, delays=17/0.03/0/2.2, dsn=2.0.0, status=sent (del
Apr 22 07:07:26 pi-box postfix/qmgr[1703]: 075253EA1E: removed
Apr 22 07:07:26 pi-box postfix/qmgr[1703]: CDD9D3EA20: from=, size=20333, nrcpt=1 (queue ac
Apr 22 07:07:26 pi-box postfix/smtpd[4290]: disconnect from out083.topica-gold-y.com[66.227.61.3]
Apr 22 07:07:26 pi-box dovecot[1515]: lmtp(4302): Connect from local
Apr 22 07:07:27 pi-box dovecot[1515]: lmtp(4302, ronel): LydHOy4B+1jOEAAA6C2HcQ: sieve: msgid=<1351457768-1463747838-1492843864@cloud.b.topica.com>: stored ma
Apr 22 07:07:27 pi-box postfix/lmtp[4301]: CDD9D3EA20: to=, relay=pi-box.co.uk[private/dovecot-lmtp], delay=0.53, delays=0.15/0.02/0.06/0.3
Apr 22 07:07:27 pi-box dovecot[1515]: lmtp(4302): Disconnect from local: Successful quit
Apr 22 07:07:27 pi-box postfix/qmgr[1703]: CDD9D3EA20: removed
Apr 22 07:09:01 pi-box CRON[4307]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 22 07:09:02 pi-box CRON[4311]: (root) CMD ( [ -x /usr/lib/php5/sessionclean ] && /usr/lib/php5/sessionclean)
Apr 22 07:09:03 pi-box CRON[4307]: pam_unix(cron:session): session closed for user root
Apr 22 07:10:47 pi-box postfix/anvil[4292]: statistics: max connection rate 1/60s for (smtp:66.227.61.3) at Apr 22 07:07:03
Apr 22 07:10:47 pi-box postfix/anvil[4292]: statistics: max connection count 1 for (smtp:66.227.61.3) at Apr 22 07:07:03
Apr 22 07:10:47 pi-box postfix/anvil[4292]: statistics: max cache size 1 at Apr 22 07:07:03
Apr 22 07:17:01 pi-box CRON[4352]: pam_unix(cron:session): session opened for user root by (uid=0)
Any ideas? When I restart my Pi everything is ok for about a day, but I had to restart my system every day for the last three or four days.
Please help.
Thx,
Jo
Segmentation fault
Segmentation fault daily cron job?
Hi Sam,
Had a look but couldn't see any errors, but also saw the times correlated to my daily cron jobs that are emailed to me and that showed a segmentation issue:
/etc/cron.daily/ddclient:
SUCCESS: updating @: good: IP address set to 212.56.88.217
/etc/cron.daily/logrotate:
Warning: Unit file of apache2.service changed on disk, 'systemctl daemon-reload'
recommended.
/etc/cron.daily/spamassassin-learn:
/etc/cron.daily/spamassassin-learn: line 29: 6800 Segmentation fault sa-learn
--no-sync --spam /home/ronel/Maildir/.Spam/{cur,new}
/etc/cron.daily/spamassassin-learn: line 30: 6824 Segmentation fault sa-learn
--no-sync --spam /home/ronel.pieterse/Maildir/.Spam/{cur,new}
/etc/cron.daily/spamassassin-learn: line 31: 6825 Segmentation fault sa-learn
--no-sync --spam /home/ronel-fords/Maildir/.Spam/{cur,new}
/etc/cron.daily/spamassassin-learn: line 32: 6826 Segmentation fault sa-learn
--no-sync --spam /home/mumofone/Maildir/.Spam/{cur,new}
/etc/cron.daily/spamassassin-learn: line 37: 6827 Segmentation fault sa-learn
--sync
run-parts: /etc/cron.daily/spamassassin-learn exited with return code 139
I also see two of my mailboxes are not there, I assume they're not affected. The above ones are my wife's...
So the question is, how do I fix it? Theoretically I assume I don't need the setting of an ip address anymore as I have a static ip...
Please advise.
Regards,
Jo
Not sure if it will be a bug
segmentation fault
Hi Sam,
I ran the commands manually and it gives me some sort of Permission Denied error for all the accounts apart from my admin account which seemed to work fine:
Admin account:
admin@pi-box:~ $ sa-learn --no-sync --spam /home/admin/Maildir/.Spam/{cur,new}
Learned tokens from 3522 message(s) (3522 message(s) examined)
One of the other accounts:
admin@pi-box:~ $ sa-learn --no-sync --spam /home/ronel/Maildir/.Spam/{cur,new}
archive-iterator: no access to /home/ronel/Maildir/.Spam/cur: Permission denied at /usr/share/perl5/Mail/SpamAssassin/ArchiveIterator.pm line 588.
archive-iterator: no access to /home/ronel/Maildir/.Spam/cur: Permission denied at /usr/share/perl5/Mail/SpamAssassin/ArchiveIterator.pm line 830.
archive-iterator: no access to /home/ronel/Maildir/.Spam/new: Permission denied at /usr/share/perl5/Mail/SpamAssassin/ArchiveIterator.pm line 588.
archive-iterator: no access to /home/ronel/Maildir/.Spam/new: Permission denied at /usr/share/perl5/Mail/SpamAssassin/ArchiveIterator.pm line 830.
archive-iterator: unable to open /home/ronel/Maildir/.Spam/cur: Permission denied
archive-iterator: unable to open /home/ronel/Maildir/.Spam/new: Permission denied
Learned tokens from 0 message(s) (0 message(s) examined)
Any idea?
Thanks for any help.
Regards,
Jo
Try with sudo
possible mailbox isolated?
Hi Sam,
I ran the commands as suggested with sudo and they all worked until I got to 'ronel' (my wife's email...). She does get a lot of spam! It wouldn't complete the spam learning with hers. I ran it again with the same result. Then I checked to see if SA is still running, but it was not. See results below:
admin@pi-box:~ $ sudo sa-learn --no-sync --spam /home/jo/Maildir/.Spam/{cur,new}
[sudo] password for admin:
Learned tokens from 0 message(s) (979 message(s) examined)
admin@pi-box:~ $ sudo sa-learn --no-sync --spam /home/mumofone/Maildir/.Spam/{cur,new}
Learned tokens from 16 message(s) (692 message(s) examined)
admin@pi-box:~ $ sudo sa-learn --no-sync --spam /home/ronel.pieterse/Maildir/.Spam/{cur,new}
Learned tokens from 11 message(s) (131 message(s) examined)
admin@pi-box:~ $ sudo sa-learn --no-sync --spam /home/ronel-fords/Maildir/.Spam/{cur,new}
Learned tokens from 0 message(s) (364 message(s) examined)
admin@pi-box:~ $ sudo sa-learn --no-sync --spam /home/ronel/Maildir/.Spam/{cur,new}
admin@pi-box:~ $ sudo sa-learn --no-sync --spam /home/ronel/Maildir/.Spam/{cur,new}
[sudo] password for admin:
admin@pi-box:~ $ systemctl status -l spamassassin.service
● spamassassin.service - Perl-based spam filter using text analysis
Loaded: loaded (/lib/systemd/system/spamassassin.service; enabled)
Active: failed (Result: signal) since Fri 2017-04-28 20:56:20 UTC; 2min 32s ago
Process: 618 ExecStart=/usr/sbin/spamd -d --pidfile=/var/run/spamassassin.pid $OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 1540 (code=killed, signal=SEGV)
admin@pi-box:~ $ sudo ls -al /var/lib/spamassassin/.spamassassin/
total 5632
drwx------ 3 debian-spamd debian-spamd 4096 Apr 28 20:26 .
drwxr-xr-x 5 debian-spamd debian-spamd 4096 Feb 28 2016 ..
-rw------- 1 debian-spamd debian-spamd 67392 Apr 28 20:51 bayes_journal
-rw------- 1 debian-spamd debian-spamd 2433024 Apr 28 20:26 bayes_seen
-rw------- 1 debian-spamd debian-spamd 5406720 Apr 28 20:26 bayes_toks
drwxr-xr-x 2 debian-spamd debian-spamd 4096 Feb 28 2016 sa-compile.cache
-rw-r--r-- 1 debian-spamd debian-spamd 1869 Feb 28 2016 user_prefs
I've also noticed that you've updated the spamassassin tutorial to include the database path (--dbpath /var/lib/spamassassin/.spamassassin/) which I don't have. Should I update mine as well?
Please advise.
Thank you for your time.
Regards,
Jo
Hi Jo,
DB path set up
Hi Sam,
Thanks, that sounds logical and makes sense. I have added the DB path and can see Spamassassin 'learning' already:
admin@pi-box:~ $ sudo nano /etc/cron.daily/spamassassin-learn
[sudo] password for admin:
admin@pi-box:~ $ sudo sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --spam /home/mumofone/Maildir/.Spam/{cur,new}
Learned tokens from 737 message(s) (738 message(s) examined)
admin@pi-box:~ $ sudo sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --spam /home/ronel-fords/Maildir/.Spam/{cur,new}
Learned tokens from 427 message(s) (427 message(s) examined)
admin@pi-box:~ $ sudo sa-learn --dbpath /var/lib/spamassassin/.spamassassin/ --no-sync --spam /home/ronel.pieterse/Maildir/.Spam/{cur,new}
Learned tokens from 171 message(s) (171 message(s) examined)
I might incrementally delete the emails from her spam folder (after backup as suggested) and test until the error doesn't show anymore...
Regards,
Jo
Spamassassin seems good now
Hi Sam,
I done as suggested and removed all the emails from the spam folder and then I incrementally put them back month by month and tested it. It seems like April's spam is where the issue lies. Without April's spam it runs fine:
#================================ 2017-04-29 ================================#
Learning HAM from Inbox
Learned tokens from 1 message(s) (367 message(s) examined)
Learned tokens from 2 message(s) (1223 message(s) examined)
Learned tokens from 10 message(s) (866 message(s) examined)
Learned tokens from 0 message(s) (688 message(s) examined)
Learned tokens from 0 message(s) (269 message(s) examined)
Learned tokens from 0 message(s) (24 message(s) examined)
Learning SPAM from Spam folder
Learned tokens from 71 message(s) (3594 message(s) examined)
Learned tokens from 38 message(s) (1015 message(s) examined)
Learned tokens from 3 message(s) (6516 message(s) examined)
Learned tokens from 1 message(s) (172 message(s) examined)
Learned tokens from 0 message(s) (427 message(s) examined)
Learned tokens from 0 message(s) (738 message(s) examined)
Syncing
bayes: synced databases from journal in 1 seconds: 846 unique entries (846 total entries)
At some point I will try to isolate the exact email that's causing the issue...
Thanks again for your help!
Regards,
Jo
You're welcome
Pi Failure - Spamassassin?
Hi Sam,
MY Pi crashed last week...it frose and when I restarted it would not boot up at all...only a few lines of the boot sequence (1 page worth) then it hangs...
Anyway, I tried the SD card in a different Pi, because I have a Pi 3 now, but the same issue. Luckily (I think) I had a backup of my SD card from a month ago and I tried that, which worked fine and regarding spamassassin, updated the database path (--dbpath /var/lib/spamassassin/.spamassassin/), which needed to be done as per our previous discussion.
Everything went swimmingly until this morning, when I got the following from the daily cron:
/etc/cron.daily/ddclient:
SUCCESS: updating @: good: IP address set to 212.56.88.217
/etc/cron.daily/spamassassin-learn:
locker: error accessing /var/lib/spamassassin/.spamassassin/bayes.lock: No such file
or directory at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 190.
locker: safe_unlock: lock on /var/lib/spamassassin/.spamassassin/bayes.lock was lost
due to expiry at /usr/share/perl5/Mail/SpamAssassin/Locker/UnixNFSSafe.pm line 219.
The only thing different from yesterday is that I updated as usual once or twice a week (apt-get update/upgrade).
Journalctl for the last 15 or so minutes show the following:
May 19 20:17:01 pi-box CRON[29805]: pam_unix(cron:session): session opened for user root by (uid=0)
May 19 20:17:01 pi-box CRON[29809]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
May 19 20:17:01 pi-box CRON[29805]: pam_unix(cron:session): session closed for user root
May 19 20:20:07 pi-box sudo[29837]: admin : TTY=pts/0 ; PWD=/home/admin ; USER=root ; COMMAND=/bin/journalctl
May 19 20:20:07 pi-box sudo[29837]: pam_unix(sudo:session): session opened for user root by (uid=0)
May 19 20:21:18 pi-box postfix/smtpd[29846]: warning: database /etc/postfix/helo_access.db is older than source file /etc/postfix/helo_access
May 19 20:21:18 pi-box postfix/smtpd[29846]: connect from d60-65-212-208.col.wideopenwest.com[65.60.208.212]
May 19 20:21:19 pi-box postfix/smtpd[29846]: NOQUEUE: reject: RCPT from d60-65-212-208.col.wideopenwest.com[65.60.208.212]: 550 5.1.1 : Recipient address rejected: User unknown in local re
May 19 20:21:19 pi-box postfix/smtpd[29846]: C117843F06: client=d60-65-212-208.col.wideopenwest.com[65.60.208.212]
May 19 20:21:20 pi-box postfix/cleanup[29850]: C117843F06: message-id=<848552993046736-TIBUMHKAZJATRXHQPWND@ndneqh3.corrievanbinsbergen.com>
May 19 20:21:20 pi-box postfix/qmgr[24088]: C117843F06: from=, size=49161, nrcpt=1 (queue active)
May 19 20:21:20 pi-box spamd[14681]: spamd: connection from localhost [::1]:58282 to port 783, fd 6
May 19 20:21:20 pi-box spamd[14681]: spamd: setuid to debian-spamd succeeded
May 19 20:21:21 pi-box spamd[14681]: spamd: processing message <848552993046736-TIBUMHKAZJATRXHQPWND@ndneqh3.corrievanbinsbergen.com> for debian-spamd:114
May 19 20:21:21 pi-box postfix/smtpd[29846]: disconnect from d60-65-212-208.col.wideopenwest.com[65.60.208.212]
May 19 20:21:21 pi-box spamd[14681]: bayes: cannot write to /var/lib/spamassassin/.spamassassin/bayes_journal, bayes db update ignored: Permission denied
May 19 20:21:22 pi-box spamd[14681]: bayes: cannot write to /var/lib/spamassassin/.spamassassin/bayes_journal, bayes db update ignored: Permission denied
May 19 20:21:26 pi-box spamd[14681]: spamd: identified spam (21.1/3.0) for debian-spamd:114 in 5.6 seconds, 47967 bytes.
May 19 20:21:26 pi-box spamd[14681]: spamd: result: Y 21 - BAYES_99,HELO_DYNAMIC_IPADDR,HTML_FONT_LOW_CONTRAST,HTML_MESSAGE,MIME_HTML_MOSTLY,MIME_QP_LONG_LINE,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_BRBL_LASTEXT,RCVD_
May 19 20:21:26 pi-box spamd[1322]: prefork: child states: II
May 19 20:21:26 pi-box postfix/pickup[29681]: DD2BD44379: uid=114 from=
May 19 20:21:26 pi-box postfix/pipe[29851]: C117843F06: to=, relay=spamassassin, delay=7.9, delays=1.9/0.02/0/5.9, dsn=2.0.0, status=sent (delivered via spamassassin service)
May 19 20:21:26 pi-box postfix/cleanup[29850]: DD2BD44379: message-id=<848552993046736-TIBUMHKAZJATRXHQPWND@ndneqh3.corrievanbinsbergen.com>
May 19 20:21:26 pi-box postfix/qmgr[24088]: C117843F06: removed
May 19 20:21:26 pi-box postfix/qmgr[24088]: DD2BD44379: from=, size=51729, nrcpt=1 (queue active)
May 19 20:21:27 pi-box dovecot[25068]: lmtp(29857): Connect from local
May 19 20:21:27 pi-box dovecot[25068]: lmtp(29857, ronel): DG9PAcdTH1mhdAAA6C2HcQ: sieve: msgid=<848552993046736-TIBUMHKAZJATRXHQPWND@ndneqh3.corrievanbinsbergen.com>: stored mail into mailbox 'Spam'
May 19 20:21:27 pi-box postfix/lmtp[29856]: DD2BD44379: to=, relay=pi-box.co.uk[private/dovecot-lmtp], delay=0.65, delays=0.39/0.01/0.03/0.23, dsn=2.0.0, status=sent (250 2.0.0 : Helo command rejected: Host not found; from= to=
May 19 20:23:43 pi-box postfix/smtpd[29953]: NOQUEUE: reject: RCPT from unknown[187.94.99.194]: 450 4.7.1 : Helo command rejected: Host not found; from= to=
May 19 20:23:43 pi-box postfix/smtpd[29953]: disconnect from unknown[187.94.99.194]
May 19 20:27:04 pi-box postfix/anvil[29848]: statistics: max connection rate 1/60s for (smtp:65.60.208.212) at May 19 20:21:18
May 19 20:27:04 pi-box postfix/anvil[29848]: statistics: max connection count 1 for (smtp:65.60.208.212) at May 19 20:21:18
May 19 20:27:04 pi-box postfix/anvil[29848]: statistics: max cache size 1 at May 19 20:21:18
May 19 20:29:11 pi-box dovecot[25068]: imap-login: Login: user=, method=PLAIN, rip=212.56.88.217, lip=192.168.1.100, mpid=29985, TLS, session=
May 19 20:31:00 pi-box dovecot[25068]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=30022, secured, session=
May 19 20:31:00 pi-box dovecot[25068]: imap(admin): Disconnected: Logged out in=44 out=831
May 19 20:31:01 pi-box dovecot[25068]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=30027, secured, session=
May 19 20:31:01 pi-box dovecot[25068]: imap(admin): Disconnected: Logged out in=261 out=2132
May 19 20:31:01 pi-box dovecot[25068]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=30033, secured, session=<5JBQZuZP2gB/AAAB>
May 19 20:31:02 pi-box dovecot[25068]: imap(admin): Disconnected: Logged out in=297 out=231134
May 19 20:31:07 pi-box dovecot[25068]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=30038, secured, session=
May 19 20:31:08 pi-box dovecot[25068]: imap(admin): Disconnected: Logged out in=152 out=2400
May 19 20:31:20 pi-box dovecot[25068]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=30051, secured, session=
May 19 20:31:22 pi-box dovecot[25068]: imap(admin): Disconnected: Logged out in=297 out=231134
May 19 20:32:59 pi-box postfix/smtpd[30053]: warning: database /etc/postfix/helo_access.db is older than source file /etc/postfix/helo_access
May 19 20:33:00 pi-box postfix/smtpd[30053]: warning: hostname 59.162.181.202.static.vsnl.net.in does not resolve to address 59.162.181.202: Name or service not known
May 19 20:33:00 pi-box postfix/smtpd[30053]: connect from unknown[59.162.181.202]
May 19 20:33:00 pi-box postfix/smtpd[30053]: NOQUEUE: reject: RCPT from unknown[59.162.181.202]: 450 4.7.1 <59.162.181.202.static.vsnl.net.in>: Helo command rejected: Host not found; from=
May 19 20:33:01 pi-box postfix/smtpd[30053]: lost connection after DATA from unknown[59.162.181.202]
May 19 20:33:01 pi-box postfix/smtpd[30053]: disconnect from unknown[59.162.181.202]
There's a few lines about spamassassin and permissions regarding bayes? I see it also occurred a few times during the week although no errors in daily cron...
Any ideas?
Regards,
Jo
What are the permissions of that folder?
/var/lib/spamassassin/.spamassassin/
? Sam.spamassassin permissions?
Hi Sam,
Here are the permissions:
admin@pi-box:/var/lib/spamassassin $ ls -al
total 20
drwxr-xr-x 5 debian-spamd debian-spamd 4096 Feb 28 2016 .
drwxr-xr-x 59 root root 4096 Apr 13 12:08 ..
drwxrwxr-x 3 debian-spamd debian-spamd 4096 Feb 28 2016 compiled
drwx------ 2 debian-spamd debian-spamd 4096 Feb 28 2016 sa-update-keys
drwx------ 3 debian-spamd debian-spamd 4096 May 22 22:14 .spamassassin
Regards,
Jo
So I can see inside the
Spamassassin tree permissions
Hi Sam,
Here's the info:
admin@pi-box:~ $ tree -augp /var/lib/spamassassin
/var/lib/spamassassin
├── [drwxrwxr-x debian-spamd debian-spamd] compiled
│ └── [drwxrwxr-x debian-spamd debian-spamd] 5.020
│ └── [drwxrwxr-x debian-spamd debian-spamd] 3.004000
│ ├── [drwxrwxr-x debian-spamd debian-spamd] auto
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] Mail
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] SpamAssassin
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] CompiledRegexps
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] body_0
│ │ ├── [-r-xr-xr-x debian-spamd debian-spamd] body_0.so
│ │ └── [-rw-rw-r-- debian-spamd debian-spamd] .packlist
│ ├── [-rw-r--r-- debian-spamd debian-spamd] bases_body_0.pl
│ ├── [drwxrwxr-x debian-spamd debian-spamd] Mail
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] SpamAssassin
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] CompiledRegexps
│ │ └── [-r--r--r-- debian-spamd debian-spamd] body_0.pm
│ └── [-rw-rw-r-- debian-spamd debian-spamd] perllocal.pod
├── [drwx------ debian-spamd debian-spamd] sa-update-keys [error opening dir]
└── [drwx------ debian-spamd debian-spamd] .spamassassin [error opening dir]
13 directories, 5 files
admin@pi-box:~ $
Regards,
Jo
same again with sudo
sudo tree spamassassin
Hi Sam,
Congratulations! I hope you will have many happy years together!!!
Here's the tree:
admin@pi-box:~ $ sudo tree -augp /var/lib/spamassassin
[sudo] password for admin:
/var/lib/spamassassin
├── [drwxrwxr-x debian-spamd debian-spamd] compiled
│ └── [drwxrwxr-x debian-spamd debian-spamd] 5.020
│ └── [drwxrwxr-x debian-spamd debian-spamd] 3.004000
│ ├── [drwxrwxr-x debian-spamd debian-spamd] auto
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] Mail
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] SpamAssassin
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] CompiledRegexps
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] body_0
│ │ ├── [-r-xr-xr-x debian-spamd debian-spamd] body_0.so
│ │ └── [-rw-rw-r-- debian-spamd debian-spamd] .packlist
│ ├── [-rw-r--r-- debian-spamd debian-spamd] bases_body_0.pl
│ ├── [drwxrwxr-x debian-spamd debian-spamd] Mail
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] SpamAssassin
│ │ └── [drwxrwxr-x debian-spamd debian-spamd] CompiledRegexps
│ │ └── [-r--r--r-- debian-spamd debian-spamd] body_0.pm
│ └── [-rw-rw-r-- debian-spamd debian-spamd] perllocal.pod
├── [drwx------ debian-spamd debian-spamd] sa-update-keys
│ ├── [-rw------- debian-spamd debian-spamd] pubring.gpg
│ ├── [-rw------- debian-spamd debian-spamd] pubring.gpg~
│ ├── [-rw------- debian-spamd debian-spamd] secring.gpg
│ └── [-rw------- debian-spamd debian-spamd] trustdb.gpg
└── [drwx------ debian-spamd debian-spamd] .spamassassin
├── [-rw------- root root ] bayes_journal
├── [-rw------- debian-spamd debian-spamd] bayes_seen
├── [-rw------- debian-spamd debian-spamd] bayes_toks
├── [drwxr-xr-x debian-spamd debian-spamd] sa-compile.cache
│ └── [-rw-r--r-- debian-spamd debian-spamd] rules.body_0
└── [-rw-r--r-- debian-spamd debian-spamd] user_prefs
14 directories, 14 files
Regards,
Jo
Thanks :)
bayes_journal
should be owned bydebian-spamd
, notroot
. I guess what happened is that the file didn't already exist when you ran the command as root for the first time, so it was created by root (with root ownership and group). If it had already existed (e.g. because the command had been run bydebian-spamd
) it should just have been read by / written to by root without changing the ownership etc. Run this command to change the ownership back and see if that solves your problems: SamNo incoming mail
Hi Sam,
since I followed along wiht squirrel everything worked fine, but after installing spamassassin no mail comes in. Outgoing seems to be fine, gmail receives mails from the server.
I don't know any further ...
Thanks.
Roman
Check the logs for messages
Error during spamassassin installation: sa-compile
Hi Sam,
I follow your great guide thus far. And all seems to be working fine. But I have no idea how to solve the following problem during the installation process of spamassassin itself.
May you know the problem?
After 'sudo apt-get install spamassassin' everything went fine till
…
spamassassin (3.4.2-1~deb9u1) wird eingerichtet ...
Lege Systembenutzer »debian-spamd« (UID 116) an ...
Lege neue Gruppe »debian-spamd« (GID 122) an ...
Lege neuen Benutzer »debian-spamd« (UID 116) mit Gruppe »debian-spamd« an ...
Erstelle Home-Verzeichnis »/var/lib/spamassassin« nicht.
sa-compile (3.4.2-1~deb9u1) wird eingerichtet ...
Running sa-compile (may take a long time)
arm-linux-gnueabihf-gcc: error trying to exec 'as': execvp: Keine Berechtigung
make: *** [body_0.o] Fehler 2
command 'make PREFIX=/tmp/.spamassassin20035ZsAjs9tmp/ignored INSTALLSITEARCH=/var/lib/spamassassin/compiled/5.024/3.004002 >>/tmp/.spamassassin20035ZsAjs9tmp/log' failed: exit 2
dpkg: Fehler beim Bearbeiten des Paketes sa-compile (--configure):
Unterprozess installiertes post-installation-Skript gab den Fehlerwert 25 zurück
Trigger für systemd (232-25+deb9u9) werden verarbeitet ...
Fehler traten auf beim Bearbeiten von:
sa-compile
needrestart is being skipped since dpkg has failed
E: Sub-process /usr/bin/dpkg returned an error code (1)
When I follow the path var/lib/spamassassin/ I currently can’t find the folder ‘compiled’.
Can you help me please?
Regards
Should update /etc/default/spamassassin section
"Now edit /etc/default/spamassassin and set:
ENABLED=1"
This is not supported anymore. The file contains some additional infos
Gmail complains about unencrypted mail
Hello Sam,
Thanks for your excellent tutorial. I followed it three years ago, and now -migrating to Buster- it is still of great help.
Today I found out that gmail currently marks mails that are not TLS encrypted as in error. I fixed this by adding the next line to /etc/postfix/main.cf:
smtp_tls_security_level = may
Perhaps you can add this to your tutorial so others can use this too.
Regards,
Hans.
Hans,
Add new comment