Raspberry Pi Email Server Part 2: Dovecot

Powered by Drupal
Submitted by Sam Hobbs on

Dovecot Logo This is the second part of a five part tutorial that will show you how to install a full featured email server on your Raspberry Pi. This tutorial covers Dovecot, which provides SASL authentication and IMAP capabilities.

The parts are:

The Introduction & Contents Page (read first)

Raspberry Pi Email Server Part 1: Postfix

Raspberry Pi Email Server Part 2: Dovecot

Raspberry Pi Email Server Part 3: Squirrelmail

Raspberry Pi Email Server Part 4: Spam Detection with Spamassassin

Raspberry Pi Email Server Part 5: Spam Sorting with LMTP & Sieve

Fixing the errors that appeared during dovecot installation

In part 1, when you installed Dovecot I mentioned that you might see some errors like this:

Creating config file /etc/dovecot/conf.d/20-imap.conf with new version
[....] Restarting IMAP/POP3 mail server: dovecotError: socket() failed: Address family not supported by protocol
Error: service(imap-login): listen(::, 143) failed: Address family not supported by protocol
Error: socket() failed: Address family not supported by protocol
Error: service(imap-login): listen(::, 993) failed: Address family not supported by protocol
Fatal: Failed to start listeners
 failed!
invoke-rc.d: initscript dovecot, action "restart" failed.
dpkg: error processing dovecot-imapd (--configure):
 subprocess installed post-installation script returned error exit status 1
Setting up dovecot-ldap (1:2.1.7-7) ...

These errors are caused by the lack of IPv6 support, which I mentioned in the previous tutorial. To remove the errors, open the main dovecot configuration file (/etc/dovecot/dovecot.conf) and find this line:

listen = *, ::

And change it to:

listen = *

The * means “all IPv4 addresses”, the :: means “all IPv6 addresses”. Now restart Dovecot, and you shouldn’t get any errors:

sudo service dovecot restart

Note: since I wrote this tutorial, there have been a few small changes to the default configuration file - you may find that the line is commented (with a # at the start of the line). If so, remember to uncomment it when you make your changes!

Tell Dovecot where your Mailbox is

Open /etc/dovecot/conf.d/10-mail.conf and find this line:

mail_location = mbox:~/mail:INBOX=/var/mail/%u

Change it to this:

mail_location = maildir:~/Maildir

Instruct Postfix to use Dovecot SASL

Now we need to tell Postfix that we would like to use Dovecot for SASL authentication. Open /etc/postfix/main.cf and add these lines:

smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes

Now tell Dovecot to listen for SASL authentication requests from Postfix. Open /etc/dovecot/conf.d/10-master.conf and comment out the current block that begins with service auth (place a # at the start of each line). Replace it with this:

service auth {
        unix_listener /var/spool/postfix/private/auth {
                mode = 0660
                user = postfix
                group = postfix
        }
}

Now you want to enable plain text logins. Do it by adding these two lines to /etc/dovecot/conf.d/10-auth.conf. Make sure they are not already present in the file, or your settings may be overwritten with the default ones if the default is declared later in the file than the lines you add. If the parameters are already present, you can either modify the existing lines or comment them out and add these new ones:

disable_plaintext_auth = no
auth_mechanisms = plain login

Note that although the logins are in plain text, we will be setting Postfix up later so that it only allows you to use plaintext logins from within SSL/TLS. This means that your login and password will sent in an encrypted session - you wouldn't see them in plain text if you used a packet sniffer, for example. For now, we’re allowing unencrypted plain text logins so that we can test logging in with Telnet. Since the connection is local (from the Pi to the Pi), your password isn’t being sent over any insecure networks so this is fine.

Testing SASL

Creating a new user for testing purposes is a good idea. Let’s call this temporary user testmail and give it the password test1234 Use this command to add the user, and follow the prompts including setting a password.

sudo adduser testmail

Now restart Postfix and Dovecot:

sudo service postfix restart
sudo service dovecot restart

We’re now going to try and send an email after authenticating with SASL. The server is expecting to see a base64 encoded version of your username and password, so we have to convert it first. There are three ways of doing this, so I've given examples below using the testmail username and test1234 password:

#Method No.1
echo -ne '\000testmail\000test1234' | openssl base64

#Method No.2
perl -MMIME::Base64 -e 'print encode_base64("\0testmail\0test1234");'

#Method No.3
printf '\0%s\0%s' 'testmail' 'test1234' | openssl base64

I have discovered that if your password starts with a number, methods 1 and 2 don’t work. Assuming the username and password are testmail and test1234, the commands produce this:

AHRlc3RtYWlsAHRlc3QxMjM0

WARNING: If you’re having problems with authentication and you paste examples to forums or mailing lists, be aware that it is really easy to convert this back into your username and password (hence the creation of a test user). If you're using your real username and password to test, redact it before posting! Now, still logged into the Pi via SSH, you can telnet port 25 to test whether or not SASL is working. There’s only one extra step, which is the AUTH PLAIN command that comes after ehlo but before mail from. For testing, the permit_mynetworks parameter should be commented out under your postfix smtpd_recipient_restrictions block in /etc/postfix/main.cf. If you’re following on from Raspberry Pi Email Server Part 1: Postfix then this should already be the case. If you have to change it, remember to reload postfix (sudo service postfix reload) after you change the value. Here’s an example:

telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 samhobbs ESMTP Postfix (Debian/GNU)
ehlo facebook.com
250-samhobbs
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-AUTH PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
AUTH PLAIN AHRlc3RtYWlsAHRlc3QxMjM0
235 2.7.0 Authentication successful
mail from:testmail
250 2.1.0 Ok
rcpt to:me@externalemail.com
250 2.1.5 Ok
data
354 End data with .
Subject: This is my first email that has been authenticated with Dovecot SASL
Woop woop
.
250 2.0.0 Ok: queued as B87133F768
quit
221 2.0.0 Bye
Connection closed by foreign host.

Now try again but enter the username/password incorrectly (base64 encode something random) – you should get an error message and the email won’t send. If everything went to plan, then SASL is working properly! You can now uncomment permit_mynetworks again.

Separating Incoming email (unauthenticated) from Outgoing Email (SASL authenticated)

It’s probably a good idea to have a dedicated port for sending outgoing email…here’s why: Port 25 doesn’t require (but does offer) SSL/TLS encryption. If you mess up configuring your mail client you could end up letting it authenticate with SASL over insecure connections. Using a different port that only accepts SSL/TLS connections removes the risk that a poorly configured email client could be sending your password unencrypted over dodgy networks. There are two ports you can use for this:

  1. 465: SMTP over SSL
  2. 587: Email submission

587 is the “official” port for email clients (like K9 mail, Thunderbird and Outlook) to use when submitting messages to the Mail Submission Agent (your email server) – the submission may be encrypted or unencrypted depending on the server configuration. 465 was a port that was assigned for SMTP with SSL/TLS before the STARTTLS protocol was introduced, back in the days when you chose your port and that decided on the type of connection you were going to get (encrypted or unencrypted). STARTTLS changed things because it allows you to connect with an unencrypted connection (like the one you get with Telnet), and then upgrade to an encrypted connection without changing port… so when STARTTLS was introduced, SMTPS on port 465 was removed from the standard because you could do the same thing with a single port (25). However, I think there is some value in specifying a port for submission that only accepts SSL/TLS encrypted connections, and won’t work if the connection isn’t encrypted. This means that if you misconfigure your email client it just won’t work, instead of working and sending your password in an unencrypted format. So, anyway… Here’s how to set up Postfix to listen on port 465 for encrypted connections. The first step is telling Postfix to listen on port 465, so open /etc/postfix/master.cf and uncomment the line:

smtps     inet  n       -       -       -       -       smtpd

Now restart Postfix:

sudo service postfix restart

Test whether Postfix is listening on port 465:

telnet localhost 465
Trying 127.0.0.1...                                                                           
Connected to localhost.                                                                       
Escape character is '^]'.
220 samhobbs.co.uk ESMTP Postfix (Debian/GNU)
ehlo samhobbs.co.uk
250-samhobbs
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-AUTH PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
quit
221 2.0.0 Bye
Connection closed by foreign host.

OK, so now it’s listening on the right port, but it’s allowing unencrypted connections. Here’s how you force TLS on port 465: open /etc/postfix/master.cf and find the line you uncommented earlier. Below it are some options, you want to edit them so that they look like this (i.e. uncomment lines 2 and 3):

smtps     inet  n       -       -       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes

Line 3 is forcing TLS on port 465, and line 2 means that connections to port 465 have a different label in the logs, which can be useful for debugging.

sudo service postfix restart

Now try connecting with Telnet again… you should be able to establish a connection, but not receive any prompts from the server:

telnet localhost 465                                            
Trying 127.0.0.1...                                                                           
Connected to localhost.
Escape character is '^]'.
exit
exit
Connection closed by foreign host.

Now try openssl:

openssl s_client -connect localhost:465 -quiet
depth=0 CN = samhobbs
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = samhobbs
verify return:1
220 samhobbs.co.uk ESMTP Postfix (Debian/GNU)
quit
221 2.0.0 Bye

Good: we are able to start a TLS encrypted connection. We got some errors because the certificate is self-signed (it's not signed by a certificate that is in the trusted root store on the server) but this is OK because we're just using the certificate for testing for now. When you come back later to set up a proper certificate, you can use this command to verify it. The -CApath option tells openssl where the trusted certificates are stored on your system:

openssl s_client -connect localhost:465 -quiet -CApath /etc/ssl/certs

Successful validation looks something like this:

sam@samhobbs:~$ openssl s_client -connect localhost:465 -quiet -CApath /etc/ssl/certs
depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root
verify return:1                                                                              
depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
verify return:1                                                                              
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA
verify return:1                                                                              
depth=0 OU = Domain Control Validated, OU = PositiveSSL, CN = samhobbs.co.uk                 
verify return:1                                                                              
220 samhobbs.co.uk ESMTP Postfix (Ubuntu)                                                    
quit                                                                                         
221 2.0.0 Bye

There are a couple more changes we want to make here: first, tell Postfix to only advertise SASL authentication over encrypted connections (so that you don’t accidentally send your password in the clear). Open /etc/postfix/main.cf and add this line:

smtpd_tls_auth_only = yes
sudo service postfix reload

Now connect to port 25 and you shouldn’t see AUTH advertised:

telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 samhobbs.co.uk ESMTP Postfix (Debian/GNU)
ehlo samhobbs.co.uk
250-samhobbs.co.uk
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

Lastly, we want to override the smtp_recipient_restrictions for port 465 so that it doesn't accept incoming messages from unauthenticated users. At first, I didn't make this change and I noticed that some spam emails were coming in on port 465 and bypassing my spam filter, which I configured to scan all incoming email on port 25, but not 465 because I only expected it to be used for outgoing email. We can do this by overriding the smtp_recipient_restrictions list for port 465 in /etc/postfix/master.cf. Open master.cf and find the smtps line. Add a new recipient restrictions list option like this:

smtps     inet  n       -       -       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject

Now reload postfix:

sudo service postfix reload

Perfect! Postfix configuration is now complete.

Testing IMAP

There are two main protocols for fetching mail: POP and IMAP. The main difference between them is what they do with emails when they collect them: a POP client will fetch email from your server and remove it from the server when it’s done. This is inconvenient if you want to connect with two or more devices (like a phone and a computer) and have complete copies of all your emails on both. IMAP, on the other hand, makes a copy of the emails on the server and leaves the originals there. For this reason, I think IMAP is much more useful than POP and I didn’t even bother to set up POP on my server. We can now test the IMAP server with Telnet in a similar way to SMTP & SASL testing earlier. This time, we’ll be using port 143, the standard port for IMAP. The stages are:

  1. establish a connection with telnet localhost 143
  2. log in with a login "USERNAME" "PASSWORD"" (not base64 encoded this time)
  3. select inbox to see messages inside b select inbox
  4. logout with c logout

In case you're wondering, the "a b c" thing is done because a client can send multiple commands to the server at once, and they might not come back in the same order depending on what they are. So, the responses have the same letter as the commands they are responding to so that the client doesn't get muddled. Here’s an example, using the testmail user we created earlier:

telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
a login "testmail" "test1234"
a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE] Logged in
b select inbox
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
* 1 EXISTS
* 0 RECENT
* OK [UNSEEN 1] First unseen.
* OK [UIDVALIDITY 1385217480] UIDs valid
* OK [UIDNEXT 2] Predicted next UID
* OK [NOMODSEQ] No permanent modsequences
b OK [READ-WRITE] Select completed.
c logout
* BYE Logging out
c OK Logout completed.
Connection closed by foreign host.

Adding TLS support

Now that we know IMAP is working, we need to enable IMAPS (imap with SSL/TLS). The standard port for this is 993. Many other tutorials that were written for older versions of dovecot will tell you to do this in different ways that won’t work, I tried 3 different methods before I ended up with a working one. First, edit /etc/dovecot/conf.d/10-master.conf, find the “service imap-login” block and uncomment the port and SSL lines so that it looks like this:

service imap-login {
  inet_listener imap {
    port = 143
  } 
  inet_listener imaps {
    port = 993
    ssl = yes
  }
}

Edit 14/10/2015: the default dovecot configuration files changed recently after Jessie became the new stable distribution of Debian, which caused some users problems; TLS on port 993 used to be enabled by default but now it isn't. We need to re-enable it. In /etc/dovecot/conf.d/10-ssl.conf, find ssl = no and change it to:

ssl = yes

There have been some security vulnerabilities discovered in older versions of the SSL protocol in recent times. SSLv2 is disabled by default, but it doesn't harm to explicitly disable it again. SSLv3 is vulnerable to an attack called POODLE, so we will disable it too. In the same file, find the ssl_protocols parameter line, uncomment it and add !SSLv3 to the end, like this:

ssl_protocols = !SSLv2 !SSLv3

Edit 02/09/2017: if you're using Debian Stretch or later, or one of its derivatives, then you will need to edit that line to match the following. The SSLv2 option is no longer recognised as an option for ssl_protocols because it has been removed entirely:

ssl_protocols = !SSLv3

For some bizarre reason, the Dovecot package for Raspberry Pi (and possibly newer versions of Ubuntu) does not create a self-signed certificate during installation like it used to. So, we have to create one manually. If you look in /usr/share/dovecot/ you will find the script that used to be used to generate the certificate; we can use it ourselves to simplify the process. The script is located at /usr/share/dovecot/mkcert.sh and looks like this:

#!/bin/sh

# Generates a self-signed certificate.
# Edit dovecot-openssl.cnf before running this.

OPENSSL=${OPENSSL-openssl}
SSLDIR=${SSLDIR-/etc/ssl}
OPENSSLCONFIG=${OPENSSLCONFIG-dovecot-openssl.cnf}

CERTDIR=/etc/dovecot
KEYDIR=/etc/dovecot/private

CERTFILE=$CERTDIR/dovecot.pem
KEYFILE=$KEYDIR/dovecot.pem

if [ ! -d $CERTDIR ]; then
  echo "$SSLDIR/certs directory doesn't exist"
  exit 1
fi

if [ ! -d $KEYDIR ]; then
  echo "$SSLDIR/private directory doesn't exist"
  exit 1
fi

if [ -f $CERTFILE ]; then
  echo "$CERTFILE already exists, won't overwrite"
  exit 1
fi

if [ -f $KEYFILE ]; then
  echo "$KEYFILE already exists, won't overwrite"
  exit 1
fi

$OPENSSL req -new -x509 -nodes -config $OPENSSLCONFIG -out $CERTFILE -keyout $KEYFILE -days 365 || exit 2
chmod 0600 $KEYFILE
echo 
$OPENSSL x509 -subject -fingerprint -noout -in $CERTFILE || exit 2

If you were going to use this certificate for any significant length of time, it would be worth editing the parameters in the config file it uses (/usr/share/dovecot/dovecot-openssl.cnf) to set the proper common name and contact details on the certificate. However, I suggest you leave the defaults as they are, use this certificate just for testing, and then come back later and generate a new cert when everything is working (more on that later). You must be in the same folder as the configuration file when you run the script, or it will not find the config and the certificate generation will fail. The following two commands will change to the right folder and then execute the script:

cd /usr/share/dovecot
sudo ./mkcert.sh

You should see a message "writing new private key to '/etc/dovecot/private/dovecot.pem'" and then some details about the certificate. Next, find the following two lines in /etc/dovecot/conf.d/10-ssl.conf and uncomment them:

#ssl_cert = </etc/dovecot/dovecot.pem
#ssl_key = </etc/dovecot/private/dovecot.pem

Now reload dovecot to apply the changes:

sudo service dovecot reload

Since IMAPS is a connection over SSL/TLS, we can’t use Telnet to test it. Instead, we use openssl to create a secure connection. There are two versions of the command, one will show you LOADS of information about the certificate used to encrypt the connection, and the other will suppress this info. I recommend trying the long version out of interest, but both will work the same for the test: For full information:

openssl s_client -connect localhost:993

For minimal information:

openssl s_client -connect localhost:993 -quiet

I won’t print the output of the first command, because it’s ridiculously long. Here’s an example of the second, including a login test:

admin@samhobbs /etc/dovecot/conf.d $ openssl s_client -connect localhost:993 -quiet
depth=0 O = Dovecot mail server, OU = samhobbs, CN = samhobbs, emailAddress = root@samhobbs.co.uk
verify error:num=18:self signed certificate
verify return:1
depth=0 O = Dovecot mail server, OU = samhobbs, CN = samhobbs, emailAddress = root@samhobbs.co.uk
verify return:1
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
a login "testmail" "test1234"
a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE] Logged in
b logout
* BYE Logging out
b OK Logout completed.
Connection closed by foreign host.

Good stuff: SSL/TLS is working on port 993, and you can log in successfully. Note that by default Dovecot uses a “snakeoil” self-signed certificate. SSL/TLS certificates are used for two purposes: encryption and verification. The “snakeoil” certificate will encrypt your content but it won’t verify that you’re talking to your server – you could be talking to someone imitating your server (anyone can create a self-signed certificate claiming to be any website). If you’d like to get your certificate signed without forking out loads of money to a cert signing authority, I’d recommend CAcert. I've written a tutorial explaining how to generate your own cert and get it signed here. If you opt for a commercial certificate, you can use the CAcert tutorial to generate the certificate and then this tutorial will explain the differences in the installation/configuration of commercial certificates once you have it signed. If you're testing a proper certificate, use this command to tell openssl where the trusted root certificates are stored:

openssl s_client -connect localhost:993 -quiet -CApath /etc/ssl/certs

Tidying up and enabling WAN access

Before opening the ports on your router to the world, it’s a good idea to delete that test user because the password is so easy to guess.

sudo userdel testmail

Also, if you still use the "pi" login, for goodness' sake change the password from "raspberry"! You can do this using the passwd command when logged in as pi:

passwd

Or you can achieve the same thing when logged in as another user by using sudo to gain root privileges:

sudo passwd pi

Now you can open a few ports on your router’s firewall. Make sure your Pi has a static LAN IP address and then forward these ports from WAN to its LAN IP address:

  • Port 25 for SMTP (used for receiving emails)
  • Port 465 for secure SMTP (used for sending emails after SASL authentication)
  • Port 993 for IMAPS (used to receive emails on your phone/tablet/computer)

Here’s an example on my router, running OpenWrt: openwrt-port-forwards-raspberry-pi-email-server.png

Setting up IMAP Email Clients

I’m now going to run through setting up IMAP email clients quickly, using K9 Mail on Android and Thunderbird on GNU/Linux as examples. The setup for Thunderbird on Windows and Mac OSX should be very similar. The basics are this:

  • Select an IMAP connection
  • Your login is your username only (omit @yourdomain.com), and you password is…your password!
  • For incoming emails: select use SSL/TLS always and the program should automatically select port 993
  • For outgoing emails: select SSL/TLS always. The program may suggest port 587, but you want port 465

K9 Mail

Open K9 Mail and select add new account. Type in your account information (you@yourdomain.com and password) and then select manual setup. Select IMAP and then enter your information as follows… Incoming email: K9 Incoming Email Settings Outgoing email: K9 Outgoing Email Settings

Thunderbird

Open Thunderbird, and then click Account Actions –> Add Mail Account. Fill in your password and email address, which is your username followed by your fully qualified domain name (FQDN), i.e. username@yourdomain.com: Thunderbird Step 1: Mail Account Setup Thunderbird will try to auto-detect settings and fail. Don’t worry, this is normal. Select “manual config”:  Thunderbird Step 2: TB will try to autodetect settings, and fail. Select “Manual Config" Now edit the settings as appropriate. I had to remove a period (.) from in front of my “server hostname”, and edit the SSL and Authentication settings. If you select “SSL/TLS” for both incoming and outgoing, ports 993 and 465 are automatically selected: Thunderbird Step 3: Edit the settings so that they match these (but change them to match your username and domain name!) Now try emailing yourself from your external email address, and see if your email gets through. If you are having problems, be sure to check you’ve set up an MX record as well as a DNS A record.

Stuck in spam filters?

A few people have contacted me recently to say that their email server is working fine but their emails are getting sent to Gmail's spam folder. If you are experiencing problems like this (or even if you're not), try setting up an SPF and/or PTR record as explained in my DNS basics tutorial. You might also want to check if your domain name or IP address are on any blacklists. There's a handy website called MX toolbox that lets you do this (choose blacklist check from the dropdown menu).

Almost done…

Good news! If you’ve reached this far and everything is working, then you’re almost done. The next step (Webmail with Squirrelmail) is optional but by far the easiest of the three steps. If you’ve hit a rut, please post a comment and I’ll try and help you out. If not… continue to Raspberry Pi Email Server Part 3: Squirrelmail

Comments

Hmm, what do you see in the postfix error log when you connect? Does it happen instantly, or only when you're inactive for a while? The log is at /var/log/mail.err, if there's nothing in there check /var/log/mail.log to get a feel for what's happening. Sam

Simon East

Mon, 07/20/2015 - 23:10

Hi Sam

I am having trouble testing IMAP

I seem to be able to log in using the testmail details, but I get an error when I select inbox, see below

telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
a login "testmail" "test1234"
a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE] Logged in
b select inbox
b NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2015-07-20 22:55:24]

Be grateful for any thoughts on the matter.

Many thanks

Simon

Hi simon, Normally when people have an error like this it's one of the following:
  • They have missed the bit about telling Dovecot where the mailbox is (so Dovecot is still looking in /var/mail)
  • The Maildir for that user doesn't exist (either because the user existed before the skel files were changed and the Maildir wasn't added manually, or because the skel step was missed).
If you check the logs at /var/log/mail.log you should see some useful information (e.g. less /var/log/mail.log and skip to the most recent messages with ctrl+G). Hope that helps, Sam

Hi Sam

Thanks for the quick reply!

I have pasted a portion of the log filev below, not quite sure what it is telling me;

Jul 21 15:43:11 pi2 dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=24340, secured, session=
Jul 21 15:43:34 pi2 dovecot: imap(testmail): Error: open(/var/mail/testmail) failed: Permission denied (euid=1001(testmail) egid=1004(testmail) missing +w perm:$
Jul 21 15:43:34 pi2 dovecot: imap(testmail): Error: Opening INBOX failed: Mailbox doesn't exist: INBOX

Can you point me in the right direction now?

Many thanks

Simon

Sam Hobbs

Tue, 07/21/2015 - 20:16

In reply to by Simon East

What a guess :D I think you missed this step:

Tell Dovecot where your Mailbox is

Open /etc/dovecot/conf.d/10-mail.conf and find this line: mail_location = mbox:~/mail:INBOX=/var/mail/%u Change it to this: mail_location = maildir:~/Maildir
Sam

Simon East

Fri, 07/24/2015 - 08:51

Hi Sam

Sorry to bother you but I have another problem, when setting up K9 Mail to receive emails I get the following error.

Username or password incorrect.
(Command: AUTHENTICATION PLAIN; response: #1# [NO, [AUTHENTICATIONFAILED], Authentication failed.])

I am using cheapname.com as recommended, any idea how I can solve this problem?

Thanks for your help.

Simon East

Mon, 07/27/2015 - 08:51

Hi Sam

Still Struggling

Below is the return from the abbreviated authentication test

openssl s_client -connect localhost:993 -quiet
depth=0 O = Dovecot mail server, OU = pi2, CN = pi2, emailAddress = root@simonaeast.com
verify error:num=18:self signed certificate
verify return:1
depth=0 O = Dovecot mail server, OU = pi2, CN = pi2, emailAddress = root@simonaeast.com
verify return:1
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
a login "testmail" "test1234"
a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE] Logged in

I have also included my mail log below.

Jul 26 06:25:19 pi2 postfix/pickup[15036]: 00A4269E86: uid=0 from=
Jul 26 06:25:19 pi2 postfix/cleanup[15447]: 00A4269E86: message-id=<20150726052519.00A4269E86@simonaeast.com>
Jul 26 06:25:19 pi2 postfix/qmgr[3981]: 00A4269E86: from=, size=725, nrcpt=1 (queue active)
Jul 26 06:25:19 pi2 postfix/local[15449]: 00A4269E86: to=, orig_to=, relay=local, delay=0.2, delays=0.14/0.03/0/0.03, dsn=2.0.0, status=sent (delivered to maildir)
Jul 26 06:25:19 pi2 postfix/qmgr[3981]: 00A4269E86: removed
Jul 26 14:50:12 pi2 postfix/pickup[21444]: 56A1A69E86: uid=33 from=
Jul 26 14:50:12 pi2 postfix/cleanup[21758]: 56A1A69E86: message-id=<8471ae176f2a1c768d513fb3e6ef0446@localhost>
Jul 26 14:50:12 pi2 postfix/qmgr[3981]: 56A1A69E86: from=, size=1082, nrcpt=1 (queue active)
Jul 26 14:50:13 pi2 postfix/smtp[21760]: 56A1A69E86: to=, relay=gmail-smtp-in.l.google.com[64.233.184.27]:25, delay=0.93, delays=0.22/0.05/0.34/0.31, dsn=2.0.0, status=sent (250 2.0.0 OK 1437918613 q6si8709046wix.94 - gsmtp)
Jul 26 14:50:13 pi2 postfix/qmgr[3981]: 56A1A69E86: removed
Jul 26 15:33:01 pi2 postfix/master[3962]: terminating on signal 15
Jul 26 15:33:02 pi2 dovecot: master: Warning: Killed with signal 15 (by pid=24157 uid=0 code=kill)
Jul 26 15:34:13 pi2 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled)
Jul 26 15:34:17 pi2 postfix/master[3292]: daemon started -- version 2.9.6, configuration /etc/postfix
Jul 26 18:09:01 pi2 postfix/master[3292]: terminating on signal 15
Jul 26 18:09:04 pi2 dovecot: master: Warning: Killed with signal 15 (by pid=6027 uid=0 code=kill)
Jul 26 18:10:13 pi2 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled)
Jul 26 18:10:22 pi2 postfix/master[3224]: daemon started -- version 2.9.6, configuration /etc/postfix
Jul 26 18:16:58 pi2 postfix/master[3224]: terminating on signal 15
Jul 26 18:16:58 pi2 dovecot: master: Warning: Killed with signal 15 (by pid=4145 uid=0 code=kill)
Jul 26 18:18:00 pi2 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled)
Jul 26 18:18:05 pi2 postfix/master[3289]: daemon started -- version 2.9.6, configuration /etc/postfix
Jul 27 08:01:28 pi2 dovecot: imap-login: Disconnected: Inactivity (no auth attempts in 180 secs): user=<>, rip=127.0.0.1, lip=127.0.0.1, TLS, session=
Jul 27 08:10:14 pi2 postfix/master[3289]: terminating on signal 15
Jul 27 08:10:17 pi2 postfix/master[8913]: daemon started -- version 2.9.6, configuration /etc/postfix
Jul 27 08:10:29 pi2 dovecot: master: Warning: Killed with signal 15 (by pid=8943 uid=0 code=kill)
Jul 27 08:10:30 pi2 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled)
Jul 27 08:36:38 pi2 postfix/master[8913]: terminating on signal 15
Jul 27 08:36:40 pi2 postfix/master[9349]: daemon started -- version 2.9.6, configuration /etc/postfix
Jul 27 08:36:46 pi2 dovecot: master: Warning: Killed with signal 15 (by pid=9380 uid=0 code=kill)
Jul 27 08:36:46 pi2 dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled)
Jul 27 08:42:00 pi2 dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=9484, TLS, session=
Jul 27 08:43:36 pi2 dovecot: imap(testmail): Connection closed in=0 out=297

Any suggestions gratefully received, thanks foe your help.

Simon

Sam Hobbs

Mon, 07/27/2015 - 19:32

In reply to by Simon East

Nothing jumps out, most of the lines are from you restarting/reloading the server. Your openssl test looks fine... are you sure this is the part of the log covering your failed Thunderbird login? You can watch the logs in realtime if you do tail -f /var/log/mail.log and then try to log in with Thunderbird. Sam

Hi Sam and thanks for the great guide.

I've gone over everything twice but still having the same issue

I can log into squirrel mail as pi with no issue but i've created a new user and can't login.
I'm getting : dovecot: imap-login: Aborted login (auth failed, 1 attempts in 2 secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, session=


main.cf
# See /usr/share/postfix/main.cf.dist for a commented, more complete version
smtpd_tls_auth_only = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
# Debian specific: Specifying a file name will cause the first
# line of that file to be used as the name. The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

myhostname = ***************
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydestination = piserver, localhost.localdomain, , localhost, ***************
mynetworks = 10.10.10.0/24 127.0.0.1/8
mailbox_size_limit = 0
recipient_delimiter = +
inet_protocols = ipv4
home_mailbox = maildir/
mailbox_command =
smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination
smtpd_helo_required = yes
smtpd_helo_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_invalid_helo_hostname,
reject_non_fqdn_helo_hostname,
reject_unknown_helo_hostname
smtpd_helo_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_invalid_helo_hostname,
reject_non_fqdn_helo_hostname,
reject_unknown_helo_hostname,
check_helo_access hash:/etc/postfix/helo_access
myorigin = ******************
smtpd_tls_security_level = encrypt

10-master.conf
#default_process_limit = 100
#default_client_limit = 1000

# Default VSZ (virtual memory size) limit for service processes. This is mainly
# intended to catch and kill processes that leak memory before they eat up
# everything.
#default_vsz_limit = 256M

# Login user is internally used by login processes. This is the most untrusted
# user in Dovecot system. It shouldn't have access to anything at all.
#default_login_user = dovenull

# Internal user is used by unprivileged processes. It should be separate from
# login user, so that login processes can't disturb other processes.
#default_internal_user = dovecot

service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}

# Number of connections to handle before starting a new process. Typically
# the only useful values are 0 (unlimited) or 1. 1 is more secure, but 0
# is faster.
#service_count = 1

# Number of processes to always keep waiting for more connections.
#process_min_avail = 0

# If you set service_count=0, you probably need to grow this.
#vsz_limit = $default_vsz_limit
}

service pop3-login {
inet_listener pop3 {
#port = 110
}
inet_listener pop3s {
#port = 995
#ssl = yes
}
}

service lmtp {
unix_listener lmtp {
#mode = 0666
}

# Create inet listener only if you can't use the above UNIX socket
#inet_listener lmtp {
# Avoid making LMTP visible for the entire internet
#address =
#port =
#}
}

service imap {
# Most of the memory goes to mmap()ing files. You may need to increase this
# limit if you have huge mailboxes.
#vsz_limit = $default_vsz_limit

# Max. number of IMAP processes (connections)
#process_limit = 1024
}

service pop3 {
# Max. number of POP3 processes (connections)
#process_limit = 1024
}

service auth {
# auth_socket_path points to this userdb socket by default. It's typically
# used by dovecot-lda, doveadm, possibly imap process, etc. Users that have
# full permissions to this socket are able to get a list of all usernames and
# get the results of everyone's userdb lookups.
#
# The default 0666 mode allows anyone to connect to the socket, but the
# userdb lookups will succeed only if the userdb returns an "uid" field that
# matches the caller process's UID. Also if caller's uid or gid matches the
# socket's uid or gid the lookup succeeds. Anything else causes a failure.
#
# To give the caller full permissions to lookup all users, set the mode to
# something else than 0666 and Dovecot lets the kernel enforce the
# permissions (e.g. 0777 allows everyone full permissions).
# unix_listener auth-userdb {
unix_listener /var/spool/postfix/private/auth {
mode = 0666
user = postfix
group = postfix
}

# Postfix smtp-auth
#unix_listener /var/spool/postfix/private/auth {
# mode = 0666
#}

# Auth process is run as this user.
#user = $default_internal_user
}

I've tried changing the password and changing where the server checks for authentication databases.

I wonder if there could be a problem with the Maildir creation for new users (all the stuff changed in /etc/skel). Have a look in /var/log/mail.log and see if there's anything related, and check that the Maildir exists for the new user you created (ls -l /home/newuser/). Sam

That all appears to be correct same on the user PI and the user account i made afterwards. I've had a bit of a play today commenting out lines etc and still the same message i also found i couldn't log the user account in on the pi but found that i stupidly forgot to add the user to the users group. I've also added the user to the mail, postfix & dovecot groups and still no luck or change in error, as i need to get this up and running asap i'm going to set up hmailserver on a windows server and work on the pi when i get time as it seems to be taking a while. Theres also nothing in the log other than the above message.

Simon East

Thu, 07/30/2015 - 23:55

Hi Sam

Done some more work on this and I have watched the log as I have tested the IMAP settings as you suggest. I can see that it is logging in to Dovecott, so I am confident I have the right user name and password on my pi (locally).

I think the problem is between namecheap and the pi, I can send an email to my namecheap email account and I can view it using their webmail server.

However I am still getting and incorrect user/password error when I try and configure K9 on my Android tablet!

I feel as though I have missed something probably connected to the configuration in namecheap?

Any thoughts gratefully received!

Hi Simon, that's some good logical problem solving :) Did you read this tutorial? https://samhobbs.co.uk/2015/02/dns-basics-websites-and-email-servers If you tell me your domain name i can help you check your dns if you like (btw if you haven't already deleted the testmail account, now is the time! You don't want to leave it enabled because a spammer could use it to send loads of mail from your server. Sam

Simon East

Fri, 07/31/2015 - 07:05

Hi Sam

My domain is simonaeast.com

I did look at your tutorial but I may benefit from a second read.

Thanks for your help

Simon

Your MX record points to mail.simoneast.com, and the A record for that subdomain points to 116.0.19.203. I tested with openssl s_client -connect 116.0.19.203 -CApath /etc/ssl/certs, and it seems the certificate presented by that server is for adonis3.instanthosting.com.au. You need to update your A record for mail.simoneast.com to point to your WAN ip address. Sam

Ruben Hidalgo

Mon, 08/24/2015 - 04:36

Hello Sam
First sorry if my english is very bad.
In the second part, after to apply the SASL configuration and testing SASL, telnet rejects my connection. The message is: "Connection closed by foreign host."

My configuration of service auth is:

service auth {
# auth_socket_path points to this userdb socket by default. It's typically
# used by dovecot-lda, doveadm, possibly imap process, etc. Users that have
# full permissions to this socket are able to get a list of all usernames and
# get the results of everyone's userdb lookups.
#
# The default 0666 mode allows anyone to connect to the socket, but the
# userdb lookups will succeed only if the userdb returns an "uid" field that
# matches the caller process's UID. Also if caller's uid or gid matches the
# socket's uid or gid the lookup succeeds. Anything else causes a failure.
#
# To give the caller full permissions to lookup all users, set the mode to
# something else than 0666 and Dovecot lets the kernel enforce the
# permissions (e.g. 0777 allows everyone full permissions).
# unix_listener /var/spool/postfix/private/auth {
#mode = 0666
#user = postfix
#group = postfix
# }
# }

and my configuration of /etc/dovecot/conf.d/10-auth.conf: is (last lines):

#!include auth-deny.conf.ext
#!include auth-master.conf.ext

!include auth-system.conf.ext
#!include auth-sql.conf.ext
#!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
#!include auth-checkpassword.conf.ext
#!include auth-vpopmail.conf.ext
#!include auth-static.conf.ext
disable_plaintext_auth = no
auth_mechanisms = plain login

The method of authenticating with SASL it´s OK. the message is AHRlc3RtYWlsAHRlc3QxMjM0 but the telnet session reject me.

regards.

Hi Sam

I had to reload and reconfigure my Raspberry PI again. While running through your tutorial again, I double checked everything to make sure I got the same results. One issue I've now got is when I test the openssl connection, instead of getting a similar result to you, I still get the raspberry pi cn name ??

openssl s_client -connect localhost:465 -quiet
depth=0 CN = raspberrypi

I've edited my hosts and hostname files accordingly and when i run the rest of the tutorial, my correct fqdn appears. Just not in the above step ?
Any ideas ?

Hi, That's the common name on the sel-signed "sakeoil" certificate (the auto generated one). Have a look at the CAcert tutorial if you want to know how to specify a different cert and key. Sam

Anonymous

Thu, 08/27/2015 - 11:02

In reply to by Sam Hobbs

Ah ok, I get it. I'll do that once I can actually connect to my mail server.
I've had a look around and found a few ways to check that I can actually connect to my mail server.
I used : telnet imap.example.com 143 and openssl s_client -connect imap.example.com:993 I just got a connection timed out error.
Obviously I used my own domain name instead "example.com"
Testing it using your tutorial works but not if I use my FQDN ?
I've also checked that my ports are being forwarded.

Lars van Balderen

Fri, 08/28/2015 - 18:21

Hi Sam,
I have added this to /etc/dovecot/conf.d/10-master.conf:
service auth {
unix_listener /var/spool/postfix/private/auth {
mode = 0660
user = postfix
group = postfix
}
}
after restarting dovecot, i get this warning:
doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/10-master.conf line 95: Unknown setting: unix_listener
[....] Restarting IMAP/POP3 mail server: dovecotdoveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/10-master.conf line 95: Unknown setting: unix_listener
Is there anything i can do about it?
Lars

Hi Lars, When people have had that problem beore, it has been because the outer block (service auth) is commented but the inner is not. unix_listener is only valid within the service auth block. If you commented the original block, make sure you commented the whole thing... Sam

I have set up the server up to part 2, but when I try to connect with the android app it keeps saying "cannot connect to server". Either my DNS isn't set up correctly or the username and password is not set up correctly. Any way I can find out which?

-I believe the DNS is not set up correclty;Please help thanks.

Did you read my DNS tutorial? I can't check your DNS settings without knowing your domain name! You might also have forgotten to set up port forwards on your router. Doesn't sound like a username/pw problem because the message doesn't say anything about authentication errors. Sam

Add new comment

The content of this field is kept private and will not be shown publicly.

Filtered HTML

  • Web page addresses and email addresses turn into links automatically.
  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.