DNS Basics for Websites and Email Servers

DNS-signpost.jpg

This article aims to explain what various DNS records are and how to use them. It is aimed at people hosting websites and email servers on a home server such as a raspberry pi.

How DNS Works

DNS stands for Domain Name System. In a nutshell, it's the system that we use to translate human readable domain names (e.g. samhobbs.co.uk) into the Internet Protocol (IP) addresses for those services (e.g. 195.166.151.235).

The DNS system has a strict hierarchy, and lookups are performed recursively using a client/server model. This means that when your computer asks for the IP address for "subdomain.example.com", the DNS resolver you are using starts at the top and works backwards to find the address. The servers at the top of the pyramid (root name servers) direct clients to the name servers reponsibe for Top Level Domains (TLD, e.g. .com), and that server in turn is able to tell the client which name server is authoritative for the second level domain (e.g. example.com), and so on. Usually for small sites, the DNS records for your domain are managed by your Domain Name Registrar.

The DNS resolver doesn't have to perform all these lookups every time, because each record has a Time To Live (TTL) in seconds, which tells the client how long they can cache the information for until it should be refreshed. That's Time To Live (live free), not Live (live electrical circuit), which is how I first read it!

There are loads of different types of DNS record, so I've picked the most important ones you might need or want to use. These are:

  1. DNS A - used to map a host name to an IP address
  2. Mail Exchanger (MX) - used to tell clients which hostnames are used for email services
  3. Sender Policy Framework (SPF) - used to define which servers are allowed to send email from your domain name
  4. Pointer (PTR) - the opposite of DNS A, this record maps an IP address to the hostname

For a website, you only need a DNS A record. For an email server, you need at least a DNS A record and an MX record; PTR and SPF records will help you get your email through spam filters.

I'll be illustrating how to set up the different types of record using Namecheap, my Domain Name Registrar, as an example. If you haven't registered a domain name yet, I'd recommend Namecheap -avoid GoDaddy if you can, since they have some pretty horrible pricing practices and consistently back the wrong side when it comes to internet censorship acts like SOPA and PIPA.

DNS A

This record is the most basic type. It maps a domain name to an IP address.

Most home internet connections have IP addresses that are dynamically allocated from a pool controlled by your Internet Service Provider (ISP). This means that your Wide Area Network (WAN) IP address can change periodically, or when you power cycle your router. To get around this you can use dynamic dns to update your DNS records when your IP address changes. If you can get one, you should use a static IP address because it cuts out this additional layer of complexity.

A DNS A record is really simple. This is what it looks like in the Namecheap control panel:

DNS-A.png

The image above actually shows two separate records, one for my root domain name (that's the @, which you can read as "no subdomain"), and another for my domain name preceded by www..

The IP address is my static WAN IP, and I chose a high value for the TTL of 60000 seconds (over 16.5 hours) because it never changes, so it's helpful to allow DNS resolvers to cache the data for as long as possible. If you have a dynamic IP address you should choose a much lower value like 1800s.

If you want to do a DNS lookup yourself from the terminal, you can install a utility called dig:

sudo apt-get update
sudo apt-get install dnsutils

DNS A records can be looked up using this command:

dig samhobbs.co.uk

You should get output like this:

; <<>> DiG 9.9.5-4.3ubuntu0.1-Ubuntu <<>> samhobbs.co.uk
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48863
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;samhobbs.co.uk.                        IN      A

;; ANSWER SECTION:
samhobbs.co.uk.         45464   IN      A       195.166.151.235

;; Query time: 3 msec
;; SERVER: 127.0.1.1#53(127.0.1.1)
;; WHEN: Sun Feb 01 14:39:41 GMT 2015
;; MSG SIZE  rcvd: 59

You can see in the answer section the IP address and the remaining number of seconds the information is valid for, as determined by the TTL. If you do a second lookup, you should see this number decrease.

Mail Exchanger (MX)

The MX record specifies the hostnames that are available to receive email for your domain. Since a domain can have more than one SMTP server, this is a prioritised list, where each domain name is given a number. The lower the number is, the higher the priority.

The image below shows my MX settings:
MX.png

These settings are probably more complicated than yours will be - a basic MX record will have just one entry pointing to the hostname of your email server (ususally yourdomain.com or mail.yourdomain.com). You can see from the settings that I have two servers that can receive mail for my domain: samhobbs.co.uk and backup.samhobbs.co.uk (which is a different physical server in a different location, with a separate internet connection).

When a client wants to know where to send an email addressed to foo@samhobbs.co.uk, they look up the MX record, which tells them to try samhobbs.co.uk first and backup.samhobbs.co.uk if that fails (backup has a higher number = lower priority). They then look up the DNS A record for the hostname to get the IP address, and then make a connection and perform the transaction.

My backup server is actually an MX backup - if my main server is unavailable, it accepts email from other servers addressed to my domain and forwards it on to the primary server when it is back online. In addition to performing MX backup services for my main server, the backup server accepts email sent to it directly (foo@backup.samhobbs.co.uk) which is what the third record is for.

To look up MX records, use this command:

dig samhobbs.co.uk MX

The output should look like this:

; <<>> DiG 9.9.5-4.3ubuntu0.1-Ubuntu <<>> samhobbs.co.uk MX                                  
;; global options: +cmd                                                                      
;; Got answer:                                                                               
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35900                                    
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1                         
                                                                                             
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;samhobbs.co.uk.                        IN      MX

;; ANSWER SECTION:
samhobbs.co.uk.         60000   IN      MX      10 samhobbs.co.uk.
samhobbs.co.uk.         1800    IN      MX      20 backup.samhobbs.co.uk.

;; Query time: 208 msec
;; SERVER: 127.0.1.1#53(127.0.1.1)
;; WHEN: Sun Feb 01 15:16:01 GMT 2015
;; MSG SIZE  rcvd: 82

The MX record for backup.samhobbs.co.uk is separate (dig backup.samhobbs.co.uk):

; <<>> DiG 9.9.5-4.3ubuntu0.1-Ubuntu <<>> backup.samhobbs.co.uk MX
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33203
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;backup.samhobbs.co.uk.         IN      MX

;; ANSWER SECTION:
backup.samhobbs.co.uk.  1800    IN      MX      10 backup.samhobbs.co.uk.

;; Query time: 218 msec
;; SERVER: 127.0.1.1#53(127.0.1.1)
;; WHEN: Sun Feb 01 15:33:26 GMT 2015
;; MSG SIZE  rcvd: 66

Sender Policy Framework (SPF)

The SPF record is used to tell other servers which IP addresses and hostnames are allowed to send email from your domain name. Some services such as gmail look for either a PTR record or an SPF record and flag email as spam if neither of these is found, so it's quite important to define one for your domain.

The record itself is defined as a TXT record:

SPF-record.png

The important part of the record is:

v=spf1 mx a ~all

The first part v=spf1 says that this record is SPF version 1. The middle of the record mx a says that servers should accept email from servers with a matching MX record or DNS record. The end part ~all defines what to do with sources that haven't matched one of the earlier statements, the tilde (~) means soft fail. This lets the recipient server decide what to do with the message, normally this means it will be accepted but flagged as spam. A minus (-) means the recipient server should reject the message, but most services don't implement this so it's a bit pointless.

Note that earlier versions allowed you to specify "PTR" as a match in the middle part, but this is default behaviour now and the option is depreciated.

If you want to do a DNS lookup for a SPF record, you can use this command:

dig samhobbs.co.uk txt

The output should look like this:

; <<>> DiG 9.9.5-4.3ubuntu0.1-Ubuntu <<>> samhobbs.co.uk txt
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61893
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;samhobbs.co.uk.                        IN      TXT

;; ANSWER SECTION:
samhobbs.co.uk.         1800    IN      TXT     "v=spf1 mx a ~all"

;; Query time: 170 msec
;; SERVER: 127.0.1.1#53(127.0.1.1)
;; WHEN: Sun Feb 01 15:49:41 GMT 2015
;; MSG SIZE  rcvd: 153

Pointer (PTR)

A pointer record is the opposite of a DNS A record: it maps an IP address to a hostname. If you can set one of these, it pretty much guarantees that you won't have any problems with your email being marked as spam.

Unlike the other DNS records, you can't change this record with your DNS provider. For this one, you're at the mercy of your ISP, and since it changes the information associated with an IP address you will only be able to do this if you have a static IP address.

Many ISPs won't let you change your PTR record. Luckily, my ISP (Plusnet) is one of the good ones, and they changed it to samhobbs.co.uk without a charge :)

To look up a PTR record you can use this command along with the IP address you got from the DNS A lookup:

dig -x 195.166.151.235

The output should look something like this:

; <<>> DiG 9.9.5-4.3ubuntu0.1-Ubuntu <<>> -x 195.166.151.235
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44108
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;235.151.166.195.in-addr.arpa.  IN      PTR

;; ANSWER SECTION:
235.151.166.195.in-addr.arpa. 43200 IN  PTR     samhobbs.co.uk.

;; Query time: 37 msec
;; SERVER: 127.0.1.1#53(127.0.1.1)
;; WHEN: Sun Feb 01 16:07:32 GMT 2015
;; MSG SIZE  rcvd: 85

That's it! Those four records are everything you should need to get your website and email server running properly.

If you have any problems, let me know!

Type: 

Comments

Hi Sam,

No luck so far...

Tried to send to and from K9 but it say failed to send, some server error, but can't see the error message properly to check.

All I can see is "Negative SMTP reply: 451 4.3.5 Server config.."

Jo

Have a look in /var/log/mail.log and find the part that matches when you tried to send mail. What does it say?

Sam

Hi Sam,

Just attempted to send another email so I can accurately copy that part of the log file...here it is:

ct 17 12:22:15 raspberrypi postfix/qmgr[769]: 20E283FD6D: from=<testmail@courseworkresources.com>, size=427, nrcpt=$
Oct 17 12:22:16 raspberrypi postfix/smtp[4387]: 20E283FD6D: host mx01.1and1.com[74.208.5.21] refused to talk to me: $
Oct 17 12:22:17 raspberrypi postfix/smtp[4387]: 20E283FD6D: to=<me@externalmail.com>, relay=mx00.1and1.com[74.208.5.$
Oct 17 12:23:00 raspberrypi postfix/smtps/smtpd[4395]: connect from unknown[213.205.251.56]
Oct 17 12:23:01 raspberrypi postfix/smtps/smtpd[4395]: warning: unknown smtpd restriction: "permit_sasl_auththentica$
Oct 17 12:23:01 raspberrypi postfix/smtps/smtpd[4395]: NOQUEUE: reject: RCPT from unknown[213.205.251.56]: 451 4.3.5$
Oct 17 12:23:01 raspberrypi postfix/cleanup[4406]: 6A80D3FD75: message-id=<20151017122301.6A80D3FD75@courseworkresou$
Oct 17 12:23:01 raspberrypi postfix/qmgr[769]: 6A80D3FD75: from=<double-bounce@courseworkresources.com>, size=1049, $
Oct 17 12:23:01 raspberrypi postfix/smtps/smtpd[4395]: disconnect from unknown[213.205.251.56]
Oct 17 12:23:01 raspberrypi postfix/local[4407]: 6A80D3FD75: to=<root@courseworkresources.com>, orig_to=<postmaster>$
Oct 17 12:23:01 raspberrypi postfix/qmgr[769]: 6A80D3FD75: removed

I can see something about a testmail email which I thought I removed yesterday, and I don't recognise the ip address (might be my android)...

Obviously I don't understand much of the log, so any advice is appreciated. At least I can make out that the data goes through the internet via my router to the RasPi at least...

Regards,

Jo

You must have a typo in your postfix config:

Oct 17 12:23:01 raspberrypi postfix/smtps/smtpd[4395]: warning: unknown smtpd restriction: "permit_sasl_auththentica$

Chrck your smtpd restrictions for typos.

Sam

Sam,

You're a star!!!!!

Corrected the typo in the main.cf and reloaded/restarted the postfix service and ran send and receive tests, and finally its working. I learned so much following your tutorials and am very appreciative.

Regards,

Jo

You did it! Your new mail server puts you in control, enjoy :)

Sam

Hello Sam.

Once again: Great tutorials. I got squirrelmail up and running perfectly, but I'm having issues with my e-mails always ending up in spam filters. I think the issue has something to do with DDNS. I have the following settings at my DDNS provider:

fedthund.dynu.com SPF v=spf1 mx a ~all 90
fedthund.dynu.com MX fedthund.dynu.com [Priority: 10] 90

Unfortunately, the PTR from my ISP points to a different hostname. Do you know of any other options to make sure mails are delivered properly?

Bummer. I don't really have any more tricks up my sleeve, you basically have two options:

  • Get a static IP address - where you live affects the cost of this quite a bit. For me (UK, PlusNet) it was a one-off £5 fee, but in the USA it's more because the ISPs can get away with it (very little competition).
  • Use an SMTP relay run by your ISP for outgoing mail (but in my opinion this defeats the purpose of running your own server). I can't help you with this because I've never had to set it up, and the requirements for authentication vary depending on your ISP.

Sam

Thanks for giving it some thought, Sam.

Unfortunately, where I live, it's a monthly fee for a static IP. I will have to consider if it's worth it to set up. As I understand it right now, it ought to solve my problem - and looking around the web, I find that most (If not all) who have attempted to set up mail servers with dynamic hostnames have failed, because of the reverse DNS lookup not matching the hostname. I got the following spam score from port25, so I assume everything else is OK:


==========================================================
Summary of Results
==========================================================
SPF check: pass
DomainKeys check: neutral
DKIM check: neutral
Sender-ID check: pass
SpamAssassin check: ham

(I did the test here):
http://blog.postmarkapp.com/post/6529156244/port25s-authentication-and-s...

I'm not listed on any blacklist either.

Sam, thanks once again for all your incredible guides.

I thought I'd update in case others have also tried the dynamic DNS angle.

I'm extremely stubborn so I've continued my attempts setting up the mailserver with dynamic DNS, and I've had a degree of success. As of now, the mailserver works (!!), and thanks to using dynamic DNS as the /etc/mailname I no longer end up in spam folders. It sends mails as user@host.ddns.com rather than user@host.com - which isn't ideal, but it's good enough. I can still recieve e-mails from my primary domain, I just can't send them from there.

Something about sending from my main hostname but being redirected to a dynamic dns hostname made the spam filters suspect my e-mails. There may be some kind of fix for that, which I will continue to look into.

- Martin

Hello again Sam.

As of right now, I've managed to get everything I wanted up and running. I can receive e-mails, I can send them - and when I set up my personal information in squirrel mail (Options -> Personal Information) it shows up fine in whatever mail client is used. I had a few issues with resolving hostnames earlier today, but I found it was because I forgot to add dns-nameserver information to my /etc/network/interfaces file when setting my interface to static IP.

The only test left for me is to see what happens when my WAN IP changes - other than that I'm pretty confident everything works as it's supposed ot. I'm really impressed with the Raspberry Pi so far. It is currently running: Seedbox, mailserver and squirrelmail (Thanks to you), VPN server, mediaserver (Samba), SSH server, and a Minecraft server (Using Cuberite - a C+ open source compatible minecraft server, as the standard Java server is a bit too ressource intensive).

I will say though: I would not recommend using dynamic DNS while setting up a mailserver. It's a real pain in the neck, and DNS stuff is difficult to troubleshoot because it takes so long for nameservers to update. I still need to set up a few things: I want to run https by default, and I want to make sure that the security on the server is as good as it gets. After that, I'll probably go ahead and do your wordpress tutorial.

- Martin

Hi Sam,

Not sure about DNS entries that you show in places. I have 4 DNS records (which I think are correct?)

DNS_A @ pointing to my domain xxx.co.uk
DNS_A www ditto xxx.co.uk
DNS_TXT v=spf1 mx a ~all
DNS_A backup pointing to my WAN backup 1.2.3.4

If I have just two MX records as follows, this should work?
MX @ xxx.co.uk 10
MX @ backup.xxx.co.uk 20

It presumably means I cannot send mail direct me@backup.xxx.co.uk. If I did want to do this (why?) I would need the third as show in your list.

In the MX backup tutorial you show three and I don't follow why it is useful to send mail direct to the backup server. Can you give me some guidance here.

MX @ your.co.uk 10
MX @ backup.your.co.uk 20
MX backup backup.your.co.uk 10

Thanks...John

Hi John,

That seems correct. Are you having problems or were you just looking for clarity?

Sending email direct to the backup server is useful for backing things up, e.g. with this backup script I wrote for Drupal - I'm not expecting you to use it as an email address to communicate with other people (although you could)!

Sam

Hello Sam,

Thank you again for your excellent tutorials. I'm so delighted with the e-mail server that I'm just one step before moving my main e-mail from gmail to my raspi. So I want to kindly ask you some tips for a backup mail server:

I assume that the backup server has the same configurations as the main one. Is this correct?

There is replication of all mailboxes/users in the backup server, or it’s just a storage until the mail sync with the main server?

What I should configure to get the received mail in the backup server synced with the main server after a system failure and restore in the main one?

Thank you very much.

Ricardo.

Hi Ricardo,

Depends what you mean by a backup server - you can use an "MX backup", which receives mail if your main server is offline and then forwards it to the main server when it comes back online. If that's what you mean, read my mx backup tutorial.

If you just need a way to back up the maildirs, you can use any method you like to make a regular copy of the directory, it's just a bunch of files.

Sam

Hi Sam,

For some reason I didn't saw the MX backup tutorial... Now I've made an overview and yes, it's there what I was looking for!

Thank you once again.

Ricardo.

Hi Sam,
I recently received a new ip adres and since then my mail either ends up in spamboxes or doesn't arrive at all (not being returned however).
I found out I'm listed on spamhaus.org zen and sorbs duhl but really it's like I'm reading chinese. I tried to delist myzelf at spamhaus and sorbs but the last one for sure didnt work.
Where do I go from here, any ideas?

If you have a dynamic IP address, it's probably simplest to just turn your router off, wait 30 seconds, and then reboot it. Normally this will assign you a different IP address, which you can check on mxtoolbox.com to see if it's on any blacklists.

Sam

Hi Sam,

Could you comment on the usefulness (or otherwise) of updating the postfix/dovecot system to include DKIM signing. There are several URL's which indicate how to do this, but not necessarily why it is worth doing.

Would this be the first step to implementing DMARC?

Thanks for your advice...

John

Hi John,

I think both of these have value and are worth implementing, but I'm no expert in them - implementation is on my "to do" list!

I plan on writing a new email server tutorial in the future that integrates with amavisd (allowing clamav to be used along with spamassassin). I think this would be less hacky than the method I've used to pipe messages through spamassassin here, but I think it would require virtual users with email stored at /var/mail instead of in /home.

Anyway, I've been putting off looking into DKIM signing until I have written that tutorial, because I think it will also be easier to integrate with a virtual user setup.

No ETA on the tutorial yet I'm afraid, I'm quite busy these days!

Sam

Hi Sam,

Thanks for the reply. I will go ahead and setup DKIM and when (if) it is tested and working I will let you know what I did.

John

Pages

Add new comment