Postfix with DKIM, Domainkeys, SPf and Sender-ID

Posted: April 3rd, 2010 | Author: | Filed under: Server side | Tags: , , , , | 9 Comments »

The authentication methods are a great tool for fighting against spammers and every server admin should have it implemented on their systems. But since it is not that simple to configure them, they often prove to be a headache with not enough documentation available. I had to pay myself the tribute (in hours spent) for making the system work in the configuration I wanted and for the others having the same problems here is my story.My configuration is : CentOS 5, Postfix 2.3.3

The server is running some websites and I wanted to be capable of sending signed emails within PHP and also using some email client through SMTP. Since there are more than one website I wanted also a simple way to add/modify domains in the filters' configuration.

Whenever I write "domain.tld" you should use your own domain name (mysite.com, wikitiki.org, etc)

The tools I installed are listed here under the Authentication section. They are:

  • dkfilter - a SMTP-proxy signing filter. Quite easy to install but, as the author says, it is good for sending authenticated email served by the port 587 not 25 which is why I give up on it. My server must be able to send emails from PHP and act as a SMTP mail server for another server.
  • dkim-milter – DKIM signer/verifier (http://sourceforge.net/projects/dkim-milter/)
  • dk-milter – Domainkeys sender/verifier. (http://sourceforge.net/projects/dk-milter/) Pay attention to these names because it is a lot of confusion because not only DK and DKIM sounds alike but their executable are called "filters": dkim-filter and dk-filter. Couldn't be much more complicated for newbies.
  • sid-milter – Sender ID verifier. Note that it's a verifier only ! (http://sourceforge.net/projects/sid-milter/)

1. DKIM

Installing dkim. For this follow one of these links:

How to Setup DomainKeys Identified Mail (DKIM) with Postfix and Ubuntu Server

Setting up DKIM, SPF, Domainkeys DNS, Regular DNS on CentOS 5.3

I won't go through the installation steps you can read about in tutorials as above. Instead I will focus on what should be checked, customized, verified so that will help in custom installations or troubleshooting.

So, for the installation I will focus on:

subject value description
username/usergroup domainkeys.domainkeys This is the owner of the signer/verifier processes. Doesn't matter the names but once created will be used for all filters
folders for configuration /etc/mail/dkim/
/etc/mail/domainkeys/
These must be owned by the user above. When adding new domains I have to modify some files here
initialization scripts /etc/init.d/dkim
/etc/init.d/domainkeys
/etc/init.d/sender-id
Essentially is the same start-up script configured for each filter. Good to know where they are when testing the configuration
folders for private keys /var/db/domainkeys/domain.tld/ These must be owned by the user above. Here I'll place the private keys
folders & files for processes' PIDs /var/run/dkim-milter/pid
/var/run/dk-milter/pid
/var/run/dk-milter/pid
These also have to be owned by the user above. The filters will create/remove the "pid" files as they need

OK, so here there are some background information. DKIM and Domainkeys uses two DNS records and a private key to sign the emails. The DNS records are

  1. policy record. It is defined as a TXT record :
    _domainkey.domain.tld IN TXT "t=y;o=~"

    t=y means the domain is in test mode. When everything works it can be changed to "n"

    o=~ means that some emails will be signed. "o=-" means that all domains are signed

  2. selector record. Also a TXT record storing the key and other parameters:
    default._domainkey.domain.tld IN TXT  "g=*; k=rsa; p=MIGfMA0...ABC"

    "default" is the selector name and will be the same as the one used for generating the keys – see below

    "_domainkey" is a keyword preceding the domain name

    "k=" key type. Optional. Default is rsa

    "p=" public key data.

    "g=" granularity of the key. Optional, default is "*"

    "v=DKIM1" version. It is recommended but I got some errors when I tried to use the same record for Domainkeys because it is defined for DKIM only so I did not use it.  Here is the yahoo tool I tested with

Note: in the two dns records above I used "domain.tld" only because my external nameserver service required it in its web interface. If you own the namesever you will probably not use it as will be appended automatically by named/bind

Important : the server may sign ALL emails with its own domain name or may sign it with other domains but in this case all other domains must have the two DNS records maintained. I went for the first option.

Generating the keys

there are three ways to generate them:

  • online Domain Key / DKIM Key Generation Wizard
  • basic:
    openssl genrsa -out private.key 1024
    openssl rsa -in private.key -out public.key -pubout -outform PEM
  • script included with the dkim:
    dkim-genkey -r -d domain.tld

The result will always be two files, one private key to copy into /var/db/domainkeys/domain.tld/default and one to use it in the DNS record as value for "p=" parameter. Pay attention that I used "default" as filename for the private key in order to use that file for a selector named "default" and defined as such in the DNS record. It is possible to have different selectors which would handle different emails or domains and so I can have:

/var/db/domainkeys/domain.tld/m1.pem – this is created for the selectod m1 and I added an extension just to remember it is a private key file. the extension, if exists, must be "pem" or "private".

/var/db/domainkeys/otherdomain.tld/default – this one will serve emails for another domain with another selector named "default"

The init script

The init script will start|stop the filter and there is where I put the command to do that. It may have two forms, either with the configuration given as parameters to the executable app or with it uses a single parameter which points to a configuration file containing all settings. I used the 2nd option:

COMMAND="dkim-filter -x /etc/dkim.conf"

Then I put in the dkim.conf file all the configuration parameters I needed. Don't forget to make it readable by the user defined above (domainkeys). Some of the most important parameters I put there are:

KeyList                 /etc/mail/dkim/keylist
MTA                     MSA
Selector                default
UserID                  domainkeys:domainkeys
Socket                  inet:8891@localhost

Don't forget that the port numbers will be used in your /etc/postfix/main.cf file:

smtpd_milters = inet:localhost:9967,inet:localhost:8891,inet:localhost:8892
non_smtpd_milters = inet:localhost:9967,inet:localhost:8891,inet:localhost:8892
milter_protocol = 2
[[code]]czozMDpcIm1pbHRlcl9kZWZhdWx0X2FjdGlvbiA9IGFjY2VwdFwiO3tbJiomXX0=[[/code]]

I have here 3 filters, dkim, domainkeys and sender-id, each on its own port. You will find below the entire configuration file I created.

The keylist file

the keylist file contains the relation between the sender-pattern, the signing domain and the path to the private key. I use this keylist file in order to manage easier the domains on my webhost server. For a singe domain one would only use the dkim's configuration file. My keylist file's location is defined in the configuration :

KeyList /etc/mail/dkim/keylist

And contains :

*:domain.tld:/var/db/domainkeys/domain.tld/default

..meaning: all emails (*) will be signed with the domain "domain.tld" using the private key from the file "/var/…/default".

The selector used in  the signature will be the filename portion of  keypath. If the  file referenced by keypath cannot be opened, the  filter will try  again by appending ".pem" and then ".private" before giving up. (man dkim-filter.conf)

Useful commands, part I

  • postsuper -d ALL

    Delete all mail queue. Sometimes due to too many unsent emails the logs are filled in continuously by the emails in queue. For a clean start it's good to have an empty mail queue. Be careful on a production server (you may just ignore it)
  • mail myaccount@yahoo.com

    Send emails from command line. good for a quick test. Verify the results on a yahoo account for both DKIM and Domainkeys
  • telnet localhost 25
    EHLO domain.tld
    MAIL FROM: root@domain.tld
    RCPT TO: myaccount@yahoo.com
    data
    bla bla
    .
    quit

    This is a suite of commands to test sending email through SMTP. Or you can use an email client set for sending through SMTP protocol
  • php -r "mail('myaccount@yahoo.com','test mail','bla bla','From: root@domain.tld');"

    Let the email be sent within PHP. This will test PHP mail() configuration

    A note here: At a certain point my mail sent from PHP was not signed with Domainkeys, only with DKIM. After several tests I found that the $headers in my php script were using Windows separator for new like (\r\n) instead of Linux (\n). I never thought it matters in headers !

  • Online tools: DomainKey Implementor's Tools and Library for email servers & clients especially "DomainKey Policy Record Tester." and "DomainKey Selector Record Tester."
  • tail -f /var/log/maillog

    this is where I look for errors

2. Domainkeys

As above, start with the download & installation from the given links from sourceforge or with the ones below:

Quick And Easy Setup For DomainKeys Using Ubuntu, Postfix And Dkim-Filter

man dk-filter – DomainKeys filter for sendmail online version of the man page for dk-filter

The DNS records follows the same rules as the DKIM above. In fact both DKIM and Domainkeys will use the same DNS records

The init script

The dk-filter does not support all parameters supported by DKIM and especially the one which would allow me to put all configuration in one file therefore the command from this initialization script has all parameters in one line:

COMMAND="dk-filter -u $USER -p inet:$PORT@localhost -l -P $PIDFILE -d $DOMAIN -S $SELECTOR -c simple -k -s $KEYFILE -i $IFILE -m $SUBMISSION_DAEMON -D"
Parameters:

  • -u userid it runs under
  • -p socket to be used by the filter for allowing connections from postfix. It must use a different port number than dkim (of course)
  • -l log calls to syslog
  • -P pidfile – the file used to write the process ID into
  • -d domain list or path to a file containing the domain list one per line (wildcard * allowed in the domain name )
  • -S selector, the name of the selector to be used
  • -k Causes -s to be interpreted as the location of a key list
  • -s keyfile or keyfilepath. Since -k is used this one will be a path to my keyfile (/etc/mail/domainkeys/keylist)
  • -i a file of internal hosts whose mail should be signed rather than verified (/etc/mail/domainkeys/ilist)
  • -D Sign subdomains too

The content of the keylist (differs from DKIM's !):

*@domain.tld:/var/db/domainkeys/domain.tld/default

The ilist contains some IPs (local network's):
192.168.1.0/24127.0.0.1

Note that I am using "/etc/mail/dkim/trusted-hosts" as value for the -d (domain) parameter. This is a path pointing to a dkim folder because I want to make as little changes as possible when adding or removing a domain so I use it for both filters.

The keylist file

This is another small difference than the DKIM's keylist file content. It looks like this (/etc/mail/domainkeys/keylist) :

*@domain.tld:/var/db/domainkeys/domain.tld/default

Each line is in the form "sender-pattern:keypath" where sender-pattern is compared against the sender and keypath is the path to the private key file. The filename portion of this keypath is also the selector name

Some problems I run into

At some moment I got in maillog : "can't read SMFIC_EOH reply packet header: Success" which was solved after setting the SUBMISSION_DAEMON above as MSA (instead of smtp as I saw somewhere)

Playing with milter_protocol=4 in /etc/postfix/main.cf I got "can't read SMFIC_DATA reply packet header: Success" which was because I've given a value which was not supported. Also milter_protocol=6 thrown error so I let it set on : milter_protocol = 2

Useful commands, part II

  • postconf -d

    Display the default postfix configuration
  • postconf -n

    Display the non-default parameters of the postfix configuration
  • /usr/sbin/sendmail -t -f root@domain.tld myaccount@yahoo.com
    From: root@domain.tld
    Subject: test it
    bla bla
    .

    Test sending email using the sendmail command as defined in the php.ini . Note that php.ini has an additional parameter for sendmail, "-i" which suppress the use of a dot as "end of input" data.
  • ps aux |grep dk

    This should confirm that the filters are running (dk-filter and dkim-filter). It was useful to check this because when I got errors from filters they had just ended silently.

3. Sender-id

This would be the 3rd filter installed and is aimed for hotmail accounts. There is a confusion between Sender-ID and SPF because they both use the same DNS registration record but the implementation of the Sender-ID protocol is similar with DKIM above. Read more about SPF vs Sender ID

Note that in order to sign the emails correctly one would only need the SPF record – see next section. At least this should work for 80% of the emails sent to hotmail/MSN according to the link above.

The init script

I named the init script "/etc/init.d/sender-id" and the command used to start the filter is:

COMMAND="sid-filter -u $USER -t -p inet:$PORT@localhost -l -P $PIDFILE -d $DOMAIN -D -h"

The parameters means:

  • -u username to run under
  • -t test mode (never reject messages)
  • -p socketspecs the socket for postfix to communicate with
  • -l log via syslog (/var/log/maillog)
  • -P the filename which will store the PID of the current process
  • -d a list of domains whose emails should be ignored by this filter.
    Warning : this is a list of the domains to be ignored so if you want to see this filter's signature in the header of your emails then use an empty value for DOMAIN. Somehow strange in my opinion.
  • -D the way to handle the DNS errors (soft failures)
  • -h adds a header indicating the presence of this filter

4. SPF

The SPF authentication is used by google and is quite simple requiring no software installation on the server. In order to generate a SPF record for your domain you will probably have to go to the most-used tool :

Setup wizard to create SPF records for your domain

The record it will create should be placed as a TXT record at your domain and looks like this:

v=spf1 a a:external.domain.tld ~all

Where:

  • "v=spf1" is the SPF protocol version
  • "a" mechanism matching current domain (e.g. my own domain.tld)
  • "a:external.domain.tld" mechanism matching an external domain which may use my own domain.tld to send emails
  • "mx" mechanism matches the current domain's MX records. Useful if I want to have a secondary server act only as mail server
  • "all" mechanism matches everything else. It usually stay at the end. The character which prefixes this mechanism means "SoftFail" and it only indicates that all other domains than the one defined won't be recognizable by this SPF record

Definition: Mechanisms can be used to describe the set of hosts which are designated outbound mailers for the domain.

Observation: the SPF standard specify that there should be used the new DNS registration record type SPF instead of TXT but because of compatibility with the current DNS implementations it is recommended to put both TXT and SPF records with the same content.

Useful commands, part III

Here are some of the tools and commands I used to verify that the record is in place and the configurations above are working:

  • dig domain.tld TXT

    This would query the nameserver for the TXT record and should return something like:
    ;; ANSWER SECTION:
    domain.tld.             300     IN      TXT     "v=spf1 a a:external.domain.tld ~all"
  • Send an email with an empty subject to check-auth@verifier.port25.com . I strongly recommend using an email client because sending from comand prompt resulted sometime in failed Domainkey verification due to probably some extra characters added into the headers. The result will arrive in email and should look like this: Thank you for using the verifier,
    The Port25 Solutions, Inc. team
    ==========================================================
    Summary of Results
    ==========================================================
    SPF check: pass
    DomainKeys check: pass
    DKIM check: pass
    Sender-ID check: pass
    SpamAssassin check: ham
    ==========================================================
  • cd /etc/init.d/; chkconfig --add sender-id

    this adds the script to the list of programs which should run after restart. Do this for each init script

Last thoughts

I invite any reader to help improving this newbie-guide based on their experience and knowledge. There is plenty room for suggestions and additions to this article but this is an area where with all documentation available it seems that it is not enough or not covers all configuration or problems which may occur when setting up such a system. I spent myself several (many) good hours reading and learning, trying and failing or succeeding and finally writing this article hoping that will help others to save some of their time and neurons while trying to follow the latest recommendation of the war against spammers.

Although not required, a donation is always welcome end encouraging ;)

Donation



using PayPal

Resources

Files

DKIM configuration file /etc/dkim.conf

DKIM startup script /etc/init.d/dkim

Domainkeys startup script /etc/init.d/domainkeys

Sender-id startup script /etc/init.d/sender-id

Links

How to Setup DomainKeys Identified Mail (DKIM) with Postfix and Ubuntu Server

http://www.unibia.com/unibianet/systems-networking/how-setup-domainkeys-identified-mail-dkim-postfix-and-ubuntu-server

Setting up DKIM, SPF, Domainkeys DNS, Regular DNS on CentOS 5.3 at Pacificrack.com

http://palma-seo.com/setting-dkim-spf-domainkeys-dns-bind

Set Up DKIM On Postfix With dkim-milter (CentOS 5.2)

http://www.howtoforge.com/set-up-dkim-on-postfix-with-dkim-milter-centos-5.2

DomainKey Implementor's Tools and Library for email servers & clients

http://domainkeys.sourceforge.net/

Set Up DKIM For Multiple Domains On Postfix With dkim-milter 2.8.x (CentOS 5.3)

http://www.howtoforge.com/set-up-dkim-for-multiple-domains-on-postfix-with-dkim-milter-2.8.x-centos-5.3

Setting Up SPF, SenderId, Domain Keys, and DKIM

http://www.digitalsanctuary.com/tech-blog/debian/setting-up-spf-senderid-domain-keys-and-dkim.html

Domain Key / DKIM Key Generation Wizard

http://www.socketlabs.com/services/dkwiz

Domainkeys and DKIM with Postfix – Gentoo Linux wiki

http://en.gentoo-wiki.com/wiki/Domainkeys_and_DKIM_with_Postfix

Postfix Add-on Software

http://www.postfix.org/addon.html

DomainKeys Identified Mail (DKIM) Signatures – RFC

http://www.ietf.org/rfc/rfc4871.txt

Postfix / DKIM

https://help.ubuntu.com/community/Postfix/DKIM

How To Implement SPF In Postfix

http://www.howtoforge.com/postfix_spf

Email authentication guide made by BITS

http://www.bits.org/downloads/Publications%20Page/BITSSenderAuthenticationDeploymentFINALJun09.pdf

Return Path’s Best Practice Guide

http://www.returnpath.net/downloads/resources/Email%20Authentication%20v080107.pdf

Last update: May 5th, 2010


9 Comments on “Postfix with DKIM, Domainkeys, SPf and Sender-ID”

  1. 1 Spiro said at 3:32 pm on May 7th, 2010:

    Is it possible with DomainKeys that all mails (*) get signed with the domain "domain.tld". I know it is with DKIM, but haven't found any resources indicating whether this is possible with DomainKeys.

    Thanks in advance

  2. 2 Alex Iordache said at 4:35 pm on May 7th, 2010:

    DKIM is a newer "Domainkeys" so it should be possible.
    Please let us know how it went for your case

    Cheers

  3. 3 Spiro said at 4:40 pm on May 7th, 2010:

    I have have no luck on this. I think i'm going to settle for running DKIM and SPF. It should be good enough.

  4. 4 Alex Iordache said at 4:57 pm on May 7th, 2010:

    Here's what to look for. Send an email from otherdomain.tld signed with the key of domain.tld to yahoo and watch the headers for a line like:

    DomainKey-Signature: a=rsa-sha1; s=default; d=*domain.tld*; c=simple; q=dns; b=Gz/c19…

    If you see this then the first step is working, e.g. signing all mails with one key.

    Next, watch for the header:
    Authentication-Results: domainkeys=pass (ok);

    If this is not working then you will look into setting a proper selector record for the otherdomain.tld

    good luck

  5. 5 Spiro said at 5:17 pm on May 7th, 2010:

    I have not been able to get it to say d=*domain.tld*. I have only manage to get it to say "otherdomain.tld" or at least the sender address domain.

    I want to be able to set domainkey so that regardless of the senders domain address it always puts the domain as "domain.tld".

    I just can't find how to do this.

    With DKIM it's simple you just put "*:domain.tld:/var/db/domainkeys/domain.tld/default" in the keylist file. With Domainkeys i have no idea where i put this information. I tried to put the same statement in the DomainKeys keyfile but it failed.

  6. 6 aa said at 5:25 pm on August 29th, 2010:

    about removing v=DKIM1, is this a correct way to do it?
    or should we better made 2 similar entries with one without v=DKIM?
    or skip using DomainKeys altogether for the sake of new DKIM?

  7. 7 Brian said at 6:47 am on October 1st, 2010:

    Hello,

    In the DKIM section, under the Policy Record description, you have this information:

    "t=y means the domain is in test mode. When everything works it can be changed to "n""

    This is incorrect. The letter "n" is not a valid value for the t= parameter in either the DomainKeys or DKIM spec. If you do not want to be in test mode, then you leave out the t= parameter entirely.

    See here for the DKIM spec:
    http://www.dkim.org/specs/rfc4871-dkimbase.html

    Otherwise, a very good and helpful article. Thanks!

  8. 8 Cody said at 1:15 pm on December 24th, 2010:

    Just a comment about one thing you said (regarding named/bind).

    ".. If you own the namesever you will probably not use it as will be appended automatically by named/bind"

    Just want to point out that this is true but it depends on the record (line) and in particular the end of the record.

    Example :

    sub.domain.tld.

    (observe the final dot)

    versus

    sub

    The second one would be translated to sub.domain.tld (if the zone is domain.tld). The first would be sub.domain.tld also.

    But if you were to put in :

    sub.domain.tld (w/o final dot) it'd be something else and definitely not what you intended (or I would be surprised if it was what you wanted anyway..). Cannot recall off hand what the '.' stands for (in named terminology that is) but it basically means it's the complete name (versus no extra . which means indeed append the zone name to it).

    Just pointing this out as it could be confusing to some and it's an easy mistake to make.

    Oh, and as an example of why or when one might be useful.

    PTR record (point ip to name, rather than name to ip, ie reverse pointer) :

    1 PTR localhost.

    You should have the extra dot here.
    But say you wanted, as well a "localhost.domain.tld" too.

    The A record could be :

    localhost IN A

    Thanks for the blog, btw.. I have had SPF for ages but was looking into dkim and I think I will be implementing it when it's actually a sane hour (2 am here).

  9. 9 Creating a fully ‘safe’ and trusted e-mail domain for Hotmail/Gmail | Wieger Bontekoe said at 6:22 pm on September 8th, 2012:

    [...] allow the signing domain to claim responsibility for an email. You can find a very good how-to here and here using Postfix, Sender ID, SPF and [...]


Leave a Reply