updated December 2023

the perfect email server

How to set up exim4 on Debian Bookworm, with spamassassin and dovecot. There is another writeup for an alternative java based approach, using apache james; it's out of date, but may still be helpful. Both of these pages should be read in conjunction with notes on ensuring outgoing email is delivered to inboxes rather than spam boxes by mega providers such as hotmail and gmail.

There are some thoughts on the use of online blocklists aka DNSBLs.

And here's some notes on setting up java based webmail access to an imap server, such as the one we're building here with jwma running under either tomcat or jetty. Even if you're not going this route, it's worth lookig at for the sections on apache modsecurity and fail2ban, which I now consider to be essential.

I'm available for hire, but once you read this you probably don't need me!

What we want to achieve

We are setting up a full email system on a Debian Bookworm server. We want users to be able to access and send email securely, either by webmail, dedicated desktop email client or phone / tablet. We want to keep spam to a minimum and we do not want the software we're installing to compromise our server in any other way. The end goal is not to have something that Ed Snowden would feel comfortable to use, but we want to feel happy that the average user will have a good experience. Typically this setup is one that could be used on a virtual cloud server, with an always on fast internet connection. It's ideally suited to someone who wants to host their own email and maybe in addition that of a few customers also.

What’s not covered in any detail

Firewalling, but tcp ports 25 (smtp) and 465/587 (starttls/submission - for access other than by webmail) must be open. Ports 143/993 (imap/imaps) do not need to be open if they are only used for webmail access.

A SSL/TLS certificate for https access to webmail letsencrypt is strongly suggested

Software

Debian Packages - exim4, exim4-base, exim4-config, exim4-daemon-heavy (or exim4-daemon-light), spamassassin, dovecot-core, dovecot-imapd, libmail-spf-perl, libspf2-2, spf-tools-perl, cronic, fail2ban.

There may be a couple of issues if upgrading from Bullseye. Firstly if you've followed what I suggest you will have a substantially edited exim4.conf.template file and you will not want the packager's new version to overwrite it until the new version has your changes. If you are checking SPF on incoming mail (see below), you will need to temporaily stop that. The exim4 implementation has changed, SPF checks will fail and you will lose email.

Online blocklists

For many years I configured exim4 to deny access to all email that fell foul of zen.spamhaus.org. It worked very well. I'm unaware of ever losing any ham, close to zero spam in inbox and very little even in a spam folder, with spamassassin seldom seeing any dubious email. Recently (late 2022) they seemed to have changed acceptable use policy and suddenly a bunch of email got lost (I do glance at logs!). It seems that a refusal to check an email (over allowance / acceptable use issues) or failure to contact the blocklist is to exim, the same as the blocklist returning a fail.

So be aware that blocklists come and go and even Spamhaus, which seems to have the best and longest reputation can change its mind about what it wants to offer. I still use Spamhaus, but only now I've got a free account and my own DQS. The exim configuration I now suggest warns (which is the default) instead of denies any mail that Spamhaus identifies as spam. I've introduced bits of exim configuration to see that such email is delivered directly to a spam box and no resource expensive spamassassin check is duplicating the Spamhaus verdict. Only exim, not spamassassin, is configured to consult a blocklist. As an alternative to Spamhaus, Spamcop seems to have a fair reputation these days.

If you do go the same route as myself, getting a Spamhaus account, be aware that when you test your setup 7 out of their 10 tests will fail. This is because we are only configuring exim to check the RCPT ip address and none of the other tests on offer are run. Provided you configure the acl/30_exim4-config_check_rcpt section of the template file, the exim4.conf.localmacros file (both described in the exim4 section) and set up a spam box as I suggest, anything falling foul of Spamhaus' ip database will not trouble your inbox.

Setting up spamassassin

I should start by admitting that I've dropped the use of spamassassin as I've found that outsourcing my spam checking as described above works so well as to make this section of the notes redundent. As a bonus there will be a very significant drop in memory use, you can replace the exim4-daemon-heavy package with exim4-daemon-light and there will be a little less work to do with the exim4.conf.template file. If you prefer to keep spam checking in house or persist with the belt and braces approach, that's what the rest of this write up assumes.

Since we are going to configure exim4 in such a way that it will not accept mail unless it can connect with spam checking software, we start with spamassassin. You may edit the /etc/default/spamassassin to allow the rules to be automatically updated. This is probably best done by creating an entry in /etc/spamassassin/local.cf, but this is untested as I prefer to configue this in a crontab ( crontab -e as root) since an overly frequent update results in mail to let you know there were no updates available. So either of the two following strategies.

CRON=1
or
# m h  dom mon dow   command
 50 1  * * 1 cronic sa-update

You may also want to do some other tinkering in /etc/spamassassin/local.cf. I reduced the spam threshold to 4.0.

# required_score 5.0
required_score 3.5
I don't make further changes to Debian's default spamassassin since black and white listing is better done in exim4. Run update-rc.d spamd enable, restart with /etc/init.d/spamd restart and check with /etc/init.d/spamd status

Setting up exim4

I stick with exim4 over postfix due to the devil you know being better than the devil you don't. Due to similar force of habit, I stick with the monolithic exim4.conf.template. If you think the split configuration option looks more manageable you should be able to adapt the schemes suggested here to that.

The dpkg-reconfigure exim4-config routine runs as apart of the Debian install, but you can run it manually as root at any time if settings need to change. There's a handful of questions to be answered, some by selecting possible answers from a list and some by supplying relevant responses yourself.

Please select the mail server configuration type that best meets your needs
Select Internet site and confirm using the Tab key

Supply the system mail name				
This can be your own email domain or the output of the hostname -f command

IP-addresses to listen on			
Leave blank to allow Exim4 to listen for connections on all available network interfaces

Other destinations for which mail is accepted   
Supply a semi-colon separated list of domains not included in the system mail name answer

Domains to relay mail for
Machines to relay mail for
Leave answer blank for both of these

Keep number of DNS-queries minimal
Select No

Delivery method for local mail
I use the mbox format; all programs, including exim4, dovecot and email/webmail clients must be 
configured to use the same format

Split configuration into small files?		
I select No to use the single exim4.conf.template as opposed to the split config in the conf.d directory

This routine updates the update-exim4.conf.conf file, runs the update-exim4.conf procedure and restarts exim4 as this is necessary for config changes to be picked up.

Next job is to manually edit the exim4.conf.template file. At the top of the ### main/03_exim4-config_tlsoptions section enter this to allow transport Layer Security access on ports 25 and the submission ports. The submission ports are used for access by mobile and desktop email clients and are not necessay if webmail access is the only option you want to support.

MAIN_TLS_ENABLE = yes
daemon_smtp_ports=smtp : 465 : 587
if the submission ports are to be used a key and certificate file must be generated and a script at /usr/share/doc/exim4-base/examples/exim-gencert is provided for this. Running it should produce the files exim.key and exim.crt in the /etc/exim4 directory. These files should both be root:Debian-exim owned.

Provided the standard MAIN_TLS_ADVERTISE_HOSTS = * setting is left in place all host that conect with EHLO will be able to switch to an encrypted TLS connection.

The ### acl/30_exim4-config_check_rcpt includes policy concerning online blocklists (DNSBL). For many years I replaced the warn statement with deny and the lack of incoming spam made me consider removing spamassassin. I've reverted to the standard warn and already discussed the reason for that above. The standard config already adds a X-Warning header if the IP address is listed, I add a tweak to the subject line. This needs to be repeated for the second check for a listed Domain which occurs immediately after the IP address check.

add_header = X-Warning: $sender_address_domain is listed at $dnslist_domain ($dnslist_value: $dnslist_text)
add_header = Subject: ***SPAM address in DNSBL*** $bh_Subject:
log_message = $sender_address_domain is listed at $dnslist_domain ($dnslist_value: $dnslist_text)
The reason for this will be explained shortly.

Scroll down in the same section to the commented code that configures spam checking and uncomment as follows, also adding some additional code. Note that I've changed some message size settings from default. You may of course choose your own deny message or leave that unset.

   warn
    #h_X-Warning is added if DNSBL check fails, so no point running spamassassin on these email
    condition = ${if !def:header_X-Warning:}
    condition = ${if <{$message_size}{120k}{1}{0}}
     # ":true" to add headers/acl variables even if not spam
     spam = nobody:true
     add_header = X-Spam_score: $spam_score
     add_header = X-Spam_bar: $spam_bar

  # reject spam at high scores (> 7.5)
  deny
      condition = ${if !def:header_X-Warning:}
      condition = ${if <{$message_size}{120K}{1}{0}}
      condition = ${if >{$spam_score_int}{75}{1}{0}}
      spam = nobody:true
      message = $spam_score sucks, rejected RCPT : Unrouteable address.

  # add second subject line with *SPAM* marker when message is over threshold (> 3.5)
  warn
      condition = ${if !def:header_X-Warning:}
      condition = ${if <{$message_size}{120K}{1}{0}}
      condition = ${if >{$spam_score_int}{35}{1}{0}}
      spam = nobody:true
      add_header = Subject: ***SPAM (score:$spam_score)*** $bh_Subject:

The first section runs if two conditions are met. There is no point running spamassassin on mail that was already declared to be spam by a blocklist and so has a X-Warning header. Also we don't want to check large email (probably with attachments) due to resouce considerations. Both of these conditions also apply in the following two sections of code. When this section does run two additional headers are added which indicate the spam score.

The second section refuses to accept mail if spamassassin has given it a score above 7.5. If you don't think that's conservative enough you can raise that figure. Note that in this way blocklist identified spam is always accepted; the reason for that has already been explained.

The final section looks for suspect spam (score above 3.5) that was not blatent enough to be dropped by the previous section and tinkers with the subject line in a similar way to mail from an address flagged by a blocklist. The reason for this still awaits explanation.

We now insert some router code to allow fine tuning of how exim4 delivers mail to our local users, after end router/300_exim4-config_real_local. This code becomes really useful if you're hosting several domains. So for instance, if user cain fancies himself as administator for mydomain.com and wants to use the email admin@mydomain.com, that could be done by making an aliase statement admin: cain in the /etc/aliases (running the newaliases command is unnecessary). If there is another hosted domain owned by different users who want an admin@anotherdomain.co.uk address, using the aliases file is no longer workable.

Whether you choose to rely on the alias file or insert the code below, I'm assuming you will create local unix accounts for all users on the server and I'm not covering the virtual user alternatives to this scheme. You do not of course want these users to have shell access to the server and that can be prevented by a line in /etc/ssh/sshd_config such as AllowGroups people and ensuring only you are in the people group.

#####################################################
### router/350_exim4-config_vdom_aliases
#####################################################

vdom_aliases:
  driver = redirect
  allow_defer
  allow_fail
  domains = dsearch;/etc/exim4/virtualhosts
  data = ${lookup{$local_part}lsearch{/etc/exim4/virtualhosts/$domain_data}}
  retry_use_local_part
  pipe_transport = address_pipe
  file_transport = address_file
  no_more

#####################################################
### end router/350_exim4-config_vdom_aliases
#####################################################

These local or virtual domains must each have a file in the virtualhosts directory. So for mydomain.com the file /etc/exim4/virtualhosts/mydomain.com might look something like this.

admin: cain@localhost
cain:  cain@localhost
eve:   eve@localhost
adam:  adam_number1@hotmail.com
abel:  :fail: account is closed
*:     :blackhole:

Lastly in the ### auth/30_exim4-config_examples section uncomment the plain server code as this is necessary for access other than by webmail for desktop and mobile clients.

plain_server:
  driver = plaintext
  public_name = PLAIN
  server_condition = "${if crypteq{$auth3}{${extract{1}{:}{${lookup{$auth2}lsearch{CONFDIR/passwd}{$value}{*:*}}}}}{1}{0}}"
  server_set_id = $auth2
  server_prompts = :
  .ifndef AUTH_SERVER_ALLOW_NOTLS_PASSWORDS
  server_advertise_condition = ${if eq{$tls_cipher}{}{}{*}}
  .endif

If access is not to be solely by webmail, a root.Debian-exim owned file /etc/exim4/passwd must be created. It's a good plan to use the comment field as a password reminder.

#/etc/exim4/passwd allows a user to authenticate a mail submission to the Exim
# MTA without using their system password (found in /etc/shadow).
# permissions should be 0600
#
# Each line of this file should contain a "user:password:comment" field,
# where the password is encrypted and encoded using standard crypt(3)
# functions--the same format as is used in /etc/shadow.  You can disable
# a user from ever sending (authenticated) messages by using "*" as the
# password.
#
# You can use the following command line to generate the password:
#
#  mkpasswd -m sha-512 'password'
#
# (replace "password" with your password, of course).

cain:$6$Y6A0LniAqj11$UCDq8ZNfENXJpJwcs7BeX/pN31RwKay8UuMZfVAm4iW5Qk/Op6qewA1cwI3Sox8kZGTR8u9Bj7mG34.lg0TtY0:***w0**
eve:$6$b9i1FowVpJEKm1FI$AKpwriAi0co4771HdYo1J8rONoHWQlPO/.cno7FON4psslDsbyXI7tRdTPsUl1Z7rmq4VqP8T/dobcaHCHl8E/:system password

In this example there is no entry for Adam since as can be seen from the virtual hosts file, his mail is forwarded to another server and he will send his email from there.

The last thing to do is to define the MAIN_LOCAL_DOMAINS macro with this line in exim4.conf.localmacros, creating it root.root owned if necessary.

MAIN_LOCAL_DOMAINS = @:localhost:dsearch;/etc/exim4/virtualhosts

At this point we’re mostly done setting up exim4 after running update-exim4.conf and restarting it, although with a couple of extra tweaks, most spam can be caught without invoking a spamassassin check. We can check that email complies with the SPF policy of the sending domain (if it has set one) and also define which online DNSBL databases should be used. This is done with some additional macro definitions in the exim4.conf.localmacros file.

It's worth a mention here that the version of exim4 in Debian Bullseye, 4.94, introduced the concept of tainted data. For instance file paths constructed using sender input are considered to be tainted and this could result in mail being dropped. So, returning to the vdom router we coded in exim4.conf.template, the line

data = ${lookup{$local_part}lsearch{/etc/exim4/virtualhosts/$domain}}

would be considered tainted, hence the use of $domain_data instead.

MAIN_LOCAL_DOMAINS = @:localhost:dsearch;/etc/exim4/virtualhosts
CHECK_RCPT_SPF = true
CHECK_RCPT_IP_DNSBLS = your_DQS.zen.dq.spamhaus.net
#CHECK_RCPT_IP_DNSBLS = bl.spamcop.net
DKIM_CANON = relaxed
DKIM_SELECTOR = your_selector
DKIM_DOMAIN = mydomain.com
DKIM_PRIVATE_KEY = /etc/exim4/rsa.private
While we have this file open we'll also add macros necessary for exim to carry out DKIM signing. This is one part of the DKIM implementation, the other being adding a TXT record to the domain's DNS. The your_selector line, part of the TXT record, is explained here and mydomain.com will be the domain for whom exim4 is signing emails.

We also need to create the public and private keys, move them to /etc/exim4 and make them owned Debian-exim:root.

openssl genrsa -out rsa.private 1024
openssl rsa -in rsa.private -out rsa.public -pubout -outform PEM

For a more complicated setup with DKIM records for more than a single domain the From: header of outgoing email can be used to assign the DKIM domain:-

DKIM_DOMAIN = ${domain:$h_from:}
I've seen unneccessarily complicated suggestions with multiple keys, but I see no reason why multiple domains cannot share the same DKIM keys and indeed the same selector. I have tested such a set up with two domains on the same server.

One last thing to say about exim4 is how the fail2ban software can help; this is dealt with at here.

We can use telnet to run a basic send/receive test of exim4. For clarity, the SMTP commands that we issue are capitalized, but that is not necessary to run this test. The responses from the server commence with a numerical code. The HELO command can be issued to any domain for which the server is the mail exchanger, you do not have to use the canonical hostname. Entry of the message is terminated by entering a period (.) on a line alone and exim issuing an id for the message is conformation it has been sent. The SMTP session is ended with Ctrl ].

cain@eden $ telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.localdomain.
Escape character is '^]'.
220 eden.uk0.bigv.io ESMTP Exim 4.96.2 Thu, 28 Sep 2023 12:13:54 +0000
HELO mydomain.com
250 eden.uk0.bigv.io Hello localhost [127.0.0.1]
MAIL FROM:cain@mydomain.com
250 OK
RCPT TO:cain@localhost
250 Accepted
DATA
354 Enter message, ending with "." on a line by itself
SUBJECT:test
hello there!
.
250 OK id=1pVAVs-000EkN-1i
^]
telnet> quit
Connection closed.
cain@eden $ 
Once you have set up dovecot next, you will be able to login and check that the message is in your inbox.

Setting up dovecot

The mail storage format must match that of exim4. This is configured in /etc/dovecot/conf.d/10-mail.conf
mail_location = mbox:~/mail:INBOX=/var/mail/%u>
As a system user you should be able to authenticate to dovecot with your usual password; we can test that with telnet.
eve@eden:~ $ telnet localhost 143
Trying 127.0.0.1...
Connected to eden.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS AUTH=PLAIN] Dovecot (Debian) ready.
A1 LOGIN eve 'password'
A1 OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY
THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT
CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN
CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY STATUS=SIZE SAVEDATE LITERAL+ NOTIFY SPECIAL-USE]
Logged in A2 LOGOUT * BYE Logging out A2 OK Logout completed (0.001 + 0.000 secs). Connection closed by foreign host. eve@eden:~ $
I know of a hopefully rare instance when that will not work, which occurs if you use fscrypt to encrypt partitions. Such a problem can be fixed by editing the file /etc/dovecot/conf.d/10-master.conf and then restarting dovecot.
# Default VSZ (virtual memory size) limit for service processes. This is mainly
# intended to catch and kill processes that leak memory before they eat up
# everything.
#default_vsz_limit = 256M
default_vsz_limit = 4096M #added to fix fscrypt incompatibilty
With our production setup we will want to use encryption, so we want to switch from plain imap access to encrypted imaps. As with exim4, Debian provides a shell script to generate SSL certificate and key.
root@mydomain.com:~# cd /usr/share/dovecot/
root@mydomain.com:/usr/share/dovecot# ./mkcert.sh

The script must be run from it's own directory to find library files it needs to call. The necessary files may have been generated by the install process, in which case the script will inform you of that. Either way there should now be a subdirectory /etc/dovecot/ssl/ with files dovecot.pem and dovecot.key

In /etc/dovecot/conf.d/10-ssl.conf (the < characters are not a typo!):-

# SSL/TLS support: yes, no, required.
#ssl = no 
ssl = yes
ssl_cert = </etc/dovecot/ssl/dovecot.pem
ssl_key = </etc/dovecot/ssl/dovecot.key
Restart dovecot and check that imaps is available on port 993
openssl s_client -servername localhost -connect localhost:993
/\/\/\/\/\/\
    output of certificates removed
\/\/\/\/\/\/
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready.

Using an email client such as thunderbird or sylpheed, a user should be able to access their email (port 465, STARTTLS) using the account details provided for them as a system user and send mail using the details created for them in the /etc/exim4/passwd file. If there is webmail software running on the server an alternative will be to browse to the provided location. This access requires only the system user details to authenticate against dovecot; since the connection from here to exim4 is a local one there is no requirement to authenticate separately. The write up here explains how to set up jwma (java webmail app) as a dovecot frontend.

Switching from mbox to maildir storage

This just deals with the changes to be made to dovcot and exim4 configs, rather than conversion of pre-existing emails. Why would you want to use maildir on a Debian system? I would not make this change on a production server, but it's useful to know for testing purposes.

For dovecot the change is very simple. Edit /etc/dovecot/config.d/10-mail.conf and restart dovecot:

## CHANGED ####################
#mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_location = maildir:/home/%u/Maildir

For exim4, in /etc/exim4/exim4.config.template the changes need to be made in the ### transport/30_exim4-config_mail_spool section which you should make to look like this:

### transport/30_exim4-config_mail_spool

# This transport is used for local delivery to user mailboxes in traditional
# BSD mailbox format.
#
mail_spool:
  debug_print = "T: appendfile for $local_part@$domain"
  driver = appendfile
  directory = ''home''${local_part}/Maildir
  #directory = $home/Maildir
  maildir_format
  maildir_use_size_file
 # file = /var/mail/$local_part_data
  delivery_date_add
  envelope_to_add
  return_path_add
  group = mail
  mode = 0660
  mode_fail_narrower = false
  
As usual for exim4 changes, this must be followed up with the usual
root@bulawayo: # update-exim4.conf
root@bulawayo: # /etc/init.d/exim4 restart

Gilding the lily

The acl/30_exim4-config_check_rcpt section in exim4.config.template includes a section of code that allows for annoying senders to be blocked. Create the /etc/exim4/local_sender_blacklist root owned and for example:-
!cain@mydomain.com
!eve@mydomain.com
!abel@mydomain.com
*@mydomain.com
spameri@tiscali.it 
*@wesenduspam.org 

The first 3 lines ensure that the real users at mydomain.com can send email, followed by a line that blocks the spammers attempted relaying trick of setting the sender header as a fake user on your domain. The last two lines demonstrate blocking a single address and a whole domain. Anything caught here is dropped without being checked by spamassassin.

Editing this files is an exception to the rule that exim4 must be restarted after any configuration changes.

Note the couple of assumption this section now makes - firstly that mbox format is used for mailboxes and secondly that the system uses /home/<user>/mail as the location of the mailboxes other than the inbox.

Mail is identified as spam with the suggestions made in this write up when a DNSBL has the ip address or spamassassin scores it at 3.5 or above; in both cases the subject line is altered to mark it as such. Because we do not want to lose false positives, if the score is below 7.5, it is still delivered and only if the score is 7.5 or above will it be rejected.

I've never seen email that spamassassin scored between 3.5 and 7.5 that I wanted, so I like to have it delivered to a spam folder rather than my inbox. There must be a mailbox named spam for this to work - it can be created by any webmail or email client software. After that, an exim forward file - /home/<user>/.forward does the trick.

# Exim filter <<< important first line, distinguishes from ye olde .forward
if $header_subject: contains "***SPAM" 
	then
	save "/home/<user>/mail/spam"
endif
It's an unfortunate fact of life that despite our best efforts some unwanted email will slip through as spammers continue to try new tricks. Spamassassin on debian includes the sa-learn program which users can make use of to teach spamassassin to reject similar mail in future.

The user must have a suitably named mailbox such as junk into which any spam that sneaks through below the 4.0 barrier can be moved from the inbox. The user also needs to create a cron job using the crontab -e command. In the following example the command will run the sa-learn program via cronic at 3:00am every night. The cronic program is used to avoid receiving email (cram or cron spam!!) that only confirms that everything worked fine:-

0 3 * * * cronic sa-learn --mbox --spam /home/<user>/mail/junk

Rather than rely on the user to do this final set up work it can all be automated after you create the system account using this shell script. You just then need to tell the user to leave any spam they receive in the junk folder for a day or two before deleting it.