webmail with tomcat, apache2 and jwma

This documents how to set up tomcat10 as a backend to apache2, by way of an AJP connection, with a view to have it hosting the JWMA webmail app. The main focus is to describe the tomcat end of things, so much of the finer details of an apache setup are not dealt with. There is though some information on adding the modsecurity module to apache and also using fail2ban in conjunction with it. Incidentally, there is no reason why tomcat could not be employed in a stand alone style as I've described for jetty and although it's not how I choose to do things, the information is here.

Version 4.5 or later of JWMA and a version 17 or 21 java runtime require a jakarta ready version of tomcat, which is available at apache.org. Download and decompress the archive in /opt. I create a system user tomcat (without ssh access, using the AllowGroups statement in /etc/ssh/sshd_config) and change the ownership of the whole tomcat install to that user, making /opt/tomcat it's home directory.

There's a couple of things in the webapps directory which you don't really want on an internet connected production server, so I remove the docs and examples directories.

In the /opt/tomcat/bin directory I add an executable file setenv.sh:-

#! /bin/sh
JRE_HOME="/opt/jre-17.0.9"
JAVA_OPTS="$JAVA_OPTS -Djavax.net.ssl.trustStore='/opt/jre-17.0.9/lib/security/cacerts'"

CATALINA_OPTS="$CATALINA_OPTS -Xms64m"
CATALINA_OPTS="$CATALINA_OPTS -Xmx512m"
CATALINA_OPTS="$CATALINA_OPTS -XX:+UseParallelGC"
CATALINA_OPTS="$CATALINA_OPTS -XX:MaxGCPauseMillis=1500"
CATALINA_OPTS="$CATALINA_OPTS -server"
CATALINA_OPTS="$CATALINA_OPTS -XX:+DisableExplicitGC"


if [ -r "$CATALINA_BASE/bin/appenv.sh" ]; then
  . "$CATALINA_BASE/bin/appenv.sh"
fi

echo "Using CATALINA_OPTS:"
for arg in $CATALINA_OPTS
do
    echo "  " $arg
done

echo "Using JAVA_OPTS:"
for arg in $JAVA_OPTS
do
    echo "  " $arg
done

Tomcat can now be started by the tomcat user from it's home directory. On a linux server this can be automated from /etc/rc.local.

#!/bin/sh -e
cd /opt/tomcat/
su tomcat -c "bin/catalina.sh start"
exit 0
If you now pop the JWMA webmail.war file into the tomcat webapps directory it should start up on port 8080. With a working dovecot as imap server you should now be able to login using your system user credentials (ie <user>, not <user@domain>) and manage and send email.

A jwma.config file is created in the /opt/tomcat/.jwma directory even after a failed login (there need not even be the dovecot imap server running). Both dovecot and exim4, must be configured to use the same storage format as JWMA, ie mbox if you've followed this writeup. JWMA should detect which system is in use and configure itself accordingly, but if it fails to do that you can make the necessary correction. We may want to make other changes to the config file, but for now just be aware that JWMA must be restarted when it's configuration is changed. We can do that by stopping and restarting tomcat with the bin/catalina.sh command and stop|start switches, but we are going to configure a less heavy handed way to do that.

Even though we've got access to an operational email system there's a couple of things that are less than ideal. We don't want to have to restart tomcat when we just want JWMA to restart and we want to send login details over https rather then http. So the next step is to enable the manager-script role which offers similar functionality to the better known manager-gui role but from the command line. Make the conf/tomcat-user.xml file look like this:-

<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
  <role rolename="manager-script"/>
 <user username="your_user" password="your_password" roles="manager-script,admin-jmx"/>
</tomcat-users>
create conf/Catalina/localhost/manager.xml with this content:-
<?xml version='1.0' encoding='utf-8'?>
<Context path="/manager" privileged="true">
<Valve className="org.apache.catalina.valves.RemoteAddrValve" allow="127\.\d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1"/>
</Context>
and put an executable copy of manager.sh with user and password matching those in the tomcat-users.xml file above in /opt/tomcat/bin. This script file, which needs curl installed, will allow you to start and stop jwma with the running tomcat left untouched. Access to port 8080 through the firewall is necessry.
#!/bin/bash
#manager.sh to manage tomcat webapps, must be executable
#needs manager-script role configured and curl (recommended), lynx, elinks or some such browser

APP=$2
#BROWSER=lynx
BROWSER="curl -u your_user:your_password

list(){
	$BROWSER http://localhost:8080/manager/text/list
	exit 0
}

reload(){
	app
	$BROWSER http://localhost:8080/manager/text/reload?path=/$APP
	exit 0
}

stop(){
	app
	$BROWSER http://localhost:8080/manager/text/stop?path=/$APP
	exit 0
}

start(){
	app
	$BROWSER http://localhost:8080/manager/text/start?path=/$APP
	exit 0
}

help(){
	echo "manager.sh -- "
	echo "the  argument is optional"
	echo "a script to control tomcat apps"
	echo "you must configure a manager-script role in tomcat-users.xml"
	echo; echo " -l | --list"
	echo " -r  | --reload "
	echo " -o  | --stop "
	echo " -a  | --start "
}

app(){
	if [ "$APP" == "" ]	;then
		echo "which app?"
		read APP
	fi
}

if [ "$1" == "-l" ] || [ "$1" == "--list" ]	;then
	list
elif [ "$1" == "" ]    ;then
    echo "manager.sh --help"
	exit 0
elif [ "$1" == "-h" ] || [ "$1" == "--help" ]	;then
	help
	exit 0
elif [ "$1" == "-r" ] || [ "$1" == "--reload" ]	;then
	reload
elif [ "$1" == "-o" ] || [ "$1" == "--stop" ]	;then
	stop
elif [ "$1" == "-a" ] || [ "$1" == "--start" ]	;then
	start
else echo "manager.sh --help"
fi

With the manager.sh script, JWMA can quickly be restarted without disturbing tomcat or any other webapps that may be running:-

bin/manager.sh --reload webmail

adding https

On now to configuring tomcat to serve content over https, which I'll demonstrate with a self-signed certificate.

keytool -genkey -keystore /opt/tomcat/.keystore -alias mydomain.com -keyalg RSA -keysize 4096 -validity 720
Enter keystore password: your_password
Re-enter new password: your_password
What is your first and last name?
  [Unknown]:  mydomain.com	
The answer to the first question should be the name of your domain (not your own first and lastname!), but the -alias can be anything. The other questions can be left unanswered and you should not enter a second password at the end of the dialogue.

Now uncomment the Connector port="8443" block of code in /opt/tomcat/conf/server.xml and make it look like this:-

<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
           maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
           clientAuth="false" sslProtocol="TLS"
           keyAlias="mydomain.com" keystoreFile="/opt/tomcat/.keystore"
           keystorePass="your_password" 
/>
Note that keyAlias, keystoreFile and keystorePass must match the values you used in the keytool command line and the answers you gave

After restarting tomcat the JWMA login page will appear at https://mydomain.com:8443/webmail as well as http://mydomain.com:8080/webmail. There follows now a couple of different ways that we can avoid having to specify the port numbers without resorting to running tomcat as root (or administrator on windows).

One method is a firewall redirect, such as this on a linux server:-

/sbin/iptables -t nat -A OUTPUT -p tcp -d servers_ip_address --dport 80 -j REDIRECT --to-port 8080
/sbin/iptables -t nat -A OUTPUT -p tcp -d servers_ip_address --dport 443 -j REDIRECT --to-port 8443

I put another shell script in the bin directory and run it as a nightly cron job. This removes any files that got uploaded as email attachments and also prunes older tomcat log files which contain a date.

#!/bin/sh
#CATALINA_BASE/bin/tidy.sh

echo "clearing jwma uploads"
rm /opt/tomcat/.jwma/uploads/*

DATE=`date -d "2 days ago" +"%y-%m-%d"`
echo "removing $DATE logs"
rm /opt/tomcat/logs/*$DATE*
The catalina.out log should be added into the logrotate fold by creating /etc/logrotate.d/catalina
/opt/tomcat/logs/catalina.out {
    weekly
    rotate 3
    compress
    notifempty
    delaycompress
    missingok
    create 640 tomcat tomcat
}
Everything works, but big problem - it's not possible to serve static non java web pages from the same ip address using apache2. So a better solution is to use tomcat as a backend to apache2; the self signed certificate is then unnecessary as is the https connector and it's easier to take advantage of the free letsencrypt service and obtain a certificate that does not create a browser security warning.

The necessary ajp configuration in tomcat is the default in it's conf/server.xml file, so all that's needed is to uncomment this section.

<!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector protocol="AJP/1.3"
               address="::1"
               port="8009"
               redirectPort="8443"
    />

adding derby support

Copy the provided derby.jar file from the JWMA zip archive into the /opt/tomcat/lib and in the <GlobalNamingResources> section of conf/server.xml add the code below, after which, tomcat can be restarted.

<Resource name="jdbc/jwmaDB"
	type="javax.sql.DataSource"  auth="Container"
	description="Derby database for jwma"
	maxActive="100" maxIdle="30" maxWait="10000"
	username="" password="" 
	driverClassName="org.apache.derby.jdbc.EmbeddedDriver"
	url="jdbc:derby:Databases/jwmaDB"
/>

Setting up apache2

After installing apache2 check that the proxy and proxy_ajp modules are enabled with a2query -m | grep proxy and then move on to configuring a site to run the jwma tomcat webapp. The mydomain.com.conf file serves as a template but will need a fair bit of editing to suit your setup, including renaming it to match your domain name and removing the .example extension.

There are two occurences each of ServerName and the ServerAdmin email address and four instances of log directories that you must change. At the bottom of the file you should indicate locations of your certificate and key files. It's possible to do a self signed certificate as we did with tomcat, but there is no longer any point in doing this now that free certificates are available from letsencrypt with minimal difficulty. Note that apache2 no longer has access to the /srv directory on Debian by default, so if you want to put logfiles there you need to allow that in /etc/apache2/apache2.conf:-

<Directory /srv/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>
With that done and the site config completed in /etc/apache2/sites-available/ you can enable it with the a2ensite command and restart apache2. JWMA should now be availble at the default http and https ports So there is no longer a need to specify a port in the url as https://mydomain.com/webmail will suffice. Note that the smtps password we set up in exim4 are not used at all for this type of access as once a user authenticates with dovecot it's a secure local connection to exim4.

modsecurity

Particularly when combined with fail2ban, modsecurity with the freely available owasp rules is a useful addition to an apache2 install. However it may occasionally and unnecessarilly trip up a jwma seesion. As an example I had a photo which had been grabbed from a video, probably with something slightly off with its attempt at jpg format creation and modsecurity blocked the sending of an email with this file attached.

Rather than identifying the problematic rule and turning it off a better approach is to turn off modsecurity which should hopefully be totally unnecessary once a user has logged in. This is facilitated by the fact that all the URLs encountered in a jwma session have the .jwma extension. So in /etc/modsecurity/modsecurity.conf add a single line below the instruction to activate the rule engine.

SecRuleEngine On
SecRule REQUEST_BASENAME "@contains .jwma" "id:1,ctl:ruleEngine=Off"
For a general management of modsecurity in the event that a rule becomes problematical, there's a good article here.

fail2ban

Recently for the first time in 15 years, on more than one occasion I've seen a large number of hostile coonection attempts to a small mail server. Typically, for a period of several weeks there will be daily attacks from a vast range of ip addresses. Since these incidents start and stop suddenly there must be some concerted effort behind this? Concurrent with this is the drive to reduce the possibility of anonymity on the internet and large providers starting to require a phone number from their users. Whether or not the attacks and possible anonymity of using a small provider are connected, you wnt to stop them!. With dynamic additions to your firewall that fail2ban provides by blocking hostile ip addresses on a temporary basis the problem is contained even if the attacks are well resourced.

The fail2ban program comes with jails configured for modsecurity and exim4, but disabled by default. We start off by turning them on by creating a /etc/fail2ban/jail.local file. This file shoul contain just the deiations from the default config that we whish to put in place. Particularly note that fail2ban enables the ssh jail by default and this config will turn that off, so if you do likewise, ensure other measures are in place to protect shell access.

[DEFAULT]
bantime = 400h
usedns = no
maxretry = 1

[sshd]
port    = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s
enabled = false

[apache-modsecurity]
port     = http,https
#logpath = /var/log/apache2/modsec_audit.log 
#logpath = /var/log/apache2/modsec_audit.log.1
logpath  = %(apache_error_log)s
#maxretry = 2
enabled = true

[apache-badbots]
# Ban hosts which agent identifies spammer robots crawling the web
# for email addresses. The mail outputs are buffered.
enabled = true
port     = http,https
logpath  = %(apache_access_log)s
#bantime  = 48h
#maxretry = 1

[apache-noscript]
#stop search for executable scripts
enabled = true
port     = http,https
logpath  = %(apache_error_log)s

[apache-overflows]
#request for dodgy URLs
enabled = true
port     = http,https
logpath  = %(apache_error_log)s
#maxretry = 2

[exim]
# see filter.d/exim.conf for further modes supported from filter:
#mode = normal
port   = smtp,465,submission
#logpath = %(exim_main_log)s
logpath = /var/log/exim4/rejectlog
enabled = true

[exim-spam]
logencoding=utf-8
port   = smtp,465,submission
logpath = %(exim_main_log)s
enabled = true

Bad guys attacking the mail exchanger are now going to find their addresses quickly blocked, with a cooling off period of a couple of weeks. With it's standard filter, fail2ban's exim jail blocks some addresses after access attempts I had not considered to be of much interest until recent times. But the bigger the jail population the better - they can sew mail bags!

With apache-modsecurity the situation is different. Modsecurity itself blocks the most outrageous requests to the http and https ports by returning an error. I want fail2ban to block all other access attempts that modsecurity deems to be security critical and by default this does not happen in every case. So to remedy this we create /etc/fail2ban/filter.d/apache-modsecurity.local and put our own filter in this file:-

# Fail2Ban apache-modsec filter
#

[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# apache-common.local
before = apache-common.conf

[Definition]

failregex = ^%(_apache_error_client)s(?: \[client [^\]]+\])? ModSecurity:.+?severity "CRITICAL

ignoreregex = 

I also like to change the iptables response to a fail2ban hit. I do this by copying /etc/fail2ban/action.d/iptables-common.conf to iptables-common.local in the same directory. In that new file I make a single alteration
#blocktype = REJECT --reject-with icmp-port-unreachable
blocktype = DROP