Mail Client Auto-Configuration

With mail-client auto-configuration, users who setup a new mail-client or smartphone only need to provide their mail-address and password. All the complicated configurations for incoming and outgoing mailservers, port-number, encryption and login methods, address-books and calendars will then be setup automatically.

These mail clients retrieve configuration data for a mail-address trough a combination of DNS and HTTP queries from the domain-part of the address.

As there is no univeersal standard for this, different mail-clients use their own proprietary method, which service providers need to support on there servers for this to work. Other clients don’t even support such functionality.

Three different methods are known today to support automatic mail client configuration:

  • Apple Mac, iPhone and iPad:

    • iPhone and iPod touch with iOS 4 or later

    • iPad with iOS 4.3 or later or iPadOS 13.1 or later

    • Mac computers with OS X 10.7 or later

    • Apple TV with tvOS 9 or later

  • Microsoft Outlook

  • Moziila Thunderbird, also supported by:

    • Evolution (for Linux GNOME desktops)

    • FairEmail (for Android phones)

    • K9 Mail (for Android phones)

    • KMail aka Kontact (for Lunux KDE desktops)

    • NextCloud Mail App

Manually providing and mainting all these different formart is tedious.

automx2 App

automx2 is a auto-configuration web-service who can provide configuration data in all three formats to requesting clients.

Prerequesites

The following needs to be available beforehand:

Create a system user who will run the service:

$ sudo adduser --system --home /var/www/example.net/automx2 automx2

Create a database access password:

$ pwgen -s 32 1
jyHZdNnB3Fe3sTotihMTiuf51BH6EEq9YkCd0zTWU6GekkkO

Create a database and a user in MariaDB server to hold configuration data:

mysql -p

mysql> CREATE DATABASE `automx2` COLLATE 'utf8mb4_general_ci';
mysql> GRANT SELECT ON automx2.* TO 'automx2'@'127.0.0.1' \
mysql>     IDENTIFIED BY 'jyHZdNnB3Fe3sTotihMTiuf51BH6EEq9YkCd0zTWU6GekkkO';
mysql> FLUSH PRIVILEGES;
mysql> exit

Create a Python virtual environment for the software to be installed under:

$ sudo -u automx2 -Hs
$ cd /var/www/example.net/automx2
$ wget https://github.com/rseichter/automx2/raw/master/contrib/setupvenv.sh
$ chmod u+x setupvenv.sh
$ ./setupvenv.sh

Software Installation

$ sudo -u automx2 -Hs
$ source .venv/bin/activate
$ pip install automx2

Software Configuration

Create the file /etc/automx2/automx2.conf.

[automx2]

# A typical production setup would use loglevel WARNING.
loglevel = DEBUG

# Echo SQL commands into log? Used for debugging.
db_echo = no

# MySQL database on a remote server. This example does not use an encrypted
# connection and is therefore *not* recommended for production use.
#db_uri = mysql://username:password@server.example.com/db

# Database server connection
db_uri = mysql+pymysql://automx2:jyHZdNnB3Fe3sTotihMTiuf51BH6EEq9YkCd0zTWU6GekkkO@localhost/automx2?charset=utf8mb4

# Number of proxy servers between automx2 and the client (default: 0).
# If your logs only show 127.0.0.1 or ::1 as the source IP for incoming
# connections, proxy_count probably needs to be changed.
proxy_count = 1

Initialize

Initialize the database:

$ curl http://127.0.0.1:4243/initdb/

Copy and edit the file /var/www/example.net/automx2/contrib/seed-example.json

{
    "provider": "Example Net.",
    "domains": ["example.net", "example.org", "example.com"],
    "servers": [
        {"name": "mail.example.net", "type": "imap"},
        {"name": "mail.example.net", "type": "smtps"}
    ]
}

SystemD Service

Copy the provided service file /var/www/example.net/automx2/contrib/automx2.service to the /etc/systemd/system/ directory.

Ajust the file path of the ExecStart and WorkingDirectory lines to our Installation.

[Unit]
After=network.target
Description=MUA configuration service
Documentation=https://rseichter.github.io/automx2/

[Service]
Environment=FLASK_APP=automx2.server:app
Environment=FLASK_CONFIG=production
ExecStart=/var/www/example.net/automx2/bin/flask run --host=127.0.0.1 --port=4243
Restart=always
User=automx2
WorkingDirectory=/var/lib/automx2

[Install]
WantedBy=multi-user.target

Reload SystemD and enable the service:

$ sudo systemctl daemon-reload
$ sudo systemctl enable automx2

Updating

Updating the Software

$ sudo -u automx2 -Hs
$ cd /srv/web/automx2
$ source .venv/bin/activate
$ pip install --upgrade automx2

Updating the Database

$ sudo -u automx2 -Hs
$ cd /srv/web/automx2
$ export RELEASE="2021.6"
$ wget https://github.com/rseichter/automx2/archive/refs/tags/$RELEASE.zip
$ unzip $RELEASE.zip
$ cd automx2-$RELEASE/alembic

Edit the file /var/www/example.net/automx2/alembic/alembic.ini

# Database server connection
sqlalchemy.url = mysql://automx2:jyHZdNnB3Fe3sTotihMTiuf51BH6EEq9YkCd0zTWU6GekkkO@localhost/automx2?charset=utf8mb4

Do the upgrade:

$ source .venv/bin/activate
make upgrade

Mozilla Thunderbird

Thunderbird looks for configuration data in XML-format at predefined (well-known) URLs.

This method of autonconfiguration

This also works for …

Evolution and KMail have adopted this format too.

The process is desribed at the Autoconfiguration in Thunderbird page in the MDN (Mozilla Developers Network).

  1. The client tries to access the following URLs in the preferred order:

    1. https://autoconfig.example.net/mail/config-v1.1.xml

    2. https://example.net/.well-known/autoconfig/mail/config-v1.1.xml

  2. If successful, the XML data received on that URL will be used as configuration hints.

DNS Records

Add the following CNAME record to every domain you are provding mail services for:

@example.org
autoconfig  IN  CNAME autoconfig.example.net

@example.com
autoconfig  IN  CNAME autoconfig.example.net

@example.net
autoconfig  IN      A 10.195.171.241
autoconfig  IN   AAAA 2001:db8:3414:6b1d::10

TLS Certificates

Since the webserver provding the auto-configuration data, will be accessed under the hosted domain name, it needs valid certificates for every hosted domain.

In a dehydrated domains.txt configuration file, this should looks as follows:

# ------------------------------------------------------------
# Mozilla Mail Client Autoconfiguration
# ------------------------------------------------------------
autoconfig.example.net autoconfig.example.org autoconfig.example.com > autoconfig

Nginx Server

The Nginx server configuration /etc/nginx/servers-available/autconfig.conf:

# ******************************************************************************
# autconfig.urown.net
# Mozilla Mail Client Auto-Configuration Server
# ******************************************************************************

# ---------------------------------------------------------
# Secured HTTPS Server
# ---------------------------------------------------------
server {

    server_name autconfig.*;

    # IPv6 public global address
    listen [2a02:168:f405::26]:443 ssl http2 deferred;

    # IPv4 private local address
    listen 172.20.10.26:443 ssl http2 deferred;

    # IPv4 private address (Port-forwarded from NAT firewall/router)
    listen 172.20.10.30:443 ssl http2;

    # TLS certificate (chained) and private key
    ssl_certificate /etc/dehydrated/certs/autconfig/fullchain.pem;
    ssl_certificate_key /etc/dehydrated/certs/autconfig/privkey.pem;

    # Enable stapling of online certificate status protocol (OCSP) response
    include ocsp-stapling.conf;

    # TLS certificate of signing CA (to validate OCSP response when stapling)
    ssl_trusted_certificate /etc/dehydrated/certs/autconfig/chain.pem;

    # OCSP stapling response file (pre-generated)
    ssl_stapling_file /etc/dehydrated/certs/autconfig/ocsp.der;

    # TLS session cache (type:name:size)
    ssl_session_cache shared:autconfig:10m;

    # Strict Transport Security (HSTS)
    include hsts.conf;

    # Expect Certificate Transparancy with valid Signed Certificate Timestamps (SCTs)
    include expect-ct.conf;

    # Common Server Settings
    include server-conf.d/*.conf;

    # Servers Public Documents Root
    root /var/www/urown.net/autconfig;

    # Logging
    error_log /var/log/nginx/autoconfig-error.log emerg;
    access_log off;

# ---------------------------------------------------------
# Unsecured HTTP Site
# ---------------------------------------------------------
server {

    server_name autconfig.*;

    # IPv6 public global address
    listen [2a02:168:f405::26]:80 deferred;

    # IPv4 private local address
    listen 172.20.10.26:80 deferred;

    # IPv4 private address (Port-forwarded from NAT firewall/router)
    listen 172.20.10.30:80;

    # Redirect to secure server
    return 301 https://$host$request_uri;
}

# -*- mode: nginx; indent-tabs-mode: nil; tab-width: 4; -*-

XML Client Configuration Data

The XML file structure and syntax is described in the Mozilla Wiki.

Provider information and domains:

 1<?xml version="1.0"?>
 2<clientConfig version="1.1">
 3
 4    <!-- Outgoing Server -->
 5    <emailProvider id="example.net">
 6
 7        <!-- Hosted Domains -->
 8        <domain>example.net</domain>
 9        <domain>example.org</domain>
10        <domain>example.com</domain>
11
12        <displayName>Your Own Mail Provider</displayName>
13        <displayShortName>example.net</displayShortName>

73        <documentation url="http://www.example.com/help/mail/thunderbird">
74            <descr lang="en">Configure Thunderbird 2.0 for IMAP</descr>
75            <descr lang="de">Thunderbird 2.0 mit IMAP konfigurieren</descr>
76        </documentation>
77
78    </emailProvider>
79
80    <clientConfigUpdate url="https://autoconfig.example.net/autoconfig/mail/config-v1.1.xml" />
81
82</clientConfig>

Servers for incoming mail:

15        <!-- Incoming Mail Servers -->
16        <incomingServer type="imap">
17            <hostname>mail.example.net</hostname>
18            <port>993</port>
19            <socketType>SSL</socketType>
20            <authentication>password-cleartext</authentication>
21            <username>%EMAILADDRESS%</username>
22        </incomingServer>
23
24        <incomingServer type="imap">
25            <hostname>mail.example.net</hostname>
26            <port>143</port>
27            <socketType>STARTTLS</socketType>
28            <authentication>password-cleartext</authentication>
29            <username>%EMAILADDRESS%</username>
30        </incomingServer>
31
32        <incomingServer type="pop3">
33            <hostname>mail.example.net</hostname>
34            <port>995</port>
35            <socketType>SSL</socketType>
36            <authentication>password-cleartext</authentication>
37            <username>%EMAILADDRESS%</username>
38            <pop3>
39                <leaveMessagesOnServer>true</leaveMessagesOnServer>
40            </pop3>
41        </incomingServer>
42
43        <incomingServer type="pop3">
44            <hostname>mail.example.net</hostname>
45            <port>110</port>
46            <socketType>STARTTLS</socketType>
47            <authentication>password-cleartext</authentication>
48            <username>%EMAILADDRESS%</username>
49            <pop3>
50                <leaveMessagesOnServer>true</leaveMessagesOnServer>
51            </pop3>
52        </incomingServer>

Servers for outgoing mail:

55        <!-- Outgoing Mail Servers -->
56        <outgoingServer type="smtp">
57            <hostname>mail.example.net</hostname>
58            <port>465</port>
59            <socketType>SSL</socketType>
60            <authentication>password-cleartext</authentication>
61            <username>%EMAILADDRESS%</username>
62        </outgoingServer>
63
64        <outgoingServer type="smtp">
65            <hostname>mail.example.net</hostname>
66            <port>587</port>
67            <socketType>STARTTLS</socketType>
68            <authentication>password-cleartext</authentication>
69            <username>%EMAILADDRESS%</username>
70        </outgoingServer>

Place the XML file in the designated location at /var/www/example.net/autoconfig/mail/config-v1.1.xml

Microsoft Clients

Microsoft mail clients like Outlook, Outlook Express or Office 365.

DNS Records

These clients forst look up a DNS service record, to locate any available web-server who might have configuration data in store for the users mail domain:

_autodiscover._tcp SRV 10 10 443 autodiscover.example.net

autodiscover.example.org CNAME autodiscover.example.net

@example.net
autodiscover  IN   A    10.195.171.241
autodiscover  IN   AAAA 2001:db8:3414:6b1d::10

TLS Certificates

Since the webserver provding the auto-configuration data, will be accessed under the hosted domain name, it needs valid certificates for every hosted domain.

In a dehydrated domains.txt configuration file, this should looks as follows:

# ------------------------------------------------------------
# Microsoft Mail Client Auto-Discovery
# ------------------------------------------------------------
autodiscover.example.net autodiscover.example.org autodiscover.example.com > autodiscover

Nginx Server

The Nginx server configuration /etc/nginx/servers-available/autodiscover.conf:

# ******************************************************************************
# autodiscover.urown.net
# Micorosdt Mail Client Auto-Discovery Server
# ******************************************************************************

# ---------------------------------------------------------
# Secured HTTPS Server
# ---------------------------------------------------------
server {

    server_name autodiscover.*;

    # IPv6 public global address
    listen [2a02:168:f405::26]:443 ssl http2 deferred;

    # IPv4 private local address
    listen 172.20.10.26:443 ssl http2 deferred;

    # IPv4 private address (Port-forwarded from NAT firewall/router)
    listen 172.20.10.30:443 ssl http2;

    # TLS certificate (chained) and private key
    ssl_certificate /etc/dehydrated/certs/autodiscover/fullchain.pem;
    ssl_certificate_key /etc/dehydrated/certs/autodiscover/privkey.pem;

    # Enable stapling of online certificate status protocol (OCSP) response
    include ocsp-stapling.conf;

    # TLS certificate of signing CA (to validate OCSP response when stapling)
    ssl_trusted_certificate /etc/dehydrated/certs/autodiscover/chain.pem;

    # OCSP stapling response file (pre-generated)
    ssl_stapling_file /etc/dehydrated/certs/autodiscover/ocsp.der;

    # TLS session cache (type:name:size)
    ssl_session_cache shared:autconfig:10m;

    # Strict Transport Security (HSTS)
    include hsts.conf;

    # Expect Certificate Transparancy with valid Signed Certificate Timestamps (SCTs)
    include expect-ct.conf;

    # Common Server Settings
    include server-conf.d/*.conf;

    # Servers Public Documents Root
    root /var/www/urown.net/autodiscover;

    # Logging
    error_log /var/log/nginx/autodiscover-error.log emerg;
    access_log off;

# ---------------------------------------------------------
# Unsecured HTTP Site
# ---------------------------------------------------------
server {

    server_name autodiscover.*;

    # IPv6 public global address
    listen [2a02:168:f405::26]:80 deferred;

    # IPv4 private local address
    listen 172.20.10.26:80 deferred;

    # IPv4 private address (Port-forwarded from NAT firewall/router)
    listen 172.20.10.30:80;

    # Redirect to secure server
    return 301 https://$host$request_uri;
}

# -*- mode: nginx; indent-tabs-mode: nil; tab-width: 4; -*-

XML Client Configuration Data

Provider information and domains:

 1<?xml version="1.0"?>
 2<Autodiscover xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 3    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 4    xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
 5    <Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">
 6        <Account>
 7            <AccountType>email</AccountType>
 8            <Action>settings</Action>
 9            <Protocol>
10                <Type>IMAP</Type>
11                <Server>mail.example.net</Server>
12                <Port>993</Port>
13                <DirectoryPort>0</DirectoryPort>
14                <ReferralPort>0</ReferralPort>
15                <SSL>on</SSL>
16                <LoginName>john@example.net</LoginName>
17                <SPA>off</SPA>
18                <AuthRequired>on</AuthRequired>
19            </Protocol>
20            <Protocol>
21                <Type>POP3</Type>
22                <Server>mail.example.net</Server>
23                <Port>995</Port>
24                <DirectoryPort>0</DirectoryPort>
25                <ReferralPort>0</ReferralPort>
26                <SSL>on</SSL>
27                <LoginName>john@example.net</LoginName>
28                <SPA>off</SPA>
29                <AuthRequired>on</AuthRequired>
30            </Protocol>
31            <Protocol>
32                <Type>SMTP</Type>
33                <Server>mail.example.net</Server>
34                <Port>587</Port>
35                <DirectoryPort>0</DirectoryPort>
36                <ReferralPort>0</ReferralPort>
37                <SSL>off</SSL>
38                <LoginName>john@example.net</LoginName>
39                <Encryption>TLS</Encryption>
40                <SPA>off</SPA>
41                <AuthRequired>on</AuthRequired>
42            </Protocol>
43        </Account>
44    </Response>
45</Autodiscover>

Apple Mail

Android K-9 Mail

Other Mail Clients

DNS SRV Records

In theory a method exists to specify servers hostnames and ports for a particular service to use:

_imap._tcp.         IN      SRV     10 10 143 mail.example.net.
_imaps._tcp.        IN      SRV     10 10 993 mail.example.net.
_pop3._tcp.         IN      SRV     10 10 110 mail.example.net.
_pop3s._tcp.        IN      SRV     10 10 995 mail.example.net.
_smtps._tcp.        IN      SRV     10 10 465 mail.example.net.
_submission._tcp.   IN      SRV     10 10 587 mail.example.net.

In practice no application is using them.

Common Host Names

Most clients will search and try (while some will not even try, but just set, servers with common hostnames):

imap         IN     CNAME   mail.example.net.
imaps        IN     CNAME   mail.example.net.
pop3         IN     CNAME   mail.example.net.
pop3s        IN     CNAME   mail.example.net.
smtp         IN     CNAME   mail.example.net.
smtps        IN     CNAME   mail.example.net.
submission   IN     CNAME   mail.example.net.

The downside to this is, that you need TLS certificates, not only for all these hostnames, but also all these hostnames in all domains, which have their mail sevices hosted here.

In a dehydrated domains.txt configuration file, this should will as follows:

# ------------------------------------------------------------
# Postfix Mail Server Certificates
# ------------------------------------------------------------
mail.example.net smtp.example.net smtp.example.org smtp.example.com smtps.example.net smtps.example.org smtps.example.com submission.example.net submission.example.org submission.example.com > postfix

# ------------------------------------------------------------
# Dovecot Mail Server Certificates
# ------------------------------------------------------------
mail.example.net imap.example.net imap.example.org imap.example.com imaps.example.net imaps.example.org imaps.example.com pop3.example.net pop3.example.org pop3.example.com pop3s.example.net pop3s.example.org pop3s.example.com > dovecot

The example above is for three domains only. For every addiotional domain, the number of hostnames who need to be certfied by your CA increases exponentially.

Testing

Microsoft

Other Projects

References

RFCs

  • RFC 6186 - “Use of SRV Records for Locating Email Client Services”

  • RFC 6764 - “Locating Services for CalDAV and CardDAV”

Mozilla Thunderbird

Mozilla Wiki:

Ben Bucksch (Moziila Dev):

Microsoft Outlook

Microsoft Support:

Microsoft Build:

Third-Party:

Apple

Apple Support:

Apple Developers:

Third-Party: