Skip to content

Email (SMTP, IMAP, POP3)

Overview

Email relies on multiple protocols working together: SMTP (Simple Mail Transfer Protocol) handles the sending and relaying of messages between servers, while IMAP (Internet Message Access Protocol) and POP3 (Post Office Protocol) allow clients to retrieve messages from a mailbox.

SMTP is a text-based protocol that uses MX (Mail Exchange) DNS records to route messages. Understanding the distinction between an MTA (Mail Transfer Agent) that relays mail between servers and an MDA (Mail Delivery Agent) that delivers to a local mailbox is fundamental. Email security is built additively on top of this basic text protocol — encryption, authentication, and spam filtering are all separate layers added over time.

How It Works

SMTP and Mail Transfer Agents (MTA)

Within the Internet email system, a Mail Transfer Agent (MTA) is software that transfers electronic mail messages from one computer to another using the Simple Mail Transfer Protocol (SMTP).

SMTP is a text-based protocol — messages are represented in UTF-8 text. Even files, images, or encrypted content are first converted to binary, then that binary is transformed into a textual representation (Base64 encoding) before being added to an email.

There are three parts to an SMTP message:

  1. Headers — metadata such as Subject, From, and To that define the sender, receiver, and other attributes.
  2. Body — the actual message content that you see when opening the email.
  3. Attachments (optional) — large blocks of Base64-encoded text representing binary files (images, documents, etc.).

All email security works additively on top of this core methodology. For example, encrypted emails encrypt the message, convert it to binary, convert that to a textual representation, and add it to the body. Reading an encrypted email with a client that doesn't support encryption shows a scrambled mess of characters.

SMTP Ports

Port Purpose
25 Primary SMTP relay port — used for server-to-server mail transfer. Not recommended for client submission.
465 Legacy secure SMTP (SMTPS) port. Should not be used for new deployments.
587 Default mail submission port. Uses authenticated submission and can be encrypted with STARTTLS.

Port 25 usually doesn't require authentication, which is acceptable for receiving mail from other servers. However, relaying mail with no authentication on this port would allow anyone to send emails through your server (becoming an open relay).

Port 587 is used by email clients to submit outgoing mail. It requires authentication and should always be secured with TLS.

IMAP — Mailbox Access

IMAP (Internet Message Access Protocol) is the standard for accessing messages in a remote email mailbox. Because SMTP is text-based, IMAP follows the same pattern — communication between client and server uses tagged text commands:

C: <open connection>
S:   * OK IMAP4rev1 Service Ready
C:   a001 login mrc secret
S:   a001 OK LOGIN completed
C:   a002 select inbox
S:   * 18 EXISTS
S:   * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S:   * 2 RECENT
S:   a002 OK [READ-WRITE] SELECT completed
C:   a003 fetch 12 full
S:   * 12 FETCH (FLAGS (\Seen) INTERNALDATE "17-Jul-1996 02:44:25 -0700" ...)
S:   a003 OK FETCH completed
C:   a004 logout
S:   * BYE IMAP4rev1 server terminating connection
S:   a004 OK LOGOUT completed

S: stands for server and C: stands for client. Each command is prefixed with a tag (e.g., a001, a002) used for session tracking, allowing the client to match responses to their corresponding requests.

IMAP Ports

Port Purpose
143 Standard, unsecured IMAP port.
993 Secure IMAP (IMAPS) — encrypted with TLS.

POP3 — Legacy Mailbox Access

POP3 (Post Office Protocol version 3) serves the same purpose as IMAP but is much older and lacks essential features. The key difference: POP3 downloads all emails to the client device and by default removes them from the server. IMAP, in contrast, synchronizes email between client and server, keeping messages on the server.

POP3 should be avoided for new deployments. Use IMAP instead.

MX Records and Mail Routing

Sending email between servers relies on MX (Mail Exchange) DNS records. An MX record defines which server is responsible for accepting email for a domain. Without an MX record, it is impossible to send emails to that domain.

MX records include a priority value — the lower the number, the higher the priority. If multiple MX servers are defined, the server with the lowest priority value receives mail first. If it's unavailable, the next server is tried.

Example zone file entry:

example.com.    IN  MX  10  mail.example.com.
example.com.    IN  MX  20  backup-mail.example.com.

The MX hostname must resolve to an A (or AAAA) record — an MX record cannot point directly to an IP address.

LMTP — Local Mail Transfer Protocol

In a typical mail server setup, LMTP (Local Mail Transfer Protocol) acts as the handoff mechanism between the MTA (e.g., Postfix) and the mail delivery/access system (e.g., Dovecot). Rather than having the MTA deliver directly to mailboxes, the MTA passes incoming mail through an LMTP socket to Dovecot, which handles final delivery to the correct user's mailbox.

This separation of concerns allows each component to focus on what it does best: the MTA handles relaying and queuing, while Dovecot handles mailbox storage and user access.

The Full Email Flow

A complete email journey involves these steps:

  1. Sender composes a message in their email client (webmail, Thunderbird, Outlook, etc.)
  2. Client submits the message via SMTP on port 587 (authenticated) to the sender's MTA
  3. Sender's MTA looks up the recipient domain's MX record via DNS
  4. Sender's MTA relays the message via SMTP on port 25 to the recipient's MTA
  5. Recipient's MTA accepts the message and passes it via LMTP to the MDA (e.g., Dovecot)
  6. MDA delivers the message to the recipient's mailbox on disk
  7. Recipient's client retrieves the message via IMAP (port 143/993)

Key Terminology

MTA (Mail Transfer Agent)
Software that transfers email between servers using SMTP. Examples: Postfix, Sendmail, Exim.
MDA (Mail Delivery Agent)
Software that delivers email to a user's local mailbox. Often integrated with the IMAP server.
MUA (Mail User Agent)
The email client used to read and compose messages. Examples: Thunderbird, Outlook, Roundcube.
SMTP Relay
Forwarding email from one mail server to another. An open relay (no authentication required) is a severe security misconfiguration that enables spam.
Maildir
A mailbox format where each message is stored as a separate file in a directory structure (new/, cur/, tmp/). Preferred over the older mbox format (single file per mailbox) for reliability and performance.
SASL
Simple Authentication and Security Layer — a framework for authentication in Internet protocols. Used by SMTP submission (port 587) to authenticate users before allowing them to send mail.

Why It Matters

As a system administrator, you will:

  • Set up and maintain mail servers that handle sending and receiving for your organization
  • Configure MX records in DNS to route mail to the correct servers
  • Manage the MTA (Postfix) and IMAP server (Dovecot) as separate but interconnected services
  • Secure mail traffic with TLS certificates on all relevant ports
  • Troubleshoot delivery issues using mail logs (/var/log/maillog)
  • Prevent your server from being an open relay or ending up on spam blacklists

Common Pitfalls

  1. Open relay — if your SMTP server accepts and forwards mail from unauthenticated external sources, spammers will abuse it and your server IP will be blacklisted.
  2. Missing or incorrect MX records — without a valid MX record, no other server can deliver mail to your domain.
  3. Forgetting to open firewall ports — SMTP (25, 587) and IMAP (143, 993) must be accessible through the firewall for mail to flow.
  4. Not separating port 25 from 587 — port 25 is for server-to-server relay (no auth); port 587 is for client submission (with auth). Mixing these up creates security holes.
  5. Plaintext authentication without TLS — sending passwords in cleartext over the network exposes credentials. Always require TLS before accepting authentication.
  6. Mailbox permission issues — Dovecot needs correct file permissions and group membership (mail group) to access and deliver to mailboxes.
  7. Not updating DNS serial — after adding MX records, forgetting to increment the zone serial means changes won't propagate.

Further Reading