openmail/ARCHITECTURE.md

OpenMail — Architecture

Self-hosted, AI-native mailbox for agents. One Go service, Postgres, and object storage on a VPS, fronted by Caddy for TLS. This document is the design of record; it favors an MVP that actually receives and sends real mail over a perfect feature set.

0. The honest framing

An agent-mailbox API is ~5% of the work. The other 95% is mail infrastructure:

Hard part	Why it's hard
Inbound	You must run an SMTP server on port 25 with MX records pointing at it, accept from arbitrary senders, survive spam/abuse, and parse messy real-world MIME.
Outbound deliverability	Anyone can send SMTP; getting it to the inbox needs SPF + DKIM signing + DMARC alignment, PTR/rDNS, warmed IP reputation, feedback loops (FBLs), and bounce/complaint handling. This is the real moat.
MIME parsing	Multipart trees, encodings, inline vs attachment, calendar parts, broken senders.
Threading	Stitching messages into conversations from Message-ID / In-Reply-To / References plus subject heuristics.
Quoted-history stripping	The extracted_text feature (reply content without the quoted thread) is its own parsing problem.

OpenMail is structured so each of these is an isolated, swappable component, and so v1 can lean on a relay for deliverability while the self-hosted SMTP path matures.

1. Component overview

                         ┌──────────────────────────────────────────────┐
   Internet senders ───▶ │  smtpd  (inbound, port 25, go-smtp)           │
   (MX → this VPS)       │   → parse MIME → normalize → enqueue          │
                         └───────────────┬──────────────────────────────┘
                                         │ internal queue (Postgres LISTEN/NOTIFY or NATS)
                         ┌───────────────▼──────────────────────────────┐
                         │  core  (Go)                                   │
   Agents / SDKs ──REST──▶   • inbox/message/thread/draft services       │
   MCP clients ───MCP──▶ │   • threading, extraction, labels, search     │
                         │   • auth (API keys, pods), webhooks/WS        │
                         └──────┬───────────────────────┬───────────────┘
                                │                        │
                   ┌────────────▼─────┐      ┌───────────▼───────────────┐
                   │ Postgres         │      │ Object store (S3/MinIO)    │
                   │ metadata + FTS   │      │ raw MIME + attachments     │
                   └──────────────────┘      └────────────────────────────┘
                                         ▲
                         ┌───────────────┴──────────────────────────────┐
   Outbound to world ◀── │  sender  (DKIM-sign → self-host SMTP OR relay)│
                         └──────────────────────────────────────────────┘

All four roles (smtpd, core API, sender, MCP) ship in one Go binary with subcommands; deploy together or split later. Caddy terminates TLS for the HTTP API and auto-manages certs.

2. Tech stack

• Language: Go (single static binary; matches the deployment-simplicity goal and OpenMail's systems nature).
• HTTP: chi router; sqlc + pgx for typed Postgres access.
• Inbound SMTP: emersion/go-smtp (server) — listens on :25, hands raw bytes to the parser.
• MIME: emersion/go-message (+ enmime for lenient real-world parsing); a quote-stripper (port of talon/EmailReplyParser heuristics) for extracted_text/extracted_html.
• DKIM: emersion/go-msgauth/dkim for signing (outbound) and verifying (inbound).
• Outbound: pluggable Sender interface — smtp (self-host, MX delivery) or relay (SES / Postmark / Resend API). Default to relay in v1 for deliverability; self-host is opt-in.
• Storage: Postgres (metadata + full-text search via tsvector); S3-compatible object store (MinIO self-hosted, or AWS S3) for raw .eml and attachment blobs.
• Events: Postgres LISTEN/NOTIFY for the internal inbound queue and for fanning out webhooks + a WebSocket stream. (Swap to NATS if volume warrants.)
• Deploy: docker compose — caddy, openmail (one image, multiple commands), postgres, minio. MX + SPF/DKIM/DMARC DNS documented in deploy/.

3. Data model (Postgres)

Mirrors the resource shapes agent tooling already expects, so SDKs/MCP map cleanly.

pods          (id, name, created_at)                       -- tenant isolation
inboxes       (id, pod_id, address UNIQUE, display_name, client_id, metadata jsonb, created_at, updated_at)
threads       (id, inbox_id, subject, last_message_id, message_count, labels text[], updated_at, created_at)
messages      (id, inbox_id, thread_id, message_id_hdr, in_reply_to, references text[],
               from_addr, to_addrs text[], cc text[], bcc text[], subject,
               preview, text, html, extracted_text, extracted_html,
               raw_object_key,            -- pointer to raw .eml in object store
               labels text[], size_bytes, headers jsonb, ts tsvector,  -- FTS
               created_at, updated_at)
attachments   (id, message_id, filename, content_type, size_bytes, object_key, inline bool, content_id)
drafts        (id, inbox_id, thread_id, to_addrs text[], cc, bcc, subject, text, html, send_at, client_id)
api_keys      (id, pod_id, hash, scopes text[], created_at)   -- bearer tokens, store only the hash
webhooks      (id, pod_id, url, event_types text[], secret)   -- HMAC-signed deliveries
outbox        (id, message_id, status, attempts, last_error, next_attempt_at)  -- send queue + retries
events        (id, inbox_id, type, payload jsonb, created_at) -- audit + WS replay

Indexes: inboxes(address), messages(thread_id), messages(inbox_id, created_at desc), GIN on messages.ts (search) and *.labels. Raw MIME and attachments live in object storage, not the DB — rows hold pointers.

4. Request / data flows

Inbound (receive)

1. smtpd accepts on :25, enforces basic anti-abuse (rate limit, SPF check, size cap), and writes the raw .eml to the object store.
2. Parser builds the MIME tree → extracts text/html, attachments, headers; computes extracted_text (strip quoted history) and a preview.
3. Threading resolves thread_id from In-Reply-To/References → fallback to normalized-subject + participants within a window.
4. Row inserted; LISTEN/NOTIFY fires → webhook + WebSocket message.received event.

Outbound (send / reply)

1. API builds the RFC 5322 message (sets In-Reply-To/References for replies), persists a messages row + outbox entry.
2. sender DKIM-signs and dispatches via the configured backend (self-host SMTP or relay API).
3. Bounces/complaints (SMTP DSN, or relay webhook) update outbox.status and emit events.

API surface (v1 — AgentMail-shaped)

POST   /v0/inboxes                         create inbox
GET    /v0/inboxes / /v0/inboxes/{id}      list / get
POST   /v0/inboxes/{id}/messages/send      send
GET    /v0/inboxes/{id}/messages           list (limit, page_token, labels)
GET    /v0/inboxes/{id}/messages/{id}      get
POST   /v0/inboxes/{id}/messages/{id}/reply
GET    /v0/inboxes/{id}/threads /{id}      list / get
…drafts, webhooks, search…

Bearer auth (Authorization: Bearer). Keeping paths/shapes close to AgentMail's public v0 means an existing Go/Python/Node client (or MCP server) can target a self-hosted OpenMail by base-URL swap.

MCP server

A thin MCP front-end over the same core: tools create_inbox, list_messages, get_thread, send_message, reply, search — so an agent (Claude Code, etc.) drives its own mailbox directly.

5. Deliverability plan (the part that actually matters)

• v1: relay-by-default. Send through SES/Postmark/Resend for inbox placement on day one; OpenMail still owns inbound, storage, threading, API. Lowest-risk way to be useful immediately.
• Self-host SMTP send (opt-in): dedicated IP with PTR/rDNS, SPF (v=spf1 ip4:… -all), DKIM signing (publish the public key), DMARC (p=quarantine → p=reject), gradual IP warmup, FBL enrollment, and bounce/complaint suppression lists. Documented honestly as a multi-week reputation effort, not a flag flip.
• Inbound auth: verify SPF/DKIM/DMARC on receipt; expose pass/fail in message metadata; optional reject/quarantine policy per inbox.

6. MVP milestones

1. Core API + storage — inboxes/messages/threads/drafts CRUD against Postgres + MinIO; bearer auth; no real mail yet (seed via an ingest endpoint). Proves the data model + API.
2. Inbound — go-smtp on :25, MIME parse, extraction, threading, message.received webhook. MX a test domain at the VPS. Proves receive.
3. Outbound via relay — send/reply through SES/Postmark with DKIM. Proves the loop: an agent receives and replies.
4. MCP server — agent owns and operates an inbox end-to-end.
5. Self-host SMTP send + deliverability hardening — the long tail.

7. Repo layout (planned)

cmd/openmail/         # single binary: `serve`, `smtpd`, `sender`, `mcp` subcommands
internal/core/        # inbox/message/thread/draft services
internal/smtp/        # inbound server + MIME parse + extraction + threading
internal/sender/      # Sender interface + smtp & relay backends + DKIM
internal/store/       # sqlc queries, object-store client
internal/api/         # chi HTTP handlers (v0 surface)
internal/mcp/         # MCP server over core
deploy/               # docker-compose.yml, Caddyfile, DNS (MX/SPF/DKIM/DMARC) notes
db/migrations/        # SQL migrations

8. Open questions

• Relay vs self-host SMTP as the default — relay is pragmatic, self-host is the point. Likely ship relay default + self-host documented.
• How far to chase AgentMail API compatibility vs a cleaner native shape.
• Search: Postgres FTS is enough for v1; revisit (Meilisearch/Typesense) only if needed.
• Multi-tenancy depth — pods now, full RBAC later.