openmail/ARCHITECTURE.md

OpenMail — Architecture

Agent-native, self-hosted mail server. One Go binary that embeds Mox's (MIT) mail internals for the correctness-critical 95% — DKIM, SPF/DMARC, DANE + MTA-STS delivery, real-world MIME, spam filtering — and owns a native, agent-shaped data model (Postgres + object store) with a REST API and MCP server on top. Mail I/O sits behind a pluggable MailBackend interface so OpenMail is useful on day one against a relay or an existing mailbox, while embedded-Mox self-host is the flagship, in-process path. This document is the design of record. It favors an MVP that actually receives and sends real mail over a perfect feature set.

Scope: OpenMail is a self-hostable app, not a SaaS — no billing/payments, no Stripe, no hosted control plane. Everything runs on the operator's hardware.

0. The decision: embed Mox, don't rebuild and don't just wrap it

An agent-mailbox API is ~5% of the work. The other 95% is mail infrastructure (inbound :25 hardening, outbound deliverability, MIME, threading, quote-stripping, anti-abuse). Three ways to get that 95%:

1. Build from scratch — re-implement DKIM/SPF/DMARC/DANE/MTA-STS/MIME ourselves. Correct but slow, and we'd do it worse than a production server at first.
2. Sidecar an existing server (run Mox as a separate process, read it over IMAP). Fast, but we don't own the data model — threads/labels/extracted_text would be re-derived from IMAP, and we ship two coupled processes.
3. Embed Mox's packages into our own binary and own the storage + API. ← chosen.

Why (3) is viable: Mox has no internal/ directory — every package is importable — and the README states "most non-server Go packages are written to be reusable." The real boundary is not visibility, it's storage coupling: Mox's server + store packages are welded to its embedded bstore/bbolt store and global config; its protocol/crypto/parsing packages are stateless and clean. We embed the latter and replace the former with our own Postgres-backed server.

What we embed from Mox vs. what we own

Concern	Source	Notes
MIME parsing	mox message	Battle-tested on real, broken senders. Better than enmime for our needs.
DKIM sign + verify	mox dkim
SPF / DMARC	mox spf, dmarc	Inbound auth verification + policy.
Secure outbound delivery	mox smtpclient, dane, mtasts, dns	The deliverability moat — DANE/MTA-STS MX delivery, mostly solved by reusing this.
Bounces (DSN)	mox dsn	Parse/generate delivery status notifications.
Anti-abuse	mox iprev, dnsbl, ratelimit	rDNS check, blocklists, rate limiting on :25.
Spam filtering	mox junk	Built-in Bayesian filter; per-inbox train/score.
SMTP auth	mox sasl, scram	For the submission (send) path.
—	—	—
Inbound SMTP server loop	emersion/go-smtp	Clean, embeddable listener/session. We do NOT import mox smtpserver (store-coupled).
Storage	OpenMail (Postgres + S3)	Native agent data model. NOT mox store/bbolt.
API, MCP, threading, extraction, webhooks, auth, multi-tenancy	OpenMail	The agent-native layer — our differentiation.

Boundary rule: never import smtpserver, imapserver, queue, store, or mox- (config) — they assume Mox's storage/config. Importing them is the line between "embed as a library" and "fork Mox." We stay on the library side.

0.1 Competitive positioning (why OpenMail exists)

The agent-mailbox concept is already taken; the implementation niche is open.

Project	Lang	License	How mail runs	Gap OpenMail fills
agenticmail (closest competitor)	TypeScript	MIT	TS orchestrator + Stalwart in a Docker sidecar, SQLite	Multi-container, AGPL mail engine, SQLite — vs. single Go binary, all-MIT, in-process, Postgres
cloudflare/agentic-inbox	TS	Apache-2	Locked to Cloudflare Workers + Email Routing	Vendor-neutral, runs anywhere
Mox / Maddy / Stalwart	Go/Rust	MIT/GPL/AGPL	Full mail servers	Not agent-native (no agent API/MCP, no threads/extract)

OpenMail's three differentiators, all downstream of the Go + embed-Mox choice:

1. Single static binary (vs. sidecar/multi-container).
2. Fully MIT stack — Mox is MIT, vs. agenticmail's AGPL Stalwart — so others can build commercial agents on top.
3. In-process mail engine — nobody else does this; it's only possible because Mox is importable Go.

0.2 The MailBackend interface (pluggable mail I/O)

All inbound delivery and outbound sending sit behind one interface, so the agent layer (API, MCP, store, threading) never depends on how mail moves. Three backends, shipped in order of effort:

// MailBackend abstracts where mail comes from and how it leaves. The core never
// knows which implementation is active.
type MailBackend interface {
    // Send dispatches an already-built, DKIM-signable RFC 5322 message.
    Send(ctx context.Context, msg *OutgoingMessage) (SendResult, error)
    // Start begins delivering inbound messages to the sink until ctx is cancelled.
    // (relay/imap: poll or webhook; embedded: go-smtp on :25.)
    Start(ctx context.Context, sink InboundSink) error
    Capabilities() Caps // self-host? inbound-push? custom-domain? throwaway-addrs?
}

Backend	Inbound	Outbound	Ops burden	Ships
relay	provider webhook / poll	SES / Postmark / Resend API (mox dkim sign)	lowest	v1 default
imap_smtp	IMAP IDLE on a BYO mailbox	SMTP submission to BYO server	low	v1
embedded (flagship)	go-smtp :25 + mox verify/parse/junk	mox smtpclient + dane + mtasts (MX)	highest	milestone 5

This makes the deliverability slog opt-in: a user gets a working agent mailbox immediately via relay or their existing mailbox, and graduates to fully self-hosted SMTP only when they want to own the whole stack. The embedded backend is the differentiator; the other two are the on-ramp.

1. Component overview

                         ┌──────────────────────────────────────────────────────┐
   Internet senders ───▶ │  smtpd  (go-smtp on :25)                              │
   (MX → this VPS)       │   iprev/dnsbl/ratelimit → SPF/DKIM/DMARC verify       │
                         │   → mox/message parse → mox/junk score → enqueue      │   ← mox pkgs
                         └───────────────┬──────────────────────────────────────┘
                                         │ internal queue (Postgres LISTEN/NOTIFY)
                         ┌───────────────▼──────────────────────────────────────┐
                         │  core  (Go)                                            │
   Agents / SDKs ──REST──▶   • inbox/message/thread/draft services               │
   MCP clients ───MCP──▶ │   • threading, extracted_text, labels, FTS search     │
                         │   • auth (API keys, pods), webhooks/WS                 │
                         └──────┬───────────────────────────────┬────────────────┘
                                │                                │
                   ┌────────────▼─────┐            ┌─────────────▼──────────────┐
                   │ Postgres         │            │ Object store (S3/MinIO)    │
                   │ metadata + FTS   │            │ raw .eml + attachments     │
                   └──────────────────┘            └────────────────────────────┘
                                         ▲
                         ┌───────────────┴──────────────────────────────────────┐
   Outbound to world ◀── │  sender  (mox/dkim sign → backend)                    │
                         │   self-host: mox/smtpclient + dane + mtasts (MX)      │   ← mox pkgs
                         │   relay:     SES / Postmark / Resend API              │
                         └──────────────────────────────────────────────────────┘

All roles (smtpd, core API, sender, mcp) ship in one binary with subcommands; deploy together or split later. Caddy terminates TLS for the HTTP API and auto-manages certs.

2. Tech stack

• Language: Go — single static binary, and the whole mature self-hosted-mail ecosystem (Mox, go-smtp, go-message) is Go, keeping us MIT-compatible. Rust (Stalwart) is AGPL → ruled out.
• HTTP: chi router; sqlc + pgx for typed Postgres access.
• Inbound server: emersion/go-smtp listener → handed to mox packages for verify/parse/score.
• Mail correctness/delivery: mox message, dkim, spf, dmarc, dane, mtasts, dns, smtpclient, dsn, junk, iprev, dnsbl, ratelimit, sasl, scram (see §0 table).
• Quote stripping: a talon/EmailReplyParser-style heuristic pass for extracted_text.
• Storage: Postgres (metadata + tsvector FTS); S3-compatible object store (MinIO self-host, or S3) for raw .eml and attachment blobs.
• Events: Postgres LISTEN/NOTIFY for the internal inbound queue and to fan out webhooks + a WebSocket stream. (Swap to NATS only if volume warrants.)
• Deploy: docker compose — caddy, openmail (one image, multiple commands), postgres, minio. MX + SPF/DKIM/DMARC/DANE/MTA-STS DNS documented in deploy/.

3. Data model (Postgres)

Native and agent-shaped (not Mox's per-account bbolt index). Resource shapes mirror what agent tooling expects so SDKs/MCP map cleanly.

pods          (id, name, created_at)                       -- tenant isolation
inboxes       (id, pod_id, address UNIQUE, display_name, dkim_selector, metadata jsonb, created_at, updated_at)
threads       (id, inbox_id, subject, last_message_id, message_count, labels text[], updated_at, created_at)
messages      (id, inbox_id, thread_id, message_id_hdr, in_reply_to, references text[],
               from_addr, to_addrs text[], cc text[], bcc text[], subject,
               preview, text, html, extracted_text, extracted_html,
               raw_object_key,            -- pointer to raw .eml in object store
               spf, dkim, dmarc,          -- inbound auth verdicts (from mox pkgs)
               junk_score, labels text[], size_bytes, headers jsonb, ts tsvector,  -- FTS
               created_at, updated_at)
attachments   (id, message_id, filename, content_type, size_bytes, object_key, inline bool, content_id)
drafts        (id, inbox_id, thread_id, to_addrs text[], cc, bcc, subject, text, html, send_at, client_id)
api_keys      (id, pod_id, hash, scopes text[], created_at)   -- bearer tokens; store only the hash
webhooks      (id, pod_id, url, event_types text[], secret)   -- HMAC-signed deliveries
outbox        (id, message_id, status, attempts, last_error, next_attempt_at)  -- send queue + retries
events        (id, inbox_id, type, payload jsonb, created_at) -- audit + WS replay
domains       (id, pod_id, name, dkim_privkey_ref, verified bool)  -- per-domain DKIM keys + DNS state

Indexes: inboxes(address), messages(thread_id), messages(inbox_id, created_at desc), GIN on messages.ts (search) and *.labels. Raw MIME + attachments live in object storage; rows hold pointers.

4. Request / data flows

Inbound (receive)

1. go-smtp accepts on :25; anti-abuse gate via mox iprev/dnsbl/ratelimit + size cap; raw .eml written to object store.
2. mox spf/dkim/dmarc verify the message; verdicts recorded. mox message builds the MIME tree → text/html, attachments, headers. mox junk scores spam. Quote-stripper computes extracted_text + preview.
3. Threading resolves thread_id from In-Reply-To/References, falling back to normalized-subject
   • participants within a window.
4. Row inserted; LISTEN/NOTIFY fires → message.received webhook + WebSocket event.

Outbound (send / reply)

1. API builds the RFC 5322 message (sets In-Reply-To/References on replies); persists a messages row + outbox entry.
2. sender mox-dkim-signs, then dispatches via the configured backend:
   • self-host: mox smtpclient + dane + mtasts for authenticated, secure MX delivery.
   • relay: SES/Postmark/Resend API.
3. Bounces/complaints (mox dsn parse, or relay webhook) update outbox.status and emit events.

API surface (v1 — AgentMail-shaped)

POST   /v0/inboxes                         create inbox
GET    /v0/inboxes  /  /v0/inboxes/{id}    list / get
POST   /v0/inboxes/{id}/messages/send      send
GET    /v0/inboxes/{id}/messages           list (limit, page_token, labels)
GET    /v0/inboxes/{id}/messages/{id}      get
POST   /v0/inboxes/{id}/messages/{id}/reply
GET    /v0/inboxes/{id}/threads  /  /{id}  list / get
…drafts, webhooks, search…

Bearer auth (Authorization: Bearer). Keeping paths/shapes close to AgentMail's public v0 lets an existing client (or MCP server) target a self-hosted OpenMail by base-URL swap. Compatibility yields to a cleaner native shape where they conflict.

MCP server

A thin MCP front-end over core: tools create_inbox, list_messages, get_thread, send_message, reply, search — so an agent drives its own mailbox directly.

5. Deliverability plan (the moat — and why embedding Mox helps most here)

• Self-host send is achievable from day one for the protocol parts because mox smtpclient + dane + mtasts + dkim already implement authenticated, DANE/MTA-STS-secured MX delivery. What remains is operational, not code: dedicated IP with PTR/rDNS, SPF (v=spf1 ip4:… -all), published DKIM key, DMARC (p=quarantine→p=reject), gradual IP warmup, FBL enrollment, suppression lists.
• Relay backend (SES/Postmark/Resend) ships alongside for instant inbox placement while a self-host IP warms up — chosen per-domain.
• Inbound auth: SPF/DKIM/DMARC verified on receipt (mox pkgs); verdicts exposed in message metadata; optional reject/quarantine policy per inbox.

6. MVP milestones

Backend-first ordering: get an end-to-end loop on the lowest-ops backend, then add the differentiator.

1. Core API + storage — inboxes/messages/threads/drafts CRUD on Postgres + MinIO; bearer auth; ingest endpoint to seed without real mail. Proves the data model + API. MailBackend interface defined; a null/ingest backend satisfies it.
2. Relay backend (loop closed) — imap_smtp and/or relay backend: inbound via IMAP IDLE or provider webhook → mox message parse + threading + message.received; outbound via SMTP submission / relay API, mox-DKIM-signed. Proves receive+reply against real mail with near-zero ops.
3. MCP server — agent owns and operates an inbox end-to-end (tools over the core).
4. Embedded inbound — go-smtp on :25 + mox verify/parse/junk + anti-abuse; MX a test domain. Proves OpenMail can receive directly.
5. Embedded outbound + deliverability — mox smtpclient/dane/mtasts; the flagship self-host path and the IP-warmup/reputation long tail.

7. Repo layout (planned)

cmd/openmail/         # single binary: `serve`, `smtpd`, `sender`, `mcp` subcommands
internal/core/        # inbox/message/thread/draft services
internal/mail/        # MailBackend interface + shared parse/thread/DKIM helpers (mox pkgs)
internal/mail/relay/    # relay backend  (SES/Postmark/Resend)
internal/mail/imapsmtp/ # imap_smtp backend (BYO mailbox: IMAP IDLE in, SMTP submit out)
internal/mail/embedded/ # embedded backend (go-smtp :25 in; mox smtpclient+dane+mtasts out)
internal/store/             # pgx/sqlc queries, object-store client, embedded migrations
internal/store/migrations/  # SQL migrations (go:embed'd into the binary)
internal/api/         # chi HTTP handlers (v0 surface)
internal/mcp/         # MCP server over core
deploy/               # docker-compose.yml, Caddyfile, DNS (MX/SPF/DKIM/DMARC/DANE/MTA-STS) notes
spike/                # throwaway feasibility spikes (mimecheck: mox standalone parse — PASSED)

8. Licensing & attribution

OpenMail is MIT (see LICENSE). Every dependency is permissive: Mox (MIT), emersion/go-smtp and go-message (MIT), pgx/chi (MIT/BSD). We import Mox as a Go module (no vendored source in this repo), but credit it prominently in the README; if we ever vendor Mox source we must retain its MIT notice. No GPL/AGPL code anywhere — Maddy (GPL-3.0) and Stalwart (AGPL) are deliberately avoided so the project stays MIT.

9. Open questions

• mox message ⇄ go-smtp seam: confirm mox's parser consumes a raw reader cleanly outside mox's store (spike in milestone 2). If any Tier-1 package drags in mox-/store transitively, isolate behind a thin adapter or vendor just that file.
• How far to chase AgentMail API compatibility vs. a cleaner native shape.
• Search: Postgres FTS is enough for v1; revisit (Meilisearch/Typesense) only if needed.
• Multi-tenancy depth — pods now, full RBAC later.
• Whether to expose an optional IMAP read path (via mox imapserver over our store would mean forking it — more likely a thin standalone IMAP front-end later, kept out of v1).