Rewrite as a Rust, Apache-2.0 workspace
Supersedes the Go + embed-Mox design. The Go tree is removed; its
architecture doc is preserved at docs/archive/ARCHITECTURE-go-embed-mox.md
because its competitive analysis and data model still hold.
Five decisions recorded as ADRs:
0001 Rust, not Go — accepting ~5,500 lines of protocol code that Mox
would have given us free, to get the first permissively licensed
Rust mail server. Costs stated plainly.
0002 Apache-2.0, not MIT or AGPL — patent grant, trademark, CLA-free
contribution. Public on GitHub; Gitea stays as the private fallback.
0003 Stalwart's primitive crates (Apache-2.0/MIT) yes; its AGPL server
crates never. DANE and MTA-STS sit on the AGPL side of that line,
which is why we write our own.
0004 Milestones, reordered: embedded inbound is required at launch.
0005 Oracle Cloud blocks outbound :25, so direct-to-MX is impossible on
the launch host. Split delivery is mandatory, not an on-ramp.
Twelve crates in three tiers. Tier 1 (mail-dane, mail-mta-sts, mail-dsn)
is standalone and publishable — no `dane` or `mta-sts` crate exists on
crates.io at all today.
openmail-relay ships the provider table as data, with SES and Oracle from
the start. Oracle's and Resend's SPF includes are deliberately None: a
guessed include turns the DNS check green against a mechanism the provider
does not honour, and mail still fails SPF silently.
cargo check/test/clippy/fmt all green; unsafe_code is forbidden workspace
wide; cargo-deny enforces the licence policy in CI.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JkyvfNJGTshJNE9FtwPLk7
This commit is contained in:
Karti Tripathi
co-authored by
Claude Opus 5
Claude Opus 5
parent
428040d964
commit
36b15ddcaf
@@ -0,0 +1,68 @@
|
||||
# ADR 0001 — Rust, not Go
|
||||
|
||||
**Status:** Accepted, 2026-09-02. Supersedes the Go + embed-Mox design in
|
||||
[`../archive/ARCHITECTURE-go-embed-mox.md`](../archive/ARCHITECTURE-go-embed-mox.md).
|
||||
|
||||
## Context
|
||||
|
||||
The archived design chose Go in order to embed Mox (MIT) as a library, getting
|
||||
~14,000 lines of production-tested mail correctness for free:
|
||||
|
||||
```
|
||||
message 2,884 dkim 2,051 spf 1,560 smtpclient 2,012 junk 1,302
|
||||
mtasts 703 dsn 771 dane 516 scram 796 sasl 327 …
|
||||
```
|
||||
|
||||
The Rust ecosystem does not offer an equivalent. Stalwart is the only
|
||||
production Rust mail server and its server crates are `AGPL-3.0-only OR
|
||||
LicenseRef-SEL`, which we cannot use under Apache-2.0.
|
||||
|
||||
## What Rust actually costs
|
||||
|
||||
Stalwart Labs publishes its *primitives* permissively (Apache-2.0 OR MIT), and
|
||||
those cover more than expected: `mail-parser` (MIME), `mail-auth` (DKIM1,
|
||||
**DKIM2**, ARC, SPF, DMARC, ARF, TLS-RPT), `mail-builder`, `mail-send`,
|
||||
`smtp-proto`. `hickory-resolver` covers DNS and DNSSEC.
|
||||
|
||||
What no permissive Rust crate provides, and we therefore write:
|
||||
|
||||
| | LOC (Mox equivalent) | Rust prior art |
|
||||
|---|---|---|
|
||||
| DANE | ~516 | **none on crates.io** |
|
||||
| MTA-STS | ~703 | **none on crates.io** |
|
||||
| SMTP server session loop | ~3,395 (`go-smtp`) | `smtp-proto` parses only |
|
||||
| DSN | ~771 | none |
|
||||
| Junk (beyond a toy) | ~1,302 | `bayespam` has no training persistence |
|
||||
| iprev / DNSBL / rate limit | ~370 | `dnsbl` crate abandoned since 2021 |
|
||||
|
||||
**~5,500 lines of adversarial protocol code**, versus zero in Go.
|
||||
|
||||
## Decision
|
||||
|
||||
**Rust.** Accepted with eyes open.
|
||||
|
||||
## Consequences
|
||||
|
||||
Negative, and stated plainly so nobody is surprised later:
|
||||
|
||||
- v1 is roughly a quarter further out.
|
||||
- DANE and MTA-STS move from *battle-tested* to *ours*, and both **fail
|
||||
silently**: a DANE bug downgrades TLS without erroring; an MTA-STS bug defers
|
||||
mail nobody sees. That tail does not close at ship — it closes after enough
|
||||
strangers' mail has flowed through it.
|
||||
- Mitigation: every outcome in those crates is an explicit enum with no
|
||||
`Default` and no `bool`, so a caller cannot accidentally read "no policy" as
|
||||
"verified". See `mail_dane::DaneResult`.
|
||||
|
||||
Positive:
|
||||
|
||||
- We ship the first permissively licensed DANE and MTA-STS in Rust, and the
|
||||
first permissively licensed Rust mail server.
|
||||
- `mail-auth` gives us DKIM2 and ARC, which Mox does not have.
|
||||
- One language for the mail engine and the agent layer.
|
||||
|
||||
## Rejected alternative
|
||||
|
||||
**Go now, Rust later**, with the two crates published early to plant the flag
|
||||
at low cost. Rejected: it puts the strategic position — "the permissive Rust
|
||||
agent mail server" — behind a rewrite that would probably never be scheduled.
|
||||
@@ -0,0 +1,46 @@
|
||||
# ADR 0002 — Apache-2.0
|
||||
|
||||
**Status:** Accepted, 2026-09-02. Supersedes the MIT choice in the archived
|
||||
architecture (§8).
|
||||
|
||||
## Decision
|
||||
|
||||
**Apache-2.0**, and the repository is public on GitHub
|
||||
(`karti-ai/openmail`). The Gitea mirror (`OSS/openmail`) stays as the fallback
|
||||
if the project ever needs to go private.
|
||||
|
||||
## Why not MIT
|
||||
|
||||
Same freedoms, but Apache-2.0 adds three things that matter here:
|
||||
|
||||
- **§3, express patent grant.** Email authentication is a standards thicket —
|
||||
DKIM, DKIM2, ARC, DMARC. MIT's patent grant is implicit at best, and that is
|
||||
what enterprise legal review flags.
|
||||
- **§6, trademark reservation.** "OpenMail" is a generic name with at least
|
||||
three unrelated projects already using it. Apache-2.0 protects the name while
|
||||
the code stays free.
|
||||
- **§5, contributor terms.** Inbound contributions are licensed on the same
|
||||
terms without a separate CLA.
|
||||
|
||||
Inbound compatibility is clean: every dependency is Apache-2.0 or MIT.
|
||||
|
||||
## Why not AGPL
|
||||
|
||||
AGPL + a commercial exception is the standard way to protect a future hosted
|
||||
offering — it is exactly what Stalwart does (`AGPL-3.0-only OR
|
||||
LicenseRef-SEL`). We reject it because it makes us unusable by the commercial
|
||||
agent builders who are the intended audience, and because being *the*
|
||||
permissive option is the entire competitive position. Stalwart's AGPL is the
|
||||
reason its competitors must run it in a sidecar; we do not want to be that for
|
||||
someone else.
|
||||
|
||||
## Consequences
|
||||
|
||||
- "No GPL/AGPL/LGPL anywhere" remains policy, but the *reason* changed. Under
|
||||
MIT it was a compatibility fact; under Apache-2.0 it is a deliberate choice,
|
||||
since Apache-2.0 is one-way-incompatible with GPL-2-only. Enforced in CI by
|
||||
`cargo-deny`.
|
||||
- Every source file gets no licence header (the `LICENSE` + `NOTICE` pair is
|
||||
sufficient and headers rot); `NOTICE` must be shipped with any redistribution
|
||||
and lists third-party attribution.
|
||||
- Anyone may fork this closed. That is the intent, not a leak.
|
||||
@@ -0,0 +1,34 @@
|
||||
# ADR 0003 — Write our own crates; use Stalwart's primitives, never its server
|
||||
|
||||
**Status:** Accepted, 2026-09-02.
|
||||
|
||||
## The licence boundary
|
||||
|
||||
Stalwart Labs ships two distinct things:
|
||||
|
||||
| | Licence | Us |
|
||||
|---|---|---|
|
||||
| The **server** (`stalwartlabs/stalwart`, `crates/*`) | `AGPL-3.0-only OR LicenseRef-SEL` | ❌ never |
|
||||
| The **primitives** (`mail-parser`, `mail-auth`, `mail-builder`, `mail-send`, `smtp-proto`) | `Apache-2.0 OR MIT` | ✅ dependencies |
|
||||
|
||||
DANE and MTA-STS live in `crates/smtp` and `crates/common` — **on the AGPL side
|
||||
of that line.** That is precisely why we write our own.
|
||||
|
||||
## Rule
|
||||
|
||||
- Depending on the permissive primitive crates is fine and intended.
|
||||
- Reading the AGPL server crates for *understanding* is fine.
|
||||
- Copying, adapting or transliterating any line from them is **not**, and would
|
||||
contaminate the whole workspace. When implementing DANE or MTA-STS, work from
|
||||
the RFCs (7672, 8461, 6698) — not from `stalwart/crates/smtp`.
|
||||
- The research mirror at `~/Desktop/ProjectMail/mail-servers/stalwart` is
|
||||
read-only reference. Same for `maddy` (GPL-3) and `BillionMail` (AGPL).
|
||||
|
||||
## Which of our crates get published
|
||||
|
||||
Tier 1 (`mail-dane`, `mail-mta-sts`, `mail-dsn`) are published standalone: they
|
||||
depend on nothing in this workspace, they fill real holes in the ecosystem, and
|
||||
their value to us is partly that other projects audit them. Names verified
|
||||
available on crates.io 2026-09-02, as is `openmail` itself — reserve early.
|
||||
|
||||
Tier 2 and 3 stay in-workspace until their APIs settle.
|
||||
@@ -0,0 +1,51 @@
|
||||
# ADR 0004 — Milestones
|
||||
|
||||
**Status:** Accepted, 2026-09-02.
|
||||
|
||||
Reordered from the archived Go plan. The original put embedded inbound at
|
||||
milestone 4 and treated it as optional-until-later; ADR 0005 makes it
|
||||
**required at launch**, because the launch host cannot send direct-to-MX at all.
|
||||
|
||||
## v0.1 — the agent layer, provable without mail
|
||||
|
||||
Nothing here needs a working mail server, which is the point: it is all
|
||||
testable in CI.
|
||||
|
||||
- [x] Apache-2.0 workspace, 12 crates, `cargo check` green
|
||||
- [x] `openmail-relay` provider table (SES, Oracle, SendGrid, Postmark, Resend, custom)
|
||||
- [ ] `openmail-core::extract::strip_quoted` — the first real algorithm
|
||||
- [ ] `openmail-core::thread` resolution, both bases
|
||||
- [ ] `openmail-store` — Postgres schema, embedded migrations, S3 blobs
|
||||
- [ ] `openmail-api` — v0 REST, bearer auth
|
||||
- [ ] Ingest endpoint: POST a raw `.eml` and get a threaded, extracted message
|
||||
back. Closes the loop with **zero mail infrastructure.**
|
||||
|
||||
## v0.2 — receive
|
||||
|
||||
- [ ] `openmail-smtpd` on :25, real MX for a test domain
|
||||
- [ ] `openmail-guard` gate, `mail-auth` SPF/DKIM/DMARC verdicts recorded
|
||||
- [ ] `openmail-junk` scoring
|
||||
- [ ] `message.received` webhooks + WebSocket
|
||||
|
||||
## v0.3 — send, and the MCP surface
|
||||
|
||||
- [ ] Relay send via SES and Oracle, DKIM-signed locally
|
||||
- [ ] `mail-dsn` bounce handling wired to outbox state
|
||||
- [ ] `openmail-mcp` — an agent creates an inbox, receives, and replies, alone
|
||||
|
||||
## v0.4 — direct-to-MX, and the crates ship
|
||||
|
||||
- [ ] `mail-dane` and `mail-mta-sts` complete, **published to crates.io**
|
||||
- [ ] Direct MX delivery with both enforced
|
||||
- [ ] IP warmup, FBL enrolment, suppression lists
|
||||
|
||||
## v1.0
|
||||
|
||||
- [ ] IMAP front-end so humans use their own client against the same mailbox
|
||||
- [ ] Multi-tenancy beyond `pods`
|
||||
- [ ] Deliverability track record worth publishing
|
||||
|
||||
## Not in v1
|
||||
|
||||
A hosted SaaS, billing, or a webmail UI. `pods` exists so the SaaS path stays
|
||||
open architecturally — do not remove it as unused.
|
||||
@@ -0,0 +1,44 @@
|
||||
# ADR 0005 — Oracle Cloud as the launch host, and what it forbids
|
||||
|
||||
**Status:** Accepted, 2026-09-02.
|
||||
|
||||
## The constraint
|
||||
|
||||
**OCI blocks outbound TCP/25 for every tenancy created after 2021-06-23.**
|
||||
Exemption is a service-limit request, routinely refused for free tier. Inbound
|
||||
:25 is *not* blocked.
|
||||
|
||||
| | On OCI |
|
||||
|---|---|
|
||||
| Receive on :25 | ✅ works — `openmail-smtpd` is fine |
|
||||
| Relay out on 587 | ✅ works (verify: see below) |
|
||||
| Direct-to-MX | ❌ **impossible.** `mail-dane` and `mail-mta-sts` can never run there |
|
||||
|
||||
## Decision
|
||||
|
||||
Launch on OCI in **split delivery**: receive directly, relay outbound. Support
|
||||
Oracle Cloud Email Delivery *and* SES as relay providers from day one — two
|
||||
providers at launch forces the provider abstraction to be genuinely
|
||||
data-driven instead of an SES-shaped `if`.
|
||||
|
||||
Direct-to-MX (v0.4) is developed and tested on a host without the block.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Embedded inbound moves from "milestone 4, later" to **required at launch**.
|
||||
- The relay path is not a temporary on-ramp; on our own launch host it is the
|
||||
only outbound path that exists.
|
||||
- Oracle's SPF include is region-scoped (`rp` / `eu.rp` / `ap.rp`
|
||||
`.oracleemaildelivery.com`), so `providers.rs` deliberately stores `None` and
|
||||
makes the operator paste theirs. A guessed include turns the DNS check green
|
||||
against a mechanism the provider does not honour and mail still fails SPF —
|
||||
silently. Same for Resend.
|
||||
|
||||
## ⚠️ Open — verify before committing to the host
|
||||
|
||||
Oracle's docs name only port 25. Mailcow community reports claim **587 and 2525
|
||||
are also blocked outbound on free tier**, which would make OCI unable to relay
|
||||
either, and would change the host choice entirely.
|
||||
|
||||
**Test on a free instance before building on this.** It is a 20-minute check
|
||||
and it invalidates this ADR if the reports are right.
|
||||
@@ -0,0 +1,273 @@
|
||||
# OpenMail — Architecture
|
||||
|
||||
Agent-native, self-hosted mail server. **One Go binary** that embeds [Mox](https://github.com/mjl-/mox)'s
|
||||
(MIT) mail internals for the correctness-critical 95% — DKIM, SPF/DMARC, DANE + MTA-STS delivery,
|
||||
real-world MIME, spam filtering — and owns a **native, agent-shaped data model** (Postgres + object
|
||||
store) with a REST API and MCP server on top. Mail I/O sits behind a pluggable **`MailBackend`**
|
||||
interface so OpenMail is useful on day one against a relay or an existing mailbox, while embedded-Mox
|
||||
self-host is the flagship, in-process path. This document is the design of record. It favors an MVP
|
||||
that actually receives and sends real mail over a perfect feature set.
|
||||
|
||||
> **Scope:** OpenMail is a **self-hostable app, not a SaaS** — no billing/payments, no Stripe, no
|
||||
> hosted control plane. Everything runs on the operator's hardware.
|
||||
|
||||
## 0. The decision: embed Mox, don't rebuild and don't just wrap it
|
||||
|
||||
An agent-mailbox API is ~5% of the work. The other 95% is mail infrastructure (inbound :25 hardening,
|
||||
outbound deliverability, MIME, threading, quote-stripping, anti-abuse). Three ways to get that 95%:
|
||||
|
||||
1. **Build from scratch** — re-implement DKIM/SPF/DMARC/DANE/MTA-STS/MIME ourselves. Correct but slow,
|
||||
and we'd do it worse than a production server at first.
|
||||
2. **Sidecar an existing server** (run Mox as a separate process, read it over IMAP). Fast, but we
|
||||
don't own the data model — threads/labels/`extracted_text` would be re-derived from IMAP, and we
|
||||
ship two coupled processes.
|
||||
3. **Embed Mox's packages** into our own binary and own the storage + API. ← **chosen.**
|
||||
|
||||
Why (3) is viable: **Mox has no `internal/` directory** — every package is importable — and the
|
||||
README states *"most non-server Go packages are written to be reusable."* The real boundary is not
|
||||
visibility, it's **storage coupling**: Mox's *server + store* packages are welded to its embedded
|
||||
`bstore`/`bbolt` store and global config; its *protocol/crypto/parsing* packages are stateless and
|
||||
clean. We embed the latter and replace the former with our own Postgres-backed server.
|
||||
|
||||
### What we embed from Mox vs. what we own
|
||||
|
||||
| Concern | Source | Notes |
|
||||
|---|---|---|
|
||||
| MIME parsing | **mox `message`** | Battle-tested on real, broken senders. Better than enmime for our needs. |
|
||||
| DKIM sign + verify | **mox `dkim`** | |
|
||||
| SPF / DMARC | **mox `spf`, `dmarc`** | Inbound auth verification + policy. |
|
||||
| Secure outbound delivery | **mox `smtpclient`, `dane`, `mtasts`, `dns`** | The deliverability moat — DANE/MTA-STS MX delivery, mostly solved by reusing this. |
|
||||
| Bounces (DSN) | **mox `dsn`** | Parse/generate delivery status notifications. |
|
||||
| Anti-abuse | **mox `iprev`, `dnsbl`, `ratelimit`** | rDNS check, blocklists, rate limiting on :25. |
|
||||
| Spam filtering | **mox `junk`** | Built-in Bayesian filter; per-inbox train/score. |
|
||||
| SMTP auth | **mox `sasl`, `scram`** | For the submission (send) path. |
|
||||
| — | — | — |
|
||||
| Inbound SMTP **server loop** | **`emersion/go-smtp`** | Clean, embeddable listener/session. We do NOT import mox `smtpserver` (store-coupled). |
|
||||
| Storage | **OpenMail (Postgres + S3)** | Native agent data model. NOT mox `store`/bbolt. |
|
||||
| API, MCP, threading, extraction, webhooks, auth, multi-tenancy | **OpenMail** | The agent-native layer — our differentiation. |
|
||||
|
||||
> **Boundary rule:** never import `smtpserver`, `imapserver`, `queue`, `store`, or `mox-` (config) —
|
||||
> they assume Mox's storage/config. Importing them is the line between "embed as a library" and
|
||||
> "fork Mox." We stay on the library side.
|
||||
|
||||
## 0.1 Competitive positioning (why OpenMail exists)
|
||||
|
||||
The agent-mailbox concept is already taken; the *implementation niche* is open.
|
||||
|
||||
| Project | Lang | License | How mail runs | Gap OpenMail fills |
|
||||
|---|---|---|---|---|
|
||||
| **agenticmail** (closest competitor) | TypeScript | MIT | TS orchestrator + **Stalwart in a Docker sidecar**, SQLite | Multi-container, AGPL mail engine, SQLite — vs. **single Go binary, all-MIT, in-process, Postgres** |
|
||||
| cloudflare/agentic-inbox | TS | Apache-2 | **Locked to Cloudflare Workers** + Email Routing | Vendor-neutral, runs anywhere |
|
||||
| Mox / Maddy / Stalwart | Go/Rust | MIT/GPL/AGPL | Full mail servers | Not agent-native (no agent API/MCP, no threads/extract) |
|
||||
|
||||
OpenMail's three differentiators, all downstream of the Go + embed-Mox choice:
|
||||
1. **Single static binary** (vs. sidecar/multi-container).
|
||||
2. **Fully MIT stack** — Mox is MIT, vs. agenticmail's AGPL Stalwart — so others can build commercial agents on top.
|
||||
3. **In-process mail engine** — nobody else does this; it's only possible because Mox is importable Go.
|
||||
|
||||
## 0.2 The `MailBackend` interface (pluggable mail I/O)
|
||||
|
||||
All inbound delivery and outbound sending sit behind one interface, so the agent layer (API, MCP,
|
||||
store, threading) never depends on *how* mail moves. Three backends, shipped in order of effort:
|
||||
|
||||
```go
|
||||
// MailBackend abstracts where mail comes from and how it leaves. The core never
|
||||
// knows which implementation is active.
|
||||
type MailBackend interface {
|
||||
// Send dispatches an already-built, DKIM-signable RFC 5322 message.
|
||||
Send(ctx context.Context, msg *OutgoingMessage) (SendResult, error)
|
||||
// Start begins delivering inbound messages to the sink until ctx is cancelled.
|
||||
// (relay/imap: poll or webhook; embedded: go-smtp on :25.)
|
||||
Start(ctx context.Context, sink InboundSink) error
|
||||
Capabilities() Caps // self-host? inbound-push? custom-domain? throwaway-addrs?
|
||||
}
|
||||
```
|
||||
|
||||
| Backend | Inbound | Outbound | Ops burden | Ships |
|
||||
|---|---|---|---|---|
|
||||
| **relay** | provider webhook / poll | SES / Postmark / Resend API (mox `dkim` sign) | lowest | **v1 default** |
|
||||
| **imap_smtp** | IMAP IDLE on a BYO mailbox | SMTP submission to BYO server | low | v1 |
|
||||
| **embedded** (flagship) | `go-smtp` :25 + mox verify/parse/junk | mox `smtpclient` + `dane` + `mtasts` (MX) | highest | milestone 5 |
|
||||
|
||||
This makes the deliverability slog *opt-in*: a user gets a working agent mailbox immediately via
|
||||
relay or their existing mailbox, and graduates to fully self-hosted SMTP only when they want to own
|
||||
the whole stack. The `embedded` backend is the differentiator; the other two are the on-ramp.
|
||||
|
||||
## 1. Component overview
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────┐
|
||||
Internet senders ───▶ │ smtpd (go-smtp on :25) │
|
||||
(MX → this VPS) │ iprev/dnsbl/ratelimit → SPF/DKIM/DMARC verify │
|
||||
│ → mox/message parse → mox/junk score → enqueue │ ← mox pkgs
|
||||
└───────────────┬──────────────────────────────────────┘
|
||||
│ internal queue (Postgres LISTEN/NOTIFY)
|
||||
┌───────────────▼──────────────────────────────────────┐
|
||||
│ core (Go) │
|
||||
Agents / SDKs ──REST──▶ • inbox/message/thread/draft services │
|
||||
MCP clients ───MCP──▶ │ • threading, extracted_text, labels, FTS search │
|
||||
│ • auth (API keys, pods), webhooks/WS │
|
||||
└──────┬───────────────────────────────┬────────────────┘
|
||||
│ │
|
||||
┌────────────▼─────┐ ┌─────────────▼──────────────┐
|
||||
│ Postgres │ │ Object store (S3/MinIO) │
|
||||
│ metadata + FTS │ │ raw .eml + attachments │
|
||||
└──────────────────┘ └────────────────────────────┘
|
||||
▲
|
||||
┌───────────────┴──────────────────────────────────────┐
|
||||
Outbound to world ◀── │ sender (mox/dkim sign → backend) │
|
||||
│ self-host: mox/smtpclient + dane + mtasts (MX) │ ← mox pkgs
|
||||
│ relay: SES / Postmark / Resend API │
|
||||
└──────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
All roles (`smtpd`, `core` API, `sender`, `mcp`) ship in one binary with subcommands; deploy together
|
||||
or split later. Caddy terminates TLS for the HTTP API and auto-manages certs.
|
||||
|
||||
## 2. Tech stack
|
||||
|
||||
- **Language:** Go — single static binary, and the whole mature self-hosted-mail ecosystem (Mox,
|
||||
go-smtp, go-message) is Go, keeping us MIT-compatible. Rust (Stalwart) is AGPL → ruled out.
|
||||
- **HTTP:** `chi` router; `sqlc` + `pgx` for typed Postgres access.
|
||||
- **Inbound server:** `emersion/go-smtp` listener → handed to mox packages for verify/parse/score.
|
||||
- **Mail correctness/delivery:** mox `message`, `dkim`, `spf`, `dmarc`, `dane`, `mtasts`, `dns`,
|
||||
`smtpclient`, `dsn`, `junk`, `iprev`, `dnsbl`, `ratelimit`, `sasl`, `scram` (see §0 table).
|
||||
- **Quote stripping:** a `talon`/`EmailReplyParser`-style heuristic pass for `extracted_text`.
|
||||
- **Storage:** Postgres (metadata + `tsvector` FTS); S3-compatible object store (MinIO self-host, or
|
||||
S3) for raw `.eml` and attachment blobs.
|
||||
- **Events:** Postgres `LISTEN/NOTIFY` for the internal inbound queue and to fan out webhooks + a
|
||||
WebSocket stream. (Swap to NATS only if volume warrants.)
|
||||
- **Deploy:** `docker compose` — `caddy`, `openmail` (one image, multiple commands), `postgres`,
|
||||
`minio`. MX + SPF/DKIM/DMARC/DANE/MTA-STS DNS documented in `deploy/`.
|
||||
|
||||
## 3. Data model (Postgres)
|
||||
|
||||
Native and agent-shaped (not Mox's per-account bbolt index). Resource shapes mirror what agent tooling
|
||||
expects so SDKs/MCP map cleanly.
|
||||
|
||||
```sql
|
||||
pods (id, name, created_at) -- tenant isolation
|
||||
inboxes (id, pod_id, address UNIQUE, display_name, dkim_selector, metadata jsonb, created_at, updated_at)
|
||||
threads (id, inbox_id, subject, last_message_id, message_count, labels text[], updated_at, created_at)
|
||||
messages (id, inbox_id, thread_id, message_id_hdr, in_reply_to, references text[],
|
||||
from_addr, to_addrs text[], cc text[], bcc text[], subject,
|
||||
preview, text, html, extracted_text, extracted_html,
|
||||
raw_object_key, -- pointer to raw .eml in object store
|
||||
spf, dkim, dmarc, -- inbound auth verdicts (from mox pkgs)
|
||||
junk_score, labels text[], size_bytes, headers jsonb, ts tsvector, -- FTS
|
||||
created_at, updated_at)
|
||||
attachments (id, message_id, filename, content_type, size_bytes, object_key, inline bool, content_id)
|
||||
drafts (id, inbox_id, thread_id, to_addrs text[], cc, bcc, subject, text, html, send_at, client_id)
|
||||
api_keys (id, pod_id, hash, scopes text[], created_at) -- bearer tokens; store only the hash
|
||||
webhooks (id, pod_id, url, event_types text[], secret) -- HMAC-signed deliveries
|
||||
outbox (id, message_id, status, attempts, last_error, next_attempt_at) -- send queue + retries
|
||||
events (id, inbox_id, type, payload jsonb, created_at) -- audit + WS replay
|
||||
domains (id, pod_id, name, dkim_privkey_ref, verified bool) -- per-domain DKIM keys + DNS state
|
||||
```
|
||||
|
||||
Indexes: `inboxes(address)`, `messages(thread_id)`, `messages(inbox_id, created_at desc)`,
|
||||
GIN on `messages.ts` (search) and `*.labels`. Raw MIME + attachments live in object storage; rows
|
||||
hold pointers.
|
||||
|
||||
## 4. Request / data flows
|
||||
|
||||
### Inbound (receive)
|
||||
1. `go-smtp` accepts on :25; anti-abuse gate via mox `iprev`/`dnsbl`/`ratelimit` + size cap; raw `.eml`
|
||||
written to object store.
|
||||
2. mox `spf`/`dkim`/`dmarc` verify the message; verdicts recorded. mox `message` builds the MIME tree
|
||||
→ `text`/`html`, attachments, headers. mox `junk` scores spam. Quote-stripper computes
|
||||
`extracted_text` + `preview`.
|
||||
3. Threading resolves `thread_id` from `In-Reply-To`/`References`, falling back to normalized-subject
|
||||
+ participants within a window.
|
||||
4. Row inserted; `LISTEN/NOTIFY` fires → `message.received` webhook + WebSocket event.
|
||||
|
||||
### Outbound (send / reply)
|
||||
1. API builds the RFC 5322 message (sets `In-Reply-To`/`References` on replies); persists a `messages`
|
||||
row + `outbox` entry.
|
||||
2. `sender` mox-`dkim`-signs, then dispatches via the configured backend:
|
||||
- **self-host:** mox `smtpclient` + `dane` + `mtasts` for authenticated, secure MX delivery.
|
||||
- **relay:** SES/Postmark/Resend API.
|
||||
3. Bounces/complaints (mox `dsn` parse, or relay webhook) update `outbox.status` and emit events.
|
||||
|
||||
### API surface (v1 — AgentMail-shaped)
|
||||
```
|
||||
POST /v0/inboxes create inbox
|
||||
GET /v0/inboxes / /v0/inboxes/{id} list / get
|
||||
POST /v0/inboxes/{id}/messages/send send
|
||||
GET /v0/inboxes/{id}/messages list (limit, page_token, labels)
|
||||
GET /v0/inboxes/{id}/messages/{id} get
|
||||
POST /v0/inboxes/{id}/messages/{id}/reply
|
||||
GET /v0/inboxes/{id}/threads / /{id} list / get
|
||||
…drafts, webhooks, search…
|
||||
```
|
||||
Bearer auth (`Authorization: Bearer`). Keeping paths/shapes close to AgentMail's public v0 lets an
|
||||
existing client (or MCP server) target a self-hosted OpenMail by base-URL swap. Compatibility yields
|
||||
to a cleaner native shape where they conflict.
|
||||
|
||||
### MCP server
|
||||
A thin MCP front-end over `core`: tools `create_inbox`, `list_messages`, `get_thread`, `send_message`,
|
||||
`reply`, `search` — so an agent drives its own mailbox directly.
|
||||
|
||||
## 5. Deliverability plan (the moat — and why embedding Mox helps most here)
|
||||
|
||||
- **Self-host send is achievable from day one for the protocol parts** because mox `smtpclient` +
|
||||
`dane` + `mtasts` + `dkim` already implement authenticated, DANE/MTA-STS-secured MX delivery. What
|
||||
remains is *operational*, not code: dedicated IP with PTR/rDNS, SPF (`v=spf1 ip4:… -all`), published
|
||||
DKIM key, DMARC (`p=quarantine`→`p=reject`), gradual IP warmup, FBL enrollment, suppression lists.
|
||||
- **Relay backend** (SES/Postmark/Resend) ships alongside for instant inbox placement while a
|
||||
self-host IP warms up — chosen per-domain.
|
||||
- **Inbound auth:** SPF/DKIM/DMARC verified on receipt (mox pkgs); verdicts exposed in message
|
||||
metadata; optional reject/quarantine policy per inbox.
|
||||
|
||||
## 6. MVP milestones
|
||||
|
||||
Backend-first ordering: get an end-to-end loop on the *lowest-ops* backend, then add the differentiator.
|
||||
|
||||
1. **Core API + storage** — inboxes/messages/threads/drafts CRUD on Postgres + MinIO; bearer auth;
|
||||
ingest endpoint to seed without real mail. *Proves the data model + API.* `MailBackend` interface
|
||||
defined; a `null`/ingest backend satisfies it.
|
||||
2. **Relay backend (loop closed)** — `imap_smtp` and/or `relay` backend: inbound via IMAP IDLE or
|
||||
provider webhook → mox `message` parse + threading + `message.received`; outbound via SMTP
|
||||
submission / relay API, mox-DKIM-signed. *Proves receive+reply against real mail with near-zero ops.*
|
||||
3. **MCP server** — agent owns and operates an inbox end-to-end (tools over the core).
|
||||
4. **Embedded inbound** — `go-smtp` on :25 + mox verify/parse/junk + anti-abuse; MX a test domain.
|
||||
*Proves OpenMail can receive directly.*
|
||||
5. **Embedded outbound + deliverability** — mox `smtpclient`/`dane`/`mtasts`; the flagship self-host
|
||||
path and the IP-warmup/reputation long tail.
|
||||
|
||||
## 7. Repo layout (planned)
|
||||
|
||||
```
|
||||
cmd/openmail/ # single binary: `serve`, `smtpd`, `sender`, `mcp` subcommands
|
||||
internal/core/ # inbox/message/thread/draft services
|
||||
internal/mail/ # MailBackend interface + shared parse/thread/DKIM helpers (mox pkgs)
|
||||
internal/mail/relay/ # relay backend (SES/Postmark/Resend)
|
||||
internal/mail/imapsmtp/ # imap_smtp backend (BYO mailbox: IMAP IDLE in, SMTP submit out)
|
||||
internal/mail/embedded/ # embedded backend (go-smtp :25 in; mox smtpclient+dane+mtasts out)
|
||||
internal/store/ # pgx/sqlc queries, object-store client, embedded migrations
|
||||
internal/store/migrations/ # SQL migrations (go:embed'd into the binary)
|
||||
internal/api/ # chi HTTP handlers (v0 surface)
|
||||
internal/mcp/ # MCP server over core
|
||||
deploy/ # docker-compose.yml, Caddyfile, DNS (MX/SPF/DKIM/DMARC/DANE/MTA-STS) notes
|
||||
spike/ # throwaway feasibility spikes (mimecheck: mox standalone parse — PASSED)
|
||||
```
|
||||
|
||||
## 8. Licensing & attribution
|
||||
|
||||
OpenMail is **MIT** (see `LICENSE`). Every dependency is permissive: Mox (MIT), `emersion/go-smtp`
|
||||
and `go-message` (MIT), `pgx`/`chi` (MIT/BSD). We import Mox as a Go module (no vendored source in
|
||||
this repo), but credit it prominently in the README; if we ever vendor Mox source we must retain its
|
||||
MIT notice. **No GPL/AGPL** code anywhere — Maddy (GPL-3.0) and Stalwart (AGPL) are deliberately
|
||||
avoided so the project stays MIT.
|
||||
|
||||
## 9. Open questions
|
||||
|
||||
- **mox `message` ⇄ go-smtp seam:** confirm mox's parser consumes a raw reader cleanly outside mox's
|
||||
store (spike in milestone 2). If any Tier-1 package drags in `mox-`/`store` transitively, isolate
|
||||
behind a thin adapter or vendor just that file.
|
||||
- How far to chase AgentMail API compatibility vs. a cleaner native shape.
|
||||
- Search: Postgres FTS is enough for v1; revisit (Meilisearch/Typesense) only if needed.
|
||||
- Multi-tenancy depth — `pods` now, full RBAC later.
|
||||
- Whether to expose an optional IMAP read path (via mox `imapserver` over our store would mean forking
|
||||
it — more likely a thin standalone IMAP front-end later, kept out of v1).
|
||||
```
|
||||
Reference in New Issue
Block a user