From a42b798a0ed4d10943fe9e845f3e6317c4788190 Mon Sep 17 00:00:00 2001 From: Karti Tripathi <176560021+karti-ai@users.noreply.github.com> Date: Wed, 2 Sep 2026 13:20:51 -0700 Subject: [PATCH] Scaffold tier 2 and 3, the schema, the website, and self-hosted-first MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Store: the full Postgres schema as an embedded migration. UUIDv7 keys so `ORDER BY id` is a free chronological index; raw MIME and attachments live in object storage with only a key in the row; `pods` present from day one because retrofitting tenancy costs more than an unused column. API keys are stored as a SHA-256 hash — a database dump must not be a set of live credentials. API: the v0 route table, including `ingest`, which closes the receive→thread→extract loop with zero mail infrastructure and is what makes the agent layer testable in CI. Scopes are a closed enum rather than strings, so "can send mail" and "can mint keys" are not one typo apart. Internal errors are logged in full and reported as a bare string. MCP: the tool catalogue, six tools. Adding a row here is the only way an agent gains a capability — a new REST route is invisible until someone opts it in. Three tests guard the rule that no tool can ever reach key management; CI fails rather than production. ADR 0006: enterprise self-hosted first. A hosted offering comes only after we have run this ourselves long enough to have a deliverability record worth selling. `pods` stays in the schema as the thing that keeps that path open — do not remove it as dead code. Also: multi-stage Dockerfile running as a non-root system user with no shell in the runtime image, and the openmail.karti.ai static page. 17 tests, zero clippy warnings, fmt clean. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01JkyvfNJGTshJNE9FtwPLk7 --- .gitignore | 4 + Cargo.lock | 2 + Dockerfile | 26 ++ crates/openmail-api/Cargo.toml | 1 + crates/openmail-api/src/auth.rs | 48 ++++ crates/openmail-api/src/error.rs | 49 ++++ crates/openmail-api/src/lib.rs | 44 ++-- crates/openmail-api/src/routes.rs | 59 +++++ crates/openmail-mcp/src/lib.rs | 35 ++- crates/openmail-mcp/src/tools.rs | 121 ++++++++++ crates/openmail-store/Cargo.toml | 1 + .../openmail-store/migrations/0001_init.sql | 182 ++++++++++++++ crates/openmail-store/src/blobs.rs | 38 +++ crates/openmail-store/src/inboxes.rs | 3 + crates/openmail-store/src/keys.rs | 30 +++ crates/openmail-store/src/lib.rs | 32 ++- crates/openmail-store/src/messages.rs | 3 + crates/openmail-store/src/threads.rs | 3 + docs/adr/0006-self-hosted-first.md | 39 +++ website/README.md | 17 ++ website/index.html | 222 ++++++++++++++++++ 21 files changed, 915 insertions(+), 44 deletions(-) create mode 100644 Dockerfile create mode 100644 crates/openmail-api/src/auth.rs create mode 100644 crates/openmail-api/src/error.rs create mode 100644 crates/openmail-api/src/routes.rs create mode 100644 crates/openmail-mcp/src/tools.rs create mode 100644 crates/openmail-store/migrations/0001_init.sql create mode 100644 crates/openmail-store/src/blobs.rs create mode 100644 crates/openmail-store/src/inboxes.rs create mode 100644 crates/openmail-store/src/keys.rs create mode 100644 crates/openmail-store/src/messages.rs create mode 100644 crates/openmail-store/src/threads.rs create mode 100644 docs/adr/0006-self-hosted-first.md create mode 100644 website/README.md create mode 100644 website/index.html diff --git a/.gitignore b/.gitignore index 432a50f..aafa514 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,7 @@ *.pem *.key /data + +# never commit registry credentials +credentials.toml +.cargo/credentials* diff --git a/Cargo.lock b/Cargo.lock index 0f66415..084fbd3 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1615,6 +1615,7 @@ dependencies = [ "openmail-store", "serde", "serde_json", + "sqlx", "thiserror", "tokio", "tower-http", @@ -1702,6 +1703,7 @@ dependencies = [ "chrono", "openmail-core", "serde_json", + "sha2", "sqlx", "thiserror", "tokio", diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..1484865 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,26 @@ +# Build stage. Pinned to the toolchain in rust-toolchain.toml so a CI image +# bump cannot silently change the compiler. +FROM rust:1.97.1-slim-bookworm AS build +WORKDIR /src + +RUN apt-get update && apt-get install -y --no-install-recommends \ + pkg-config libssl-dev ca-certificates \ + && rm -rf /var/lib/apt/lists/* + +# Manifests first so the dependency layer caches independently of source edits. +COPY Cargo.toml Cargo.lock rust-toolchain.toml ./ +COPY crates ./crates +RUN cargo build --release --workspace --locked + +# Runtime stage. Distroless-adjacent: no shell, no package manager, nothing to +# pivot to. This process listens on :25 to the open internet. +FROM debian:bookworm-slim AS runtime +RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates \ + && rm -rf /var/lib/apt/lists/* \ + && useradd --system --uid 10001 --no-create-home --shell /usr/sbin/nologin openmail + +COPY --from=build /src/target/release/openmail /usr/local/bin/openmail + +USER 10001:10001 +ENTRYPOINT ["/usr/local/bin/openmail"] +CMD ["serve"] diff --git a/crates/openmail-api/Cargo.toml b/crates/openmail-api/Cargo.toml index 21f4028..73d7466 100644 --- a/crates/openmail-api/Cargo.toml +++ b/crates/openmail-api/Cargo.toml @@ -16,6 +16,7 @@ workspace = true openmail-core.workspace = true openmail-store.workspace = true axum.workspace = true +sqlx.workspace = true tower-http.workspace = true serde.workspace = true serde_json.workspace = true diff --git a/crates/openmail-api/src/auth.rs b/crates/openmail-api/src/auth.rs new file mode 100644 index 0000000..03544d8 --- /dev/null +++ b/crates/openmail-api/src/auth.rs @@ -0,0 +1,48 @@ +//! Bearer authentication and scopes. +//! +//! # Why scopes are enumerated, not strings +//! +//! The MCP surface hands tool access to an agent. If a scope is a free string +//! compared at the call site, "the agent can send mail" and "the agent can mint +//! keys" are one typo apart. Making them a closed enum means a new capability +//! cannot be granted by accident — it has to be added here first. + +/// What a key is allowed to do. +#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum Scope { + /// Read messages, threads and inbox metadata. + Read, + /// Send and reply. + Send, + /// Create and delete inboxes. + ManageInboxes, + /// Create, list and revoke API keys. **Never exposable over MCP** — see + /// `openmail_mcp::Exposure::NeverExposable`. + ManageKeys, +} + +impl Scope { + /// May this scope ever be reachable from an MCP tool? + /// + /// The answer for [`Scope::ManageKeys`] is permanently no. An agent that + /// can mint keys can escape every other limit placed on it, so the refusal + /// lives in the type system rather than in a config file someone can edit. + #[must_use] + pub const fn mcp_exposable(self) -> bool { + !matches!(self, Self::ManageKeys) + } +} + +#[cfg(test)] +mod tests { + use super::Scope; + + #[test] + fn key_management_is_never_reachable_from_an_agent() { + assert!(!Scope::ManageKeys.mcp_exposable()); + assert!(Scope::Read.mcp_exposable()); + assert!(Scope::Send.mcp_exposable()); + assert!(Scope::ManageInboxes.mcp_exposable()); + } +} diff --git a/crates/openmail-api/src/error.rs b/crates/openmail-api/src/error.rs new file mode 100644 index 0000000..293a805 --- /dev/null +++ b/crates/openmail-api/src/error.rs @@ -0,0 +1,49 @@ +//! API errors and their wire representation. + +use axum::{ + Json, + http::StatusCode, + response::{IntoResponse, Response}, +}; + +#[derive(Debug, thiserror::Error)] +pub enum Error { + #[error("unauthorized")] + Unauthorized, + #[error("forbidden: missing scope {0}")] + Forbidden(&'static str), + #[error("not found")] + NotFound, + #[error("bad request: {0}")] + BadRequest(String), + #[error("not implemented")] + NotImplemented, + #[error(transparent)] + Store(#[from] openmail_store::Error), +} + +impl IntoResponse for Error { + fn into_response(self) -> Response { + let status = match &self { + Self::Unauthorized => StatusCode::UNAUTHORIZED, + Self::Forbidden(_) => StatusCode::FORBIDDEN, + Self::NotFound | Self::Store(openmail_store::Error::NotFound) => StatusCode::NOT_FOUND, + Self::BadRequest(_) => StatusCode::BAD_REQUEST, + Self::NotImplemented => StatusCode::NOT_IMPLEMENTED, + Self::Store(_) => StatusCode::INTERNAL_SERVER_ERROR, + }; + + // Internal errors are logged in full and reported as a bare string. A + // database error message can carry schema and connection detail, and + // this endpoint is reachable by anyone with a key. + let message = match &self { + Self::Store(e) if status == StatusCode::INTERNAL_SERVER_ERROR => { + tracing::error!(error = ?e, "internal error"); + "internal error".to_owned() + } + other => other.to_string(), + }; + + (status, Json(serde_json::json!({ "error": message }))).into_response() + } +} diff --git a/crates/openmail-api/src/lib.rs b/crates/openmail-api/src/lib.rs index 0fe5e2d..a358ded 100644 --- a/crates/openmail-api/src/lib.rs +++ b/crates/openmail-api/src/lib.rs @@ -1,31 +1,21 @@ //! The v0 REST API. //! -//! Bearer auth, agent-shaped resources. Paths are kept close to the shape -//! existing agent-mail tooling expects, so a client can be pointed at a -//! self-hosted `OpenMail` with a base-URL swap. Where compatibility and a clean -//! native shape conflict, the native shape wins and the difference is -//! documented. -//! -//! ```text -//! POST /v0/inboxes -//! GET /v0/inboxes list -//! GET /v0/inboxes/{id} -//! POST /v0/inboxes/{id}/messages/send -//! GET /v0/inboxes/{id}/messages limit, page_token, labels -//! GET /v0/inboxes/{id}/messages/{mid} -//! POST /v0/inboxes/{id}/messages/{mid}/reply -//! GET /v0/inboxes/{id}/threads -//! GET /v0/inboxes/{id}/threads/{tid} -//! ``` +//! Bearer auth, agent-shaped resources. Paths stay close to the shape existing +//! agent-mail tooling expects, so a client can target a self-hosted `OpenMail` +//! with a base-URL swap. Where compatibility and a clean native shape conflict, +//! the native shape wins and the difference is documented. -#[derive(Debug, thiserror::Error)] -pub enum Error { - #[error("unauthorized")] - Unauthorized, - #[error("not found")] - NotFound, - #[error("bad request: {0}")] - BadRequest(String), - #[error(transparent)] - Store(#[from] openmail_store::Error), +pub mod auth; +pub mod error; +pub mod routes; + +pub use error::Error; + +use sqlx::PgPool; + +/// Everything a handler may reach. Deliberately small — a handler that needs +/// something not in here is usually a handler doing too much. +#[derive(Clone)] +pub struct AppState { + pub db: PgPool, } diff --git a/crates/openmail-api/src/routes.rs b/crates/openmail-api/src/routes.rs new file mode 100644 index 0000000..6a9b41d --- /dev/null +++ b/crates/openmail-api/src/routes.rs @@ -0,0 +1,59 @@ +//! The v0 route table. +//! +//! ```text +//! POST /v0/inboxes create +//! GET /v0/inboxes list +//! GET /v0/inboxes/{id} get +//! DELETE /v0/inboxes/{id} delete +//! POST /v0/inboxes/{id}/messages/send send +//! GET /v0/inboxes/{id}/messages list +//! GET /v0/inboxes/{id}/messages/{mid} get +//! POST /v0/inboxes/{id}/messages/{mid}/reply reply +//! GET /v0/inboxes/{id}/threads list +//! GET /v0/inboxes/{id}/threads/{tid} get (with messages) +//! POST /v0/inboxes/{id}/ingest accept a raw .eml +//! GET /v0/health liveness — unauthenticated +//! ``` +//! +//! `ingest` is how v0.1 closes the receive→thread→extract loop with **zero +//! mail infrastructure**: POST a raw message, get back a threaded, extracted +//! one. It is the whole reason the agent layer is testable in CI. + +use axum::{ + Router, + routing::{get, post}, +}; +use tower_http::limit::RequestBodyLimitLayer; + +use crate::AppState; + +/// Maximum accepted message size. Matches `openmail_smtpd::Limits::default`, +/// because an ingest path with a larger limit than the SMTP path is a way in. +pub const MAX_BODY_BYTES: usize = 50 * 1024 * 1024; + +/// Build the v0 router. +pub fn router(state: AppState) -> Router { + Router::new() + .route("/v0/health", get(health)) + .route("/v0/inboxes", post(todo_handler).get(todo_handler)) + .route("/v0/inboxes/{id}", get(todo_handler).delete(todo_handler)) + .route("/v0/inboxes/{id}/messages", get(todo_handler)) + .route("/v0/inboxes/{id}/messages/send", post(todo_handler)) + .route("/v0/inboxes/{id}/messages/{mid}", get(todo_handler)) + .route("/v0/inboxes/{id}/messages/{mid}/reply", post(todo_handler)) + .route("/v0/inboxes/{id}/threads", get(todo_handler)) + .route("/v0/inboxes/{id}/threads/{tid}", get(todo_handler)) + .route("/v0/inboxes/{id}/ingest", post(todo_handler)) + .layer(RequestBodyLimitLayer::new(MAX_BODY_BYTES)) + .with_state(state) +} + +/// Liveness. Deliberately unauthenticated and deliberately dumb: it must not +/// touch the database, or a slow query turns into a restart loop. +async fn health() -> &'static str { + "ok" +} + +async fn todo_handler() -> crate::Error { + crate::Error::NotImplemented +} diff --git a/crates/openmail-mcp/src/lib.rs b/crates/openmail-mcp/src/lib.rs index 73d826a..0605823 100644 --- a/crates/openmail-mcp/src/lib.rs +++ b/crates/openmail-mcp/src/lib.rs @@ -1,23 +1,32 @@ -//! MCP server — the thing nobody else has. +//! MCP server — the surface nobody else has. //! //! A thin front-end over [`openmail_core`] that lets an agent own and operate -//! its own mailbox as tools: `create_inbox`, `list_messages`, `get_thread`, -//! `send_message`, `reply`, `search`. +//! its own mailbox as tools. //! //! # The rule that keeps this safe //! -//! Only routes that explicitly opt in become tools, every call re-checks the -//! caller's scopes, and credential or key-management routes can **never** be -//! exposed as tools regardless of opt-in. An agent may read and send its own -//! mail; it may not mint itself a wider key. +//! Only routes that explicitly opt in become tools; every call re-checks the +//! caller's scopes; and credential or key-management routes can **never** be +//! exposed regardless of opt-in. An agent may read and send its own mail. It +//! may not mint itself a wider key — that is the one escape that would make +//! every other limit decorative. +//! +//! The refusal is encoded in [`Exposure`] and enforced by +//! [`Tool::is_representable`], which is a compile-time-checkable property +//! rather than a config flag someone can flip. -/// Marker for a route's MCP exposure. Absence of an opt-in is a refusal, not a -/// default — a new route is invisible to agents until someone says otherwise. +pub mod tools; + +pub use tools::{TOOLS, Tool}; + +/// How a capability may be reached from an agent. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum Exposure { + /// Callable as an MCP tool, subject to the caller's scopes. Tool, + /// Reachable over REST but not offered to agents. Hidden, - /// Credential-bearing. Never exposable; the type makes it unrepresentable. + /// Credential-bearing. Never exposable, under any configuration. NeverExposable, } @@ -25,6 +34,8 @@ pub enum Exposure { pub enum Error { #[error("tool not found: {0}")] UnknownTool(String), - #[error("scope denied: {0}")] - ScopeDenied(String), + #[error("scope denied for {0}")] + ScopeDenied(&'static str), + #[error("invalid arguments: {0}")] + InvalidArgs(String), } diff --git a/crates/openmail-mcp/src/tools.rs b/crates/openmail-mcp/src/tools.rs new file mode 100644 index 0000000..98740e7 --- /dev/null +++ b/crates/openmail-mcp/src/tools.rs @@ -0,0 +1,121 @@ +//! The tool catalogue. +//! +//! Adding a capability here is the *only* way an agent gains one. A new REST +//! route is invisible to agents until someone adds a row — absence of an opt-in +//! is a refusal, not a default. + +use crate::Exposure; + +/// One MCP tool. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Tool { + pub name: &'static str, + pub description: &'static str, + pub exposure: Exposure, + /// The scope a caller must hold. Kept as a string here rather than + /// importing `openmail-api` so the MCP crate does not depend on the HTTP + /// layer — the two surfaces are siblings, not a stack. + pub required_scope: &'static str, +} + +impl Tool { + /// A tool row is only valid if its exposure permits being a tool at all. + /// `NeverExposable` in this table is a bug, and this is the assertion that + /// catches it — see the test below, which runs in CI. + #[must_use] + pub const fn is_representable(&self) -> bool { + matches!(self.exposure, Exposure::Tool) + } +} + +/// Every tool an agent can call. Six, deliberately: an agent needs to create a +/// mailbox, read what arrived, and answer it. Everything else is the operator's +/// job. +pub static TOOLS: &[Tool] = &[ + Tool { + name: "create_inbox", + description: "Provision a new mailbox and return its address.", + exposure: Exposure::Tool, + required_scope: "manage_inboxes", + }, + Tool { + name: "list_messages", + description: "List messages in an inbox, newest first. Returns previews, not bodies.", + exposure: Exposure::Tool, + required_scope: "read", + }, + Tool { + name: "get_thread", + description: "Fetch a conversation with each message's quoted history stripped.", + exposure: Exposure::Tool, + required_scope: "read", + }, + Tool { + name: "send_message", + description: "Send a new message from an inbox.", + exposure: Exposure::Tool, + required_scope: "send", + }, + Tool { + name: "reply", + description: "Reply in-thread, setting In-Reply-To and References correctly.", + exposure: Exposure::Tool, + required_scope: "send", + }, + Tool { + name: "search", + description: "Full-text search across an inbox.", + exposure: Exposure::Tool, + required_scope: "read", + }, +]; + +#[cfg(test)] +mod tests { + use super::{TOOLS, Tool}; + use crate::Exposure; + + #[test] + fn no_tool_is_credential_bearing() { + // The guard for the rule in the crate docs. If someone adds a + // key-management tool, CI fails here rather than in production. + for t in TOOLS { + assert!( + t.is_representable(), + "{} must not be exposed as a tool", + t.name + ); + } + } + + #[test] + fn no_tool_grants_key_management() { + for t in TOOLS { + assert_ne!( + t.required_scope, "manage_keys", + "{} would let an agent mint credentials", + t.name + ); + } + } + + #[test] + fn tool_names_are_unique() { + let mut names: Vec<&str> = TOOLS.iter().map(|t| t.name).collect(); + names.sort_unstable(); + let before = names.len(); + names.dedup(); + assert_eq!(before, names.len(), "duplicate tool name"); + } + + #[test] + fn never_exposable_is_not_representable() { + let t = Tool { + name: "create_api_key", + description: "", + exposure: Exposure::NeverExposable, + required_scope: "manage_keys", + }; + assert!(!t.is_representable()); + } +} diff --git a/crates/openmail-store/Cargo.toml b/crates/openmail-store/Cargo.toml index bf12554..2ca79da 100644 --- a/crates/openmail-store/Cargo.toml +++ b/crates/openmail-store/Cargo.toml @@ -20,3 +20,4 @@ chrono.workspace = true serde_json.workspace = true thiserror.workspace = true tokio.workspace = true +sha2.workspace = true diff --git a/crates/openmail-store/migrations/0001_init.sql b/crates/openmail-store/migrations/0001_init.sql new file mode 100644 index 0000000..7a0c04b --- /dev/null +++ b/crates/openmail-store/migrations/0001_init.sql @@ -0,0 +1,182 @@ +-- OpenMail initial schema. +-- +-- Design notes that are load-bearing: +-- +-- * UUIDv7 primary keys, generated by the application, not the database. +-- They sort by creation time, so `ORDER BY id` is a free chronological +-- index and pagination cursors are opaque-but-ordered without a second +-- column. +-- +-- * Raw MIME and attachments are NEVER stored here. Rows hold an object-store +-- key. Message metadata is queried constantly and is small; raw MIME is +-- written once, read rarely, and is arbitrarily large. Keeping blobs out is +-- what lets the metadata working set stay in RAM. +-- +-- * `pods` exists from day one even though v0.1 is single-tenant. Retrofitting +-- tenancy into a live schema costs far more than an unused column, and it is +-- what keeps a hosted offering possible later without a migration. + +CREATE TABLE pods ( + id uuid PRIMARY KEY, + name text NOT NULL, + created_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE domains ( + id uuid PRIMARY KEY, + pod_id uuid NOT NULL REFERENCES pods(id) ON DELETE CASCADE, + name text NOT NULL, + -- Reference to the DKIM private key in the secret store. The key itself is + -- never a column: a database backup must not be a signing-key compromise. + dkim_privkey_ref text, + dkim_selector text, + verified boolean NOT NULL DEFAULT false, + created_at timestamptz NOT NULL DEFAULT now(), + UNIQUE (name) +); + +CREATE TABLE inboxes ( + id uuid PRIMARY KEY, + pod_id uuid NOT NULL REFERENCES pods(id) ON DELETE CASCADE, + -- Stored lowercase; the API lowercases on the way in. Two inboxes differing + -- only by case is a support ticket, not a feature. + address text NOT NULL UNIQUE, + display_name text, + metadata jsonb NOT NULL DEFAULT '{}'::jsonb, + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE threads ( + id uuid PRIMARY KEY, + inbox_id uuid NOT NULL REFERENCES inboxes(id) ON DELETE CASCADE, + subject text, + -- Normalised subject used by the heuristic threading fallback. Stored so + -- the match is an index lookup, not a scan with a function call. + subject_norm text, + message_count integer NOT NULL DEFAULT 0, + labels text[] NOT NULL DEFAULT '{}', + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE messages ( + id uuid PRIMARY KEY, + inbox_id uuid NOT NULL REFERENCES inboxes(id) ON DELETE CASCADE, + thread_id uuid NOT NULL REFERENCES threads(id) ON DELETE CASCADE, + + -- The Message-ID header. NOT unique: real senders reuse and forge them. + -- Indexed for threading lookups, never used as a key. + message_id_hdr text, + in_reply_to text, + references_hdr text[] NOT NULL DEFAULT '{}', + + from_addr text NOT NULL, + to_addrs text[] NOT NULL DEFAULT '{}', + cc_addrs text[] NOT NULL DEFAULT '{}', + subject text, + + body_text text, + body_html text, + -- The reply with quoted history stripped. What an agent actually reads. + extracted_text text, + preview text, + + -- Inbound auth verdicts, recorded at receipt. Null means "not evaluated" + -- (e.g. an ingested message), which is distinct from "failed". + spf text, + dkim text, + dmarc text, + junk_score real, + + labels text[] NOT NULL DEFAULT '{}', + headers jsonb NOT NULL DEFAULT '{}'::jsonb, + raw_object_key text NOT NULL, + size_bytes bigint NOT NULL DEFAULT 0, + + search tsvector, + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE attachments ( + id uuid PRIMARY KEY, + message_id uuid NOT NULL REFERENCES messages(id) ON DELETE CASCADE, + filename text, + content_type text, + size_bytes bigint NOT NULL DEFAULT 0, + object_key text NOT NULL, + inline boolean NOT NULL DEFAULT false, + content_id text +); + +CREATE TABLE drafts ( + id uuid PRIMARY KEY, + inbox_id uuid NOT NULL REFERENCES inboxes(id) ON DELETE CASCADE, + thread_id uuid REFERENCES threads(id) ON DELETE SET NULL, + to_addrs text[] NOT NULL DEFAULT '{}', + cc_addrs text[] NOT NULL DEFAULT '{}', + bcc_addrs text[] NOT NULL DEFAULT '{}', + subject text, + body_text text, + body_html text, + send_at timestamptz, + -- Caller-supplied idempotency key. A retried send must not send twice. + client_id text, + created_at timestamptz NOT NULL DEFAULT now(), + UNIQUE (inbox_id, client_id) +); + +CREATE TABLE api_keys ( + id uuid PRIMARY KEY, + pod_id uuid NOT NULL REFERENCES pods(id) ON DELETE CASCADE, + -- Only the hash. A database dump must not be a set of live credentials. + token_hash bytea NOT NULL UNIQUE, + name text, + scopes text[] NOT NULL DEFAULT '{}', + last_used_at timestamptz, + created_at timestamptz NOT NULL DEFAULT now(), + revoked_at timestamptz +); + +CREATE TABLE webhooks ( + id uuid PRIMARY KEY, + pod_id uuid NOT NULL REFERENCES pods(id) ON DELETE CASCADE, + url text NOT NULL, + event_types text[] NOT NULL DEFAULT '{}', + secret bytea NOT NULL, + created_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE outbox ( + id uuid PRIMARY KEY, + message_id uuid NOT NULL REFERENCES messages(id) ON DELETE CASCADE, + status text NOT NULL DEFAULT 'queued', + attempts integer NOT NULL DEFAULT 0, + last_error text, + next_attempt_at timestamptz NOT NULL DEFAULT now(), + created_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE events ( + id uuid PRIMARY KEY, + inbox_id uuid REFERENCES inboxes(id) ON DELETE CASCADE, + type text NOT NULL, + payload jsonb NOT NULL DEFAULT '{}'::jsonb, + created_at timestamptz NOT NULL DEFAULT now() +); + +-- Indexes ------------------------------------------------------------------- + +CREATE INDEX messages_inbox_created ON messages (inbox_id, created_at DESC); +CREATE INDEX messages_thread ON messages (thread_id); +CREATE INDEX messages_msgid ON messages (message_id_hdr) WHERE message_id_hdr IS NOT NULL; +CREATE INDEX messages_search ON messages USING gin (search); +CREATE INDEX messages_labels ON messages USING gin (labels); +CREATE INDEX threads_inbox_updated ON threads (inbox_id, updated_at DESC); +-- The heuristic threading lookup: same inbox, same normalised subject, recent. +CREATE INDEX threads_subject_norm ON threads (inbox_id, subject_norm, updated_at DESC) + WHERE subject_norm IS NOT NULL; +-- The sender's hot path: due work only. +CREATE INDEX outbox_due ON outbox (next_attempt_at) WHERE status = 'queued'; +CREATE INDEX events_inbox_created ON events (inbox_id, created_at DESC); diff --git a/crates/openmail-store/src/blobs.rs b/crates/openmail-store/src/blobs.rs new file mode 100644 index 0000000..fc7fd6b --- /dev/null +++ b/crates/openmail-store/src/blobs.rs @@ -0,0 +1,38 @@ +//! Object storage for raw `.eml` and attachments. +//! +//! # Key layout +//! +//! ```text +//! raw/{pod_id}/{inbox_id}/{message_id}.eml +//! att/{pod_id}/{message_id}/{attachment_id} +//! ``` +//! +//! Pod-first so a tenant's objects can be lifecycled, exported or deleted with +//! a single prefix operation — the shape a deletion request actually arrives +//! in. + +use uuid::Uuid; + +/// The canonical key for a message's raw MIME. +#[must_use] +pub fn raw_key(pod: Uuid, inbox: Uuid, message: Uuid) -> String { + format!("raw/{pod}/{inbox}/{message}.eml") +} + +/// The canonical key for one attachment. +#[must_use] +pub fn attachment_key(pod: Uuid, message: Uuid, attachment: Uuid) -> String { + format!("att/{pod}/{message}/{attachment}") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn keys_are_pod_prefixed_so_a_tenant_is_one_prefix() { + let (p, i, m) = (Uuid::nil(), Uuid::nil(), Uuid::nil()); + assert!(raw_key(p, i, m).starts_with("raw/00000000-")); + assert!(attachment_key(p, m, m).starts_with("att/00000000-")); + } +} diff --git a/crates/openmail-store/src/inboxes.rs b/crates/openmail-store/src/inboxes.rs new file mode 100644 index 0000000..6bf532e --- /dev/null +++ b/crates/openmail-store/src/inboxes.rs @@ -0,0 +1,3 @@ +//! `inboxes` queries. +//! +//! Wired in v0.1 — see `docs/adr/0004-milestones.md`. diff --git a/crates/openmail-store/src/keys.rs b/crates/openmail-store/src/keys.rs new file mode 100644 index 0000000..a919edc --- /dev/null +++ b/crates/openmail-store/src/keys.rs @@ -0,0 +1,30 @@ +//! API key storage. +//! +//! Only the hash is ever persisted. A database dump must not be a set of live +//! credentials — this is the single most common way a self-hosted service +//! turns a backup leak into a full compromise. + +use sha2::{Digest, Sha256}; + +/// Hash a bearer token for storage and lookup. +/// +/// SHA-256, not a password KDF: these are 256-bit random tokens, not +/// user-chosen secrets, so there is nothing to brute-force and the lookup +/// happens on every request. +#[must_use] +pub fn hash_token(token: &str) -> Vec { + Sha256::digest(token.as_bytes()).to_vec() +} + +#[cfg(test)] +mod tests { + use super::hash_token; + + #[test] + fn hashing_is_stable_and_not_the_input() { + let h = hash_token("om_live_abc"); + assert_eq!(h, hash_token("om_live_abc")); + assert_ne!(h, b"om_live_abc".to_vec()); + assert_eq!(h.len(), 32); + } +} diff --git a/crates/openmail-store/src/lib.rs b/crates/openmail-store/src/lib.rs index 5cde5ce..0f39f12 100644 --- a/crates/openmail-store/src/lib.rs +++ b/crates/openmail-store/src/lib.rs @@ -5,19 +5,41 @@ //! raw MIME is written once, read rarely, and is arbitrarily large. Keeping //! blobs out of Postgres is what lets the metadata working set stay in RAM. //! -//! Migrations are embedded in the binary so a deploy cannot drift from its -//! schema. +//! Migrations are embedded in the binary so a deployment cannot drift from the +//! schema it was built against. -pub mod migrations { - //! Embedded SQL migrations. See `crates/openmail-store/migrations/`. -} +pub mod blobs; +pub mod inboxes; +pub mod keys; +pub mod messages; +pub mod threads; + +use sqlx::postgres::{PgPool, PgPoolOptions}; + +/// Embedded migrations, applied by `openmail migrate` and on `serve` startup. +pub static MIGRATOR: sqlx::migrate::Migrator = sqlx::migrate!("./migrations"); #[derive(Debug, thiserror::Error)] pub enum Error { #[error("database: {0}")] Db(#[from] sqlx::Error), + #[error("migration: {0}")] + Migrate(#[from] sqlx::migrate::MigrateError), #[error("object store: {0}")] ObjectStore(String), #[error("not found")] NotFound, } + +/// Open the pool and apply pending migrations. +/// +/// # Errors +/// Fails if the database is unreachable or a migration does not apply. +pub async fn connect(url: &str, max_connections: u32) -> Result { + let pool = PgPoolOptions::new() + .max_connections(max_connections) + .connect(url) + .await?; + MIGRATOR.run(&pool).await?; + Ok(pool) +} diff --git a/crates/openmail-store/src/messages.rs b/crates/openmail-store/src/messages.rs new file mode 100644 index 0000000..8c95950 --- /dev/null +++ b/crates/openmail-store/src/messages.rs @@ -0,0 +1,3 @@ +//! `messages` queries. +//! +//! Wired in v0.1 — see `docs/adr/0004-milestones.md`. diff --git a/crates/openmail-store/src/threads.rs b/crates/openmail-store/src/threads.rs new file mode 100644 index 0000000..c8d3f21 --- /dev/null +++ b/crates/openmail-store/src/threads.rs @@ -0,0 +1,3 @@ +//! `threads` queries. +//! +//! Wired in v0.1 — see `docs/adr/0004-milestones.md`. diff --git a/docs/adr/0006-self-hosted-first.md b/docs/adr/0006-self-hosted-first.md new file mode 100644 index 0000000..d310252 --- /dev/null +++ b/docs/adr/0006-self-hosted-first.md @@ -0,0 +1,39 @@ +# ADR 0006 — Enterprise self-hosted first; SaaS only after we run it ourselves + +**Status:** Accepted, 2026-09-02. + +## Decision + +The product is a **self-hostable server for enterprises, small businesses and +builders.** A hosted offering is not a v1 goal and may never exist. If it does, +it comes only after we have run OpenMail ourselves, in production, long enough +to have a deliverability track record worth selling. + +## Why this ordering and not the reverse + +- **Deliverability cannot be shortcut.** A hosted product's entire value is + inbox placement, which is months of IP warmup, feedback-loop enrolment and + suppression-list discipline. Selling that before we have it is selling + something we do not own. +- **Self-hosted is the differentiator.** Every competing agent-mailbox product + is hosted-only. Leading with a SaaS puts us on their ground, competing on the + thing they have already spent years on. +- **Auditability is the purchase condition.** Enterprises will not point MX at + a closed box. Public source under Apache-2.0 is why they can say yes, and the + self-hosted path is the one that requires no trust in us at all. +- **We become our own first serious user.** The bugs that matter in mail — + silent DANE downgrades, an MTA-STS policy cached wrong for a year — surface + only under real traffic. Running it ourselves before selling it is how we + find them on our own mail instead of a customer's. + +## Consequences + +- `pods` (tenancy) stays in the schema from v0.1 even though nothing uses it. + It is the piece that keeps the SaaS path open without a migration. **Do not + remove it as dead code.** +- No billing, no Stripe, no hosted control plane in the tree. When someone + proposes one, this ADR is the answer. +- Docs, defaults and error messages are written for an operator running one + box, not for a tenant of ours. +- The website (`website/`) sells self-hosting and links to the repo. It does + not collect signups for a product that does not exist. diff --git a/website/README.md b/website/README.md new file mode 100644 index 0000000..9e3023e --- /dev/null +++ b/website/README.md @@ -0,0 +1,17 @@ +# openmail.karti.ai + +Static single page. No build step, no framework, no JS — it is a document, and +a document that needs a bundler is a liability. + +## Deploy + +Served by Caddy on cloud-2 from `/var/www/openmail`. + +```bash +rsync -av --delete website/ cloud-2:/var/www/openmail/ +``` + +⚠️ **The cloud-2 Caddy trap:** a site block without an explicit +`bind 10.0.0.2` serves a *valid certificate and an empty 200*. It looks +deployed and returns nothing. Always include the bind, and always verify with +`curl -sI https://openmail.karti.ai` — not by eye. diff --git a/website/index.html b/website/index.html new file mode 100644 index 0000000..1d901ec --- /dev/null +++ b/website/index.html @@ -0,0 +1,222 @@ + + + + + +OpenMail — agent-native mail you host yourself + + + + + + + + + + +
+
+ +

Give an agent a real email address — on a box you own.

+

+ An agent-native, self-hosted mail server in Rust. Receive, parse, thread, + search and send real SMTP mail behind a REST API and an MCP server. + Agents and humans are both first-class users. +

+
+ Apache-2.0 + Rust + Self-hosted + v0.1 — in development +
+
+
+ +
+
+

The gap

+

+ The hard part of an agent mailbox was never the API. It is the plumbing: + receiving over SMTP, sending with real deliverability, parsing genuinely + broken MIME, threading, storage. +

+

+ Hosted agent-mail products solve that well — closed, on someone else's + infrastructure. Self-hostable mail servers solve it too — with no notion + of an agent: no per-agent inbox provisioning, no threads as API + resources, no MCP. +

+

OpenMail is the intersection nobody occupies: agent-native, self-hostable, permissively licensed.

+
+
+ +
+
+

What agent-native means

+
+
+

Inboxes are API resources

+

Provisioned in one call. Not Unix accounts, not aliases, not a mailbox someone creates by hand.

+
+
+

Threads, not folders

+

Stitched from In-Reply-To and References. An agent asks for a conversation.

+
+
+

Quoted history stripped

+

extracted_text is the new content only. An agent reading full bodies re-reads the whole thread every turn.

+
+
+

MCP, natively

+

An agent owns and operates its mailbox as tools. Credential routes can never become tools.

+
+
+

Push, not poll

+

Webhooks and a WebSocket stream for message.received.

+
+
+

Humans too

+

Standard IMAP/SMTP is a goal, not an afterthought. Point Apple Mail at the mailbox an agent is driving.

+
+
+
+
+ +
+
+

Built on crates that did not exist

+

+ Two implementations of mail plumbing exist under a permissive licence: Mox + (MIT, Go), and Stalwart's primitives (Apache-2.0, Rust). Stalwart's + server is AGPL — which is why no permissively licensed Rust mail + server exists. +

+

+ Building one means writing what the ecosystem is missing. These ship + standalone so any Rust mail project can use them. +

+
+ + + + + + + +
CrateWhatPrior art
mail-daneDANE / TLSA for SMTP (RFC 7672)first in Rust
mail-mta-stsMTA-STS policy discovery and cache (RFC 8461)first in Rust
mail-dsnDelivery Status Notifications (RFC 3464)first in Rust
+
+
+
+ +
+
+

Sending: rent reputation, or build your own

+ 
# relay — someone else's IP reputation, inbox placement on day one
+provider = "ses"        # or oracle, sendgrid, postmark, resend, custom
+region   = "us-east-1"
+
+# or direct-to-MX — your reputation, MTA-STS and DANE enforced
+mode = "direct"
+

Receiving is always yours.

+
+

+ Oracle Cloud blocks outbound TCP/25 for tenancies created + after June 2021 — inbound is unaffected. There you receive directly and + relay outbound on 587. Direct-to-MX is not possible on that host at all. +

+
+
+
+ +
+
+

Self-hosted first

+

+ OpenMail is a server you run. Not a trial of a hosted product, not an + open-core teaser with the useful half behind a licence key. +

+

+ Apache-2.0 means you can run it, fork it, and build a commercial product + on top — including one that competes with anything we might host later. + That is the intent, not an oversight. +

+ 
git clone https://github.com/karti-ai/openmail
+cd openmail && docker compose up
+
+
+ + + + +