Real weather, real aircraft, a heightfield off the main thread, and instruments
Three things that were built and never connected, connected.
**The weather was already there.** `observe()` has always taken a
`WeatherObservation` and `main.ts` has always passed null, so the cloud,
precipitation, visibility and marine-layer paths in atmosphere.ts had never run
outside a test. The server already shipped NWS, met.no and Open-Meteo, all
configured off. What was actually missing was that a single TERA_ORIGIN_LAT/LNG
served one metro and lied to the other — so weather and traffic are per-region
now, derived from the city's own bounds, and the Bay Area gets its fog while
Long Beach gets its own sky. The route takes ?city= or a validated ?lat=&lng=
and refuses to become an open geocoding proxy for the planet.
**The heightfield moved to a Worker.** 2.3 s of blocked main thread at boot, and
another ~950 ms of point-in-polygon on top of it: the park mask is filled in the
worker now, and block placement samples four corners and only runs the exact
test on a cell that straddles an edge — 8 buildings differ out of 185,036.
createScene is async and takes a Stage as a consequence, and there is a
main-thread fallback because "clone it and it works" has no exception clause.
**Spaces is a chunk you fetch when you reach for the door**, not one everybody
downloads. Same for the godmode tools. The entry chunk is 722 kB rather than
772; three.js is most of what is left and splitting it is a different job.
**Godmode is an instrument panel now** rather than one slider: the date and the
season, not just the hour, so the Meeus moon and the sun's seasonal arc become
visible instead of merely correct; a weather override that says on screen when
it is lying; a frame-time and draw-call readout; and a pose editor that emits a
paste-ready Chapter block, which is the thing that makes adding New York cheap.
Two blockers the review caught:
- Every city switch leaked 8 GPU textures — one of them a 2048x2048 shadow map
— and ~10.5 shader programs, and deleteTexture had never been called once in
the app's lifetime. The renderer was being built per scene; it belongs to the
canvas, for the life of the page.
- An upstream fetch that threw rather than returning null skipped the cache
stamp, so the TTL — the only rate limit on outbound calls — collapsed to one
upstream request per inbound request, and the caller got a 500.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+93
-10
@@ -23,23 +23,72 @@ without breaking the typecheck, which is the wrong order to find out.
|
||||
|
||||
## Routes
|
||||
|
||||
| route | body | cached |
|
||||
| --- | --- | --- |
|
||||
| `GET /api/v1/health` | `HealthBody` | never |
|
||||
| `GET /api/v1/flights` | `FlightsBody` | public, `TERA_FLIGHTS_TTL` |
|
||||
| `GET /api/v1/weather` | `WeatherBody` | public, `TERA_WEATHER_TTL` |
|
||||
| `GET /api/v1/markers` | `MarkersBody` | public, `TERA_MARKERS_TTL` |
|
||||
| `GET /api/v1/offices/:id` | `OfficeDoc` | public offices only |
|
||||
| route | query | body | cached |
|
||||
| --- | --- | --- | --- |
|
||||
| `GET /api/v1/health` | — | `HealthBody` | never |
|
||||
| `GET /api/v1/flights` | `?city=` or `?lat=&lng=`; **400** for a place this box does not serve | `FlightsBody` | public, `TERA_FLIGHTS_TTL` |
|
||||
| `GET /api/v1/weather` | `?city=` or `?lat=&lng=`; **400** for a place this box does not serve | `WeatherBody` | public, `TERA_WEATHER_TTL` |
|
||||
| `GET /api/v1/markers` | — | `MarkersBody` | **private; 401 unless signed in** — public empty body when `TERA_MARKERS_SOURCE=none` |
|
||||
| `GET /api/v1/offices/:id` | — | `OfficeDoc` | public offices only |
|
||||
|
||||
Every body is declared once, in `src/server/wire.ts` in the **root** package —
|
||||
type-only, so it compiles to nothing and both the browser build and this service
|
||||
import the same declarations without either becoming a dependency of the other.
|
||||
|
||||
Both location parameters are optional and omitting them answers for the default
|
||||
region, which is the first entry in `TERA_REGIONS`. Giving both `city` and a
|
||||
coordinate is a 400, as is a coordinate this deployment has nothing to say
|
||||
about; see **Regions** below for why that is a refusal and not a lookup.
|
||||
`radiusNm` is accepted on `/flights` and deliberately ignored — `routes/flights.ts`
|
||||
explains what a caller-chosen cache key would dissolve.
|
||||
|
||||
`Cache-Control` is fail-closed: a global hook stamps `private, no-store` on every
|
||||
reply before any route runs, and a route opts in explicitly. A request that
|
||||
arrived with an `Authorization` header or a cookie never gets a public policy,
|
||||
whatever the route asked for.
|
||||
|
||||
## Regions
|
||||
|
||||
**A caller's coordinate is never forwarded upstream. It only selects among the
|
||||
points the operator configured.** Weather and flights answer for a *resolved
|
||||
region*, and a request for somewhere this box does not serve is a 400 naming
|
||||
what it does.
|
||||
|
||||
That is an allowlist rather than a lookup because the obvious version — take
|
||||
`?lat=&lng=` and hand it to NWS — turns an unauthenticated endpoint into a free
|
||||
geocoding proxy for the planet: an amplifier pointed at somebody else's
|
||||
public-good API, from an address they will blame, with the operator's own
|
||||
contact string on every request. It is also what bounds everything downstream,
|
||||
since the upstream key space *is* the region list: the per-region caches, the
|
||||
NWS station cache and the adsb.lol poll budget are all bounded by the
|
||||
environment file and cannot be grown by anybody sending requests.
|
||||
(`src/regions.ts` has the full reasoning, including why snapping to a coarse
|
||||
grid was rejected.)
|
||||
|
||||
| variable | default | what it does |
|
||||
| --- | --- | --- |
|
||||
| `TERA_REGIONS` | *(empty)* | `id:lat,lng[:radiusKm]`, separated by `;` or newlines. The list, in the operator's order; the first is the default. |
|
||||
|
||||
```ini
|
||||
TERA_REGIONS=sf:37.7749,-122.4194;socal:33.82,-118.05:150
|
||||
```
|
||||
|
||||
Ids are the same ones the browser's city packs use. `radiusKm` defaults to 120,
|
||||
which covers both shipped boards with room to spare and leaves them disjoint. A
|
||||
malformed entry is dropped with a line in `degraded`, and a spec in which
|
||||
nothing parses falls back to the shipped pair — a typo is a demotion, never a
|
||||
refusal to boot.
|
||||
|
||||
Left empty, the box serves the two cities the map ships with. An operator who
|
||||
pointed `TERA_ORIGIN_LAT/_LNG` somewhere else additionally gets that point as a
|
||||
region named `origin`, first in the list and therefore the default, so a bare
|
||||
`GET /api/v1/weather` on their box answers exactly as it did before regions
|
||||
existed.
|
||||
|
||||
`GET /api/v1/health` publishes the resolved list as `regions`, in the same
|
||||
order, so a client can pick its default the way the server does instead of
|
||||
guessing a `?city=` and getting a 400 it cannot explain.
|
||||
|
||||
## Configuration
|
||||
|
||||
Everything is `TERA_*`, everything is optional, and **nothing is fatal**. A
|
||||
@@ -53,7 +102,7 @@ missing weather contact string; this is the correction. (CONTRACT.md §5.1.)
|
||||
| `TERA_HOST` | `127.0.0.1` | Bind address. `0.0.0.0` inside a container, nowhere else. |
|
||||
| `TERA_PORT` | `8431` | |
|
||||
| `TERA_LOG_LEVEL` | `info` | |
|
||||
| `TERA_ORIGIN_LAT` / `_LNG` | SF | The city this box serves. Weather point and flight-plan centre. |
|
||||
| `TERA_ORIGIN_LAT` / `_LNG` | SF | Default region only, and superseded entirely by `TERA_REGIONS`. Nothing per-request reads it. |
|
||||
| `TERA_CORS_ORIGIN` | *(empty)* | Comma-separated. Empty means same-origin only. |
|
||||
| `TERA_PUBLIC_MAX_AGE` | `60` | `max-age` for routes without their own TTL. |
|
||||
|
||||
@@ -83,9 +132,9 @@ is a supported steady state, not an error path.
|
||||
| --- | --- | --- |
|
||||
| `TERA_FLIGHTS_SOURCE` | `sim` | `sim`, `adsb`, `dump1090`. |
|
||||
| `TERA_ADSB_ENDPOINT` | `https://api.adsb.lol` | Also works with airplanes.live. |
|
||||
| `TERA_ADSB_RADIUS_NM` | `40` | |
|
||||
| `TERA_ADSB_RADIUS_NM` | `40` | Clamped to 1–250 nm, which is what both feeds accept, with a `degraded` line. |
|
||||
| `TERA_DUMP1090_PATH` | *(empty)* | Path to your receiver's `aircraft.json`. |
|
||||
| `TERA_FLIGHTS_TTL` | `300` | Clamped to 15 s for live sources. |
|
||||
| `TERA_FLIGHTS_TTL` | `300` | For live sources, clamped **into 5–15 s**. |
|
||||
| `TERA_FLIGHTS_SEED` | `4711` | |
|
||||
|
||||
The simulated source is served as a **route plan**, not as positions: the routes,
|
||||
@@ -93,6 +142,17 @@ a fixed phase origin and a seed, which every browser evaluates in closed form
|
||||
against wall-clock time. One cacheable request replaces a poll per second, and
|
||||
two people on different machines see the same aircraft in the same places.
|
||||
|
||||
The **floor** under the live TTL is the load-bearing half of that clamp, and it
|
||||
is why the cell above reads as a range. adsb.lol asks for no more than one
|
||||
request per second and airplanes.live publishes the same ceiling; both are
|
||||
volunteer-fed. `TERA_FLIGHTS_TTL=0` used to mean one upstream request per
|
||||
inbound request — the exact flood the limit exists to stop, delivered by a
|
||||
setting that reads like "as fresh as possible". With the floor the worst case is
|
||||
arithmetic rather than a guess: **regions ÷ 5 requests per second** with every
|
||||
region under continuous load, which for the two shipped here is 0.4/s. An
|
||||
operator configuring more than five regions and keeping them all warm is the
|
||||
case to watch.
|
||||
|
||||
There is no FlightRadar24 client and there will not be one — their terms forbid
|
||||
scraping and forbid redistribution, so shipping one in an Apache-2.0 repo would
|
||||
be publishing instructions for breaking a ToS. An RTL-SDR and `dump1090` on a box
|
||||
@@ -112,6 +172,29 @@ The API serves a file. It holds no database and no credential, and private
|
||||
per-user markers are never proxied through it — an authenticated browser calls
|
||||
Workie directly with its own token, so a private row never enters this process.
|
||||
|
||||
**Where a marker feed is configured, it takes a session.** This is the one route
|
||||
the `member` tier is about, and until it refused somebody the tier was a word in
|
||||
a type union that no server behaviour corresponded to. An anonymous caller gets
|
||||
401 with `WWW-Authenticate: Bearer`; a member gets the snapshot with no public
|
||||
cache policy on it, because a body that took a credential to obtain must not sit
|
||||
in a shared cache waiting for the next caller. There is deliberately no
|
||||
`TERA_MARKERS_PUBLIC` escape hatch. (401 rather than the 404 an office answers
|
||||
with: nothing here is enumerable — one feed, one path, and `/api/v1/health`
|
||||
already publishes `sources.markers` — so the caller is told the useful thing,
|
||||
which is "sign in and ask again".)
|
||||
|
||||
Two consequences worth knowing before you configure it:
|
||||
|
||||
- **A box with no feed still answers 200 and an empty list.** `source: none` is
|
||||
the zero-config default, there is nothing there to protect, and making a
|
||||
stranger sign in to be told "no markers" would fail the acceptance test at the
|
||||
top of this file.
|
||||
- **`TERA_MARKERS_SOURCE=file` with `TERA_AUTH_MODE=none` is unreachable by
|
||||
everyone**, because nobody on such a box is ever authenticated. That is the
|
||||
fail-closed direction and it is the one private offices already take; the
|
||||
config pushes a line into `degraded` saying so, rather than letting it be
|
||||
discovered from an empty map.
|
||||
|
||||
**Every row must declare where its coordinate came from, and the gate refuses
|
||||
anything not on the allowlist.** Serving a snapshot of geocoded coordinates is
|
||||
Public Use of a Derivative Database; if those coordinates came from Nominatim,
|
||||
|
||||
+61
-8
@@ -19,6 +19,7 @@
|
||||
|
||||
import { readFileSync } from "node:fs";
|
||||
import { parseScryptHash, type ScryptHash } from "./auth/password.ts";
|
||||
import { loadRegions, type RegionSet } from "./regions.ts";
|
||||
import type {
|
||||
AuthMode,
|
||||
FlightsSourceId,
|
||||
@@ -114,8 +115,17 @@ export interface Config {
|
||||
port: number;
|
||||
logLevel: string;
|
||||
version: string;
|
||||
/** The city this box is serving, in degrees. Used by weather and by flights. */
|
||||
/**
|
||||
* The default place this box serves, in degrees.
|
||||
*
|
||||
* Kept because it is what a self-hoster already wrote in their env file, and
|
||||
* because `regions[0]` is derived from it. Nothing per-request reads it any
|
||||
* more: weather and flights answer for a *resolved region*, since one origin
|
||||
* cannot describe two cities six hundred kilometres apart. See `regions.ts`.
|
||||
*/
|
||||
origin: { lat: number; lng: number };
|
||||
/** Every place this box will answer for. The first one is the default. */
|
||||
regions: RegionSet;
|
||||
/** Allowed CORS origins. Empty means same-origin only, which is the default. */
|
||||
corsOrigins: string[];
|
||||
/** `max-age` for routes that opt in to public caching. */
|
||||
@@ -136,18 +146,29 @@ export function loadConfig(env: Env = process.env): Config {
|
||||
|
||||
const weather = loadWeather(env, degraded);
|
||||
const flights = loadFlights(env, degraded);
|
||||
const markers = loadMarkers(env, degraded);
|
||||
const auth = loadAuth(env, degraded);
|
||||
// After auth, because a marker feed with nobody able to sign in is worth a
|
||||
// sentence and the sentence is only true once `mode` has finished demoting.
|
||||
const markers = loadMarkers(env, auth.mode, degraded);
|
||||
|
||||
const origin = {
|
||||
lat: num(env, "TERA_ORIGIN_LAT", 37.7749, degraded),
|
||||
lng: num(env, "TERA_ORIGIN_LNG", -122.4194, degraded),
|
||||
};
|
||||
|
||||
return {
|
||||
host: str(env, "TERA_HOST", "127.0.0.1"),
|
||||
port: num(env, "TERA_PORT", 8431, degraded),
|
||||
logLevel: str(env, "TERA_LOG_LEVEL", "info"),
|
||||
version: readVersion(),
|
||||
origin: {
|
||||
lat: num(env, "TERA_ORIGIN_LAT", 37.7749, degraded),
|
||||
lng: num(env, "TERA_ORIGIN_LNG", -122.4194, degraded),
|
||||
},
|
||||
origin,
|
||||
regions: loadRegions({
|
||||
spec: str(env, "TERA_REGIONS", ""),
|
||||
origin,
|
||||
originConfigured:
|
||||
str(env, "TERA_ORIGIN_LAT", "") !== "" || str(env, "TERA_ORIGIN_LNG", "") !== "",
|
||||
degraded,
|
||||
}),
|
||||
corsOrigins: list(env, "TERA_CORS_ORIGIN"),
|
||||
publicMaxAge: num(env, "TERA_PUBLIC_MAX_AGE", 60, degraded),
|
||||
weather,
|
||||
@@ -236,7 +257,7 @@ function loadFlights(env: Env, degraded: string[]): FlightsConfig {
|
||||
return {
|
||||
source,
|
||||
endpoint: str(env, "TERA_ADSB_ENDPOINT", "https://api.adsb.lol"),
|
||||
radiusNm: num(env, "TERA_ADSB_RADIUS_NM", 40, degraded),
|
||||
radiusNm: radius(num(env, "TERA_ADSB_RADIUS_NM", 40, degraded), degraded),
|
||||
dump1090Path,
|
||||
epochMs: num(env, "TERA_FLIGHTS_EPOCH_MS", PLAN_EPOCH_MS, degraded),
|
||||
seed: num(env, "TERA_FLIGHTS_SEED", 4711, degraded),
|
||||
@@ -244,12 +265,31 @@ function loadFlights(env: Env, degraded: string[]): FlightsConfig {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The largest circle the hosted feeds will answer for: adsb.lol and
|
||||
* airplanes.live both cap `/v2/point/:lat/:lng/:radius` at 250 nautical miles
|
||||
* and reject anything larger outright. Clamping here rather than letting the
|
||||
* request fail keeps the failure legible — an operator who typed 2500 gets a map
|
||||
* with aircraft on it and a sentence in `degraded`, not a silently empty sky.
|
||||
*/
|
||||
const MAX_ADSB_RADIUS_NM = 250;
|
||||
|
||||
function radius(asked: number, degraded: string[]): number {
|
||||
if (asked >= 1 && asked <= MAX_ADSB_RADIUS_NM) return asked;
|
||||
const clamped = Math.min(MAX_ADSB_RADIUS_NM, Math.max(1, asked));
|
||||
degraded.push(
|
||||
`TERA_ADSB_RADIUS_NM=${asked} is outside the 1–${MAX_ADSB_RADIUS_NM} nautical ` +
|
||||
`miles the hosted feeds answer for; using ${clamped}.`,
|
||||
);
|
||||
return clamped;
|
||||
}
|
||||
|
||||
const MARKER_SOURCES: MarkersSourceId[] = ["none", "file"];
|
||||
|
||||
/** CONTRACT.md §8. Adding to this list is a licence decision, not a config tweak. */
|
||||
export const DEFAULT_PROVENANCE_ALLOWLIST = ["us-census", "hand-placed", "synthetic"];
|
||||
|
||||
function loadMarkers(env: Env, degraded: string[]): MarkersConfig {
|
||||
function loadMarkers(env: Env, authMode: AuthMode, degraded: string[]): MarkersConfig {
|
||||
const asked = str(env, "TERA_MARKERS_SOURCE", "none");
|
||||
let source = oneOf(asked, MARKER_SOURCES);
|
||||
if (source === null) {
|
||||
@@ -268,6 +308,19 @@ function loadMarkers(env: Env, degraded: string[]): MarkersConfig {
|
||||
source = "none";
|
||||
}
|
||||
|
||||
// Not a demotion — the feed stays configured and the route keeps refusing
|
||||
// correctly — but an operator who mounted a marker file on a box where nobody
|
||||
// can sign in has built something no browser will ever see, and one sentence
|
||||
// now is cheaper than an afternoon. `routes/markers.ts` has the reasoning for
|
||||
// why the feed is members-only with no public escape hatch.
|
||||
if (source === "file" && authMode === "none") {
|
||||
degraded.push(
|
||||
"TERA_MARKERS_SOURCE=file needs an authentication mode: the marker feed is " +
|
||||
"refused to anonymous callers, and with TERA_AUTH_MODE=none nobody is ever " +
|
||||
"anything else, so it will answer 401 to everybody. Set TERA_AUTH_MODE.",
|
||||
);
|
||||
}
|
||||
|
||||
const allowlist = list(env, "TERA_MARKERS_PROVENANCE_ALLOWLIST");
|
||||
return {
|
||||
source,
|
||||
|
||||
@@ -38,15 +38,36 @@ export interface FlightsSnapshot {
|
||||
observedAt: number;
|
||||
}
|
||||
|
||||
/** Just enough of the service's logger to say a feed came back oversized. */
|
||||
export interface AdsbLog {
|
||||
warn(msg: string): void;
|
||||
}
|
||||
|
||||
/**
|
||||
* One circle of sky, from a hosted feed.
|
||||
*
|
||||
* `center` is a **region centre from `regions.ts`**, never a coordinate off the
|
||||
* wire. That is the whole reason the routes validate before they get here: this
|
||||
* function will happily fetch anywhere, and the thing standing between it and
|
||||
* being an open proxy pointed at a community feed is that its callers can only
|
||||
* hand it points an operator configured. The radius is clamped to the 250 nm
|
||||
* both feeds accept, in `config.ts`.
|
||||
*
|
||||
* How often this is allowed to be called, and the arithmetic that keeps it
|
||||
* inside adsb.lol's one-request-per-second ceiling, is in `flights/index.ts`.
|
||||
*/
|
||||
export async function fetchAdsb(
|
||||
endpoint: string,
|
||||
center: { lat: number; lng: number },
|
||||
radiusNm: number,
|
||||
log?: AdsbLog,
|
||||
): Promise<FlightsSnapshot | null> {
|
||||
const url = `${endpoint.replace(/\/$/, "")}/v2/point/${center.lat.toFixed(4)}/${center.lng.toFixed(4)}/${Math.round(radiusNm)}`;
|
||||
const body = await getJson<AircraftEnvelope>(url);
|
||||
if (body === null) return null;
|
||||
return normalise(body);
|
||||
return normalise(body, (dropped) =>
|
||||
log?.warn(`flights:adsb: feed sent ${dropped + MAX_ROWS} aircraft; kept the first ${MAX_ROWS}`),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -57,19 +78,75 @@ export async function fetchAdsb(
|
||||
* returns `null` and lets the caller keep the previous snapshot instead of
|
||||
* emptying the sky for one tick.
|
||||
*/
|
||||
export async function readDump1090(path: string): Promise<FlightsSnapshot | null> {
|
||||
export async function readDump1090(path: string, log?: AdsbLog): Promise<FlightsSnapshot | null> {
|
||||
try {
|
||||
const text = await readFile(path, "utf8");
|
||||
return normalise(JSON.parse(text) as AircraftEnvelope);
|
||||
return normalise(JSON.parse(text) as AircraftEnvelope, (dropped) =>
|
||||
log?.warn(
|
||||
`flights:dump1090: ${path} held ${dropped + MAX_ROWS} aircraft; ` +
|
||||
`kept the first ${MAX_ROWS}`,
|
||||
),
|
||||
);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function normalise(body: AircraftEnvelope): FlightsSnapshot {
|
||||
const rows = body.ac ?? body.aircraft ?? [];
|
||||
/**
|
||||
* How many aircraft one snapshot may carry.
|
||||
*
|
||||
* Not a limit anybody should ever meet: a 250 nm circle over the busiest
|
||||
* airspace on earth is a few thousand contacts, and the board draws a dart for
|
||||
* each. It is here because the row count is set by somebody else's server and
|
||||
* the parsed array is held in this process and served, cached, to every
|
||||
* anonymous caller for the length of the TTL — a feed answering with 200,000
|
||||
* rows was measured at a 14.9 MB public body. The excess is dropped and
|
||||
* counted rather than silently trimmed, so an operator whose sky is suddenly
|
||||
* capped finds out from the log rather than from a map that looks thin.
|
||||
*/
|
||||
const MAX_ROWS = 5000;
|
||||
|
||||
/**
|
||||
* One envelope, turned into a snapshot — or `null` if it is not an envelope.
|
||||
*
|
||||
* ### Checked, not assumed
|
||||
*
|
||||
* This was `body.ac ?? body.aircraft ?? []` followed by a `for…of`, so a
|
||||
* well-formed JSON 200 with a number in `ac` threw "rows is not iterable" out
|
||||
* of `fetchAdsb` — past `upstream.ts`, which then neither stamped its clock nor
|
||||
* caught it, and out to the route as a 500. `markers/gate.ts` has always done
|
||||
* this for its own input; the hosted-feed path was the one that trusted the
|
||||
* wire.
|
||||
*
|
||||
* ### Why an unreadable body is `null` and not an empty sky
|
||||
*
|
||||
* `?? []` would still be here and would still not throw, and it would be the
|
||||
* wrong answer. `flights/index.ts` reads `null` as "fall back to the plan" and
|
||||
* reads a snapshot — *including an empty one* — as observed truth, which is
|
||||
* correct: three in the morning over a small city really is an empty sky. A
|
||||
* garbled body coerced to zero aircraft is therefore served as `mode: "live"`
|
||||
* with nothing in it, and the operator who has just turned on ADS-B gets a
|
||||
* blank map that claims to be real. That is precisely the outcome the top of
|
||||
* this file says the fallback exists to prevent.
|
||||
*
|
||||
* So the test is on the *array*, not on its length: a body carrying `ac: []`
|
||||
* is an empty circle and is live; a body carrying neither `ac` nor `aircraft`
|
||||
* as an array is not an answer at all.
|
||||
*/
|
||||
function normalise(
|
||||
body: AircraftEnvelope,
|
||||
onDrop?: (dropped: number) => void,
|
||||
): FlightsSnapshot | null {
|
||||
const raw = body.ac ?? body.aircraft;
|
||||
if (!Array.isArray(raw)) return null;
|
||||
const rows = raw.length > MAX_ROWS ? raw.slice(0, MAX_ROWS) : raw;
|
||||
if (raw.length > rows.length) onDrop?.(raw.length - rows.length);
|
||||
|
||||
const aircraft: WireAircraft[] = [];
|
||||
for (const a of rows) {
|
||||
// A row that is not an object at all reaches this from the same wire that
|
||||
// sent a number where the array was.
|
||||
if (a === null || typeof a !== "object") continue;
|
||||
if (typeof a.lat !== "number" || typeof a.lon !== "number") continue;
|
||||
const callsign = a.flight?.trim();
|
||||
const id = a.hex ?? callsign;
|
||||
|
||||
+80
-34
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Which sky this box serves.
|
||||
* Which sky this box serves, and for which city.
|
||||
*
|
||||
* The simulated source answers from a plan and never touches the network, so it
|
||||
* is free and it is the default. The two real sources are polled on a timer and
|
||||
@@ -7,15 +7,48 @@
|
||||
* answering serves its last snapshot, and a feed that has never answered falls
|
||||
* back to the plan rather than to an empty sky. An operator who turned on ADS-B
|
||||
* and got a blank map would reasonably conclude the renderer was broken.
|
||||
*
|
||||
* Everything here is **per region**. `TERA_ADSB_RADIUS_NM` around one origin
|
||||
* could only ever describe one city, and the Bay Area wants SFO, OAK and SJC
|
||||
* while the Southland wants LAX, BUR, LGB and SNA — six hundred kilometres of
|
||||
* empty coastline apart, with no sane radius that covers both. The requested
|
||||
* region supplies the centre; the radius stays what the operator configured.
|
||||
*
|
||||
* ### Staying inside adsb.lol's limits
|
||||
*
|
||||
* adsb.lol asks for **no more than one request per second** from a client and
|
||||
* says outright that it will rate-limit callers who ignore it; airplanes.live,
|
||||
* which serves the same shape at `TERA_ADSB_ENDPOINT`, publishes the same
|
||||
* ceiling. Both are volunteer-fed community feeds paid for by people who did not
|
||||
* sign up to host somebody's map.
|
||||
*
|
||||
* This stays inside the limit structurally rather than by hoping:
|
||||
*
|
||||
* - One poll per region per `liveTtl`, never per request. Concurrent misses on a
|
||||
* region collapse into one upstream call (`upstream.ts`).
|
||||
* - `liveTtl` is **floored at 5 seconds** as well as capped at 15. The floor is
|
||||
* the load-bearing half and it is new: `TERA_FLIGHTS_TTL=0` used to mean one
|
||||
* upstream request per inbound request, which is exactly the flood the limit
|
||||
* exists to stop, delivered by a config value that reads like "as fresh as
|
||||
* possible".
|
||||
* - The region set is a fixed allowlist from the environment, so the worst case
|
||||
* is arithmetic rather than a guess: **regions ÷ 5 requests per second**, with
|
||||
* every region under continuous load. With the two this repo ships that is
|
||||
* 0.4/s against a ceiling of 1/s. An operator who configures more than five
|
||||
* regions and keeps them all warm is the case to watch, and writing the number
|
||||
* down here is how they find that out before adsb.lol does.
|
||||
* - A box nobody is looking at polls nothing at all.
|
||||
*/
|
||||
|
||||
import type { Config } from "../config.ts";
|
||||
import type { Region } from "../regions.ts";
|
||||
import type { FlightsBody } from "../../../src/server/wire.ts";
|
||||
import { createUpstream } from "../upstream.ts";
|
||||
import { fetchAdsb, readDump1090, type FlightsSnapshot } from "./adsb.ts";
|
||||
import { planFor } from "./plan.ts";
|
||||
|
||||
export interface FlightsService {
|
||||
current(): Promise<FlightsBody>;
|
||||
current(region: Region): Promise<FlightsBody>;
|
||||
}
|
||||
|
||||
export interface FlightsLog {
|
||||
@@ -24,48 +57,61 @@ export interface FlightsLog {
|
||||
|
||||
/**
|
||||
* How long a live snapshot may be cached. Aircraft move; the plan does not, so
|
||||
* only the live path is clamped.
|
||||
* only the live path is clamped. See the rate-limit note above for the floor.
|
||||
*/
|
||||
const LIVE_MAX_TTL_SECONDS = 15;
|
||||
const LIVE_MIN_TTL_SECONDS = 5;
|
||||
|
||||
/**
|
||||
* The one cache key every `dump1090` region shares.
|
||||
*
|
||||
* A receiver has one antenna. Asking a Bay Area rooftop for the Southland gets
|
||||
* the Bay Area sky whatever the query said, so keying this per region would
|
||||
* multiply the file reads without changing a byte of the answer. The honest
|
||||
* thing is one key and a snapshot whose aircraft carry their own coordinates —
|
||||
* the renderer puts them where they actually are.
|
||||
*/
|
||||
const RECEIVER_KEY = "receiver";
|
||||
|
||||
export function createFlightsService(config: Config, log: FlightsLog): FlightsService {
|
||||
const { source, endpoint, radiusNm, dump1090Path, epochMs, seed, ttlSeconds } = config.flights;
|
||||
const routes = planFor(config.origin);
|
||||
|
||||
const plan = (): FlightsBody => ({
|
||||
mode: "plan",
|
||||
source: "sim",
|
||||
t0: epochMs,
|
||||
seed,
|
||||
routes,
|
||||
ttlSeconds,
|
||||
// Built once per region and kept: the plan is a pure function of the centre,
|
||||
// and it is handed out on every cacheable request.
|
||||
const plans = new Map<string, FlightsBody>();
|
||||
const plan = (region: Region): FlightsBody => {
|
||||
const existing = plans.get(region.id);
|
||||
if (existing !== undefined) return existing;
|
||||
const body: FlightsBody = {
|
||||
mode: "plan",
|
||||
source: "sim",
|
||||
t0: epochMs,
|
||||
seed,
|
||||
routes: planFor(region),
|
||||
ttlSeconds,
|
||||
};
|
||||
plans.set(region.id, body);
|
||||
return body;
|
||||
};
|
||||
|
||||
const liveTtl = Math.min(LIVE_MAX_TTL_SECONDS, Math.max(LIVE_MIN_TTL_SECONDS, ttlSeconds));
|
||||
const upstream = createUpstream<FlightsSnapshot>({
|
||||
label: `flights:${source}`,
|
||||
ttlSeconds: liveTtl,
|
||||
log,
|
||||
});
|
||||
|
||||
const liveTtl = Math.min(ttlSeconds, LIVE_MAX_TTL_SECONDS);
|
||||
let snapshot: FlightsSnapshot | null = null;
|
||||
let polledAt = 0;
|
||||
|
||||
async function poll(): Promise<void> {
|
||||
const fresh =
|
||||
source === "dump1090"
|
||||
? await readDump1090(dump1090Path)
|
||||
: await fetchAdsb(endpoint, config.origin, radiusNm);
|
||||
polledAt = Date.now();
|
||||
if (fresh !== null) {
|
||||
snapshot = fresh;
|
||||
return;
|
||||
}
|
||||
log.warn(
|
||||
`flights: ${source} did not answer; serving ${snapshot === null ? "the simulated plan" : "the last snapshot"}`,
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
async current(): Promise<FlightsBody> {
|
||||
if (source === "sim") return plan();
|
||||
async current(region: Region): Promise<FlightsBody> {
|
||||
if (source === "sim") return plan(region);
|
||||
|
||||
if (Date.now() - polledAt > liveTtl * 1000) await poll();
|
||||
if (snapshot === null) return plan();
|
||||
const key = source === "dump1090" ? RECEIVER_KEY : region.id;
|
||||
const snapshot = await upstream.get(key, () =>
|
||||
source === "dump1090"
|
||||
? readDump1090(dump1090Path, log)
|
||||
: fetchAdsb(endpoint, region, radiusNm, log),
|
||||
);
|
||||
if (snapshot === null) return plan(region);
|
||||
|
||||
return {
|
||||
mode: "live",
|
||||
|
||||
+107
-17
@@ -11,11 +11,12 @@
|
||||
*
|
||||
* ### Where the coordinates came from
|
||||
*
|
||||
* The three Bay Area airport positions are published FAA airport reference
|
||||
* points, typed in by hand. They are US government facts in the public domain,
|
||||
* and — as with every other coordinate in this repo — emphatically not derived
|
||||
* from OpenStreetMap. Everything else here is a waypoint someone made up so the
|
||||
* legs go the right way. See ARCHITECTURE.md §3.2 and CONTRACT.md §8.
|
||||
* The seven airport positions — SFO, OAK and SJC in the north, LAX, BUR, LGB and
|
||||
* SNA in the south — are published FAA airport reference points, typed in by
|
||||
* hand. They are US government facts in the public domain, and — as with every
|
||||
* other coordinate in this repo — emphatically not derived from OpenStreetMap.
|
||||
* Everything else here is a waypoint someone made up so the legs go the right
|
||||
* way. See ARCHITECTURE.md §3.2 and CONTRACT.md §8.
|
||||
*
|
||||
* The callsigns are invented, with operator prefixes that belong to nobody. A
|
||||
* repo that refuses to ship other people's logos should not ship their flight
|
||||
@@ -95,17 +96,100 @@ const BAY_AREA: WireSimRoute[] = [
|
||||
},
|
||||
];
|
||||
|
||||
/** Degrees, roughly the distance from downtown SF to the far end of the bay. */
|
||||
const BAY_AREA_RADIUS = 0.75;
|
||||
const LAX: [number, number] = [33.9425, -118.4081];
|
||||
const BUR: [number, number] = [34.2007, -118.3585];
|
||||
const LGB: [number, number] = [33.8177, -118.1516];
|
||||
const SNA: [number, number] = [33.6757, -117.8682];
|
||||
|
||||
/**
|
||||
* The Southland, which is a different shape of airspace and not a translated
|
||||
* copy of the Bay Area.
|
||||
*
|
||||
* Almost everything here runs east–west, because the terrain does: the basin is
|
||||
* walled by the San Gabriels to the north, so departures go out over the water
|
||||
* and turn, and arrivals come down the length of the valley. Four fields instead
|
||||
* of three, and the two small ones matter more than they do in the north — a
|
||||
* board four times the area at a third of the scale reads as empty very fast if
|
||||
* everything is at cruise.
|
||||
*/
|
||||
const SOCAL: WireSimRoute[] = [
|
||||
// Departures — the standard west-over-the-ocean climb, the north-east haul
|
||||
// over the Cajon Pass, and the two valley fields going their own way.
|
||||
{ callsign: "LMB604", from: LAX, to: [33.6, -119.3], fromAlt: 20, toAlt: 10500, duration: 440 },
|
||||
{ callsign: "PAC712", from: LAX, to: [34.6, -117.1], fromAlt: 20, toAlt: 11500, duration: 520 },
|
||||
{ callsign: "SIE288", from: BUR, to: [34.9, -118.9], fromAlt: 20, toAlt: 9800, duration: 460 },
|
||||
{ callsign: "BAY1516", from: SNA, to: [33.1, -118.6], fromAlt: 20, toAlt: 9600, duration: 420 },
|
||||
{ callsign: "GLD843", from: LGB, to: [34.5, -116.9], fromAlt: 20, toAlt: 10200, duration: 500 },
|
||||
|
||||
// Arrivals — the long straight-in from the east, the coastal descent from the
|
||||
// north-west, and one down the back of the mountains into Burbank.
|
||||
{ callsign: "LMB1170", from: [34.1, -116.8], to: LAX, fromAlt: 6200, toAlt: 20, duration: 560 },
|
||||
{ callsign: "RDW425", from: [34.7, -119.4], to: LAX, fromAlt: 6800, toAlt: 20, duration: 600 },
|
||||
{ callsign: "PAC96", from: [34.8, -117.6], to: BUR, fromAlt: 5400, toAlt: 20, duration: 480 },
|
||||
{ callsign: "SIE1901", from: [33.2, -117.2], to: SNA, fromAlt: 5000, toAlt: 20, duration: 450 },
|
||||
|
||||
// Overflights, level the whole way: the coastal corridor and the desert one.
|
||||
{
|
||||
callsign: "GLD1330",
|
||||
from: [34.9, -119.2],
|
||||
to: [32.9, -117.0],
|
||||
fromAlt: 11200,
|
||||
toAlt: 11200,
|
||||
duration: 690,
|
||||
},
|
||||
{
|
||||
callsign: "RDW78",
|
||||
from: [33.0, -119.1],
|
||||
to: [34.9, -116.8],
|
||||
fromAlt: 10600,
|
||||
toAlt: 10600,
|
||||
duration: 720,
|
||||
},
|
||||
|
||||
// Low across the basin. General aviation is most of what anybody standing in
|
||||
// Culver City actually sees, and it is the only traffic that reads as *near*.
|
||||
{
|
||||
callsign: "BAY3312",
|
||||
from: [33.78, -118.42],
|
||||
to: [34.16, -117.75],
|
||||
fromAlt: 850,
|
||||
toAlt: 850,
|
||||
duration: 540,
|
||||
},
|
||||
{
|
||||
callsign: "LMB2044",
|
||||
from: [34.22, -118.72],
|
||||
to: [33.72, -117.98],
|
||||
fromAlt: 1300,
|
||||
toAlt: 1300,
|
||||
duration: 570,
|
||||
},
|
||||
];
|
||||
|
||||
/**
|
||||
* The airspaces this file has actually laid out, and how close a region's centre
|
||||
* has to be to get one.
|
||||
*
|
||||
* Degrees rather than kilometres because the comparison is a box, not a circle,
|
||||
* and the radii are generous: the Bay Area's is roughly downtown to the far end
|
||||
* of the bay, and the Southland's spans a basin that runs from Ventura to
|
||||
* Riverside. A region centre that lands inside one of these is close enough that
|
||||
* the real airports are the right answer; anything else gets spokes.
|
||||
*/
|
||||
const AIRSPACES: { centre: [number, number]; radius: number; routes: WireSimRoute[] }[] = [
|
||||
{ centre: [37.7749, -122.4194], radius: 0.75, routes: BAY_AREA },
|
||||
{ centre: [33.82, -118.05], radius: 1.0, routes: SOCAL },
|
||||
];
|
||||
|
||||
/**
|
||||
* A plan for a city this file has never heard of.
|
||||
*
|
||||
* A self-hoster pointing `TERA_ORIGIN_LAT/LNG` at somewhere that is not San
|
||||
* Francisco should get moving aircraft rather than an empty sky, so eight legs
|
||||
* are laid out on evenly-spaced bearings through their origin. It is not their
|
||||
* city's real airspace and does not pretend to be — it is motion in the right
|
||||
* kind of place, which is all the map ever wanted from this.
|
||||
* A self-hoster pointing `TERA_REGIONS` (or `TERA_ORIGIN_LAT/LNG`) at somewhere
|
||||
* that is neither of the two boards should get moving aircraft rather than an
|
||||
* empty sky, so eight legs are laid out on evenly-spaced bearings through the
|
||||
* region centre. It is not their city's real airspace and does not pretend to be
|
||||
* — it is motion in the right kind of place, which is all the map ever wanted
|
||||
* from this.
|
||||
*/
|
||||
function genericPlan(lat: number, lng: number): WireSimRoute[] {
|
||||
const routes: WireSimRoute[] = [];
|
||||
@@ -127,9 +211,15 @@ function genericPlan(lat: number, lng: number): WireSimRoute[] {
|
||||
return routes;
|
||||
}
|
||||
|
||||
export function planFor(origin: { lat: number; lng: number }): WireSimRoute[] {
|
||||
const nearSf =
|
||||
Math.abs(origin.lat - 37.7749) < BAY_AREA_RADIUS &&
|
||||
Math.abs(origin.lng + 122.4194) < BAY_AREA_RADIUS;
|
||||
return nearSf ? BAY_AREA : genericPlan(origin.lat, origin.lng);
|
||||
export function planFor(centre: { lat: number; lng: number }): WireSimRoute[] {
|
||||
for (const airspace of AIRSPACES) {
|
||||
const [lat, lng] = airspace.centre;
|
||||
if (
|
||||
Math.abs(centre.lat - lat) < airspace.radius &&
|
||||
Math.abs(centre.lng - lng) < airspace.radius
|
||||
) {
|
||||
return airspace.routes;
|
||||
}
|
||||
}
|
||||
return genericPlan(centre.lat, centre.lng);
|
||||
}
|
||||
|
||||
+66
-3
@@ -9,28 +9,91 @@
|
||||
|
||||
const DEFAULT_TIMEOUT_MS = 6000;
|
||||
|
||||
/**
|
||||
* The largest upstream body this service will read.
|
||||
*
|
||||
* "Every outbound call is bounded" was true of the *time* and not of the size:
|
||||
* a bare `res.json()` reads whatever arrives, and what arrives is chosen by
|
||||
* somebody else's server. An ADS-B endpoint answering with 200,000 aircraft was
|
||||
* measured at 14.9 MB, parsed into an array this process then held and served
|
||||
* — cached, publicly — to every anonymous caller for the length of the TTL.
|
||||
*
|
||||
* Four megabytes is roughly two orders of magnitude above any honest answer
|
||||
* from the four upstreams here (a busy adsb.lol circle is tens of kilobytes, an
|
||||
* NWS observation is under ten) and well under anything that would trouble the
|
||||
* heap. It bounds the damage; `flights/adsb.ts` caps the row count, which
|
||||
* bounds what is kept.
|
||||
*/
|
||||
const MAX_BODY_BYTES = 4 * 1024 * 1024;
|
||||
|
||||
export interface GetJsonOptions {
|
||||
headers?: Record<string, string>;
|
||||
timeoutMs?: number;
|
||||
/** Body-size ceiling in bytes. Defaults to `MAX_BODY_BYTES`. */
|
||||
maxBytes?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* `null` on any failure at all — transport, status, or unparseable body. The
|
||||
* caller decides what a missing answer means; nothing here does.
|
||||
* `null` on any failure at all — transport, status, an oversized body, or one
|
||||
* that will not parse. The caller decides what a missing answer means; nothing
|
||||
* here does.
|
||||
*/
|
||||
export async function getJson<T>(url: string, opts: GetJsonOptions = {}): Promise<T | null> {
|
||||
const maxBytes = opts.maxBytes ?? MAX_BODY_BYTES;
|
||||
try {
|
||||
const res = await fetch(url, {
|
||||
headers: { accept: "application/json", ...opts.headers },
|
||||
signal: AbortSignal.timeout(opts.timeoutMs ?? DEFAULT_TIMEOUT_MS),
|
||||
});
|
||||
if (!res.ok) return null;
|
||||
return (await res.json()) as T;
|
||||
|
||||
/**
|
||||
* The header first, because it is free and it is the one that stops the
|
||||
* transfer before it happens. It is only advisory — a chunked response
|
||||
* sends none — so the body is counted as it streams as well, and the
|
||||
* `cancel()` closes the socket on a server that lied or did not say.
|
||||
*/
|
||||
const declared = Number(res.headers.get("content-length"));
|
||||
if (Number.isFinite(declared) && declared > maxBytes) {
|
||||
await res.body?.cancel();
|
||||
return null;
|
||||
}
|
||||
|
||||
const text = await readBounded(res, maxBytes);
|
||||
if (text === null) return null;
|
||||
return JSON.parse(text) as T;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** The body as text, or `null` the moment it goes over `maxBytes`. */
|
||||
async function readBounded(res: Response, maxBytes: number): Promise<string | null> {
|
||||
const body = res.body;
|
||||
// Undici always gives a stream; a test double or a `fetch` polyfill may not,
|
||||
// and falling back to `res.text()` there is still bounded by the header check
|
||||
// above and by the timeout.
|
||||
if (!body) {
|
||||
const text = await res.text();
|
||||
return text.length > maxBytes ? null : text;
|
||||
}
|
||||
const reader = body.getReader();
|
||||
const decoder = new TextDecoder();
|
||||
let size = 0;
|
||||
let out = "";
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
size += value.byteLength;
|
||||
if (size > maxBytes) {
|
||||
await reader.cancel();
|
||||
return null;
|
||||
}
|
||||
out += decoder.decode(value, { stream: true });
|
||||
}
|
||||
return out + decoder.decode();
|
||||
}
|
||||
|
||||
/**
|
||||
* A User-Agent that identifies this software and the operator running it.
|
||||
*
|
||||
|
||||
+86
-10
@@ -9,9 +9,42 @@
|
||||
* Private per-user markers never appear here at all. An authenticated browser
|
||||
* calls Workie directly with its own token, so private rows never transit this
|
||||
* process. CONTRACT.md §5.
|
||||
*
|
||||
* ### What the file looks like
|
||||
*
|
||||
* ```json
|
||||
* { "generatedAt": "2026-08-05T09:00:00Z",
|
||||
* "markers": [
|
||||
* { "id": "ferry-building", "label": "Ferry Building", "colorKey": "sector.civic",
|
||||
* "lat": 37.7955, "lng": -122.3937, "provenance": "hand-placed" }
|
||||
* ] }
|
||||
* ```
|
||||
*
|
||||
* A bare array is accepted too, because it is the obvious thing to hand-write
|
||||
* and a self-hoster's first marker file should not need a wrapper object. Every
|
||||
* row goes through `gate.ts`, which refuses — row by row, never repairing —
|
||||
* anything carrying a field nobody reviewed and anything whose `provenance` is
|
||||
* not on the allowlist. The counts come back on the wire in `refused`, because a
|
||||
* silent drop looks exactly like an empty database.
|
||||
*
|
||||
* ### Reloading without a restart
|
||||
*
|
||||
* `TERA_MARKERS_TTL` is the reload interval, not just a cache lifetime: once it
|
||||
* expires, the next request `stat`s the file and re-reads it only if the mtime
|
||||
* or the size moved. That is what makes a short TTL affordable — five seconds on
|
||||
* an unchanged file costs one `stat` every five seconds — so `npm run sync`
|
||||
* followed by its atomic rename is visible on the map within seconds, with
|
||||
* nothing restarted and nothing signalled.
|
||||
*
|
||||
* The other half is that **a read that fails keeps the last good snapshot**.
|
||||
* This used to replace the cache with an empty body, so a single unreadable read
|
||||
* — the file being rewritten by something less careful than the sync oneshot, a
|
||||
* permissions change, a full disk — emptied the map for a whole TTL. A stale
|
||||
* marker set is a far cheaper mistake than a map that quietly lost its markers,
|
||||
* which is the same trade the sync oneshot makes when it refuses to write.
|
||||
*/
|
||||
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { readFile, stat } from "node:fs/promises";
|
||||
import type { Config } from "../config.ts";
|
||||
import type { MarkersBody } from "../../../src/server/wire.ts";
|
||||
import { assertPublicShape } from "./gate.ts";
|
||||
@@ -29,24 +62,43 @@ interface Snapshot {
|
||||
markers?: unknown;
|
||||
}
|
||||
|
||||
/**
|
||||
* What a configured-but-unreadable file serves before it has ever been read.
|
||||
*
|
||||
* It goes out as a refusal rather than as a plain empty list on purpose: an
|
||||
* operator looking at `/api/v1/markers` and seeing nothing is owed the
|
||||
* difference between "there are no markers" and "this box cannot read the file
|
||||
* you pointed it at". Same reasoning as the gate's counts.
|
||||
*/
|
||||
function unreadable(file: string): MarkersBody {
|
||||
return {
|
||||
markers: [],
|
||||
generatedAt: new Date().toISOString(),
|
||||
refused: [{ reason: `the snapshot at ${file} could not be read`, count: 1 }],
|
||||
};
|
||||
}
|
||||
|
||||
export function createMarkerStore(config: Config, log: MarkerLog): MarkerStore {
|
||||
const { source, file, provenanceAllowlist, ttlSeconds } = config.markers;
|
||||
|
||||
let cached: MarkersBody | null = null;
|
||||
let readAt = 0;
|
||||
/** mtime and size of the file `cached` was built from. Empty means "unknown". */
|
||||
let signature = "";
|
||||
|
||||
async function load(): Promise<MarkersBody> {
|
||||
const now = new Date().toISOString();
|
||||
/** `null` when the file could not be read or parsed; the caller keeps what it has. */
|
||||
async function load(): Promise<MarkersBody | null> {
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(await readFile(file, "utf8"));
|
||||
} catch (err) {
|
||||
log.warn(`markers: cannot read ${file} (${String(err)}); serving no markers`);
|
||||
return { markers: [], generatedAt: now, refused: [] };
|
||||
log.warn(
|
||||
`markers: cannot read ${file} (${String(err)}); serving ` +
|
||||
`${cached === null ? "no markers" : "the last snapshot"}`,
|
||||
);
|
||||
return null;
|
||||
}
|
||||
|
||||
// A bare array is accepted because it is the obvious thing to hand-write,
|
||||
// and a self-hoster's first marker file should not need a wrapper object.
|
||||
const snapshot: Snapshot =
|
||||
Array.isArray(parsed) ? { markers: parsed } : ((parsed ?? {}) as Snapshot);
|
||||
|
||||
@@ -57,21 +109,45 @@ export function createMarkerStore(config: Config, log: MarkerLog): MarkerStore {
|
||||
|
||||
return {
|
||||
markers: accepted,
|
||||
generatedAt: snapshot.generatedAt ?? now,
|
||||
generatedAt: snapshot.generatedAt ?? new Date().toISOString(),
|
||||
refused,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the file on disk differs from the one behind `cached`.
|
||||
*
|
||||
* A failed `stat` answers "yes", so the read is attempted and reports the real
|
||||
* error once, rather than this function inventing a reason of its own.
|
||||
*/
|
||||
async function changed(): Promise<boolean> {
|
||||
try {
|
||||
const info = await stat(file);
|
||||
const next = `${info.mtimeMs}:${info.size}`;
|
||||
if (next === signature) return false;
|
||||
signature = next;
|
||||
return true;
|
||||
} catch {
|
||||
signature = "";
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
async current(): Promise<MarkersBody> {
|
||||
if (source === "none") {
|
||||
return { markers: [], generatedAt: new Date().toISOString(), refused: [] };
|
||||
}
|
||||
|
||||
if (cached === null || Date.now() - readAt > ttlSeconds * 1000) {
|
||||
cached = await load();
|
||||
if (cached === null || (await changed())) {
|
||||
const fresh = await load();
|
||||
if (fresh !== null) cached = fresh;
|
||||
}
|
||||
readAt = Date.now();
|
||||
}
|
||||
return cached;
|
||||
|
||||
return cached ?? unreadable(file);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,332 @@
|
||||
/**
|
||||
* The places this deployment will answer for, and the refusal of everywhere else.
|
||||
*
|
||||
* `TERA_ORIGIN_LAT/LNG` was one point, and the map has two cities six hundred
|
||||
* kilometres apart with genuinely different skies — the marine-layer comment in
|
||||
* `engine/atmosphere.ts` makes the point exactly: Los Angeles gets its own
|
||||
* weather, not San Francisco's fog. So weather and flights have to answer for a
|
||||
* *requested* place rather than for the box's one origin.
|
||||
*
|
||||
* ### Why this is an allowlist and not a lookup
|
||||
*
|
||||
* The obvious implementation — take `?lat=&lng=` and hand it to NWS — turns an
|
||||
* unauthenticated endpoint into a free geocoding proxy for the whole planet.
|
||||
* Two things go wrong with that, and neither is hypothetical. It is an
|
||||
* amplification vector: one cheap request here becomes one expensive request to
|
||||
* somebody else's public-good API, from an address they will blame. And it is
|
||||
* how a deployment's User-Agent gets blocked, because NWS's fair-use policy is
|
||||
* written against exactly this pattern and the contact string in
|
||||
* `TERA_WEATHER_CONTACT` is the operator's own name on the request.
|
||||
*
|
||||
* The rule here is therefore: **a caller's coordinate is never forwarded
|
||||
* upstream. It only selects among the points the operator configured.** A
|
||||
* request for Berkeley resolves to the San Francisco region and fetches San
|
||||
* Francisco's centre; a request for Fresno is refused with a 400 naming what is
|
||||
* served. The upstream key space is the region list, so it is bounded by the
|
||||
* environment file and cannot be grown by anybody sending requests — which is
|
||||
* the property that makes per-region caching, the NWS station cache and the
|
||||
* adsb.lol poll budget in `flights/adsb.ts` all bounded too.
|
||||
*
|
||||
* A coarse grid was the other candidate and was rejected: snapping to 0.5° still
|
||||
* leaves a caller able to name a hundred thousand distinct cells, which bounds
|
||||
* nothing that matters. Refusing is the honest answer, and a self-hoster who
|
||||
* wants their own city writes one line of `TERA_REGIONS`.
|
||||
*/
|
||||
|
||||
/** Kilometres per degree of latitude. Good to a tenth of a percent anywhere. */
|
||||
const KM_PER_DEGREE = 111.195;
|
||||
|
||||
/**
|
||||
* How far from a region's centre a request may land and still resolve there,
|
||||
* when the operator did not say.
|
||||
*
|
||||
* 120 km covers both shipped boards with room to spare — the far corner of the
|
||||
* Bay Area pack is 89 km from its centre and SoCal's is 97 km — while leaving
|
||||
* the two regions comfortably disjoint, since their centres are about 440 km
|
||||
* apart. It is deliberately not tight: the point of the radius is to refuse
|
||||
* somewhere this deployment has nothing to say about, not to police the edge of
|
||||
* the rendered board.
|
||||
*/
|
||||
export const DEFAULT_RADIUS_KM = 120;
|
||||
|
||||
export interface Region {
|
||||
/** Url-safe, and the same id the browser's city pack uses: `sf`, `socal`. */
|
||||
id: string;
|
||||
lat: number;
|
||||
lng: number;
|
||||
/** Kilometres. See `DEFAULT_RADIUS_KM`. */
|
||||
radiusKm: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* At least one region, always, with the first one being the default.
|
||||
*
|
||||
* A tuple rather than an array because "the default region" is read on every
|
||||
* request that omits a query, and a `Region | undefined` there would be a
|
||||
* falsehood the type system made everybody handle.
|
||||
*/
|
||||
export type RegionSet = [Region, ...Region[]];
|
||||
|
||||
/**
|
||||
* The two cities this repo ships, with the centres their packs declare —
|
||||
* `SAN_FRANCISCO_CITY.center` and `SOCAL_CITY.center` in `src/cities/`.
|
||||
*
|
||||
* They are restated rather than imported for the same reason `wire.ts` restates
|
||||
* `SimRoute`: a city pack is three thousand lines of coastline for a renderer
|
||||
* that owns three.js, and the API has one runtime dependency and intends to keep
|
||||
* it. Two numbers each, and the day a pack moves its centre the weather resolves
|
||||
* to a point a few kilometres off, which is a rounding error against a radius of
|
||||
* 120 km.
|
||||
*/
|
||||
const SHIPPED: Region[] = [
|
||||
{ id: "sf", lat: 37.7749, lng: -122.4194, radiusKm: DEFAULT_RADIUS_KM },
|
||||
{ id: "socal", lat: 33.82, lng: -118.05, radiusKm: DEFAULT_RADIUS_KM },
|
||||
];
|
||||
|
||||
/**
|
||||
* `id:lat,lng` with an optional `:radiusKm`, which is the whole grammar.
|
||||
*
|
||||
* The decimal places are capped in the pattern rather than checked afterwards
|
||||
* because it is the same cap the query parser applies, and the two agreeing is
|
||||
* the point: a coordinate nobody could ask for is a coordinate nobody should be
|
||||
* able to configure either.
|
||||
*/
|
||||
const ENTRY = /^([a-z0-9][a-z0-9-]{0,31}):(-?\d{1,3}(?:\.\d{1,6})?),(-?\d{1,3}(?:\.\d{1,6})?)(?::(\d{1,4}(?:\.\d{1,3})?))?$/;
|
||||
|
||||
export interface LoadRegionsOptions {
|
||||
/** `TERA_REGIONS`, raw. Empty means "work it out from the defaults". */
|
||||
spec: string;
|
||||
/** `TERA_ORIGIN_LAT/LNG`, already read and defaulted. */
|
||||
origin: { lat: number; lng: number };
|
||||
/** Whether the operator actually wrote an origin, as opposed to inheriting SF. */
|
||||
originConfigured: boolean;
|
||||
degraded: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* The region set for this box.
|
||||
*
|
||||
* Three cases, in order, and the ordering is what keeps every existing
|
||||
* deployment answering exactly as it did:
|
||||
*
|
||||
* 1. `TERA_REGIONS` is set — that list is the answer, in the operator's order.
|
||||
* 2. Otherwise the two shipped cities, because that is what the bundled map
|
||||
* draws and serving only one of them is wrong for half the board.
|
||||
* 3. On top of case 2, an operator who pointed `TERA_ORIGIN_LAT/LNG` somewhere
|
||||
* else gets that point as a region of its own, first in the list and
|
||||
* therefore the default. A bare `GET /api/v1/weather` on their box answers
|
||||
* for their origin, exactly as it did before this file existed. If their
|
||||
* origin already falls inside a shipped region, that region moves to the
|
||||
* front instead of being duplicated.
|
||||
*
|
||||
* Malformed entries are dropped with a line in `degraded` rather than being
|
||||
* fatal, and a spec in which *nothing* parses falls all the way back to case 2.
|
||||
* CONTRACT.md §5.1: a typo is a demotion, never a refusal to boot.
|
||||
*/
|
||||
export function loadRegions(opts: LoadRegionsOptions): RegionSet {
|
||||
const configured = parseSpec(opts.spec, opts.degraded);
|
||||
if (configured.length > 0) return asSet(configured, configured[0] as Region);
|
||||
|
||||
const shipped = SHIPPED.map((region) => ({ ...region }));
|
||||
const containing = shipped.findIndex((region) => contains(region, opts.origin));
|
||||
|
||||
if (containing > 0) {
|
||||
const moved = shipped[containing] as Region;
|
||||
shipped.splice(containing, 1);
|
||||
shipped.unshift(moved);
|
||||
} else if (containing === -1 && opts.originConfigured) {
|
||||
shipped.unshift({
|
||||
id: "origin",
|
||||
lat: opts.origin.lat,
|
||||
lng: opts.origin.lng,
|
||||
radiusKm: DEFAULT_RADIUS_KM,
|
||||
});
|
||||
}
|
||||
|
||||
return asSet(shipped, shipped[0] as Region);
|
||||
}
|
||||
|
||||
/**
|
||||
* `Region[]` to `RegionSet`, with the caller supplying the head it has already
|
||||
* proved is there. The alternative is a cast on the whole array, which would
|
||||
* also silence the empty case this type exists to rule out.
|
||||
*/
|
||||
function asSet(regions: Region[], head: Region): RegionSet {
|
||||
return [head, ...regions.slice(1)];
|
||||
}
|
||||
|
||||
function parseSpec(spec: string, degraded: string[]): Region[] {
|
||||
const trimmed = spec.trim();
|
||||
if (trimmed === "") return [];
|
||||
|
||||
const regions: Region[] = [];
|
||||
for (const raw of trimmed.split(/[;\n]/)) {
|
||||
const entry = raw.trim();
|
||||
if (entry === "") continue;
|
||||
|
||||
const match = ENTRY.exec(entry);
|
||||
if (match === null) {
|
||||
degraded.push(
|
||||
`TERA_REGIONS entry "${entry}" is not \`id:lat,lng\` with an optional ` +
|
||||
`\`:radiusKm\` (try \`sf:37.7749,-122.4194\`); ignoring it.`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
const id = match[1] as string;
|
||||
const lat = Number(match[2]);
|
||||
const lng = Number(match[3]);
|
||||
const radiusKm = match[4] === undefined ? DEFAULT_RADIUS_KM : Number(match[4]);
|
||||
|
||||
if (Math.abs(lat) > 90 || Math.abs(lng) > 180 || radiusKm <= 0) {
|
||||
degraded.push(`TERA_REGIONS entry "${entry}" is not a place on Earth; ignoring it.`);
|
||||
continue;
|
||||
}
|
||||
if (regions.some((region) => region.id === id)) {
|
||||
degraded.push(`TERA_REGIONS names "${id}" twice; keeping the first one.`);
|
||||
continue;
|
||||
}
|
||||
|
||||
regions.push({ id, lat, lng, radiusKm });
|
||||
}
|
||||
|
||||
if (regions.length === 0 && trimmed !== "") {
|
||||
degraded.push(
|
||||
"TERA_REGIONS was set but nothing in it parsed; serving the two cities the " +
|
||||
"map ships with instead.",
|
||||
);
|
||||
}
|
||||
return regions;
|
||||
}
|
||||
|
||||
// ---- Answering a request --------------------------------------------------
|
||||
|
||||
/**
|
||||
* What a route hands in. `unknown` throughout because this is untrusted query
|
||||
* input: Fastify hands back a string for `?lat=1`, an **array** for
|
||||
* `?lat=1&lat=2`, and `undefined` for absent, and a signature that claimed
|
||||
* `string` would be a lie the first time somebody repeated a parameter.
|
||||
*/
|
||||
export interface RegionQuery {
|
||||
city?: unknown;
|
||||
lat?: unknown;
|
||||
lng?: unknown;
|
||||
}
|
||||
|
||||
export type RegionResolution =
|
||||
| { ok: true; region: Region }
|
||||
/** Already a sentence. The route sends it as `ErrorBody.message`. */
|
||||
| { ok: false; message: string };
|
||||
|
||||
/** Six decimal places is about 0.1 m. Anything finer is a bug or a probe. */
|
||||
const DEGREES = /^-?\d{1,3}(?:\.\d{1,6})?$/;
|
||||
|
||||
const CITY_ID = /^[a-z0-9][a-z0-9-]{0,31}$/;
|
||||
|
||||
/**
|
||||
* Turn a query into one of the configured regions, or into a refusal.
|
||||
*
|
||||
* Every branch that is not a resolved region is a **400**, not a degraded body.
|
||||
* That is the one place this file departs from the "degrade, never fail"
|
||||
* convention, and the distinction is who made the mistake: an upstream that is
|
||||
* down is not the caller's fault and must not become their problem, whereas
|
||||
* `?lat=banana` is a client bug that a 200 full of clear sky would hide until
|
||||
* somebody wondered why the fog never rolls in.
|
||||
*/
|
||||
export function resolveRegion(regions: RegionSet, query: RegionQuery): RegionResolution {
|
||||
const city = query.city;
|
||||
const hasCity = city !== undefined && city !== "";
|
||||
const hasLat = query.lat !== undefined && query.lat !== "";
|
||||
const hasLng = query.lng !== undefined && query.lng !== "";
|
||||
|
||||
if (hasCity && (hasLat || hasLng)) {
|
||||
return { ok: false, message: "Ask with ?city= or with ?lat=&lng=, not both." };
|
||||
}
|
||||
|
||||
if (hasCity) {
|
||||
if (typeof city !== "string" || !CITY_ID.test(city)) {
|
||||
return { ok: false, message: `city must be one of: ${served(regions)}.` };
|
||||
}
|
||||
const region = regions.find((candidate) => candidate.id === city);
|
||||
if (region === undefined) {
|
||||
return { ok: false, message: `This deployment serves: ${served(regions)}.` };
|
||||
}
|
||||
return { ok: true, region };
|
||||
}
|
||||
|
||||
if (hasLat !== hasLng) {
|
||||
return { ok: false, message: "lat and lng have to be given together." };
|
||||
}
|
||||
|
||||
if (!hasLat) return { ok: true, region: regions[0] };
|
||||
|
||||
const lat = degrees(query.lat, 90);
|
||||
const lng = degrees(query.lng, 180);
|
||||
if (lat === null || lng === null) {
|
||||
return {
|
||||
ok: false,
|
||||
message:
|
||||
"lat and lng must be plain decimal degrees within ±90 and ±180, " +
|
||||
"with at most six decimal places.",
|
||||
};
|
||||
}
|
||||
|
||||
const region = nearest(regions, lat, lng);
|
||||
if (region === null) {
|
||||
return {
|
||||
ok: false,
|
||||
message:
|
||||
`Nothing this deployment serves is near ${lat},${lng}. It serves: ` +
|
||||
`${served(regions)}. Ask for one of those by id, or add yours to ` +
|
||||
`TERA_REGIONS on the server.`,
|
||||
};
|
||||
}
|
||||
return { ok: true, region };
|
||||
}
|
||||
|
||||
function served(regions: RegionSet): string {
|
||||
return regions.map((region) => region.id).join(", ");
|
||||
}
|
||||
|
||||
function degrees(raw: unknown, limit: number): number | null {
|
||||
if (typeof raw !== "string") return null;
|
||||
const trimmed = raw.trim();
|
||||
// The pattern is doing the work `Number()` would do badly: it rejects `NaN`,
|
||||
// `Infinity`, `1e400`, `0x2f`, `37.7749deg` and the empty string, all of which
|
||||
// `Number()` either accepts or turns into a value that then has to be
|
||||
// re-checked. One regex, one meaning.
|
||||
if (!DEGREES.test(trimmed)) return null;
|
||||
const value = Number(trimmed);
|
||||
return Number.isFinite(value) && Math.abs(value) <= limit ? value : null;
|
||||
}
|
||||
|
||||
/** The closest region that claims the point, or `null` if none of them does. */
|
||||
function nearest(regions: RegionSet, lat: number, lng: number): Region | null {
|
||||
let best: Region | null = null;
|
||||
let bestKm = Infinity;
|
||||
for (const region of regions) {
|
||||
const km = distanceKm(region, lat, lng);
|
||||
if (km <= region.radiusKm && km < bestKm) {
|
||||
best = region;
|
||||
bestKm = km;
|
||||
}
|
||||
}
|
||||
return best;
|
||||
}
|
||||
|
||||
/**
|
||||
* Equirectangular rather than haversine, because at these distances the error is
|
||||
* under half a percent and the comparison it feeds is against a radius chosen to
|
||||
* the nearest ten kilometres. Trigonometry accurate to the metre would be
|
||||
* decorating a threshold that is deliberately fuzzy.
|
||||
*/
|
||||
function distanceKm(region: Region, lat: number, lng: number): number {
|
||||
const dLat = lat - region.lat;
|
||||
const meanLat = (((lat + region.lat) / 2) * Math.PI) / 180;
|
||||
const dLng = (lng - region.lng) * Math.cos(meanLat);
|
||||
return Math.hypot(dLat, dLng) * KM_PER_DEGREE;
|
||||
}
|
||||
|
||||
function contains(region: Region, point: { lat: number; lng: number }): boolean {
|
||||
return distanceKm(region, point.lat, point.lng) <= region.radiusKm;
|
||||
}
|
||||
@@ -1,18 +1,48 @@
|
||||
/**
|
||||
* `GET /api/v1/flights`.
|
||||
* `GET /api/v1/flights` — for a city, not for the box.
|
||||
*
|
||||
* `?city=socal`, or `?lat=&lng=` resolved against the same allowlist the weather
|
||||
* route uses, or neither for the default region. `regions.ts` owns the
|
||||
* validation, the refusal, and the reasoning behind refusing at all: a live
|
||||
* traffic endpoint that will fetch any coordinate on demand is an amplifier
|
||||
* pointed at a volunteer-funded feed.
|
||||
*
|
||||
* Publicly cacheable, because the whole design of the plan is that one response
|
||||
* serves every viewer for its whole TTL. Aircraft are not personal data and this
|
||||
* body never varies by who asked.
|
||||
* serves every viewer of a region for its whole TTL. Aircraft are not personal
|
||||
* data and this body never varies by who asked — only by where.
|
||||
*
|
||||
* ### `radiusNm` on the query is ignored, deliberately
|
||||
*
|
||||
* The browser sends one. It is dropped, and the size of the circle stays
|
||||
* `TERA_ADSB_RADIUS_NM`, for a reason that is not stubbornness: the cache key
|
||||
* would have to include it, and a caller who can choose the key can make this
|
||||
* box hold an unbounded number of entries and issue an unbounded number of
|
||||
* distinct upstream requests — each one more expensive than the last, since a
|
||||
* wider circle is more work for the feed to answer. Every bound in
|
||||
* `flights/index.ts` and `regions.ts` rests on the key space being the operator's
|
||||
* region list, and a query parameter that widens it dissolves all of them.
|
||||
*
|
||||
* An operator whose board is bigger than the circle raises
|
||||
* `TERA_ADSB_RADIUS_NM`; 60 nm covers both shipped cities. The client already
|
||||
* discards aircraft outside the region it drew, so a circle that is too large
|
||||
* costs a little bandwidth and nothing else.
|
||||
*/
|
||||
|
||||
import type { FastifyInstance } from "fastify";
|
||||
import { publicCache } from "../cache.ts";
|
||||
import { resolveRegion, type RegionQuery } from "../regions.ts";
|
||||
import type { ErrorBody } from "../../../src/server/wire.ts";
|
||||
import type { Services } from "../services.ts";
|
||||
|
||||
export function registerFlights(app: FastifyInstance, services: Services): void {
|
||||
app.get("/api/v1/flights", async (req, reply) => {
|
||||
const body = await services.flights.current();
|
||||
app.get<{ Querystring: RegionQuery }>("/api/v1/flights", async (req, reply) => {
|
||||
const resolved = resolveRegion(services.config.regions, req.query);
|
||||
if (!resolved.ok) {
|
||||
const error: ErrorBody = { error: "bad_request", message: resolved.message };
|
||||
return reply.code(400).send(error);
|
||||
}
|
||||
|
||||
const body = await services.flights.current(resolved.region);
|
||||
publicCache(req, reply, body.ttlSeconds);
|
||||
return body;
|
||||
});
|
||||
|
||||
@@ -10,17 +10,37 @@
|
||||
* `degraded` is what makes it more than a liveness probe: every demotion the
|
||||
* config made is printed here, so "why is the weather always clear" has an
|
||||
* answer that does not require log access.
|
||||
*
|
||||
* `regions` joins it for the same reason. Weather and flights now refuse a place
|
||||
* this box does not serve, so a client that guesses `?city=` and a 400 it cannot
|
||||
* explain is the failure this field prevents: ask health once, learn what may be
|
||||
* asked for, and an operator diagnosing "why is there no SoCal weather" reads
|
||||
* the answer instead of the env file. Publishing the allowlist gives nothing
|
||||
* away — knowing what is served is not the same as widening it, and the ids are
|
||||
* the names of the cities the map already draws.
|
||||
*/
|
||||
|
||||
import type { FastifyInstance } from "fastify";
|
||||
import type { Region } from "../regions.ts";
|
||||
import type { HealthBody } from "../../../src/server/wire.ts";
|
||||
import type { Services } from "../services.ts";
|
||||
|
||||
/**
|
||||
* `HealthBody` plus the served regions.
|
||||
*
|
||||
* The field belongs in `src/server/wire.ts` beside the body it extends, and it
|
||||
* is stated here only because that file is the browser side of this change and
|
||||
* lands with it. Fold `regions: Region[]` into `HealthBody` and this alias goes
|
||||
* away; nothing else has to move, because the shape is already exactly what the
|
||||
* route serves.
|
||||
*/
|
||||
type HealthBodyWithRegions = HealthBody & { regions: Region[] };
|
||||
|
||||
export function registerHealth(app: FastifyInstance, services: Services): void {
|
||||
const { config, startedAt } = services;
|
||||
|
||||
app.get("/api/v1/health", async () => {
|
||||
const body: HealthBody = {
|
||||
const body: HealthBodyWithRegions = {
|
||||
ok: true,
|
||||
service: "tera-api",
|
||||
version: config.version,
|
||||
@@ -36,6 +56,10 @@ export function registerHealth(app: FastifyInstance, services: Services): void {
|
||||
? config.auth.entryUrl
|
||||
: null,
|
||||
},
|
||||
// In the config's order, so the first entry is the region a request with
|
||||
// no query gets. A client reading this can pick its default the same way
|
||||
// the server does.
|
||||
regions: config.regions,
|
||||
degraded: config.degraded,
|
||||
};
|
||||
return body;
|
||||
|
||||
@@ -1,24 +1,73 @@
|
||||
/**
|
||||
* `GET /api/v1/markers`.
|
||||
* `GET /api/v1/markers` — the one route the `member` tier is about.
|
||||
*
|
||||
* The public snapshot, and nothing else. Private per-user markers are never
|
||||
* proxied through this box — an authenticated browser calls Workie directly with
|
||||
* its own token, so a private row never enters this process and cannot leave it.
|
||||
* CONTRACT.md §5.
|
||||
* CONTRACT.md §5. Every row served here has been through the provenance gate in
|
||||
* `markers/gate.ts`, which is what keeps a public snapshot from quietly becoming
|
||||
* a Publicly Used Derivative Database under ODbL.
|
||||
*
|
||||
* Every row served here has been through the provenance gate in `markers/gate.ts`,
|
||||
* which is what keeps a public snapshot from quietly becoming a Publicly Used
|
||||
* Derivative Database under ODbL.
|
||||
* ### Why a configured feed is refused to anonymous callers
|
||||
*
|
||||
* `src/access.ts` has three tiers and, until this route, the middle one gated
|
||||
* nothing at all: `member` was a word in a type union that no server behaviour
|
||||
* corresponded to. A tier that never refuses anybody anything is not a tier, and
|
||||
* one that lives only in the client is worse — it is a UI hiding a control over
|
||||
* a body the API hands to whoever asks. **So: where a marker feed is configured,
|
||||
* it takes a session.** That refusal is the whole of what makes membership real,
|
||||
* it is enforced here rather than drawn in the browser, and there is
|
||||
* deliberately no `TERA_MARKERS_PUBLIC` escape hatch to undo it.
|
||||
*
|
||||
* Two consequences worth stating out loud:
|
||||
*
|
||||
* - **A box with no feed still answers 200 and an empty list.** `source: none`
|
||||
* is the zero-config default and there is nothing there to protect; making a
|
||||
* stranger sign in to be told "no markers" would fail the acceptance test in
|
||||
* `boot.test.ts` and gain nobody anything.
|
||||
* - **A feed with `TERA_AUTH_MODE=none` is unreachable by everyone**, because
|
||||
* nobody on such a box is ever authenticated. That is the fail-closed
|
||||
* direction, it is the one private offices already take, and `config.ts`
|
||||
* pushes a line into `degraded` saying so rather than letting an operator
|
||||
* discover it from an empty map.
|
||||
*
|
||||
* ### 401 here, 404 for an office
|
||||
*
|
||||
* `offices.ts` answers 404 for a private office because a 403 there is an
|
||||
* enumeration oracle — walk the id space, read the status codes, learn every
|
||||
* tenant. Nothing is enumerable here: there is one feed, at a fixed path, and
|
||||
* `/api/v1/health` already publishes `sources.markers`, so whether this
|
||||
* deployment has markers is not a secret being kept. What the caller needs to
|
||||
* know is "sign in and ask again", which is what 401 with `WWW-Authenticate`
|
||||
* says. A 404 would be a lie told to protect nothing.
|
||||
*/
|
||||
|
||||
import type { FastifyInstance } from "fastify";
|
||||
import { publicCache } from "../cache.ts";
|
||||
import type { ErrorBody } from "../../../src/server/wire.ts";
|
||||
import type { Services } from "../services.ts";
|
||||
|
||||
const UNAUTHORIZED: ErrorBody = {
|
||||
error: "unauthorized",
|
||||
message: "The marker feed is for signed-in members.",
|
||||
};
|
||||
|
||||
export function registerMarkers(app: FastifyInstance, services: Services): void {
|
||||
app.get("/api/v1/markers", async (req, reply) => {
|
||||
const body = await services.markers.current();
|
||||
publicCache(req, reply, services.config.markers.ttlSeconds);
|
||||
return body;
|
||||
if (services.config.markers.source === "none") {
|
||||
const body = await services.markers.current();
|
||||
publicCache(req, reply, services.config.markers.ttlSeconds);
|
||||
return body;
|
||||
}
|
||||
|
||||
const viewer = await services.auth.resolve(req);
|
||||
if (!viewer.authenticated) {
|
||||
return reply.code(401).header("www-authenticate", "Bearer").send(UNAUTHORIZED);
|
||||
}
|
||||
|
||||
// No `publicCache`. This body took a credential to obtain, and a shared
|
||||
// cache holding it would hand one member's copy to the next caller — the
|
||||
// exact thing the fail-closed default in `cache.ts` exists to prevent.
|
||||
return services.markers.current();
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1,19 +1,40 @@
|
||||
/**
|
||||
* `GET /api/v1/weather`.
|
||||
* `GET /api/v1/weather` — for a place, not for the box.
|
||||
*
|
||||
* Always 200, always a body. A source that is down, misconfigured or absent
|
||||
* produces `synthetic: true` and a clear day — there is no failure mode here in
|
||||
* which the caller has to decide what to render, because the answer to "what is
|
||||
* the sky doing" is never allowed to be a 503.
|
||||
* `?city=sf`, or `?lat=&lng=` which resolves to whichever configured region
|
||||
* claims the point, or neither, which answers for the default region. The
|
||||
* validation and the refusal both live in `regions.ts`; the paragraph there on
|
||||
* why this is an allowlist rather than a lookup is the one worth reading before
|
||||
* touching this file.
|
||||
*
|
||||
* Two kinds of answer, and the split is deliberate:
|
||||
*
|
||||
* - **A bad request is a 400.** `?lat=banana`, or a coordinate in a city this
|
||||
* deployment does not serve, is a client bug, and a 200 full of clear sky
|
||||
* would hide it until somebody wondered why the fog never rolled in.
|
||||
* - **Everything else is a 200 with a body.** A source that is down,
|
||||
* misconfigured or absent produces `synthetic: true` and a clear day. There is
|
||||
* no failure mode in which the caller has to decide what to render, because
|
||||
* the answer to "what is the sky doing" is never allowed to be a 503.
|
||||
*/
|
||||
|
||||
import type { FastifyInstance } from "fastify";
|
||||
import { publicCache } from "../cache.ts";
|
||||
import { resolveRegion, type RegionQuery } from "../regions.ts";
|
||||
import type { ErrorBody } from "../../../src/server/wire.ts";
|
||||
import type { Services } from "../services.ts";
|
||||
|
||||
export function registerWeather(app: FastifyInstance, services: Services): void {
|
||||
app.get("/api/v1/weather", async (req, reply) => {
|
||||
const body = await services.weather.current();
|
||||
app.get<{ Querystring: RegionQuery }>("/api/v1/weather", async (req, reply) => {
|
||||
const resolved = resolveRegion(services.config.regions, req.query);
|
||||
if (!resolved.ok) {
|
||||
const error: ErrorBody = { error: "bad_request", message: resolved.message };
|
||||
return reply.code(400).send(error);
|
||||
}
|
||||
|
||||
const body = await services.weather.current(resolved.region);
|
||||
// Query strings are part of a shared cache's key, so two regions cannot
|
||||
// collide here and no extra `Vary` is owed.
|
||||
publicCache(req, reply, services.config.weather.ttlSeconds);
|
||||
return body;
|
||||
});
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
/**
|
||||
* Live traffic, and the budget that keeps it welcome.
|
||||
*
|
||||
* adsb.lol is a volunteer-fed community feed that asks for no more than one
|
||||
* request per second and will rate-limit a caller who ignores it. Most of this
|
||||
* file is therefore about **how often this server is capable of calling out**,
|
||||
* not about what comes back: one poll per region per TTL, a TTL with a floor
|
||||
* under it, and a refused query that costs the upstream nothing. The arithmetic
|
||||
* those tests pin down is written out in `flights/index.ts`.
|
||||
*
|
||||
* The one that is not about rate is the URL assertion. A caller's coordinate
|
||||
* must select a configured region and must never itself be fetched.
|
||||
*/
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { after, before, beforeEach, describe, it } from "node:test";
|
||||
import { buildApp } from "../app.ts";
|
||||
import { loadConfig } from "../config.ts";
|
||||
import type { FlightsBody } from "../../../src/server/wire.ts";
|
||||
|
||||
const realFetch = globalThis.fetch;
|
||||
let calls: string[] = [];
|
||||
let feedIsUp = true;
|
||||
/** A body to serve instead of the usual one, for the wrong-shape tests. */
|
||||
let feedOverride: unknown = undefined;
|
||||
|
||||
/** One aircraft, whose id encodes the point that was asked about. */
|
||||
function feed(url: string): unknown | undefined {
|
||||
if (!feedIsUp) return undefined;
|
||||
const point = /\/v2\/point\/(-?[\d.]+)\/(-?[\d.]+)\/(\d+)$/.exec(url);
|
||||
if (point === null) return undefined;
|
||||
if (feedOverride !== undefined) return feedOverride;
|
||||
return {
|
||||
now: 1_770_000_000_000,
|
||||
ac: [{ hex: `a${point[1]}`, flight: "LMB1 ", lat: 37.5, lon: -122.3, alt_baro: 10_000, track: 90 }],
|
||||
};
|
||||
}
|
||||
|
||||
globalThis.fetch = (async (input: unknown) => {
|
||||
const url = String(input);
|
||||
calls.push(url);
|
||||
const body = feed(url);
|
||||
if (body === undefined) return new Response("nope", { status: 503 });
|
||||
return new Response(JSON.stringify(body), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}) as unknown as typeof globalThis.fetch;
|
||||
|
||||
after(() => {
|
||||
globalThis.fetch = realFetch;
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
calls = [];
|
||||
feedIsUp = true;
|
||||
feedOverride = undefined;
|
||||
});
|
||||
|
||||
function appWith(env: Record<string, string>) {
|
||||
const config = loadConfig(env);
|
||||
config.logLevel = "silent";
|
||||
return buildApp(config);
|
||||
}
|
||||
|
||||
const adsbEnv = { TERA_FLIGHTS_SOURCE: "adsb" };
|
||||
|
||||
describe("the ADS-B source", () => {
|
||||
it("polls each city's own centre, and never the caller's coordinate", async () => {
|
||||
const app = appWith(adsbEnv);
|
||||
after(() => app.close());
|
||||
|
||||
// Pasadena, which is inside the Southland region and is not a point this
|
||||
// deployment serves.
|
||||
await app.inject({ method: "GET", url: "/api/v1/flights?lat=34.1478&lng=-118.1445" });
|
||||
await app.inject({ method: "GET", url: "/api/v1/flights?city=sf" });
|
||||
|
||||
assert.deepEqual(calls, [
|
||||
"https://api.adsb.lol/v2/point/33.8200/-118.0500/40",
|
||||
"https://api.adsb.lol/v2/point/37.7749/-122.4194/40",
|
||||
]);
|
||||
});
|
||||
|
||||
it("clamps a radius the feeds would reject, and says so", async () => {
|
||||
const config = loadConfig({ ...adsbEnv, TERA_ADSB_RADIUS_NM: "2500" });
|
||||
config.logLevel = "silent";
|
||||
const app = buildApp(config);
|
||||
after(() => app.close());
|
||||
|
||||
assert.equal(config.flights.radiusNm, 250);
|
||||
assert.match(config.degraded[0] ?? "", /TERA_ADSB_RADIUS_NM/);
|
||||
|
||||
await app.inject({ method: "GET", url: "/api/v1/flights" });
|
||||
assert.equal(calls[0], "https://api.adsb.lol/v2/point/37.7749/-122.4194/250");
|
||||
});
|
||||
|
||||
it("polls once per region per TTL, whatever the request rate is", async () => {
|
||||
const app = appWith(adsbEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const ask = (city: string) =>
|
||||
app.inject({ method: "GET", url: `/api/v1/flights?city=${city}` });
|
||||
await Promise.all([ask("sf"), ask("sf"), ask("socal"), ask("sf"), ask("socal")]);
|
||||
for (let i = 0; i < 6; i++) await ask("sf");
|
||||
|
||||
// Two regions, eleven requests, two upstream calls.
|
||||
assert.equal(calls.length, 2);
|
||||
});
|
||||
|
||||
it("floors the poll interval so TERA_FLIGHTS_TTL=0 is not a flood", async () => {
|
||||
// This is the bug the floor exists for: a zero TTL used to mean one
|
||||
// outbound request per inbound request, straight through the rate limit,
|
||||
// from a setting that reads like "as fresh as possible".
|
||||
const app = appWith({ ...adsbEnv, TERA_FLIGHTS_TTL: "0" });
|
||||
after(() => app.close());
|
||||
|
||||
let body: FlightsBody | null = null;
|
||||
for (let i = 0; i < 8; i++) {
|
||||
body = (await app.inject({ method: "GET", url: "/api/v1/flights" })).json<FlightsBody>();
|
||||
}
|
||||
assert.equal(calls.length, 1);
|
||||
assert.ok(body !== null && body.mode === "live" && body.ttlSeconds === 5);
|
||||
});
|
||||
|
||||
it("caps the poll interval too, because aircraft move", async () => {
|
||||
const app = appWith({ ...adsbEnv, TERA_FLIGHTS_TTL: "3600" });
|
||||
after(() => app.close());
|
||||
|
||||
const body = (await app.inject({ method: "GET", url: "/api/v1/flights" })).json<FlightsBody>();
|
||||
assert.ok(body.mode === "live" && body.ttlSeconds === 15);
|
||||
});
|
||||
|
||||
it("costs the upstream nothing when the query is refused", async () => {
|
||||
const app = appWith(adsbEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const res = await app.inject({ method: "GET", url: "/api/v1/flights?city=atlantis" });
|
||||
assert.equal(res.statusCode, 400);
|
||||
assert.deepEqual(calls, []);
|
||||
});
|
||||
|
||||
it("falls back to the simulated plan rather than to an empty sky", async () => {
|
||||
feedIsUp = false;
|
||||
const app = appWith(adsbEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const body = (await app.inject({ method: "GET", url: "/api/v1/flights" })).json<FlightsBody>();
|
||||
assert.equal(body.mode, "plan");
|
||||
assert.ok(body.mode === "plan" && body.routes.length > 0);
|
||||
});
|
||||
|
||||
it("credits the feed it took the positions from", async () => {
|
||||
const app = appWith(adsbEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const body = (await app.inject({ method: "GET", url: "/api/v1/flights" })).json<FlightsBody>();
|
||||
assert.ok(body.mode === "live");
|
||||
assert.match(body.attribution?.[0] ?? "", /adsb\.lol/);
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* The failure that turns a rate limit inside out.
|
||||
*
|
||||
* A feed that goes *down* is handled everywhere. A feed that stays up and
|
||||
* answers 200 with a field of the wrong type used to throw out of `normalise`,
|
||||
* past `upstream.ts` — which then never stamped its clock — and out to the
|
||||
* route as a 500. The TTL is the only rate limit on outbound calls, so losing it
|
||||
* meant one request to adsb.lol per inbound request, from the operator's
|
||||
* address, for as long as the feed stayed broken.
|
||||
*/
|
||||
describe("a feed that changed shape", () => {
|
||||
it("answers with the plan instead of a 500", async () => {
|
||||
// Valid JSON, wrong shape: `ac` is a number where an array belongs.
|
||||
feedOverride = { ac: 5, now: 1 };
|
||||
const app = appWith(adsbEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const res = await app.inject({ method: "GET", url: "/api/v1/flights" });
|
||||
assert.equal(res.statusCode, 200);
|
||||
assert.equal(res.json<FlightsBody>().mode, "plan");
|
||||
});
|
||||
|
||||
it("still polls once per TTL, which is the whole of the rate limit", async () => {
|
||||
feedOverride = { ac: 5, now: 1 };
|
||||
const app = appWith(adsbEnv);
|
||||
after(() => app.close());
|
||||
|
||||
for (let i = 0; i < 5; i++) await app.inject({ method: "GET", url: "/api/v1/flights" });
|
||||
assert.equal(calls.length, 1);
|
||||
});
|
||||
|
||||
it("survives rows that are not objects", async () => {
|
||||
feedOverride = { ac: [null, 7, "LMB1", { hex: "ok", lat: 37.5, lon: -122.3 }], now: 1 };
|
||||
const app = appWith(adsbEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const body = (await app.inject({ method: "GET", url: "/api/v1/flights" })).json<FlightsBody>();
|
||||
assert.ok(body.mode === "live" && body.aircraft.length === 1);
|
||||
});
|
||||
|
||||
it("caps how much of somebody else's sky it will hold and serve", async () => {
|
||||
feedOverride = {
|
||||
now: 1,
|
||||
ac: Array.from({ length: 6000 }, (_, i) => ({
|
||||
hex: `x${i}`,
|
||||
lat: 37.5,
|
||||
lon: -122.3,
|
||||
alt_baro: 1000,
|
||||
})),
|
||||
};
|
||||
const app = appWith(adsbEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const body = (await app.inject({ method: "GET", url: "/api/v1/flights" })).json<FlightsBody>();
|
||||
assert.ok(body.mode === "live" && body.aircraft.length === 5000);
|
||||
});
|
||||
});
|
||||
|
||||
describe("a local receiver", () => {
|
||||
let path = "";
|
||||
|
||||
before(async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "tera-flights-"));
|
||||
path = join(dir, "aircraft.json");
|
||||
await writeFile(
|
||||
path,
|
||||
JSON.stringify({
|
||||
now: 1_770_000_000,
|
||||
aircraft: [{ hex: "abc123", flight: "LMB9 ", lat: 37.6, lon: -122.4, alt_baro: 3000 }],
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("has one antenna, so every region reads the same snapshot once", async () => {
|
||||
const app = appWith({ TERA_FLIGHTS_SOURCE: "dump1090", TERA_DUMP1090_PATH: path });
|
||||
after(() => app.close());
|
||||
|
||||
const sf = (
|
||||
await app.inject({ method: "GET", url: "/api/v1/flights?city=sf" })
|
||||
).json<FlightsBody>();
|
||||
assert.ok(sf.mode === "live" && sf.aircraft.length === 1);
|
||||
|
||||
// Take the file away, then ask for the other city inside the TTL. A live
|
||||
// answer proves the two regions share one cache entry — a per-region key
|
||||
// would have gone back to disk here and found nothing.
|
||||
await rm(path);
|
||||
const socal = (
|
||||
await app.inject({ method: "GET", url: "/api/v1/flights?city=socal" })
|
||||
).json<FlightsBody>();
|
||||
assert.ok(socal.mode === "live");
|
||||
assert.deepEqual(socal.aircraft, sf.aircraft);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,263 @@
|
||||
/**
|
||||
* The marker feed: the refusal that makes the `member` tier real, and a file an
|
||||
* operator can actually use.
|
||||
*
|
||||
* The first half is one assertion said several ways. `src/access.ts` has three
|
||||
* tiers and the middle one gated nothing — `member` was a word in a type union
|
||||
* with no server behaviour behind it. A tier that never refuses anybody is not a
|
||||
* tier, so a configured feed takes a session, and the test that matters is the
|
||||
* negative: an anonymous caller gets 401 and no rows, with no query parameter,
|
||||
* header or cleared cookie that changes it.
|
||||
*
|
||||
* The second half is the file itself — what a malformed row does to the rest of
|
||||
* the snapshot, and the reload path, which has to work without a restart because
|
||||
* the sync oneshot runs on a timer and nothing signals this process.
|
||||
*
|
||||
* Follows `offices.test.ts`: a temp directory, `buildApp` over a fake
|
||||
* environment, `inject()` rather than a socket, and HS256 by hand.
|
||||
*/
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { createHmac } from "node:crypto";
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { after, before, beforeEach, describe, it } from "node:test";
|
||||
import { setTimeout as sleep } from "node:timers/promises";
|
||||
import { buildApp } from "../app.ts";
|
||||
import { loadConfig } from "../config.ts";
|
||||
import type { ErrorBody, MarkersBody } from "../../../src/server/wire.ts";
|
||||
|
||||
const SECRET = "not-a-real-secret-and-never-was";
|
||||
|
||||
const FERRY = {
|
||||
id: "ferry-building",
|
||||
label: "Ferry Building",
|
||||
colorKey: "sector.civic",
|
||||
lat: 37.7955,
|
||||
lng: -122.3937,
|
||||
provenance: "hand-placed",
|
||||
};
|
||||
|
||||
let file = "";
|
||||
|
||||
before(async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "tera-markers-"));
|
||||
file = join(dir, "markers.json");
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await write({ generatedAt: "2026-08-05T09:00:00Z", markers: [FERRY] });
|
||||
});
|
||||
|
||||
async function write(snapshot: unknown): Promise<void> {
|
||||
await writeFile(file, JSON.stringify(snapshot));
|
||||
}
|
||||
|
||||
/** A feed that is configured, with an issuer that can produce a member. */
|
||||
const feedEnv = {
|
||||
TERA_MARKERS_SOURCE: "file",
|
||||
TERA_AUTH_MODE: "jwt",
|
||||
TERA_AUTH_JWT_SECRET: SECRET,
|
||||
};
|
||||
|
||||
function appWith(env: Record<string, string>) {
|
||||
const config = loadConfig({ TERA_MARKERS_FILE: file, ...env });
|
||||
config.logLevel = "silent";
|
||||
return buildApp(config);
|
||||
}
|
||||
|
||||
function member(): string {
|
||||
const encode = (value: unknown): string =>
|
||||
Buffer.from(JSON.stringify(value)).toString("base64url");
|
||||
const claims = { sub: "someone", exp: Math.floor(Date.now() / 1000) + 600 };
|
||||
const signed = `${encode({ alg: "HS256", typ: "JWT" })}.${encode(claims)}`;
|
||||
return `${signed}.${createHmac("sha256", SECRET).update(signed).digest("base64url")}`;
|
||||
}
|
||||
|
||||
function asMember(app: ReturnType<typeof buildApp>) {
|
||||
return app.inject({
|
||||
method: "GET",
|
||||
url: "/api/v1/markers",
|
||||
headers: { authorization: `Bearer ${member()}` },
|
||||
});
|
||||
}
|
||||
|
||||
describe("a configured marker feed", () => {
|
||||
it("refuses an anonymous caller", async () => {
|
||||
const app = appWith(feedEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const res = await app.inject({ method: "GET", url: "/api/v1/markers" });
|
||||
assert.equal(res.statusCode, 401);
|
||||
assert.equal(res.json<ErrorBody>().error, "unauthorized");
|
||||
assert.equal(res.headers["www-authenticate"], "Bearer");
|
||||
// No rows anywhere in the refusal, and nothing shared may keep it.
|
||||
assert.ok(!res.body.includes("Ferry"));
|
||||
assert.equal(res.headers["cache-control"], "private, no-store");
|
||||
});
|
||||
|
||||
it("refuses every shape of not-being-signed-in", async () => {
|
||||
const app = appWith(feedEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const attempts = [
|
||||
{},
|
||||
{ authorization: "Bearer not-a-jwt" },
|
||||
{ authorization: `Bearer ${member()}tampered` },
|
||||
{ authorization: "Basic Zm9vOmJhcg==" },
|
||||
{ cookie: "tera_session=" },
|
||||
{ cookie: "tera_session=%zz" },
|
||||
];
|
||||
for (const headers of attempts) {
|
||||
const res = await app.inject({ method: "GET", url: "/api/v1/markers", headers });
|
||||
assert.equal(res.statusCode, 401, `${JSON.stringify(headers)} must not get the feed`);
|
||||
}
|
||||
});
|
||||
|
||||
it("serves the snapshot to a member", async () => {
|
||||
const app = appWith(feedEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const res = await asMember(app);
|
||||
assert.equal(res.statusCode, 200);
|
||||
const body = res.json<MarkersBody>();
|
||||
assert.equal(body.markers.length, 1);
|
||||
assert.equal(body.markers[0]?.id, "ferry-building");
|
||||
assert.equal(body.generatedAt, "2026-08-05T09:00:00Z");
|
||||
});
|
||||
|
||||
it("never lets a shared cache keep a member's copy", async () => {
|
||||
const app = appWith(feedEnv);
|
||||
after(() => app.close());
|
||||
assert.equal((await asMember(app)).headers["cache-control"], "private, no-store");
|
||||
});
|
||||
|
||||
it("is unreachable, loudly, on a box where nobody can sign in", async () => {
|
||||
const config = loadConfig({ TERA_MARKERS_FILE: file, TERA_MARKERS_SOURCE: "file" });
|
||||
config.logLevel = "silent";
|
||||
const app = buildApp(config);
|
||||
after(() => app.close());
|
||||
|
||||
// Fail-closed, and it says so rather than leaving an operator to work it out
|
||||
// from an empty map.
|
||||
assert.equal(config.auth.mode, "none");
|
||||
assert.ok(config.degraded.some((line) => line.includes("TERA_MARKERS_SOURCE=file")));
|
||||
|
||||
const anonymous = await app.inject({ method: "GET", url: "/api/v1/markers" });
|
||||
assert.equal(anonymous.statusCode, 401);
|
||||
// Even a token that would be valid elsewhere: mode=none verifies nothing.
|
||||
assert.equal((await asMember(app)).statusCode, 401);
|
||||
});
|
||||
|
||||
it("still answers the public empty body on a box with no feed", async () => {
|
||||
// The acceptance test's box. There is nothing here to protect, and making a
|
||||
// stranger sign in to be told "no markers" would gain nobody anything.
|
||||
const app = appWith({ TERA_MARKERS_SOURCE: "none" });
|
||||
after(() => app.close());
|
||||
|
||||
const res = await app.inject({ method: "GET", url: "/api/v1/markers" });
|
||||
assert.equal(res.statusCode, 200);
|
||||
assert.deepEqual(res.json<MarkersBody>().markers, []);
|
||||
assert.match(String(res.headers["cache-control"]), /^public, max-age=/);
|
||||
});
|
||||
});
|
||||
|
||||
describe("the marker file itself", () => {
|
||||
it("accepts a bare array, because that is what a person writes first", async () => {
|
||||
await write([FERRY]);
|
||||
const app = appWith(feedEnv);
|
||||
after(() => app.close());
|
||||
|
||||
assert.equal((await asMember(app)).json<MarkersBody>().markers.length, 1);
|
||||
});
|
||||
|
||||
it("refuses a bad row without losing the good ones, and reports the count", async () => {
|
||||
await write({
|
||||
markers: [
|
||||
FERRY,
|
||||
{ ...FERRY, id: "osm-row", provenance: "nominatim" },
|
||||
{ ...FERRY, id: "leaky", ownerEmail: "someone@example.com" },
|
||||
{ ...FERRY, id: "broken", lat: 200 },
|
||||
],
|
||||
});
|
||||
const app = appWith(feedEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const body = (await asMember(app)).json<MarkersBody>();
|
||||
assert.deepEqual(
|
||||
body.markers.map((marker) => marker.id),
|
||||
["ferry-building"],
|
||||
);
|
||||
assert.equal(body.refused.length, 3);
|
||||
assert.ok(body.refused.some((entry) => entry.reason.includes("allowlist")));
|
||||
assert.ok(body.refused.some((entry) => entry.reason.includes("ownerEmail")));
|
||||
});
|
||||
|
||||
it("picks up a rewritten file without a restart", async () => {
|
||||
// TERA_MARKERS_TTL is the reload interval. Zero means every request checks,
|
||||
// which is what makes this assertable without sleeping for five minutes.
|
||||
const app = appWith({ ...feedEnv, TERA_MARKERS_TTL: "0" });
|
||||
after(() => app.close());
|
||||
|
||||
assert.equal((await asMember(app)).json<MarkersBody>().markers.length, 1);
|
||||
|
||||
await write({ markers: [FERRY, { ...FERRY, id: "coit-tower", label: "Coit Tower" }] });
|
||||
// A zero TTL means "check on the next millisecond", not "check twice inside
|
||||
// the same one" — `inject()` is fast enough that both requests can land on
|
||||
// the same `Date.now()`, which is a property of the test and not of the
|
||||
// reload.
|
||||
await sleep(5);
|
||||
const reloaded = (await asMember(app)).json<MarkersBody>();
|
||||
assert.deepEqual(
|
||||
reloaded.markers.map((marker) => marker.id),
|
||||
["ferry-building", "coit-tower"],
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps the last good snapshot when a read fails", async () => {
|
||||
const app = appWith({ ...feedEnv, TERA_MARKERS_TTL: "0" });
|
||||
after(() => app.close());
|
||||
|
||||
assert.equal((await asMember(app)).json<MarkersBody>().markers.length, 1);
|
||||
|
||||
// The file goes away mid-flight — a rewrite by something less careful than
|
||||
// the sync oneshot, a permissions change, a full disk. Emptying the map for
|
||||
// a whole TTL is the expensive mistake; serving what was there is the cheap
|
||||
// one.
|
||||
await rm(file);
|
||||
await sleep(5);
|
||||
const survived = (await asMember(app)).json<MarkersBody>();
|
||||
assert.equal(survived.markers.length, 1);
|
||||
|
||||
// And it recovers on its own once the file comes back.
|
||||
await write({ markers: [{ ...FERRY, id: "coit-tower" }] });
|
||||
await sleep(5);
|
||||
const recovered = (await asMember(app)).json<MarkersBody>();
|
||||
assert.deepEqual(
|
||||
recovered.markers.map((marker) => marker.id),
|
||||
["coit-tower"],
|
||||
);
|
||||
});
|
||||
|
||||
it("says it cannot read the file rather than pretending there are no markers", async () => {
|
||||
await rm(file);
|
||||
const app = appWith(feedEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const body = (await asMember(app)).json<MarkersBody>();
|
||||
assert.deepEqual(body.markers, []);
|
||||
assert.equal(body.refused.length, 1);
|
||||
assert.match(body.refused[0]?.reason ?? "", /could not be read/);
|
||||
});
|
||||
|
||||
it("survives a file that is not JSON at all", async () => {
|
||||
await writeFile(file, "{ this is not json");
|
||||
const app = appWith(feedEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const res = await asMember(app);
|
||||
assert.equal(res.statusCode, 200);
|
||||
assert.deepEqual(res.json<MarkersBody>().markers, []);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,280 @@
|
||||
/**
|
||||
* The served-region allowlist, and everything it refuses.
|
||||
*
|
||||
* Two halves. The first is that a box answers for the *places it was configured
|
||||
* for* rather than for one origin, because the map has two cities six hundred
|
||||
* kilometres apart and the Bay Area's fog is not Los Angeles's weather.
|
||||
*
|
||||
* The second is the one with teeth: **a caller's coordinate must never reach an
|
||||
* upstream.** An endpoint that fetches any point on demand is an amplifier
|
||||
* aimed at somebody else's public-good API, with this deployment's contact
|
||||
* string on every request. So `?lat=&lng=` selects among configured points and
|
||||
* nothing else, and everything it cannot select is a 400. See the header of
|
||||
* `regions.ts`; the assertion that this refusal actually holds at the HTTP layer
|
||||
* is `weather.test.ts`.
|
||||
*/
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { after, describe, it } from "node:test";
|
||||
import { buildApp } from "../app.ts";
|
||||
import { loadConfig } from "../config.ts";
|
||||
import { loadRegions, resolveRegion, type RegionSet } from "../regions.ts";
|
||||
import type { ErrorBody, FlightsBody, WeatherBody } from "../../../src/server/wire.ts";
|
||||
|
||||
function appWith(env: Record<string, string>) {
|
||||
const config = loadConfig(env);
|
||||
config.logLevel = "silent";
|
||||
return buildApp(config);
|
||||
}
|
||||
|
||||
/** The two shipped cities, which is what an empty environment resolves to. */
|
||||
function shipped(): RegionSet {
|
||||
return loadConfig({}).regions;
|
||||
}
|
||||
|
||||
describe("the region set a box ends up with", () => {
|
||||
it("serves both shipped cities when handed nothing, San Francisco first", () => {
|
||||
const config = loadConfig({});
|
||||
assert.deepEqual(
|
||||
config.regions.map((region) => region.id),
|
||||
["sf", "socal"],
|
||||
);
|
||||
assert.deepEqual(config.degraded, []);
|
||||
// The default region is still the origin a pre-existing env file named, so
|
||||
// a bare GET answers exactly as it did before regions existed.
|
||||
assert.equal(config.regions[0].lat, config.origin.lat);
|
||||
});
|
||||
|
||||
it("promotes the shipped city an operator's origin falls inside", () => {
|
||||
const config = loadConfig({ TERA_ORIGIN_LAT: "34.05", TERA_ORIGIN_LNG: "-118.24" });
|
||||
assert.deepEqual(
|
||||
config.regions.map((region) => region.id),
|
||||
["socal", "sf"],
|
||||
);
|
||||
assert.deepEqual(config.degraded, []);
|
||||
});
|
||||
|
||||
it("adds an origin that is neither city, and makes it the default", () => {
|
||||
const config = loadConfig({ TERA_ORIGIN_LAT: "39.7392", TERA_ORIGIN_LNG: "-104.9903" });
|
||||
assert.deepEqual(
|
||||
config.regions.map((region) => region.id),
|
||||
["origin", "sf", "socal"],
|
||||
);
|
||||
assert.equal(config.regions[0].lat, 39.7392);
|
||||
});
|
||||
|
||||
it("takes TERA_REGIONS literally, in order, with an optional radius", () => {
|
||||
const config = loadConfig({
|
||||
TERA_REGIONS: "pdx:45.5152,-122.6784:80; sea:47.6062,-122.3321",
|
||||
});
|
||||
assert.deepEqual(config.regions, [
|
||||
{ id: "pdx", lat: 45.5152, lng: -122.6784, radiusKm: 80 },
|
||||
{ id: "sea", lat: 47.6062, lng: -122.3321, radiusKm: 120 },
|
||||
]);
|
||||
assert.deepEqual(config.degraded, []);
|
||||
});
|
||||
|
||||
it("drops a malformed entry with a sentence and keeps the good ones", () => {
|
||||
const config = loadConfig({ TERA_REGIONS: "pdx:45.5152,-122.6784; nowhere; sea:200,0" });
|
||||
assert.deepEqual(
|
||||
config.regions.map((region) => region.id),
|
||||
["pdx"],
|
||||
);
|
||||
assert.equal(config.degraded.length, 2);
|
||||
assert.match(config.degraded[0] ?? "", /nowhere/);
|
||||
assert.match(config.degraded[1] ?? "", /not a place on Earth/);
|
||||
});
|
||||
|
||||
it("falls back to the shipped cities when nothing in TERA_REGIONS parses", () => {
|
||||
const config = loadConfig({ TERA_REGIONS: "?????" });
|
||||
assert.deepEqual(
|
||||
config.regions.map((region) => region.id),
|
||||
["sf", "socal"],
|
||||
);
|
||||
assert.ok(config.degraded.some((line) => line.includes("nothing in it parsed")));
|
||||
});
|
||||
|
||||
it("keeps the first of two entries sharing an id", () => {
|
||||
const config = loadConfig({ TERA_REGIONS: "sf:37.7749,-122.4194; sf:0,0" });
|
||||
assert.equal(config.regions.length, 1);
|
||||
assert.ok(config.degraded.some((line) => line.includes("twice")));
|
||||
});
|
||||
|
||||
it("never ends up with an empty set, whatever it was handed", () => {
|
||||
for (const spec of ["", " ", ";;;", "sf:", "@:1,2", "x".repeat(200)]) {
|
||||
const regions = loadRegions({
|
||||
spec,
|
||||
origin: { lat: 37.7749, lng: -122.4194 },
|
||||
originConfigured: false,
|
||||
degraded: [],
|
||||
});
|
||||
assert.ok(regions.length > 0, `"${spec}" produced no regions`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolving a request to a region", () => {
|
||||
const regions = shipped();
|
||||
|
||||
it("answers for the default when asked for nothing", () => {
|
||||
const resolved = resolveRegion(regions, {});
|
||||
assert.ok(resolved.ok && resolved.region.id === "sf");
|
||||
});
|
||||
|
||||
it("answers for a city by id", () => {
|
||||
const resolved = resolveRegion(regions, { city: "socal" });
|
||||
assert.ok(resolved.ok && resolved.region.id === "socal");
|
||||
});
|
||||
|
||||
it("snaps a nearby coordinate to the region that claims it", () => {
|
||||
// Berkeley, Pasadena: neither is a configured point, and both resolve to the
|
||||
// configured point that will actually be fetched.
|
||||
const berkeley = resolveRegion(regions, { lat: "37.8715", lng: "-122.2730" });
|
||||
assert.ok(berkeley.ok && berkeley.region.id === "sf");
|
||||
const pasadena = resolveRegion(regions, { lat: "34.1478", lng: "-118.1445" });
|
||||
assert.ok(pasadena.ok && pasadena.region.id === "socal");
|
||||
});
|
||||
|
||||
it("refuses a coordinate this deployment has nothing to say about", () => {
|
||||
for (const point of [
|
||||
{ lat: "36.7378", lng: "-119.7871" }, // Fresno, between the two boards.
|
||||
{ lat: "40.7128", lng: "-74.0060" }, // New York.
|
||||
{ lat: "0", lng: "0" }, // Null Island, the classic probe.
|
||||
]) {
|
||||
const resolved = resolveRegion(regions, point);
|
||||
assert.ok(!resolved.ok, `${point.lat},${point.lng} must be refused`);
|
||||
assert.match(resolved.message, /serves: sf, socal/);
|
||||
}
|
||||
});
|
||||
|
||||
it("refuses a coordinate that is not a coordinate", () => {
|
||||
for (const lat of [
|
||||
"banana",
|
||||
"NaN",
|
||||
"Infinity",
|
||||
"1e5",
|
||||
"0x2f",
|
||||
"37.7749deg",
|
||||
"91",
|
||||
"37.77490001",
|
||||
"",
|
||||
" ",
|
||||
]) {
|
||||
const resolved = resolveRegion(regions, { lat, lng: "-122.4194" });
|
||||
assert.ok(!resolved.ok, `lat=${lat} must be refused`);
|
||||
}
|
||||
assert.ok(!resolveRegion(regions, { lat: "37.7", lng: "181" }).ok);
|
||||
});
|
||||
|
||||
it("refuses a repeated parameter rather than picking one", () => {
|
||||
// `?lat=1&lat=2` arrives as an array, and quietly taking either half is how
|
||||
// a parser disagreement becomes a security bug somewhere downstream.
|
||||
assert.ok(!resolveRegion(regions, { lat: ["37.7", "0"], lng: "-122.4" }).ok);
|
||||
assert.ok(!resolveRegion(regions, { city: ["sf", "socal"] }).ok);
|
||||
});
|
||||
|
||||
it("refuses half a coordinate", () => {
|
||||
assert.ok(!resolveRegion(regions, { lat: "37.7749" }).ok);
|
||||
assert.ok(!resolveRegion(regions, { lng: "-122.4194" }).ok);
|
||||
});
|
||||
|
||||
it("refuses a request that asks two ways at once", () => {
|
||||
const resolved = resolveRegion(regions, { city: "sf", lat: "37.7", lng: "-122.4" });
|
||||
assert.ok(!resolved.ok);
|
||||
assert.match(resolved.message, /not both/);
|
||||
});
|
||||
|
||||
it("refuses an unknown city without pretending it might exist elsewhere", () => {
|
||||
const resolved = resolveRegion(regions, { city: "atlantis" });
|
||||
assert.ok(!resolved.ok);
|
||||
assert.match(resolved.message, /sf, socal/);
|
||||
});
|
||||
});
|
||||
|
||||
describe("the routes that take a region", () => {
|
||||
for (const route of ["weather", "flights"]) {
|
||||
it(`serves ${route} for either city, from that city's own point`, async () => {
|
||||
const app = appWith({});
|
||||
after(() => app.close());
|
||||
|
||||
const sf = await app.inject({ method: "GET", url: `/api/v1/${route}?city=sf` });
|
||||
const socal = await app.inject({ method: "GET", url: `/api/v1/${route}?city=socal` });
|
||||
assert.equal(sf.statusCode, 200);
|
||||
assert.equal(socal.statusCode, 200);
|
||||
assert.notDeepEqual(sf.json(), socal.json());
|
||||
});
|
||||
|
||||
it(`refuses a nonsense ${route} query with a 400 and no cached copy of it`, async () => {
|
||||
const app = appWith({});
|
||||
after(() => app.close());
|
||||
|
||||
const res = await app.inject({ method: "GET", url: `/api/v1/${route}?lat=banana&lng=0` });
|
||||
assert.equal(res.statusCode, 400);
|
||||
assert.equal(res.json<ErrorBody>().error, "bad_request");
|
||||
// The fail-closed default still applies: nothing shared may keep a refusal.
|
||||
assert.equal(res.headers["cache-control"], "private, no-store");
|
||||
});
|
||||
|
||||
it(`refuses a ${route} request for somewhere this box does not serve`, async () => {
|
||||
const app = appWith({});
|
||||
after(() => app.close());
|
||||
|
||||
const res = await app.inject({
|
||||
method: "GET",
|
||||
url: `/api/v1/${route}?lat=51.5072&lng=-0.1276`,
|
||||
});
|
||||
assert.equal(res.statusCode, 400);
|
||||
assert.match(res.json<ErrorBody>().message, /sf, socal/);
|
||||
});
|
||||
}
|
||||
|
||||
it("puts the weather where it was asked for, not where the box is", async () => {
|
||||
const app = appWith({});
|
||||
after(() => app.close());
|
||||
|
||||
const res = await app.inject({ method: "GET", url: "/api/v1/weather?city=socal" });
|
||||
const body = res.json<WeatherBody>();
|
||||
assert.deepEqual(body.location, { lat: 33.82, lng: -118.05 });
|
||||
// Still the synthetic clear day, because a zero-config box has no source.
|
||||
assert.equal(body.synthetic, true);
|
||||
});
|
||||
|
||||
it("flies the Southland's own airports over the Southland", async () => {
|
||||
const app = appWith({});
|
||||
after(() => app.close());
|
||||
|
||||
const body = (
|
||||
await app.inject({ method: "GET", url: "/api/v1/flights?city=socal" })
|
||||
).json<FlightsBody>();
|
||||
assert.ok(body.mode === "plan");
|
||||
// LAX's published reference point, as a departure or an arrival. The Bay
|
||||
// Area plan cannot produce it, which is the whole point of the assertion.
|
||||
const lax = body.routes.some(
|
||||
(leg) =>
|
||||
(leg.from[0] === 33.9425 && leg.from[1] === -118.4081) ||
|
||||
(leg.to[0] === 33.9425 && leg.to[1] === -118.4081),
|
||||
);
|
||||
assert.ok(lax, "the SoCal plan should fly out of LAX");
|
||||
});
|
||||
|
||||
it("lays out spokes for a city it has never heard of", async () => {
|
||||
const app = appWith({ TERA_REGIONS: "pdx:45.5152,-122.6784" });
|
||||
after(() => app.close());
|
||||
|
||||
const body = (await app.inject({ method: "GET", url: "/api/v1/flights" })).json<FlightsBody>();
|
||||
assert.ok(body.mode === "plan" && body.routes.length === 8);
|
||||
});
|
||||
|
||||
it("publishes the allowlist on health so a client can stop guessing", async () => {
|
||||
const app = appWith({});
|
||||
after(() => app.close());
|
||||
|
||||
const body = (await app.inject({ method: "GET", url: "/api/v1/health" })).json<{
|
||||
regions: { id: string }[];
|
||||
}>();
|
||||
assert.deepEqual(
|
||||
body.regions.map((region) => region.id),
|
||||
["sf", "socal"],
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,294 @@
|
||||
/**
|
||||
* Weather, once a source is actually turned on.
|
||||
*
|
||||
* `api.weather.gov` is stood up as a stub here rather than called, for the
|
||||
* obvious reason and for a less obvious one: the assertions that matter are
|
||||
* about **which URL this server constructs**, and a real upstream would answer
|
||||
* the right way for the wrong reason. The load-bearing one is that a caller
|
||||
* asking for Berkeley causes a fetch of San Francisco's point and never of
|
||||
* Berkeley's — the difference between a map and an open proxy pointed at a
|
||||
* public-good API with this deployment's contact address on it.
|
||||
*
|
||||
* Global `fetch` is replaced for the file. `node --test` runs each test file in
|
||||
* its own process, so nothing here leaks into another one, and `after` puts the
|
||||
* real one back anyway.
|
||||
*/
|
||||
|
||||
import assert from "node:assert/strict";
|
||||
import { after, beforeEach, describe, it } from "node:test";
|
||||
import { buildApp } from "../app.ts";
|
||||
import { loadConfig } from "../config.ts";
|
||||
import type { WeatherBody } from "../../../src/server/wire.ts";
|
||||
|
||||
const CONTACT = "ops@example.com";
|
||||
|
||||
const nwsEnv = { TERA_WEATHER_SOURCE: "nws", TERA_WEATHER_CONTACT: CONTACT };
|
||||
|
||||
interface Call {
|
||||
url: string;
|
||||
userAgent: string;
|
||||
}
|
||||
|
||||
const realFetch = globalThis.fetch;
|
||||
let calls: Call[] = [];
|
||||
/** Flipped by a test that wants to watch the upstream go away mid-flight. */
|
||||
let upstreamIsUp = true;
|
||||
/**
|
||||
* Flipped by a test that wants the upstream to stay *up* and answer 200 with
|
||||
* something the parser cannot walk. `cloudLayers` becomes a number, which is
|
||||
* the shape that used to throw straight through the cache.
|
||||
*/
|
||||
let upstreamIsGarbled = false;
|
||||
|
||||
/**
|
||||
* The three hops NWS makes you take, and nothing else: an unrecognised URL is a
|
||||
* 404, so a request this server should not be making shows up as a failure
|
||||
* rather than as a plausible answer.
|
||||
*/
|
||||
function nws(url: string): unknown | undefined {
|
||||
if (!upstreamIsUp) return undefined;
|
||||
|
||||
const point = /\/points\/(-?[\d.]+),(-?[\d.]+)$/.exec(url);
|
||||
if (point !== null) {
|
||||
return { properties: { observationStations: `https://api.weather.gov/zones/${point[1]}` } };
|
||||
}
|
||||
const zone = /\/zones\/(-?[\d.]+)$/.exec(url);
|
||||
if (zone !== null) {
|
||||
// One station id per latitude, so a body can be traced back to the point
|
||||
// that was asked about.
|
||||
return { features: [{ properties: { stationIdentifier: `K${zone[1]}` } }] };
|
||||
}
|
||||
const station = /\/stations\/K(-?[\d.]+)\/observations\/latest$/.exec(url);
|
||||
if (station !== null) {
|
||||
return {
|
||||
properties: {
|
||||
timestamp: "2026-08-05T09:00:00+00:00",
|
||||
temperature: { value: Number(station[1]), unitCode: "wmoUnit:degC" },
|
||||
windSpeed: { value: 9, unitCode: "wmoUnit:km_h-1" },
|
||||
cloudLayers: upstreamIsGarbled ? 7 : [{ amount: "BKN" }],
|
||||
presentWeather: [],
|
||||
visibility: { value: 16_000, unitCode: "wmoUnit:m" },
|
||||
},
|
||||
};
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
globalThis.fetch = (async (input: unknown, init?: { headers?: Record<string, string> }) => {
|
||||
const url = String(input);
|
||||
// `http.ts` always passes a plain object, so this needs no `Headers` dance.
|
||||
calls.push({ url, userAgent: init?.headers?.["user-agent"] ?? "" });
|
||||
|
||||
const body = nws(url);
|
||||
if (body === undefined) return new Response("nope", { status: 503 });
|
||||
return new Response(JSON.stringify(body), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}) as unknown as typeof globalThis.fetch;
|
||||
|
||||
after(() => {
|
||||
globalThis.fetch = realFetch;
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
calls = [];
|
||||
upstreamIsUp = true;
|
||||
upstreamIsGarbled = false;
|
||||
});
|
||||
|
||||
function appWith(env: Record<string, string>) {
|
||||
const config = loadConfig(env);
|
||||
config.logLevel = "silent";
|
||||
return buildApp(config);
|
||||
}
|
||||
|
||||
function observations(): string[] {
|
||||
return calls.filter((call) => call.url.includes("/observations/")).map((call) => call.url);
|
||||
}
|
||||
|
||||
describe("a configured weather source", () => {
|
||||
it("fetches the region centre and never the coordinate the caller sent", async () => {
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
|
||||
// Berkeley. Inside the Bay Area region, and not a point this deployment
|
||||
// serves — so it selects San Francisco and San Francisco is what gets asked.
|
||||
const res = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/v1/weather?lat=37.8715&lng=-122.2730",
|
||||
});
|
||||
assert.equal(res.statusCode, 200);
|
||||
|
||||
assert.ok(
|
||||
calls.every((call) => !call.url.includes("37.8715")),
|
||||
`the caller's coordinate reached the upstream: ${calls.map((c) => c.url).join(" ")}`,
|
||||
);
|
||||
assert.ok(calls.some((call) => call.url.endsWith("/points/37.7749,-122.4194")));
|
||||
assert.deepEqual(res.json<WeatherBody>().location, { lat: 37.7749, lng: -122.4194 });
|
||||
});
|
||||
|
||||
it("holds one observation per city rather than one per box", async () => {
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const sf = (
|
||||
await app.inject({ method: "GET", url: "/api/v1/weather?city=sf" })
|
||||
).json<WeatherBody>();
|
||||
const socal = (
|
||||
await app.inject({ method: "GET", url: "/api/v1/weather?city=socal" })
|
||||
).json<WeatherBody>();
|
||||
|
||||
// The stub encodes the latitude in the temperature, so two different
|
||||
// numbers here means two different stations were actually consulted.
|
||||
assert.equal(sf.temperatureC, 37.7749);
|
||||
assert.equal(socal.temperatureC, 33.82);
|
||||
assert.equal(observations().length, 2);
|
||||
assert.equal(sf.source, "nws");
|
||||
assert.equal(sf.synthetic, false);
|
||||
});
|
||||
|
||||
it("asks once per region per TTL however many callers turn up", async () => {
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const ask = () => app.inject({ method: "GET", url: "/api/v1/weather?city=sf" });
|
||||
// Concurrent, then sequential: the first is the single-flight collapse, the
|
||||
// second is the TTL. Both used to be one fetch each and only one of them was.
|
||||
await Promise.all([ask(), ask(), ask(), ask()]);
|
||||
await ask();
|
||||
await ask();
|
||||
|
||||
assert.equal(observations().length, 1);
|
||||
});
|
||||
|
||||
it("keeps the cities apart under load, not merely on the first request", async () => {
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const bodies = await Promise.all(
|
||||
["sf", "socal", "sf", "socal", "sf"].map(async (city) =>
|
||||
(await app.inject({ method: "GET", url: `/api/v1/weather?city=${city}` })).json<WeatherBody>(),
|
||||
),
|
||||
);
|
||||
assert.deepEqual(
|
||||
bodies.map((body) => body.temperatureC),
|
||||
[37.7749, 33.82, 37.7749, 33.82, 37.7749],
|
||||
);
|
||||
assert.equal(observations().length, 2);
|
||||
});
|
||||
|
||||
/**
|
||||
* A source that is *up* and answering in a shape this build cannot read is a
|
||||
* different failure from one that is down, and it used to be a much worse
|
||||
* one: the parser threw, `upstream.ts` never reached its clock stamp, and the
|
||||
* TTL — the only thing standing between a public-good API and one outbound
|
||||
* request per inbound request — stopped existing. `current()` also stopped
|
||||
* being the thing its own header calls it, which is a function that never
|
||||
* throws.
|
||||
*/
|
||||
it("treats a body it cannot parse as a source that did not answer", async () => {
|
||||
upstreamIsGarbled = true;
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const res = await app.inject({ method: "GET", url: "/api/v1/weather?city=sf" });
|
||||
assert.equal(res.statusCode, 200);
|
||||
// The clear day, which is what "nobody answered" has always meant here.
|
||||
assert.equal(res.json<WeatherBody>().synthetic, true);
|
||||
});
|
||||
|
||||
it("keeps the TTL when the body is garbled, not just when the socket dies", async () => {
|
||||
upstreamIsGarbled = true;
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
|
||||
for (let i = 0; i < 5; i++) {
|
||||
await app.inject({ method: "GET", url: "/api/v1/weather?city=sf" });
|
||||
}
|
||||
assert.equal(observations().length, 1);
|
||||
});
|
||||
|
||||
it("identifies the operator to a source that requires it", async () => {
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
|
||||
await app.inject({ method: "GET", url: "/api/v1/weather" });
|
||||
assert.ok(calls.length > 0);
|
||||
for (const call of calls) assert.match(call.userAgent, new RegExp(CONTACT));
|
||||
});
|
||||
|
||||
it("serves the last good observation when the upstream goes away", async () => {
|
||||
// TTL 0 makes every request a refetch, which is what makes the failure
|
||||
// reachable in a test without waiting ten minutes for one.
|
||||
const app = appWith({ ...nwsEnv, TERA_WEATHER_TTL: "0" });
|
||||
after(() => app.close());
|
||||
|
||||
const first = (
|
||||
await app.inject({ method: "GET", url: "/api/v1/weather?city=sf" })
|
||||
).json<WeatherBody>();
|
||||
assert.equal(first.temperatureC, 37.7749);
|
||||
|
||||
upstreamIsUp = false;
|
||||
const second = await app.inject({ method: "GET", url: "/api/v1/weather?city=sf" });
|
||||
assert.equal(second.statusCode, 200);
|
||||
const body = second.json<WeatherBody>();
|
||||
// Not a 503, not a clear day: the observation from a minute ago, which is
|
||||
// the only answer that keeps the sky looking like the sky.
|
||||
assert.equal(body.temperatureC, 37.7749);
|
||||
assert.equal(body.synthetic, false);
|
||||
});
|
||||
|
||||
it("falls back to a clear day for a city that has never answered", async () => {
|
||||
upstreamIsUp = false;
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const res = await app.inject({ method: "GET", url: "/api/v1/weather?city=socal" });
|
||||
assert.equal(res.statusCode, 200);
|
||||
const body = res.json<WeatherBody>();
|
||||
assert.equal(body.synthetic, true);
|
||||
assert.equal(body.condition, "clear");
|
||||
assert.deepEqual(body.location, { lat: 33.82, lng: -118.05 });
|
||||
});
|
||||
|
||||
it("retries a dead source on the TTL, not on every request", async () => {
|
||||
upstreamIsUp = false;
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
|
||||
for (let i = 0; i < 5; i++) await app.inject({ method: "GET", url: "/api/v1/weather" });
|
||||
// One attempt, one failure, one clock stamp. Somebody else's outage must not
|
||||
// turn into this box's outbound flood.
|
||||
assert.equal(calls.length, 1);
|
||||
});
|
||||
|
||||
it("makes no outbound request at all until somebody asks", async () => {
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
await app.inject({ method: "GET", url: "/api/v1/health" });
|
||||
assert.deepEqual(calls, []);
|
||||
});
|
||||
|
||||
it("never calls anybody when the source is off", async () => {
|
||||
const app = appWith({});
|
||||
after(() => app.close());
|
||||
|
||||
await app.inject({ method: "GET", url: "/api/v1/weather?city=sf" });
|
||||
await app.inject({ method: "GET", url: "/api/v1/weather?city=socal" });
|
||||
assert.deepEqual(calls, []);
|
||||
});
|
||||
|
||||
it("refuses the request before it would have fetched anything", async () => {
|
||||
const app = appWith(nwsEnv);
|
||||
after(() => app.close());
|
||||
|
||||
const res = await app.inject({
|
||||
method: "GET",
|
||||
url: "/api/v1/weather?lat=51.5072&lng=-0.1276",
|
||||
});
|
||||
assert.equal(res.statusCode, 400);
|
||||
// The point of the allowlist: a refused request costs the upstream nothing.
|
||||
assert.deepEqual(calls, []);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,133 @@
|
||||
/**
|
||||
* A per-key TTL cache in front of something that is allowed to fail.
|
||||
*
|
||||
* Weather and flights had the same fifteen lines each, and once both of them
|
||||
* became *per region* they would have had the same fifteen lines with the same
|
||||
* `Map` bolted on. Three rules, and they are the three the weather service
|
||||
* already stated:
|
||||
*
|
||||
* 1. **Nothing is fetched until somebody asks.** A box nobody visits makes no
|
||||
* outbound requests at all, which is what keeps a public-good API's fair-use
|
||||
* policy satisfiable by a deployment that is mostly idle.
|
||||
* 2. **A dead upstream serves the last good answer**, per key, and only reports
|
||||
* nothing when it has never once answered for that key. Ten-minute-old
|
||||
* weather beats no weather by a mile and beats a 503 by more.
|
||||
* 3. **A failed fetch stamps the clock too**, and a fetch that *throws* is a
|
||||
* fetch that failed. A source that is down — or that has started answering
|
||||
* in a shape this build cannot read — is retried on the same cadence as one
|
||||
* that is up, not on every request, which is the bug that turns somebody
|
||||
* else's outage into your outbound flood. See `refresh`.
|
||||
*
|
||||
* Concurrent misses on one key collapse into one upstream request. Misses on
|
||||
* *different* keys do not, deliberately: they are different places and the
|
||||
* caller's rate budget is worked out per key in `flights/index.ts`.
|
||||
*
|
||||
* ### On the size of the map
|
||||
*
|
||||
* Keys are region ids, and the region set comes from the environment
|
||||
* (`regions.ts`), so the number of entries is bounded by the operator's config
|
||||
* and cannot be grown by anybody sending requests. That invariant lives in the
|
||||
* routes, which resolve a query to a configured region *before* anything here is
|
||||
* touched; this module trusts it and does not re-check it. Key this on anything
|
||||
* a caller can choose and it becomes an unbounded map.
|
||||
*/
|
||||
|
||||
export interface UpstreamLog {
|
||||
warn(msg: string): void;
|
||||
}
|
||||
|
||||
export interface Upstream<T> {
|
||||
/**
|
||||
* The freshest value for `key`, or `null` if nothing has ever answered for it.
|
||||
*
|
||||
* `fetchFresh` is passed per call rather than at construction so the caller
|
||||
* can close over the region it just resolved instead of keeping a second
|
||||
* lookup table. It must be the same function of the same key every time, which
|
||||
* is trivially true for both callers here — the key *is* the region.
|
||||
*/
|
||||
get(key: string, fetchFresh: () => Promise<T | null>): Promise<T | null>;
|
||||
}
|
||||
|
||||
interface Entry<T> {
|
||||
value: T | null;
|
||||
/** Epoch ms of the last completed attempt, successful or not. */
|
||||
fetchedAt: number;
|
||||
inFlight: Promise<void> | null;
|
||||
}
|
||||
|
||||
export interface UpstreamOptions {
|
||||
/** Prefix for the one warning line this ever logs, e.g. `weather:nws`. */
|
||||
label: string;
|
||||
ttlSeconds: number;
|
||||
log: UpstreamLog;
|
||||
}
|
||||
|
||||
export function createUpstream<T>(opts: UpstreamOptions): Upstream<T> {
|
||||
const ttlMs = Math.max(0, opts.ttlSeconds) * 1000;
|
||||
const entries = new Map<string, Entry<T>>();
|
||||
|
||||
/**
|
||||
* One attempt, and it cannot fail in a way the caller has to know about.
|
||||
*
|
||||
* The `try` is rule 3, and the `finally` is the whole of it. `fetchFresh` is
|
||||
* supposed to return `null` for every failure — `http.ts` does exactly that —
|
||||
* but "supposed to" is not a guarantee, and one upstream that answers 200
|
||||
* with a field of the wrong type is enough: `adsb.ts` iterating a non-array
|
||||
* `ac`, or `nws.ts` iterating a `cloudLayers` that came back as a number,
|
||||
* throws out of here. Before this, that throw skipped the clock stamp *and*
|
||||
* propagated to the route, so the TTL — the only rate limit on outbound calls
|
||||
* — collapsed to one upstream request per inbound request and the caller got
|
||||
* a 500. Five requests to `/api/v1/flights` produced five fetches at
|
||||
* adsb.lol, from the operator's address, and five 500s; that is the flood
|
||||
* rule 3 exists to prevent, delivered by the failure mode it exists for.
|
||||
*
|
||||
* So a throw is made indistinguishable from the `null` it should have been:
|
||||
* clock stamped, last good value kept, route answers 200 with whatever is in
|
||||
* hand. The stack goes in the log line, because a source whose *shape*
|
||||
* changed is a different problem from a source that is down and the operator
|
||||
* needs to be able to tell them apart.
|
||||
*/
|
||||
async function refresh(key: string, entry: Entry<T>, fetchFresh: () => Promise<T | null>) {
|
||||
let fresh: T | null = null;
|
||||
let threw: unknown = null;
|
||||
try {
|
||||
fresh = await fetchFresh();
|
||||
} catch (err) {
|
||||
threw = err;
|
||||
} finally {
|
||||
entry.fetchedAt = Date.now();
|
||||
}
|
||||
if (threw === null && fresh !== null) {
|
||||
entry.value = fresh;
|
||||
return;
|
||||
}
|
||||
const why =
|
||||
threw === null
|
||||
? "did not answer"
|
||||
: `answered with something this build cannot read (${threw instanceof Error ? threw.message : String(threw)})`;
|
||||
opts.log.warn(
|
||||
`${opts.label}: ${key} ${why}; serving ` +
|
||||
`${entry.value === null ? "the fallback" : "the last good answer"}`,
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
async get(key: string, fetchFresh: () => Promise<T | null>): Promise<T | null> {
|
||||
let entry = entries.get(key);
|
||||
if (entry === undefined) {
|
||||
entry = { value: null, fetchedAt: 0, inFlight: null };
|
||||
entries.set(key, entry);
|
||||
}
|
||||
|
||||
if (Date.now() - entry.fetchedAt > ttlMs) {
|
||||
const pending = entry;
|
||||
pending.inFlight ??= refresh(key, pending, fetchFresh).finally(() => {
|
||||
pending.inFlight = null;
|
||||
});
|
||||
await pending.inFlight;
|
||||
}
|
||||
|
||||
return entry.value;
|
||||
},
|
||||
};
|
||||
}
|
||||
+35
-41
@@ -1,26 +1,38 @@
|
||||
/**
|
||||
* Which source answers, how often it is asked, and what happens when it does not.
|
||||
* Which source answers, for which place, how often it is asked, and what happens
|
||||
* when it does not.
|
||||
*
|
||||
* Three rules, in order of how much trouble getting them wrong causes:
|
||||
* Four rules, in order of how much trouble getting them wrong causes:
|
||||
*
|
||||
* 1. **This never throws.** `current()` always resolves to a `WeatherBody`.
|
||||
* 2. **A dead upstream serves the last good observation**, and only falls back
|
||||
* to the clear day if there has never been one. Ten-minute-old weather is
|
||||
* better than no weather and much better than a 503.
|
||||
* 2. **A dead upstream serves the last good observation** for that place, and
|
||||
* only falls back to the clear day if there has never been one. Ten-minute-old
|
||||
* weather is better than no weather and much better than a 503.
|
||||
* 3. **Nothing is fetched until somebody asks.** A box nobody visits makes no
|
||||
* outbound requests, which matters when the source is a public good with a
|
||||
* rate limit and a fair-use policy.
|
||||
* 4. **One cache entry per region, and the regions come from the environment.**
|
||||
* San Francisco's fog and Los Angeles's sun are two different observations
|
||||
* and the old single-origin cache could only hold one of them. The keys are
|
||||
* region ids and nothing a caller sends becomes one — `routes/weather.ts`
|
||||
* resolves the query to a configured region first, which is what bounds both
|
||||
* this map and the number of stations `nws.ts` will ever look up.
|
||||
*
|
||||
* Rules 1–3 live in `upstream.ts` now, because flights wanted exactly the same
|
||||
* three.
|
||||
*/
|
||||
|
||||
import type { Config } from "../config.ts";
|
||||
import type { Region } from "../regions.ts";
|
||||
import type { WeatherBody } from "../../../src/server/wire.ts";
|
||||
import { createUpstream } from "../upstream.ts";
|
||||
import { fetchMetno } from "./metno.ts";
|
||||
import { fetchNws } from "./nws.ts";
|
||||
import { fetchOpenMeteo } from "./openmeteo.ts";
|
||||
import { clearDay } from "./synthetic.ts";
|
||||
|
||||
export interface WeatherService {
|
||||
current(): Promise<WeatherBody>;
|
||||
current(region: Region): Promise<WeatherBody>;
|
||||
}
|
||||
|
||||
export interface WeatherLog {
|
||||
@@ -28,46 +40,28 @@ export interface WeatherLog {
|
||||
}
|
||||
|
||||
export function createWeatherService(config: Config, log: WeatherLog): WeatherService {
|
||||
const { lat, lng } = config.origin;
|
||||
const { source, contact, ttlSeconds } = config.weather;
|
||||
const upstream = createUpstream<WeatherBody>({ label: `weather:${source}`, ttlSeconds, log });
|
||||
|
||||
let cached: WeatherBody | null = null;
|
||||
let fetchedAt = 0;
|
||||
let inFlight: Promise<void> | null = null;
|
||||
|
||||
async function refresh(): Promise<void> {
|
||||
const fresh =
|
||||
source === "nws"
|
||||
? await fetchNws(lat, lng, contact)
|
||||
: source === "metno"
|
||||
? await fetchMetno(lat, lng, contact)
|
||||
: source === "openmeteo"
|
||||
? await fetchOpenMeteo(lat, lng)
|
||||
: null;
|
||||
|
||||
// Stamp the clock either way. A source that is down should be retried on the
|
||||
// same cadence as one that is up, not hammered once per request.
|
||||
fetchedAt = Date.now();
|
||||
if (fresh !== null) {
|
||||
cached = fresh;
|
||||
return;
|
||||
}
|
||||
log.warn(`weather: ${source} did not answer; serving ${cached === null ? "a synthetic clear day" : "the last observation"}`);
|
||||
function fetchFor(region: Region): Promise<WeatherBody | null> {
|
||||
// The coordinate that goes upstream is the *region centre*, never the one
|
||||
// the caller sent. See the top of `regions.ts` for why that distinction is
|
||||
// the whole abuse story on this endpoint.
|
||||
const { lat, lng } = region;
|
||||
return source === "nws"
|
||||
? fetchNws(lat, lng, contact)
|
||||
: source === "metno"
|
||||
? fetchMetno(lat, lng, contact)
|
||||
: source === "openmeteo"
|
||||
? fetchOpenMeteo(lat, lng)
|
||||
: Promise.resolve(null);
|
||||
}
|
||||
|
||||
return {
|
||||
async current(): Promise<WeatherBody> {
|
||||
if (source === "none") return clearDay(lat, lng);
|
||||
|
||||
const stale = Date.now() - fetchedAt > ttlSeconds * 1000;
|
||||
if (stale) {
|
||||
// Collapse concurrent misses into one upstream request.
|
||||
inFlight ??= refresh().finally(() => {
|
||||
inFlight = null;
|
||||
});
|
||||
await inFlight;
|
||||
}
|
||||
return cached ?? clearDay(lat, lng);
|
||||
async current(region: Region): Promise<WeatherBody> {
|
||||
if (source === "none") return clearDay(region.lat, region.lng);
|
||||
const body = await upstream.get(region.id, () => fetchFor(region));
|
||||
return body ?? clearDay(region.lat, region.lng);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -10,6 +10,13 @@
|
||||
* Getting an observation takes three hops — point to station list, station list
|
||||
* to nearest station, station to latest observation — so the first two are
|
||||
* resolved once per process and kept. Stations do not move.
|
||||
*
|
||||
* That station cache is keyed on the coordinate, which means its size is exactly
|
||||
* the number of places this box will answer for. It stays bounded because the
|
||||
* only coordinates that reach here are region centres from `regions.ts`: a
|
||||
* caller cannot name a point, so a caller cannot grow this map, and the three
|
||||
* hops are paid once per region for the life of the process rather than once per
|
||||
* curious request.
|
||||
*/
|
||||
|
||||
import { getJson, userAgent } from "../http.ts";
|
||||
|
||||
Reference in New Issue
Block a user