Fix twenty findings from the Motion review
Each was raised by a reviewer and then survived an independent attempt to refute it. The four that mattered most: - A third of the starter library was invisible. Three templates authored `fields` shapes no renderer read — decisions, blockingSet, checks, steps and the rest — so about forty records rendered as no DOM at all, in the library and again on the engagement that instantiated them. Nothing failed: a renderer returns null for a key set it does not recognise, and a header-plus-body page looks like a template written that way. FieldsView now reads every key the seeds carry. - "Add a framework" opened a picker that could never match, because the dialog was seeded with both the forced kind and the deal's stage, and qualification serves only the qualification stage. The stage is now dropped when MOTION_KIND_STAGES says the pair is incoherent. - Piggy reported the promotion count as an exact figure capped at 8, against a tile showing the true count beside it. It is now counted in SQL, and all three motion tools carry a ResultScope whose denominator is shared lineages — never rows, never private drafts. - No Motion test went through createApp, so the whole feature could be unmounted with a green suite. That is the AGENTS.md §5 trap that already cost this project read-guards.ts and learn.ts. Also: both sides of the instantiate/edit race now lock, so a template cannot be rewritten under an artefact that has copied it; concurrent engagement opens queue on the deal row and get the 409 the handler already promised rather than a 500; latestScore uses DISTINCT ON instead of losing engagements past a 200-row cap; the migration adds the scored_by_user_id foreign key the schema declares; and the demo clear refunds usage_count for engagements it reaches by cascade, which otherwise left starter templates permanently un-editable. Verified on a fresh database: 16 migrations apply and re-apply as a no-op, both seeds idempotent, usage_count back to zero after --clear. 564 unit tests pass. Every Motion route measures zero horizontal overflow at 393 and 1440 in both themes, and all twelve seeded field trees are asserted onto the screen by scripts/motion-fields-check.mjs. One thing left open deliberately: the shipped qualification scorecard's five bands and MOTION_BANDS' four are calibrated differently. The framework's table is now titled as its own guidance rather than the product's verdict, which removes the contradiction on screen. Making the framework's calibration authoritative over the persisted band column is a product decision nobody has made. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -16,16 +16,25 @@
|
||||
*
|
||||
* The unit suite runs in CI BEFORE the migration step, against a database with
|
||||
* no tables, so nothing here may execute a query. The WHERE clause is rendered
|
||||
* with `PgDialect` rather than run.
|
||||
* with `PgDialect` rather than run, and the tools themselves are executed
|
||||
* against a stub handle — the second half of this file, which pins the figures
|
||||
* they report and the denominators those figures are drawn from.
|
||||
*/
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { MOTION_KINDS } from '@pig/core';
|
||||
import type { Database } from '@pig/db';
|
||||
import {
|
||||
engagementArtifacts,
|
||||
engagements,
|
||||
motionTemplates,
|
||||
qualificationScores,
|
||||
type Database,
|
||||
} from '@pig/db';
|
||||
import type { SQL } from 'drizzle-orm';
|
||||
import { PgDialect } from 'drizzle-orm/pg-core';
|
||||
import { zodToJsonSchema } from 'zod-to-json-schema';
|
||||
import { assertPigToolBoundary } from '../src/chat';
|
||||
import { createPagePigTools, motionLibraryWhere } from '../src/page-tools';
|
||||
import { createPagePigTools, motionLibraryWhere, type ResultScope } from '../src/page-tools';
|
||||
|
||||
/** Schema and SQL-shape checks only: no query is executed. */
|
||||
const db = {} as Database;
|
||||
@@ -209,3 +218,331 @@ test('an omitted motion filter arrives as the null the emitted schema asks for',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// The figures the tools report
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* A stub handle, because the unit suite still may not execute a query.
|
||||
*
|
||||
* Drizzle's builder is a promise you can keep calling methods on, so this is
|
||||
* the same: every chaining method returns itself and `then` resolves the rows.
|
||||
* What is under test here is the shaping — which figure reaches the headline
|
||||
* and what denominator sits beside it — not the SQL, which
|
||||
* `e2e/page-tools.test.ts` covers against a real book.
|
||||
*
|
||||
* The where clauses are not evaluated: a stub that reimplemented them would be
|
||||
* testing itself, so a filtered read is fixtured as the rows it returned. They
|
||||
* ARE rendered, for the one assertion that has to see them — a denominator
|
||||
* taken over the whole table rather than the shared library would publish the
|
||||
* size of everybody's private drafts, and it would look exactly like this from
|
||||
* the outside.
|
||||
*/
|
||||
interface StubMotion {
|
||||
/** What the library read returned. Shared, non-archived, versions included. */
|
||||
templates: readonly Record<string, unknown>[];
|
||||
/** `count(distinct slug)`: lineages, not rows. Fixtured, not derived. */
|
||||
lineageTotal: number;
|
||||
/** The promotion exemplars, capped by the tool, and their exact total. */
|
||||
promotionRows?: readonly Record<string, unknown>[];
|
||||
promotionTotal?: number;
|
||||
/** The open engagements the summary counts by stage. */
|
||||
openEngagements?: readonly Record<string, unknown>[];
|
||||
/** What the engagement search matched, against every engagement on the book. */
|
||||
engagementRows?: readonly Record<string, unknown>[];
|
||||
engagementTotal?: number;
|
||||
artifacts?: readonly Record<string, unknown>[];
|
||||
scores?: readonly Record<string, unknown>[];
|
||||
}
|
||||
|
||||
function stubBook(book: StubMotion): { handle: Database; denominators: string[] } {
|
||||
const denominators: string[] = [];
|
||||
|
||||
const rowsFor = (
|
||||
table: unknown,
|
||||
projection: Record<string, unknown>,
|
||||
where: string,
|
||||
): readonly unknown[] => {
|
||||
const counting = Object.hasOwn(projection, 'value');
|
||||
if (table === motionTemplates) {
|
||||
if (counting) {
|
||||
denominators.push(where);
|
||||
// Both counts are taken over `motion_templates`; only the promotion one
|
||||
// asks for an origin artefact, which is what tells them apart here.
|
||||
return where.includes('origin_artifact_id')
|
||||
? [{ value: book.promotionTotal ?? 0 }]
|
||||
: [{ value: book.lineageTotal }];
|
||||
}
|
||||
// The promotion exemplars are the only template read carrying a date.
|
||||
return Object.hasOwn(projection, 'createdAt') ? (book.promotionRows ?? []) : book.templates;
|
||||
}
|
||||
if (table === engagements) {
|
||||
if (counting) {
|
||||
denominators.push(where);
|
||||
return [{ value: book.engagementTotal ?? 0 }];
|
||||
}
|
||||
// The summary asks for a stage per open engagement; the search asks for
|
||||
// the row.
|
||||
return Object.hasOwn(projection, 'id')
|
||||
? (book.engagementRows ?? [])
|
||||
: (book.openEngagements ?? []);
|
||||
}
|
||||
if (table === engagementArtifacts) return book.artifacts ?? [];
|
||||
if (table === qualificationScores) return book.scores ?? [];
|
||||
throw new Error('the stub was asked for a table this suite does not fixture');
|
||||
};
|
||||
|
||||
const select = (projection: Record<string, unknown>) => ({
|
||||
from: (table: unknown) => {
|
||||
let where = '';
|
||||
const builder: Record<string, unknown> = {
|
||||
where: (clause: SQL | undefined) => {
|
||||
if (clause) where = dialect.sqlToQuery(clause).sql;
|
||||
return builder;
|
||||
},
|
||||
};
|
||||
for (const method of ['limit', 'orderBy', 'groupBy', 'innerJoin', 'leftJoin']) {
|
||||
builder[method] = () => builder;
|
||||
}
|
||||
builder.then = (resolve: (value: readonly unknown[]) => unknown) =>
|
||||
resolve(rowsFor(table, projection, where));
|
||||
return builder;
|
||||
},
|
||||
});
|
||||
return { handle: { select } as unknown as Database, denominators };
|
||||
}
|
||||
|
||||
type Reading = Record<string, unknown> & { headline?: string; scope?: ResultScope };
|
||||
|
||||
async function read(
|
||||
route: '/motion' | '/motion/library' | '/motion/engagements',
|
||||
handle: Database,
|
||||
input: Record<string, unknown> = {},
|
||||
): Promise<Reading> {
|
||||
const [only] = createPagePigTools(handle, route);
|
||||
assert.ok(only, `no tool for ${route}`);
|
||||
return (await only.execute(input)) as Reading;
|
||||
}
|
||||
|
||||
function template(slug: string, stage: string, version = 1) {
|
||||
return {
|
||||
slug,
|
||||
kind: 'playbook',
|
||||
stage,
|
||||
version,
|
||||
title: `Template ${slug} v${version}`,
|
||||
summary: 'A shared template.',
|
||||
usageCount: 3,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Twelve lineages in thirteen rows, covering six of the eight open stages.
|
||||
*
|
||||
* The thirteenth row is a second version of the first lineage, and it is there
|
||||
* because the two figures differ: a library reported by rows says 13 when the
|
||||
* number of pieces of practice anybody can choose from is 12.
|
||||
*/
|
||||
const COVERED_STAGES = ['qualification', 'legal', 'scoping', 'proposal', 'poc', 'expansion'];
|
||||
const SHARED_ROWS = [
|
||||
...Array.from({ length: 12 }, (_, i) =>
|
||||
template(`starter-${i}`, COVERED_STAGES[i % COVERED_STAGES.length]!),
|
||||
),
|
||||
template('starter-0', 'qualification', 2),
|
||||
];
|
||||
const LINEAGES = 12;
|
||||
|
||||
/** Twelve promotions, of which the tool may show eight. The gap is the point. */
|
||||
const PROMOTION_TOTAL = 12;
|
||||
const PROMOTION_ROWS = Array.from({ length: 8 }, (_, i) => ({
|
||||
title: `Promoted ${i}`,
|
||||
kind: 'case_study',
|
||||
version: 2,
|
||||
createdAt: new Date(Date.UTC(2026, 0, i + 1)),
|
||||
}));
|
||||
|
||||
const LIBRARY = stubBook({
|
||||
templates: SHARED_ROWS,
|
||||
lineageTotal: LINEAGES,
|
||||
promotionRows: PROMOTION_ROWS,
|
||||
promotionTotal: PROMOTION_TOTAL,
|
||||
openEngagements: Array.from({ length: 5 }, () => ({ stage: 'poc', dealName: 'DEMO — Halcyon' })),
|
||||
});
|
||||
|
||||
test('the promotion figure is the exact total, not the length of the list beside it', async () => {
|
||||
const reading = await read('/motion', LIBRARY.handle);
|
||||
|
||||
// The /motion tile counts promotions in SQL for exactly this reason. Piggy
|
||||
// reading the length of its own capped list would peg the answer at 8 the
|
||||
// moment the loop started working, and disagree with the tile on screen.
|
||||
assert.equal(reading.promotions, PROMOTION_TOTAL);
|
||||
assert.match(String(reading.headline), /12 artifact\(s\) promoted back into the library/);
|
||||
assert.doesNotMatch(String(reading.headline), /8 artifact\(s\) promoted/);
|
||||
|
||||
const scope = reading.recentPromotionsScope as ResultScope;
|
||||
assert.equal(scope.total, PROMOTION_TOTAL);
|
||||
assert.equal(scope.listed, PROMOTION_ROWS.length);
|
||||
assert.equal((reading.recentPromotions as unknown[]).length, PROMOTION_ROWS.length);
|
||||
});
|
||||
|
||||
test('the motion summary counts lineages, not versions', async () => {
|
||||
const reading = await read('/motion', LIBRARY.handle);
|
||||
const scope = reading.scope;
|
||||
assert.ok(scope);
|
||||
|
||||
// 13 rows, 12 lineages. A denominator of 13 would be a library four times the
|
||||
// size of the one anybody can choose from, in miniature.
|
||||
assert.equal(scope.total, LINEAGES);
|
||||
assert.equal(reading.sharedTemplates, LINEAGES);
|
||||
assert.notEqual(scope.total, SHARED_ROWS.length);
|
||||
assert.match(scope.totalLabel, /shared template lineage\(s\)/);
|
||||
});
|
||||
|
||||
test('no motion denominator is taken over the whole template table', async () => {
|
||||
await read('/motion', LIBRARY.handle);
|
||||
await read('/motion/library', LIBRARY.handle, {});
|
||||
assert.ok(LIBRARY.denominators.length >= 2);
|
||||
for (const where of LIBRARY.denominators) {
|
||||
// A total that counted private rows would publish the existence and the
|
||||
// size of colleagues' drafts through a figure nobody thinks of as a read.
|
||||
assert.match(where, /"visibility" = \$\d+/, where);
|
||||
assert.match(where, /"archived_at" is null/, where);
|
||||
}
|
||||
});
|
||||
|
||||
test('the stages the library misses are named when the whole library was read', async () => {
|
||||
const reading = await read('/motion', LIBRARY.handle);
|
||||
// Six stages are covered by the fixture, so the two that are not are a piece
|
||||
// of work somebody can act on — which is why this is a list and not a count.
|
||||
assert.deepEqual(reading.uncoveredStages, ['procurement', 'deployment']);
|
||||
assert.match(String(reading.headline), /No shared template covers procurement, deployment\./);
|
||||
});
|
||||
|
||||
test('stage coverage is not asserted from a capped scan', async () => {
|
||||
// SCAN_LIMIT is 500 and the read asks for one more, so 501 rows is a library
|
||||
// that certainly continues past the cap.
|
||||
const capped = stubBook({
|
||||
templates: Array.from({ length: 501 }, (_, i) => template(`over-${i}`, 'qualification')),
|
||||
lineageTotal: 501,
|
||||
promotionRows: PROMOTION_ROWS,
|
||||
promotionTotal: PROMOTION_TOTAL,
|
||||
});
|
||||
const reading = await read('/motion', capped.handle);
|
||||
|
||||
// "No shared template covers deployment" is a definite negative, and the rows
|
||||
// that would refute it are precisely the ones the cap dropped. Null says the
|
||||
// question was not answered; an empty list would say every stage is covered
|
||||
// and a full list would say seven are not, and both are inventions.
|
||||
assert.equal(reading.uncoveredStages, null);
|
||||
assert.doesNotMatch(String(reading.headline), /No shared template covers/);
|
||||
assert.doesNotMatch(String(reading.headline), /Every live demand stage/);
|
||||
assert.match(String(reading.headline), /row cap/);
|
||||
});
|
||||
|
||||
test('a filtered library search is counted against the whole shared library', async () => {
|
||||
// The stub evaluates no where clause, so what a kind filter matched is
|
||||
// fixtured: three lineages, out of a library that still holds twelve.
|
||||
const filtered = stubBook({
|
||||
templates: [
|
||||
template('proposal-blocks', 'proposal'),
|
||||
template('proposal-terms', 'procurement'),
|
||||
template('proposal-exec', 'proposal'),
|
||||
],
|
||||
lineageTotal: LINEAGES,
|
||||
});
|
||||
const reading = await read('/motion/library', filtered.handle, { kind: 'proposal' });
|
||||
const scope = reading.scope;
|
||||
assert.ok(scope);
|
||||
|
||||
assert.equal(scope.matched, 3);
|
||||
assert.equal(scope.total, LINEAGES);
|
||||
assert.equal(reading.count, 3);
|
||||
assert.equal(reading.sharedTemplates, LINEAGES);
|
||||
// The denominator has to reach the headline, because the headline is the
|
||||
// field a small model quotes: "3 shared templates" alone is the size of a
|
||||
// filter presented as the size of the library.
|
||||
assert.match(String(reading.headline), /3 of 12 shared template lineage\(s\)/);
|
||||
assert.match(scope.summary, /the total is 12/);
|
||||
assert.equal(scope.filters.kind, 'proposal');
|
||||
});
|
||||
|
||||
test('an unfiltered library search says so rather than hedging an exact figure', async () => {
|
||||
const reading = await read('/motion/library', LIBRARY.handle, {
|
||||
kind: null,
|
||||
stage: null,
|
||||
query: null,
|
||||
});
|
||||
const scope = reading.scope;
|
||||
assert.ok(scope);
|
||||
|
||||
// Nothing was filtered out, so `matched` IS the total. Reporting the nulls a
|
||||
// schema-abiding model sends as filters would teach it to distrust a figure
|
||||
// that is exact.
|
||||
assert.equal(scope.matched, scope.total);
|
||||
assert.deepEqual(scope.filters, {});
|
||||
assert.match(scope.summary, /All 12 shared template lineage\(s\) in the motion library/);
|
||||
});
|
||||
|
||||
test('an engagement search is counted against every engagement on the book', async () => {
|
||||
const book = stubBook({
|
||||
templates: [],
|
||||
lineageTotal: LINEAGES,
|
||||
engagementTotal: 9,
|
||||
engagementRows: [
|
||||
{
|
||||
id: 'engagement-1',
|
||||
status: 'open',
|
||||
summary: 'Mid POC.',
|
||||
openedAt: new Date(Date.UTC(2026, 1, 1)),
|
||||
stage: 'poc',
|
||||
dealName: 'DEMO — Halcyon Research',
|
||||
accountName: 'DEMO — Halcyon Research',
|
||||
},
|
||||
{
|
||||
id: 'engagement-2',
|
||||
status: 'closed',
|
||||
summary: 'Won.',
|
||||
openedAt: new Date(Date.UTC(2025, 10, 1)),
|
||||
stage: 'expansion',
|
||||
dealName: 'DEMO — Halcyon Expansion',
|
||||
accountName: 'DEMO — Halcyon Research',
|
||||
},
|
||||
],
|
||||
artifacts: [{ engagementId: 'engagement-1', status: 'final' }],
|
||||
scores: [],
|
||||
});
|
||||
const reading = await read('/motion/engagements', book.handle, { query: 'Halcyon' });
|
||||
const scope = reading.scope;
|
||||
assert.ok(scope);
|
||||
|
||||
assert.equal(scope.matched, 2);
|
||||
assert.equal(scope.total, 9);
|
||||
assert.equal(reading.totalEngagements, 9);
|
||||
assert.match(String(reading.headline), /2 of 9 engagement\(s\) on the book match "Halcyon"/);
|
||||
assert.equal(scope.filters.query, 'Halcyon');
|
||||
});
|
||||
|
||||
test('every motion result carries a complete scope, and no scope outruns its own total', async () => {
|
||||
const readings = [
|
||||
await read('/motion', LIBRARY.handle),
|
||||
await read('/motion/library', LIBRARY.handle, {}),
|
||||
await read('/motion/engagements', stubBook({ templates: [], lineageTotal: 0 }).handle),
|
||||
];
|
||||
|
||||
for (const reading of readings) {
|
||||
const found = Object.entries(reading)
|
||||
.filter(([key]) => key === 'scope' || key.endsWith('Scope'))
|
||||
.map(([, value]) => value as ResultScope);
|
||||
assert.ok(found.length > 0, `a motion result carries no scope at all: ${reading.headline}`);
|
||||
for (const scope of found) {
|
||||
assert.ok(scope.summary.length > 0);
|
||||
assert.ok(scope.totalLabel.length > 0);
|
||||
// The denominator has to reach the sentence, because the sentence is what
|
||||
// gets quoted.
|
||||
assert.match(scope.summary, new RegExp(`\\b${scope.total}\\b`));
|
||||
assert.ok(scope.matched <= scope.total, 'matched exceeds its own denominator');
|
||||
assert.ok(scope.listed <= scope.matched, 'more rows listed than matched');
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user