Keep the test and snapshot harness out of production installs (0.4.0) (#72)

Author: nedtwigg

PACKAGES.md

@@ -1,6 +1,6 @@
 # Consuming pgstencil
 
-Three packages share version 0.3.0: `pgstencil` (infrastructure and test primitives), `@pgstencil/auth` (email/OAuth/session services and a JSON HTTP adapter), and `@pgstencil/stripe` (SaaS billing and StripeDev). They are published to npm as 0.x releases, so a minor release may break compatibility.
+Three packages share version 0.4.0: `pgstencil` (infrastructure and test primitives), `@pgstencil/auth` (email/OAuth/session services and a JSON HTTP adapter), and `@pgstencil/stripe` (SaaS billing and StripeDev). They are published to npm as 0.x releases, so a minor release may break compatibility.
 
 Run `pnpm packages:pack` to produce the three archives in `dist/packages`. They contain ESM JavaScript, TypeScript declarations, the MIT license, and required SQL/Compose assets. Workspace development uses source exports; packing switches the exports to compiled files. Nothing runs migrations during install.
 
@@ -16,6 +16,23 @@ Applications declare pgstencil's peer dependencies themselves: `kysely` for ever
 
 Every other dependency is private: pgstencil owns its version and applications do not import it. Better Auth is deliberately private. pgstencil imports its internal subpaths and tests login and linking rules against specific releases, so it is pinned to the exact release CI tested. Even a patch can change internals pgstencil reads (1.7.7 renamed its OAuth state rows), so each Better Auth upgrade reaches applications only through a pgstencil release. A new login provider or Better Auth plugin belongs in `@pgstencil/auth`, not in an application. Private ranges start at the version pgstencil's CI tested. [`.github/renovate.json`](.github/renovate.json) raises that floor, and the next release carries it into each application.
 
+### Tooling peers
+
+pgstencil's test, migration and snapshot tooling loads optional peers, so a production install leaves them out. No entry point imports them statically, and only three load them; `scripts/runtime-deps.test.mjs` and `pnpm packages:verify` fail otherwise. An application using the tooling adds the peers it needs, usually as devDependencies:
+
+| Entry point           | Optional peer         | Loaded when                                                                                                      |
+| --------------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------- |
+| `pgstencil/database`  | `testcontainers`      | Local services start or reattach: `ensureServices`, `prepareTemplate`, `allocateDatabase`, `developmentDatabase` |
+| `pgstencil/database`  | `node-pg-migrate`     | `migrate` runs, including inside `allocateDatabase` and `developmentDatabase`                                    |
+| `pgstencil/testing`   | both of the above     | `createTestContext`                                                                                              |
+| `pgstencil/snapshots` | `cheerio`, `turndown` | The module is imported                                                                                           |
+
+```sh
+pnpm add -D testcontainers node-pg-migrate cheerio turndown
+```
+
+Reading migration sources (`defaultMigrations`, `readMigrations`, `validateMigrations`, `appliedMigrations`) needs none of them. A release step that migrates a persistent database needs `node-pg-migrate`, which peers on `pg`, but not `testcontainers`; make it a regular dependency if that step runs from a production install. A missing peer fails with an error naming the package and the entry point that needs it. The peer ranges start at the versions pgstencil's CI tests, and pgstencil widens them as for any peer.
+
 ## Releasing
 
 The three packages share one version. A release is a pull request that raises `version` in all three `package.json` files: a patch for a fix or compatible addition, a minor for a breaking change, since these are 0.x. Nothing else triggers one.
@@ -34,7 +51,7 @@ Version 0.2.0 was published by hand from commit `e79cc4d`, after its audit passe
 
 ## Application composition
 
-Production imports use `pgstencil`, `pgstencil/postgres`, `@pgstencil/auth` and `@pgstencil/stripe`. Docker startup and test fixtures are opt-in imports through `pgstencil/database`, `pgstencil/testing`, and `@pgstencil/stripe/testing`.
+Production imports use `pgstencil`, `pgstencil/postgres`, `pgstencil/diagnostics`, `@pgstencil/auth` and `@pgstencil/stripe`. Docker startup and test fixtures are opt-in imports through `pgstencil/database`, `pgstencil/testing`, and `@pgstencil/stripe/testing`; the first two need the [tooling peers](#tooling-peers).
 
 The database tooling uses the current working directory as the project root, or `PGSTENCIL_PROJECT_ROOT` when explicitly set. State goes in that project's `.pgstencil`; its Docker Compose project name derives from the project path. A project can supply `compose.yaml`, otherwise the package's bundled default is used. The default SQL directory is `migrations`, overridable by a `pgstencil.json` file containing a `migrations` directory path inside the project root; a path resolving outside it is refused. Applications composing packages should pass their complete source list explicitly:
 

README.md

@@ -32,7 +32,7 @@ Use `PORT=3000 pnpm dev` for a fixed port. Development uses real time so cooldow
 | Email              | Injected `EmailSender`, with `EmailDev` capture, waiting, unread checks and preview routes            |
 | Snapshots          | Vitest file snapshots plus local JSON, response, HTML, Markdown and email captures                    |
 
-The workspace contains `pgstencil`, `@pgstencil/auth`, and `@pgstencil/stripe`. They are published to npm as 0.x releases; [PACKAGES.md](PACKAGES.md) explains installing them and consuming unreleased changes through compiled local tarballs. The core package's `pgstencil/postgres` export is the runtime connection layer; `pgstencil/database` and `pgstencil/testing` include local Docker infrastructure. `examples/login` is a complete consumer, using Node's HTTP server and native HTML forms. No frontend framework is required.
+The workspace contains `pgstencil`, `@pgstencil/auth`, and `@pgstencil/stripe`. They are published to npm as 0.x releases; [PACKAGES.md](PACKAGES.md) explains installing them and consuming unreleased changes through compiled local tarballs. The core package's `pgstencil/postgres` export is the runtime connection layer; `pgstencil/database` and `pgstencil/testing` include local Docker infrastructure, and with `pgstencil/snapshots` they load [optional peers](PACKAGES.md#tooling-peers) that a production install leaves out. `examples/login` is a complete consumer, using Node's HTTP server and native HTML forms. No frontend framework is required.
 
 Hono and Cloudflare Workers are supported through the shared Fetch adapter and request-scoped Hyperdrive connections. See [WORKERS.md](WORKERS.md) for the deployable example, local runtime tests, and deployment preparation.
 

examples/better-auth/package.json

@@ -10,5 +10,9 @@
     "pg": "^8.16.0",
     "pgstencil": "workspace:*",
     "@pgstencil/auth": "workspace:*"
+  },
+  "devDependencies": {
+    "node-pg-migrate": "^9.0.0",
+    "testcontainers": "^12.2.1"
   }
 }

examples/login/package.json

@@ -11,5 +11,9 @@
     "@pgstencil/auth": "workspace:*",
     "hono": "^4.13.7",
     "stripe": "^22.6.1"
+  },
+  "devDependencies": {
+    "node-pg-migrate": "^9.0.0",
+    "testcontainers": "^12.2.1"
   }
 }

packages/auth/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@pgstencil/auth",
-  "version": "0.3.0",
+  "version": "0.4.0",
   "type": "module",
   "exports": {
     ".": "./src/index.ts",

packages/pgstencil/README.md

@@ -6,6 +6,8 @@ Postgres schema migrations, isolated test databases, and deterministic snapshot
 
 ```sh
 pnpm add pgstencil kysely
+# Only for pgstencil/database, pgstencil/testing and pgstencil/snapshots:
+pnpm add -D testcontainers node-pg-migrate cheerio turndown
 ```
 
-`kysely` is a peer dependency: your application owns its version and shares one copy with pgstencil. See [PACKAGES.md](https://github.com/diffplug/pgstencil/blob/main/PACKAGES.md) for usage and the dependency policy, and [SECURITY.md](https://github.com/diffplug/pgstencil/blob/main/SECURITY.md) for the audited security rules. Source: [diffplug/pgstencil](https://github.com/diffplug/pgstencil).
+`kysely` is a peer dependency: your application owns its version and shares one copy with pgstencil. The test, migration and snapshot tooling loads optional peers, so a production install of the runtime entry points never carries the Docker client or the HTML converter; [PACKAGES.md](https://github.com/diffplug/pgstencil/blob/main/PACKAGES.md#tooling-peers) lists which entry point needs which. See [PACKAGES.md](https://github.com/diffplug/pgstencil/blob/main/PACKAGES.md) for usage and the dependency policy, and [SECURITY.md](https://github.com/diffplug/pgstencil/blob/main/SECURITY.md) for the audited security rules. Source: [diffplug/pgstencil](https://github.com/diffplug/pgstencil).

packages/pgstencil/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pgstencil",
-  "version": "0.3.0",
+  "version": "0.4.0",
   "type": "module",
   "exports": {
     ".": "./src/index.ts",
@@ -11,17 +11,35 @@
     "./diagnostics": "./src/diagnostics.ts"
   },
   "dependencies": {
-    "pg": "^8.23.1",
+    "pg": "^8.23.1"
+  },
+  "peerDependencies": {
+    "kysely": "^0.29.5",
     "node-pg-migrate": "^9.0.0",
     "testcontainers": "^12.2.1",
     "cheerio": "^1.2.0",
     "turndown": "^7.2.4"
   },
-  "peerDependencies": {
-    "kysely": "^0.29.5"
+  "peerDependenciesMeta": {
+    "node-pg-migrate": {
+      "optional": true
+    },
+    "testcontainers": {
+      "optional": true
+    },
+    "cheerio": {
+      "optional": true
+    },
+    "turndown": {
+      "optional": true
+    }
   },
   "devDependencies": {
-    "kysely": "^0.29.5"
+    "kysely": "^0.29.5",
+    "node-pg-migrate": "^9.0.0",
+    "testcontainers": "^12.2.1",
+    "cheerio": "^1.2.0",
+    "turndown": "^7.2.4"
   },
   "engines": {
     "node": ">=24"

packages/pgstencil/src/database.ts

@@ -1,15 +1,12 @@
 import { mkdir, readFile, writeFile } from 'node:fs/promises';
 import { join, dirname, basename } from 'node:path';
-import {
-  DockerComposeEnvironment,
-  Wait,
-  type StartedDockerComposeEnvironment,
-} from 'testcontainers';
+import type { StartedDockerComposeEnvironment } from 'testcontainers';
 import pg from 'pg';
 import { queryDatabase, withClient } from './postgres.ts';
 export { connectDatabase, queryDatabase, withClient } from './postgres.ts';
 import { migrate, migrationFingerprint, readMigrations } from './migrations.ts';
 import { withProcessLock } from './lock.ts';
+import { importPeer } from './peer.ts';
 import {
   composeFile,
   projectName,
@@ -59,6 +56,12 @@ export function ensureServices(): Promise<Services> {
   servicesPromise ??= withProcessLock(
     join(stateDirectory, 'services.lock'),
     async () => {
+      // Loaded even when reattaching, so a missing peer fails the same way every run.
+      const { DockerComposeEnvironment, Wait } = await importPeer(
+        'testcontainers',
+        'pgstencil/database',
+        () => import('testcontainers'),
+      );
       const statePath = join(stateDirectory, 'services.json');
       try {
         const state = JSON.parse(await readFile(statePath, 'utf8')) as Services;

packages/pgstencil/src/migrations.ts

@@ -3,8 +3,8 @@ import { readdir, readFile, mkdtemp, writeFile, rm } from 'node:fs/promises';
 import { tmpdir } from 'node:os';
 import { join } from 'node:path';
 import pg from 'pg';
-import { runner } from 'node-pg-migrate';
 import { withClient } from './postgres.ts';
+import { importPeer } from './peer.ts';
 export interface MigrationFile {
   /** Ordering is enforced within each directory; package sources evolve independently. */
   source: string;
@@ -118,6 +118,11 @@ export async function migrate(
   url: string,
   files: MigrationFile[],
 ): Promise<void> {
+  const { runner } = await importPeer(
+    'node-pg-migrate',
+    'pgstencil/database',
+    () => import('node-pg-migrate'),
+  );
   const client = new pg.Client({ connectionString: url });
   await client.connect();
   let temporary: string | undefined;

packages/pgstencil/src/peer.ts

@@ -0,0 +1,27 @@
+/**
+ * Loads an optional peer dependency. Tooling entry points reach `testcontainers`,
+ * `node-pg-migrate`, `cheerio` and `turndown` only through here, so a runtime
+ * install never needs them and a tooling install that lacks one is told which
+ * package to add. `scripts/runtime-deps.test.mjs` fails on a static import.
+ */
+export async function importPeer<T>(
+  name: string,
+  entry: string,
+  load: () => Promise<T>,
+): Promise<T> {
+  try {
+    return await load();
+  } catch (error) {
+    // Only the peer itself being absent; a broken install of it surfaces as-is.
+    const code = (error as { code?: unknown } | null)?.code;
+    if (
+      code === 'ERR_MODULE_NOT_FOUND' &&
+      String((error as Error).message).includes(`'${name}'`)
+    )
+      throw new Error(
+        `${entry} needs the optional peer dependency "${name}". Add it to your project, for example: pnpm add -D ${name}`,
+        { cause: error },
+      );
+    throw error;
+  }
+}