Skip to content

Run the generated smoke test in test:init instead of the bespoke lookup harness #995

Description

@2chanhaeng

Summary

The hydrated (--no-dry-run) phase of mise run test:init verifies each generated app with its own ad-hoc harness (packages/init/src/test/lookup.ts + server.ts + port.ts): it reserves a port, patches it into the app, spawns the dev server, waits for readiness, and looks up http://localhost:<port>/users/john.

Since #990, every generated project ships a test task that does almost exactly the same thing (scripts/smokeTest.ts: spawn the dev server, detect the port from its output, wait for readiness, resolve /users/john with lookupObject()). The same logic now lives in two places -- the test harness and the template -- and any change to how generated apps are verified (readiness polling, port detection, lookup target) has to be made twice, or the two copies drift apart.

This issue proposes removing the duplication by making the harness run the generated project's own test task (deno task test / npm test / ...) instead of orchestrating the server and lookup itself. The verification logic then lives only in the template, and the harness shrinks to generating the app and invoking one command per case. The smoke test's actor lookup via lookupObject() is a sufficient replacement for the harness's fedify lookup invocation.

Proposed behavior

To keep test cases from colliding on the same port, the smoke-test script is extended to accept a port from the command line (e.g. deno task test --port 3001), overriding the port auto-detected from the server output. The harness reserves a fresh port per case (as reservePort does today) and passes it through this option. Likewise, the script's hard-coded STARTUP_TIMEOUT (15s) becomes overridable from the command line so the harness can grant CI machines a longer startup window.

In runServerAndLookupUser (or its replacement), for each generated app directory:

  1. Skip the combinations in BANNED_LOOKUP_REASONS, which is preserved as-is; the same combinations that cannot run a dev server cannot run the smoke test either.
  2. Resolve the package manager's test invocation (reuse the getDevCommand-style mapping in packages/init/src/lib.ts, e.g. deno task test, npm test, bun run test) and append the reserved port.
  3. Run it in the app directory and treat the exit code as the pass/fail signal.
  4. Keep the existing summary output (total/passed/failed) and per-case diagnostics (persisting the child process output to out.txt / err.txt as today).

The now-unused harness code (serverClosure, waitForServer, and the port detection in server.ts / port.ts) is deleted or slimmed down accordingly.

Metadata

Metadata

Assignees

Type

No type

Fields

Priority

Low

Effort

Low

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions