feat(e2e): run paired continuation workloads with Pi

[Fengzdadi]

Which issue or RFC does this PR close?

Part of #1705, the last plugin host in the plan, after Codex (#1779), Claude Code (#1800), and OpenCode (#1835). It uses the OFF definition from #1816.

Rationale for this change

#1705 asks for PowerContext OFF/ON runs across agent hosts, including an overlapping configuration. Pi has an official PowerContext package, and Harbor ships a Pi agent, so it can run the same continuation workloads, grader, and evidence as the other hosts. This run uses GLM-5.3 through OpenRouter, the same model as the OpenCode run, so the two hosts can be compared on one model.

What changes are included in this PR?

• Pi host: paired --host pi runs both arms with PowerContextPiAgent, a subclass of Harbor's Pi agent and a fourth PluginHost.
• Pi version: Harbor installs Pi from its former npm name, @mariozechner/pi-coding-agent, which ends at 0.73.1. The PowerContext Pi package targets @earendil-works/pi-coding-agent, so the agent installs that package at 0.82.1, the version the Pi package's lockfile and make pi-test use, with Harbor 0.16's own install steps.
• Arms:
  • OFF follows fix(e2e): keep PowerContext out of the OFF arm #1816: no package, no mounts, and no POWERCONTEXT_* settings.
  • ON runs pi install on the mounted package, as powercontext setup pi does. Pi records the path and loads the package in place, so the mount is read-only. The container sees only the package's package.json, extensions, src, and skills. The package reads its Server URL, Scope, plain-HTTP consent, and the harness Client's token (POWERCONTEXT_PI_AUTHORIZATION) from its environment.
• Pi's own session data: Harbor runs Pi with --no-session, so Pi saves no session. Pi's bash tool keeps the full output of a command over 2,000 lines or 50 KB as pi-bash-<id>.log in the temporary directory and does not delete it, and Harbor does not clear that directory between the steps of a trial. Before each session in both arms, the agent removes those files, so a recall session cannot read its capture session's tool output.
• Settings: the reasoning effort, medium by default, becomes Pi's --thinking. Harbor passes the key of the model's provider, such as OPENROUTER_API_KEY.
• README: documents the Pi arms, settings, and run command.

Are there any user-facing changes?

• paired --host accepts pi.
• Bub, Codex, Claude Code, and OpenCode runs are unchanged, and so are the acceptance command, its manifests, and its CI.

How was this change tested?

Pilot run

make harness-paired ARGS='--host pi --trials 2' on this PR's head commit, against a real Server, Harbor, Pi, and model:

Trial	Order	OFF recall	ON recall
1	OFF, ON	nulls, reward 0	OceanBase / 12, reward 1
2	ON, OFF	nulls, reward 0	OceanBase / 12, reward 1

• Report: OFF 0/2 and ON 2/2, with 2 scored pairs and a mean ON minus OFF of +1.00. There were no errors, timeouts, or integration failures.

• Treatment evidence (ON):

  • After the capture session, the Scope held 1 Source (the user prompt) and 1 Memory entry after the flush.
  • During the recall session, context requests rose from 1 to 2 and ready preparations from 0 to 1.
  • The agent answered from the context the package injected, without calling a package tool.

• OFF behavior: the Harbor job configs have no mounts and an empty agent environment. In the recall session both agents searched /workspace, read the README, and wrote null for both values. Neither OFF trajectory mentions PowerContext, the Server, or Pi's saved tool output.

• Authentication: the Server ran with POWERCONTEXT_SERVER_ACCESS_MODE=enforced. It answered /v1/scopes with 401 without a token, and every ON capture and context request authenticated. Neither the full Server token nor the OpenRouter key appears in the evidence.

• Token usage (from Pi's own usage records, cached tokens included):

  Session	OFF prompt tokens	ON prompt tokens	Model calls
  Capture	about 4.6k	about 54.5k	3
  Recall	about 6.5k	about 36.7k	OFF 4, ON 2

  Each ON model call carries about 18k prompt tokens, against under 2k for OFF: the package registers its tools and guidance on every request. The ON recall used fewer calls because it answered directly.

• Configuration:

  • Agent: Harbor 0.16.1, Pi 0.82.1, openrouter/z-ai/glm-5.3 with --thinking medium.
  • Server: local, on SQLite with the default coding extraction profile and enforced access. Generation used openrouter:deepseek/deepseek-v4-pro. Embeddings used openai/text-embedding-3-small (1536 dimensions) through OpenRouter's OpenAI-compatible endpoint, as in the OpenCode run; it answered in 0.3–0.6 s before the run. The agent reached the Server at http://host-gateway:8000 with POWERCONTEXT_PI_ALLOW_INSECURE_HTTP=true.
  • Runtime: OrbStack on macOS.

• Time: each arm took 30–45 s.

Pi's saved tool output on a real Pi

In a throwaway node:22-bookworm container with @earendil-works/pi-coding-agent@0.82.1 and the same model:

• A session whose bash command printed 30,000 lines ended with [Showing lines 29017-30000 of 30000 (50.0KB limit). Full output: /tmp/pi-bash-565921934bc38449.log]. After Pi exited, that file was still there, 1.5 MB with every line.
• The agent's clearing command, run as Harbor runs it (bash with set -o pipefail), removed the file and exited 0. Run again with nothing to remove, it also exited 0.
• Pi started normally afterwards.
• Under ~/.pi/agent, Pi had written only auth.json and models-store.json, neither with session content.

Checks

• make check and make harness-check pass.
• New and generalized tests:
  • The plugin-host tests now include Pi: OFF has no mounts, environment, or package, and ON gets the mounts, the Scope, and Authorization: Bearer <token>.
  • Pi installs the package only for ON.
  • The clearing command runs in a real bash against a seeded temporary directory, for both arms: the saved tool output is gone and other files stay. With nothing to remove, the session still starts.
  • Pi runs with --no-session, which the statement that Pi saves no session depends on.
  • Redaction covers POWERCONTEXT_PI_AUTHORIZATION.

Limits

• Pilot scope: one task and two trials per arm validate the pipeline and its evidence; they do not measure PowerContext's effect.
• Copied install steps: Harbor hard-codes the former package name in its install command, so the agent repeats Harbor 0.16's steps with the current name. They need a comparison when Harbor is upgraded.
• Unlisted model: Pi 0.82.1 warns that z-ai/glm-5.3 is not in its model list for OpenRouter and uses it as a custom model ID. The run is unaffected.
• Silent skip without HTTP consent: without POWERCONTEXT_PI_ALLOW_INSECURE_HTTP=true, the package stays inactive against a plain-HTTP Server, and the run shows an integration failure rather than a scored result.
• Embedding latency: the package's context request shares a 3 s timeout with the Server's embedding call, so a slow embedding provider turns ON sessions into integration failures. Check the provider's latency before a run.
• What the package captures: like the Codex, Claude Code, and OpenCode plugins, the Pi package captures only user prompts, while Bub's ON arm also captures model and tool results.
• Masked token in Harbor's job config: Harbor writes the agent environment to its job config.json with its own masking, which keeps the first four and last three characters of POWERCONTEXT_<HOST>_AUTHORIZATION (Bear**** plus three characters of the token). This applies to every plugin host since fix(e2e): keep PowerContext out of the OFF arm #1816. The results directory is not committed.
• Earlier steps' verifier stays visible: as on every host, Harbor leaves the capture step's /tests/test.sh in the container during recall. It holds no answer, but its license header names OceanBase, which is also this task's answer.
• Not run: the fixed Compose harness, other models, and providers other than OpenRouter. CI does not run Pi through Harbor.

AI usage statement

This PR was developed with Claude Code (Claude Opus 5.5 and Claude Fable 5.1), which designed and wrote the change and tests, ran the checks, the container test, and the pilot above, and analyzed the evidence. The author chose the model, approved the OFF definition and the Server configuration, reviewed the change, and provided the pilot environment.

🤖 Generated with Claude Code