Skip to content

fix(sandbox): verify a port-forward is alive before reusing it, and close the env gitignore gap - #1

Merged
AsherDLL merged 1 commit into
mainfrom
public/portforward-liveness
Aug 8, 2026
Merged

fix(sandbox): verify a port-forward is alive before reusing it, and close the env gitignore gap#1
AsherDLL merged 1 commit into
mainfrom
public/portforward-liveness

Conversation

@AsherDLL

@AsherDLL AsherDLL commented Aug 8, 2026

Copy link
Copy Markdown
Member

Why an MCP engine goes unreachable

kubectl port-forward reports per-connection faults on stderr and keeps
running
. The tunnel dies; the process does not. From outside, a dead tunnel is
indistinguishable from a healthy one.

The module already documented this at _terminate_handle"a tunnel can be
alive and useless, which looks identical to a healthy one from the outside"

and never acted on it.

Measured: 4 of ~60 recorded runs. In the worst, forwards opened at
12:11:25 and the engines were unreachable by 12:14:49 — processes still
alive, nothing reopening. Those runs produced reports blaming the analysis engine
rather than the transport.

Two halves, neither works alone

1. open() now probes before reusing. It previously returned early on
same pod + process alive, while its own docstring promised to "block until it
forwards"
:

if existing.pod == pod and exit_code is None:
    return          # never re-probed

So a stage calling prepare_* again to repair a dead tunnel got an immediate
return and the same dead tunnel. The probe is the existing HTTP readiness
check — already the right test, since a TCP connect only proves the port is open.
It runs with a single attempt: opening waits for a tunnel to come up and keeps
the full 15, but checking an existing one is a liveness question whose first
failure is the answer.

A failed probe logs its own reason (probe failed), distinct from pod replaced
and process exited — in a live log it reads as the fix firing, not a
regression.

2. triage_recon and packer_analyst gain prepare_sandbox. The tunnel is
opened twice per run, both at intake, and four agents consume those two servers
minutes apart:

agent consumes could re-establish?
triage_recon radare2 no → now yes
retriage radare2 yes
packer_analyst radare2 no → now yes
dotnet_decompile ilspy yes

LESSONS_LEARNED #6 already said each analysis agent must prepare its own engine.
It had been applied to the two agents someone noticed. A test now asserts the
property across all four — that is the shape of the original bug: a rule
applied case by case leaves holes invisible until a transport flakes.

The .gitignore gap

.gitignore had env/ — the virtualenv directory pattern — but nothing
matching a file named env. One turned up in a working tree holding live
provider keys, a single git add -A away from this now-public repository.

Now covered: the bare name, the dotted forms (.env.local) and the suffixed
forms (prod.env), with .env.example negated back in because it is the
template and must stay tracked.

env .env .env.local prod.env .env.example env/x.py
ignored ignored ignored ignored tracked ignored

Verified that no credential value appears in this repository's files, in its
history, or in this diff.

Testing

make check: 2469 passed, 1 skipped. 12 new tests, no cluster needed.

Live validation still outstanding — expect zero Optional MCP server unavailable; a replacing a stale port-forward at reason="probe failed" is
the fix working.

…lose the env gitignore gap

kubectl port-forward reports per-connection faults on stderr and keeps running,
so a dead tunnel is indistinguishable from a healthy one by poll() alone. The
module documented exactly this at _terminate_handle -- "a tunnel can be alive
and useless, which looks identical to a healthy one from the outside" -- and
then never acted on it.

Measured across recorded runs: it hit 4 of ~60. In the worst, forwards opened at
12:11:25 and the engines behind them were unreachable by 12:14:49, processes
still alive, nothing reopening. The reports those runs produced blamed the
analysis engine rather than the transport.

Two halves, and neither works alone.

open() now probes an existing tunnel before handing it back. It previously
returned early on "same pod + process alive" while its own docstring promised to
block until it forwards, so a stage calling prepare_* again to repair a dead
tunnel got an immediate return and the same dead tunnel. The probe is the
existing HTTP readiness check, which is already the right test -- a TCP connect
only proves the port is open. It runs with a single attempt: opening a tunnel
means waiting for one to come up and keeps the full fifteen, but checking an
existing one is a liveness question whose first failure is the answer. A failed
probe is logged as its own reason, distinct from a replaced pod or an exited
process, so it reads as the fix firing rather than as a regression.

triage_recon and packer_analyst gain prepare_sandbox. The tunnel is opened twice
per run, both at intake, and four agents consume those two servers minutes
apart; two of them carried no prepare tool and so depended on a tunnel another
agent had opened, with no way to repair it. LESSONS_LEARNED #6 already said each
analysis agent must prepare its own engine. It had been applied to the two
agents someone noticed. A test now asserts the property across all four
consumers rather than the two, which is the shape of the original bug: a rule
applied case by case leaves holes that stay invisible until a transport flakes.

Separately, .gitignore had `env/` -- the virtualenv directory pattern -- but
nothing matching a FILE named `env`. One turned up in a working tree holding
live provider keys, a single `git add -A` away from this now-public repository.
The bare name, the dotted forms and the suffixed forms are all covered now, with
.env.example negated back in because it is the template and must stay tracked.
@AsherDLL

AsherDLL commented Aug 8, 2026

Copy link
Copy Markdown
Member Author

Live validation complete — all three fixes proven

Port-forward liveness (this PR). Tested against a real tunnel, not a stubbed one. A SIGSTOPped kubectl port-forward is alive to poll() but no longer forwards — the exact state the old code returned early on:

2. reuse while healthy   pid 784603 -> 784603   KEPT (correct)
3. SIGSTOP               poll() says alive: True   <- what the old code trusted
4. reuse now             replacing a stale port-forward
                         exit_code=None  reason='probe failed'
                         pid 784603 -> 785695   REBUILT
                         replacement answers: True

exit_code=None is the proof: the process was alive and the tunnel was dead, and the registry now rebuilds instead of handing it back.

ILSpy sanitization. Live run: 14 of 14 ILSpy responses framed by the membrane across 8 tools, against 0 of 17 before. Also driven offline through the real re_guarded chain with the sample's actual SkiDzEX : https://discord.gg/… type name plus an injection attempt, with prepare_ilspy as a passing negative control.

Critic laundering rules. accepted=73 rejected=0 qualified=3 (previously 88/0/0). The critic now acts, and on exactly the intended class — three findings whose detail read 0 matches while the claim asserted positive content. The previous run's specific defect (a T1071 C2 mapping sourced only from VirusTotal intel but cited to search_strings) did not recur.

Full-run signals — all zero

Optional MCP server unavailable  0        Run stopped (context kill)  0
evidence_envelope_invalid        0        findings_dropped            0
ilspy_unavailable                0        Traceback                   0

Report written to disk (.md + .html), sandbox claims released, make check 2469 passed / 1 skipped.

Two notes for the record

Peak context reached 228k on dotnet_decompile — above the old 200k budget, so the raise to 400k remains load-bearing rather than precautionary.

dotnet_analyst grew to 91 calls / 11.6M tokens (from 49 / 5.19M on the same sample). Nothing failed, but it confirms the execution-count cap does not bound cost — LESSONS_LEARNED #20, still open.

@AsherDLL
AsherDLL merged commit 60010ee into main Aug 8, 2026
2 checks passed
@AsherDLL
AsherDLL deleted the public/portforward-liveness branch August 8, 2026 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant