Skip to content

ULW v1: active-cell admission failures can lead to unbounded retry/respawn loops and parent-child lifecycle divergence #143

Description

@kihyunnn

Summary

When OMO/ULW runs on the multi_agent_v1 surface and Codex starts returning
too many active cells, the workflow can continue attempting lifecycle work
(waiting, follow-up messages, review retries, and replacement workers) rather
than converge to one terminal capacity-blocked state.

This is not a request to remove V1. V1 is intentionally required for
heterogeneous routed-provider delegation in deployments where V2 can reject an
encrypted native-to-routed child task as unreadable_encrypted_agent_task.

Environment

  • LazyCodex / OMO: 4.18.2
  • Codex App: 0.145.0
  • Multi-agent surface: multi_agent_v1
  • Heterogeneous routed-provider delegation enabled
  • OpenCodex catalog intentionally forced multi_agent_version: "v1" for the
    affected sessions.

The affected Codex session records confirmed multi_agent_version: "v1" for
both gpt-5.6-sol and gpt-5.6-luna sessions.

Observed behavior

During long ULW workflows involving implementation, QA, independent review, and
gate review:

  1. Codex eventually returned too many active cells.
  2. The parent kept reporting and retrying the same capacity-blocked condition.
  3. The workflow continued attempting or planning replacement-worker/reviewer
    work after the admission failure.
  4. In one observed case, a parent was marked blocked while a delegated worker
    still appeared active.
  5. The workflow did not converge to one actionable terminal state without
    manual interruption.

The symptom appeared across multiple models using the same V1 worker surface,
so it does not appear specific to one routed model.

Why V1 is intentional here

V2 is not a drop-in workaround for this deployment. OpenCodex documents that a
native parent can deliver a child task as backend-encrypted content; a routed
non-native child cannot decrypt that content and is rejected with:

HTTP 400 unreadable_encrypted_agent_task

V1 is therefore required here for heterogeneous-provider delegation. The issue
is the lack of safe V1 admission-failure handling.

Static source evidence

1. Spawn guard has a high volume-only limit

In OMO 4.18.2, components/ulw-loop/dist/spawn-guard.js contains:

const DEFAULT_FANOUT_LIMIT = 60;

The guard maintains a per-session total spawn counter. It does not inspect
native active-cell capacity, maintain a live-worker registry, stop after an
admission failure, or enforce parent/child terminal-state consistency. A total
volume limit of 60 is not a safe admission limit when the native runtime can
have a materially lower active-cell capacity.

2. Configuration migration still forces a 1000 limit

In OMO 4.18.2, scripts/migrate-codex-config/subagent-limit-guard.mjs contains:

const CODEX_SUBAGENT_THREAD_LIMIT = "1000";

The migration can write or replace the worker-limit settings. The observed
configuration contained:

[agents]
max_threads = 1000

[features.multi_agent_v2]
max_concurrent_threads_per_session = 1000
enabled = false

This relates to the still-open #130:
#130

Even with V2 disabled, the V1 limit was raised to 1000. That removes a useful
user-side safety boundary and amplifies the retry behavior.

Expected behavior

When native Codex rejects a V1 worker operation because active-cell capacity is
exhausted:

  1. Prohibit additional workers, reviewers, gate workers, and respawns for that
    workflow.
  2. Stop repeated TASK STILL ACTIVE follow-ups and no-op/empty waits.
  3. Preserve tracked live-worker IDs; do not mark the parent terminal while a
    tracked worker is live.
  4. Record exactly one durable capacity-blocked state with the original error
    class and an actionable resume condition.
  5. Do not auto-resume a capacity-blocked workflow through stop/resume hooks.
  6. Keep V1 heterogeneous-provider delegation supported.

Suggested fix direction

This appears to need both configuration and orchestration changes:

Acceptance criteria

  • A simulated V1 admission failure prevents future spawn, respawn, and reviewer
    actions for the affected workflow.
  • The workflow records one capacity-blocked state rather than repeated status
    messages.
  • A parent cannot become terminal while a tracked child is active.
  • A lower user-configured concurrency limit survives session-start/config
    migration.
  • V1 routed-provider delegation remains supported.
  • Regression tests cover active-cell/thread-limit rejection, empty-worker-set
    behavior, and a live-child/blocked-parent race.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions