diff --git a/CLAUDE.md b/CLAUDE.md index 1f7349fd..b4ff4d0b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,7 +43,7 @@ Ralph for Claude Code — an autonomous AI development loop system enabling cont - **log_utils.sh** — `rotate_logs()` rotates `$LOG_DIR/ralph.log` at 10MB, keeping 4 archives (`.log.1`–`.log.4`); GNU `stat -c%s` with BSD `stat -f%z` fallback - **github_lifecycle.sh** — GitHub issue lifecycle management backing `ralph --github-issue` (Issue #73). `parse_issue_reference` (N | #N | owner/repo#N | URL → number+repo); gh wrappers `gh_issue_comment`/`gh_close_issue`/`gh_add_labels`/`gh_create_pr`/`gh_create_issue` (each logs + returns non-zero on failure, never exits); state primitives `init_github_lifecycle`/`lifecycle_get`/`_lifecycle_apply` (atomic temp+`mv` at `.ralph/.github_lifecycle_state`, program-first signature like `_queue_apply`); generators `generate_progress_comment`/`generate_completion_summary`/`scan_for_todos`; orchestration `lifecycle_post_progress ` (interval-gated) and `lifecycle_on_completion` (summary → PR → followups → close, each flag-guarded, always returns 0). Reuses `lib/date_utils.sh` timestamps - **sandbox_docker.sh** — Docker sandbox execution backing `ralph --sandbox docker` (Issue #74). Containerizes only the Claude CLI execution: one persistent container per run (`docker run -d … sleep infinity`, project bind-mounted rw at `/workspace`), each iteration wrapped as `docker exec -i -w /workspace claude …` via `build_sandbox_exec_args`/`wrap_claude_command_for_sandbox` (works in live and background modes); ralph orchestration, analysis, and status.json stay host-side so ralph-monitor is unaffected. `validate_sandbox_config` (image/memory/cpus/network), `init_docker_sandbox` (daemon + image checks with build/pull guidance), `setup_docker_credentials` (ANTHROPIC_API_KEY → 0600 env-file via `--env-file`, else host `~/.claude/.credentials.json` copied into a container-scoped home — never `docker secret`, which needs Swarm), `handle_sandbox_timeout` (exit 124 kills only the docker-exec client → `docker restart` reaps orphans), idempotent `cleanup_docker_sandbox` on all exit paths. State: `.ralph/.docker_sandbox_state` (atomic temp+`mv`). Setup failure is fatal — never falls back to host execution. Config: `SANDBOX_PROVIDER`, `SANDBOX_DOCKER_IMAGE/MEMORY/CPUS/NETWORK` (env > CLI > .ralphrc); sandbox `_env_*` capture sits BEFORE the lib source block in ralph_loop.sh because the lib sets defaults at source time. Default image from GHCR (`docker pull ghcr.io/frankbria/ralph-sandbox:latest`, published on `v*` tags by `.github/workflows/docker-publish.yml` — multi-arch amd64+arm64, smoke-tested before push, dry-run via workflow_dispatch `push=false`; guards in `tests/unit/test_workflow_docker_publish.bats`, Issue #298) or built locally from the repo `Dockerfile` (`docker build -t ralph-sandbox .`; install.sh copies it to `~/.ralph`). `get_sandbox_status` is the provider router for status.json (docker impl renamed `get_docker_sandbox_status`; routes to `get_e2b_sandbox_status` when `SANDBOX_PROVIDER=e2b`) -- **sandbox_e2b.sh** + **e2b_helper.py** — E2B cloud sandbox execution backing `ralph --sandbox e2b` (Issue #75). Same host-side-orchestration model as docker, but the E2B SDK is Python/JS-only, so all transport goes through `lib/e2b_helper.py` (thin CLI over `pip install e2b`: `check`/`create`/`connect`/`info`/`exec`/`upload`/`download`/`ack-download`/`write-file`/`kill`; JSON on stdout, `exec` streams remote output + propagates the remote exit code; secrets only via env/stdin, never argv — tests substitute the interpreter via `SANDBOX_E2B_PYTHON`). No bind mount in the cloud → project uploaded once at start (`git ls-files -coz` + `.ralph` control files, tar over stdin), changed files downloaded after EVERY iteration (sandbox-side `.ralph_sync_marker` mtime; marker advanced via `ack-download` only after host extraction + deletion pass succeed — a missed ack re-delivers the same changes, at-least-once delivery; download runs before git-based progress detection so cloud work counts; `.git` excluded both directions → in-sandbox commits are NOT synced, content arrives as uncommitted host changes). Sandbox-side deletions/renames propagate: each download tar carries a `.ralph_e2b_manifest` member (full workspace file list); host files in `.ralph/.e2b_synced_files` (upload list ∪ synced-down files) that left the manifest are deleted — host-only files and `.git`/`.ralph` paths are never deletion candidates (codex review finding, #75). One sandbox per run, reused across loops; expiry (session `--sandbox-timeout`, default 3600s) detected by the pre-exec liveness probe → fresh create + re-upload. Claude CLI bootstrap: `claude --version` in-sandbox, one `npm install -g @anthropic-ai/claude-code` attempt, else fatal with custom-template guidance. Credentials: `E2B_API_KEY` env or `~/.ralph/e2b_api_key` (0600, never .ralphrc); claude auth via `ANTHROPIC_API_KEY` as sandbox env at create, else host `~/.claude/.credentials.json` seeded over stdin. Cost: estimate = `accrued_cost` (prior sandbox segments) + active runtime × `SANDBOX_E2B_COST_PER_HOUR` (default 0.10); previous segment's cost folded into `accrued_cost` before epoch reset on sandbox recreation so `--sandbox-max-cost` spans the whole run; `--sandbox-cost-alert` warns once, summary in `.ralph/logs/e2b_cost.log` + status.json + monitor Sandbox panel. Exit 124 → remote `pkill -f claude`. `cleanup_e2b_sandbox` (final sync + kill, idempotent) on all exit paths; `--sandbox-keep-alive` skips the kill and logs the id for `--sandbox-id` reuse. State: `.ralph/.e2b_sandbox_state` (atomic temp+`mv`). Setup failure is fatal — never falls back to host execution. Sync filtering (Issue #76, via lib/sync.sh): upload list runs include → exclude → `.ralphignore` → large-file policy with `.ralph` control files force-included past any filter; download extraction selects members explicitly (`-T` list after hard exclusions + user patterns, tar `--exclude` kept as defense in depth) and logs filtered counts; deletion safety — filtered files never enter the synced-files baseline and excluded paths are never deletion candidates (`_apply_e2b_deletions` pipes candidates through the download filter). Every control-dir guard (upload exclusion + allowlist, deletion guard, member classifiers, extract excludes) derives `RALPH_DIR` via `_e2b_control_dir` — a workspace-relative path, trailing slashes stripped, `.`/`..` collapsed lexically — never its basename (nested, trailing-slash or `a/../b` `RALPH_DIR` slipped past the guards, Issue #376); a relative exclude git rejects as outside the repo is retried without it +- **sandbox_e2b.sh** + **e2b_helper.py** — E2B cloud sandbox execution backing `ralph --sandbox e2b` (Issue #75). Same host-side-orchestration model as docker, but the E2B SDK is Python/JS-only, so all transport goes through `lib/e2b_helper.py` (thin CLI over `pip install e2b`: `check`/`create`/`connect`/`info`/`exec`/`upload`/`download`/`ack-download`/`write-file`/`kill`; JSON on stdout, `exec` streams remote output + propagates the remote exit code; secrets only via env/stdin, never argv — tests substitute the interpreter via `SANDBOX_E2B_PYTHON`). No bind mount in the cloud → project uploaded once at start (`git ls-files -coz` + `.ralph` control files, tar over stdin), changed files downloaded after EVERY iteration (sandbox-side `.ralph_sync_marker` mtime; marker advanced via `ack-download` only after host extraction + deletion pass succeed — a missed ack re-delivers the same changes, at-least-once delivery; download runs before git-based progress detection so cloud work counts; `.git` excluded both directions → in-sandbox commits are NOT synced, content arrives as uncommitted host changes). Sandbox-side deletions/renames propagate: each download tar carries a `.ralph_e2b_manifest` member (full workspace file list); host files in `.ralph/.e2b_synced_files` (upload list ∪ synced-down files) that left the manifest are deleted — host-only files and `.git`/`.ralph` paths are never deletion candidates (codex review finding, #75). One sandbox per run, reused across loops; expiry (session `--sandbox-timeout`, default 3600s) detected by the pre-exec liveness probe → fresh create + re-upload. Claude CLI bootstrap: `claude --version` in-sandbox, one `npm install -g @anthropic-ai/claude-code` attempt, else fatal with custom-template guidance. Credentials: `E2B_API_KEY` env or `~/.ralph/e2b_api_key` (0600, never .ralphrc); claude auth via `ANTHROPIC_API_KEY` as sandbox env at create, else host `~/.claude/.credentials.json` seeded over stdin. Cost: estimate = `accrued_cost` (prior sandbox segments) + active runtime × `SANDBOX_E2B_COST_PER_HOUR` (default 0.10); previous segment's cost folded into `accrued_cost` before epoch reset on sandbox recreation so `--sandbox-max-cost` spans the whole run; `--sandbox-cost-alert` warns once, summary in `.ralph/logs/e2b_cost.log` + status.json + monitor Sandbox panel. Exit 124 → remote `pkill -f claude`. `cleanup_e2b_sandbox` (final sync + kill, idempotent) on all exit paths; `--sandbox-keep-alive` skips the kill and logs the id for `--sandbox-id` reuse. State: `.ralph/.e2b_sandbox_state` (atomic temp+`mv`). Setup failure is fatal — never falls back to host execution. Sync filtering (Issue #76, via lib/sync.sh): upload list runs include → exclude → `.ralphignore` → large-file policy with `.ralph` control files force-included past any filter; download extraction selects members explicitly (`-T` list after hard exclusions + user patterns, tar `--exclude` kept as defense in depth) and logs filtered counts; deletion safety — filtered files never enter the synced-files baseline and excluded paths are never deletion candidates (`_apply_e2b_deletions` pipes candidates through the download filter). Every control-dir guard (upload exclusion + allowlist, deletion guard, member classifiers, extract excludes) derives `RALPH_DIR` via `_e2b_control_dir` — a workspace-relative path, trailing slashes stripped, `.`/`..` collapsed lexically — never its basename (nested, trailing-slash or `a/../b` `RALPH_DIR` slipped past the guards, Issue #376); a relative exclude git rejects as outside the repo is retried without it; `validate_e2b_sandbox_config` rejects a `RALPH_DIR` at the workspace root (`.`, `a/..`, `$PWD`) or outside it (absolute, `../x`) before any sandbox is created — the guards can't separate control state from project files there (Issue #383) - **sync.sh** — backend-agnostic sandbox sync filtering (Issue #76), consumed by sandbox_e2b.sh (Docker needs none — its rw bind mount is real-time by architecture; `--sync-*` flags are rejected with docker). `.ralphignore` parsing (gitignore-like SUBSET: bare name = basename/segment at any depth, `*.ext` basename glob, `dir/` tree, `src/**` full-path glob with `*` crossing `/`; comments/blank dropped, `!negation` unsupported and dropped), `_sync_path_matches_pattern`/`_sync_path_matches_list` matchers, `sync_filter_file_list` (upload, NUL-separated: include → exclude → `.ralphignore` → size policy), `sync_filter_download_list` (download, newline-separated: exclude + `.ralphignore` only — include deliberately not applied so new artifacts come back; size policy upload-only), `validate_sync_config`, `_sync_file_size` (GNU/BSD stat), `format_sync_size`, `log_sync_summary`. Config: `SYNC_INCLUDE`/`SYNC_EXCLUDE` (comma-separated; CLI `--sync-include`/`--sync-exclude`), `SYNC_MAX_FILE_SIZE` (default 10485760, 0=unlimited), `SYNC_LARGE_FILE_ACTION` (warn|skip). `_env_*` capture sits BEFORE the lib source block in ralph_loop.sh (lib sets defaults at source time). Git-based sync + E2B realtime sync deliberately descoped (conflicts with the `.git`-exclusion safety model) — see docs/SANDBOX_SYNC.md - **queue_manager.sh** — queue state primitives backing `ralph-queue` (Issue #72). State at `.ralph/queue.json` (`{version, created_at, updated_at, repository, queue:[…]}`); entries carry `id`/`source` (`github`|`prd`)/`issue_number`/`path`/`title`/`priority`/`labels`/`milestone`/`dependencies`/`status`/timestamps. Functions: `init_queue`, `add_to_queue` (dedupe by id, fills defaults; rc 0/1/2), `remove_from_queue`, `clear_queue`, `mark_issue_status` (validates status, stamps started/completed), `get_queue_status` (counts JSON), `sort_queue_by_priority` (rank then FIFO), `get_priority_from_labels` (reuses the P0–P9 / `priority: PN` parser), `parse_issue_dependencies` (`depends on/blocked by/requires #N`), `is_dependency_satisfied`, `get_next_issue` (ready+priority+FIFO), `validate_dependencies` (jq cycle detection). All mutations are atomic temp-file+`mv` via `_queue_apply` diff --git a/docs/E2B_SANDBOX.md b/docs/E2B_SANDBOX.md index a8861a68..601f4bbb 100644 --- a/docs/E2B_SANDBOX.md +++ b/docs/E2B_SANDBOX.md @@ -188,4 +188,5 @@ dashboard for actual usage. | `Claude Code CLI is unavailable in the E2B sandbox` | Build a custom E2B template with `@anthropic-ai/claude-code` preinstalled and pass `--sandbox-template` | | Claude auth errors inside the sandbox | Export `ANTHROPIC_API_KEY`, or log in on the host first so `~/.claude/.credentials.json` exists | | Loop stops with `e2b_cost_limit` | Expected — raise `--sandbox-max-cost` or fix `SANDBOX_E2B_COST_PER_HOUR` if your template rate differs | +| `RALPH_DIR ... is the workspace root` / `... is outside the workspace` | E2B needs `RALPH_DIR` in a subdirectory of the project (default `.ralph`); unset it or point it at e.g. `.ralph` | | Orphaned sandbox after a hard kill (`kill -9`) | It expires at `--sandbox-timeout`; kill it sooner from the E2B dashboard | diff --git a/lib/sandbox_e2b.sh b/lib/sandbox_e2b.sh index 07179ac8..936f20ff 100644 --- a/lib/sandbox_e2b.sh +++ b/lib/sandbox_e2b.sh @@ -179,6 +179,23 @@ validate_e2b_sandbox_config() { echo "Error: invalid SANDBOX_E2B_COST_PER_HOUR '$SANDBOX_E2B_COST_PER_HOUR'" >&2 return 1 fi + # The control-dir guards (upload exclude + allowlist, download and + # deletion guards) need RALPH_DIR strictly inside the workspace: at the + # root they can't tell control state from project files, outside it the + # control files get wrong tar member paths (Issue #383). + # _e2b_control_dir strips the workspace prefix only from a path below it; + # an absolute RALPH_DIR equal to the workspace comes back whole, hence the + # two explicit comparisons + local rb + rb=$(_e2b_control_dir) + if [[ -z "$rb" || "$rb" == "$(_e2b_collapse_path "$PWD")" || "$rb" == "$(_e2b_collapse_path "$(pwd -P)")" ]]; then + echo "Error: RALPH_DIR '$RALPH_DIR' is the workspace root — --sandbox e2b needs it in a subdirectory (e.g. .ralph)" >&2 + return 1 + fi + if [[ "$rb" == /* || "$rb" == ".." || "$rb" == ../* ]]; then + echo "Error: RALPH_DIR '$RALPH_DIR' is outside the workspace — --sandbox e2b needs it inside the project (e.g. .ralph)" >&2 + return 1 + fi return 0 } diff --git a/tests/unit/test_sandbox_e2b.bats b/tests/unit/test_sandbox_e2b.bats index b63b6123..9fc5f5c4 100644 --- a/tests/unit/test_sandbox_e2b.bats +++ b/tests/unit/test_sandbox_e2b.bats @@ -191,6 +191,44 @@ _started_sandbox() { [[ "$output" == *"provider"* ]] } +@test "validate_e2b_sandbox_config: rejects RALPH_DIR at the workspace root (#383)" { + local d + for d in . ./ a/.. "$TEST_DIR" "$TEST_DIR/" "$(pwd -P)"; do + export RALPH_DIR="$d" + run validate_e2b_sandbox_config + assert_failure + [[ "$output" == *"workspace root"* ]] || fail "no root message for RALPH_DIR=$d: $output" + done +} + +@test "validate_e2b_sandbox_config: rejects RALPH_DIR outside the workspace (#383)" { + local d + for d in /abs/out ../out sub/../../out ..; do + export RALPH_DIR="$d" + run validate_e2b_sandbox_config + assert_failure + [[ "$output" == *"outside the workspace"* ]] || fail "no outside message for RALPH_DIR=$d: $output" + done +} + +@test "validate_e2b_sandbox_config: accepts in-workspace RALPH_DIR spellings (#383)" { + local d + for d in .ralph state/ralph "$TEST_DIR/state/ralph/"; do + export RALPH_DIR="$d" + run validate_e2b_sandbox_config + [[ "$status" -eq 0 ]] || fail "rejected valid RALPH_DIR=$d: $output" + done +} + +@test "init_e2b_sandbox: workspace-root RALPH_DIR fails before any helper call (#383)" { + _mock_e2b ok + export RALPH_DIR="." + run init_e2b_sandbox + assert_failure + [[ "$output" == *"workspace root"* ]] || fail "expected workspace root message: $output" + [[ ! -f "$TEST_DIR/e2b_args" ]] || fail "helper was called: $(cat "$TEST_DIR/e2b_args")" +} + @test "validate_e2b_sandbox_config: rejects template with shell metacharacters" { export SANDBOX_E2B_TEMPLATE='evil;rm -rf /' run validate_e2b_sandbox_config