Joysafeter v2 - #141
Open
yuzzjj wants to merge 902 commits into
Open
Conversation
GLei16
approved these changes
Jul 2, 2026
GLei16
previously approved these changes
Jul 2, 2026
asyncpg defaults to sslmode=prefer, probing /root/.postgresql/postgresql.key. Running as non-root admin user, that path is unreadable and Path.exists() raises PermissionError, crashing DB connection. Set connect_args ssl explicitly from POSTGRES_SSL env: - default (unset/disable/false) → ssl=False (internal RDS) - require/true/verify-* → ssl=True Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Failed storage reads were silently swallowed at debug level, invisible in production. Log at warn so missing/misconfigured S3 files show up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
archive_extract_dir previously created a same-named subdir (foo.zip → foo/), which double-nested when the zip already contained a top-level foo/ dir, producing /workspace/foo/foo/... Now extract into the archive's parent dir so the zip's own structure decides the layout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Previously hardcoded at 20 max connections. Now reads: - DATABASE_POOL_SIZE (default 20) - DATABASE_MIN_CONNECTIONS (default 2) The DB pool is the primary bottleneck for active sandbox capacity. Setting DATABASE_POOL_SIZE=50-100 can increase active sandbox capacity from ~100-200 to ~500-1000. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Previously hardcoded constants, now configurable: - JOYSAFETER_GRPC_MAX_CONNECTIONS (default 2000): max concurrent runner connections - JOYSAFETER_GRPC_MAX_EXECUTIONS (default 1000): max concurrent task executions - JOYSAFETER_MAX_MEMORIES_PER_STORE (default 2000): max memories per store - JOYSAFETER_SCHEDULER_BATCH_SIZE (default 10): tasks claimed per scheduler poll Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Send X-Org-Id alongside X-Project-Id on managed requests so multi-org users resolve to the correct org (backend prefers X-Org-Id, else first- found). Read the CSRF token cookie-first so a cross-tab refresh can't leave a stale in-memory token (avoids 403s). Extract session-events, session-refresh, auth-lifecycle and query-client-lifecycle modules; clear project context + non-session cache on logout/signin to stop cross-user data bleed on a shared tab. Guard project-provider and switch-context against out-of-order responses. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…ounts Replace startsWith(route) path checks in isPublicRoute, the proxy callback whitelist, and the verify redirect whitelist with exact-or- segment-boundary matching (strip query/hash first), so e.g. /verify no longer matches /verify-evil and the callback whitelist no longer accepts /managedEVIL. Add run-sequence + isMounted guards and timer cleanup to the signin, signup, reset-password and verify flows so a resolved request or delayed redirect can't fire after unmount. Defer form.handleSubmit(onSubmit) to submit time so the ref-reading handler isn't passed during render (react-hooks/refs). Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Key the session SSE state by session+org+project scope and make org/project reactive so switching context reconnects the stream instead of leaking the previous scope's events; flush the trailing buffer on stream end so the last event isn't dropped. Guard every BaseWsClient socket callback by connect-attempt id and current socket, make an in-flight connect cancellable on disconnect, and clear authExpired on a successful reconnect. Add an autoReconnect option and handler cleanup to the notification hook. Extract managed session-event merge/sort/collapse helpers. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Key the paginated-list cursor and query cache by org+project scope and only reuse previous-page placeholder data within the same scope, so switching context no longer shows another project's list; fix the goToPage backward-navigation off-by-one that loaded page 1's data on an earlier page. Add scope + lifecycle-run guards throughout the quickstart chat and skill-authoring hooks (abort in-flight streams and reset state on context switch, per-scope draft storage, guarded stream finally) so a stale stream can't write into a new scope. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Key list queries, mutations and cache invalidations across the managed pages (agents, api-keys, environments, files, members, memory-stores, projects, quickstart, secrets, sessions, settings, skills, vaults) and shared components by org+project scope, with run guards on async mutations, so switching context refetches cleanly and stale responses can't land on the wrong scope. Fix the session-detail pagination to append with after_seq != null (not truthiness) so a max-seq of 0 doesn't replace loaded pages. Adds the accompanying test suites. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…e abstraction Add new trait methods with default no-op implementations: - on_startup/on_shutdown: provider lifecycle hooks - setup_networking/teardown_networking: egress management - orchestrator_url: runner callback address - capabilities: ProviderCapabilities declaration - supported_injection_strategies: file injection strategy selection Add ProviderCapabilities struct and NetworkIsolation enum. All existing providers compile unchanged (default impls). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DockerProvider now owns all Docker-specific subsystems: - envoy_manager: Envoy sidecar lifecycle (init, add/remove sandbox, recover) - xds_service: Delta xDS server for gRPC mode - Implements new trait methods: on_startup (Envoy init + DB recovery), setup_networking, teardown_networking, orchestrator_url, capabilities, supported_injection_strategies - Exposes envoy_manager()/xds_service() getters for transition period Framework layer (main.rs, resolver) still has its own Envoy code — that will be removed in phase 3. This phase ensures DockerProvider is ready to take over before anything is removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ayer main.rs: - Remove bollard::Docker import and direct Envoy/ImageBuilder initialization - Provider startup delegated to provider.on_startup(pool) - xDS service extracted from DockerProvider before trait object wrapping sandbox_resolver.rs: - Remove envoy_manager field, use provider.setup_networking/teardown_networking - orchestrator_url from provider.orchestrator_url() instead of hardcoded host.docker.internal scheduler.rs: - Remove envoy_manager parameter, SandboxResolver no longer needs it sandbox_controller.rs: - Remove envoy_manager field and parameter - Pool skip check uses provider.capabilities().has_egress_management - teardown_networking delegates to provider file_injection.rs: - Add select_strategies_from_capabilities() using ProviderCapabilities Framework layer is now fully provider-agnostic. No bollard, EnvoyManager, or host.docker.internal references remain outside sandbox/docker.rs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Daytona:
- inject_files: POST /sandbox/{id}/files/upload/{path} (raw body)
- orchestrator_url: reads JOYSAFETER_GRPC_PUBLIC_URL (required for remote)
- capabilities: { has_host_mount: false, network_isolation: Platform }
E2B:
- inject_files: POST /sandboxes/{id}/files (JSON with base64 content)
- orchestrator_url: same as Daytona
- capabilities: same as Daytona
Both providers are now first-class citizens of the SandboxProvider trait
with real file injection, correct orchestrator URL resolution, and
proper capability declarations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- ImageBuilderBackend trait with build_environment_image() method - DockerImageBuilder: existing bollard-based implementation (renamed) - NoopImageBuilder: for cloud providers using pre-built images (E2B/Daytona) - EnvironmentPackages and sanitize_packages remain shared utilities Adding a new image build strategy (kaniko for K8s, Buildkit-as-service) only requires implementing the trait — no framework changes needed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
H1 — Runner token bypass: Remove legacy 'allow without token' backdoor.
Runners MUST present a valid token when the sandbox has one configured.
Empty token → Shutdown with 'authentication required' message.
H2 — HITL heartbeat false-positive: During HITL pause (requires_action_pending),
runner may legitimately stop heartbeating. Heartbeat timeout now resets
the deadline and continues instead of killing the task. Prevents task
failure for HITL interactions lasting >120 seconds.
H3 — Reconnect race condition: When a runner reconnects, the old connection's
multi_task_loop could continue running and claim tasks concurrently.
Added 'displaced' AtomicBool flag to SandboxBridge — set by register()
on reconnect, checked by multi_task_loop (exits) and run_single_task
(aborts mid-task). Combined with existing try_lock cancel for belt+suspenders.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
M1: increment_retry now uses CAS (optional expected_retry_count param)
to prevent scheduler+watchdog double-increment. Removed redundant
transition_task call after increment_retry in scheduler.
M2: Drop conn_permit before 120s grace period cleanup, freeing the
connection semaphore slot immediately on disconnect.
M3: notify_peers clones peer list and drops Mutex before sending gRPC
messages, preventing lock contention when channels back-pressure.
M4: transition_sandbox (non-CAS) now guards against overwriting
stopping/destroyed states in SQL WHERE clause. Added deprecation
warning for tracing.
M5: stopping sandbox now gets destroyed+cleaned up in resolver instead
of being silently skipped (which caused session unique constraint
blocks for up to 60s).
M6: Non-graceful idle sweep re-verifies disconnected_at from DB before
killing sandbox (TOCTOU guard against reconnected sandboxes).
M7: Envoy add_sandbox cleans up orphaned listeners/clusters on socket
wait timeout via remove_sandbox before returning error.
M8: S3Backend caches S3Client with tokio OnceCell (was creating new
AWS config + client on every get/exists call).
M10: Daytona and E2B inject_files now reject paths containing '..'
or null bytes (path traversal protection).
M9 (memory_sync silent errors) already covered by earlier warn! log
upgrade — no additional code change needed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add cron-style recurring task scheduling that fires agent runs through the
same path the HTTP API uses, so the Rust task engine's lease, owner_epoch
fencing, watchdog reclaim, idempotency and retry are all inherited for free.
- joysafeter_schedules table + schedule_id on joysafeter_tasks (run history
reuses the task table; no separate schedule_runs)
- SKIP LOCKED multi-worker claim; "catch up once and advance" misfire policy;
fresh session per fire; per-schedule concurrency_policy (allow/forbid/replace)
- Exactly-once firing across replicas via task idempotency_key = sched:{id}:{slot}
- Shared TaskSubmissionService: the API endpoint and the scheduler now share
one submission/enqueue definition (canonical enqueue moved to the shared
orchestrator_bridge layer) so they cannot drift from the queue contract
- SchedulerLoop runs inside the worker service, gated by settings flags
- REST API: CRUD + enable/disable/trigger/runs under /schedules
- croniter dependency; timezone/DST handling via stdlib zoneinfo
Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…edit 1. Agent edit: save button disabled when name is empty (was only checking formReadOnly + isPending). 2. Environment edit: validate egress service required fields (name, baseUrl, credentialRef, secretKey for cookie) before save. Pass errors prop to EgressServicesEditor so inline error messages display correctly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… hosts
- Permission mode (Agent create + edit): explains the bypass mode behavior
('控制 Agent 使用工具时是否需要人工确认')
- Network type (Environment create): explains limited mode restrictions
('沙箱默认无法访问外网，只有白名单和第三方服务地址可访问')
- Allowed hosts (Environment create): clarifies behavior and that
egress services are auto-allowed
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ub-fields - Env vars: '注入到沙箱的环境变量。格式：KEY=value，逗号或换行分隔' - Secret refs: '引用密钥库中的密钥名称，沙箱启动时自动解密注入为环境变量' - Access mode: explains read_only vs read_write + permission ceiling - Sub-path: explains relative path within volume + prefix constraint - Mount path: explains it must be /workspace/ + this is Agent's file path Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ed skills The skill-delete confirm dialog is a Radix AlertDialogAction, which fires onOpenChange(false) -> onCancel on the same click as onConfirm. onCancel (closeDeleteSkillDialog) bumped the shared mutationRunRef, so the just-launched delete's onSuccess guard saw itself as stale and skipped invalidateQueries: the delete landed server-side but the row lingered until a manual refresh. Stop bumping the run counter on close. Also unify the delete semantics: archived skills are now deletable (archiving retires a skill; purging a retired skill is a valid follow-up). Drop the _ensure_skill_mutable guard from delete_skill and the isSkillMutable gating on the frontend delete affordances, and reword the SKILL_DELETE_HAS_REFERENCES message (which no longer contradicts itself by suggesting "archive it"). Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
… guardrails skill.security_status was the only status field on the platform without an enum: its 6 values (not_scanned/scanning/passed/warning/failed/blocked) were scattered as string literals across the scan service, the runtime gate, the publish gate, and the worker recovery job — free to drift. - add JoySafeterSkillSecurityStatus enum (mirrors JoySafeterSkillLifecycleStatus) - converge every producer/consumer literal onto it: model default, the two policy frozensets (_RUNTIME_ALLOWED_SECURITY_STATUSES / _AUTO_DEMOTE_SCAN_STATUSES), the write-admission policy decisions, _is_blocked, mark_scanning, the stuck-scan resets (service + worker startup), and the publish blocked-gate - behavior-preserving: str-enum .value equals the previous wire strings exactly Also add tests/test_state_status_single_source.py — defensive guardrails that turn silent drift into failing checks: - the security_status policy sets must derive from the enum - the task terminal-status set and its active-task complement in the Python JoySafeterTaskStatus enum must match the SQL literals re-typed in the Rust orchestrator (task.rs / session.rs), and the sandbox status vocabulary must appear on both the Python and Rust sides Verified: new guardrails + skill security/version suites 22 passed; ruff clean; grep confirms zero remaining security_status string literals. (Pre-existing failures in test_skill_version_security_gate.py — fake repo lacks get_highest_version_str — confirmed on clean HEAD, unrelated.) Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…rusted Gate X-Forwarded-For / X-Real-IP behind a new trust_forwarded_headers setting (default off) so clients cannot spoof their source IP to bypass rate limiting; fall back to the direct socket peer otherwise. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Split the JoySafeterTriggerService god-object into five single-purpose services (config policy, runtime gate, scheduler state, fire, webhook auth), moving business-invariant validation out of the Pydantic schema into a domain policy so wire types stay permissive. Harden the webhook route (fail-closed empty auth_methods, ignore X-Request-ID for the body-hash fallback, shell-quote sample curl) and bound oversized external idempotency components / keyed session keys. Convert trigger name uniqueness to a DB constraint with a friendly conflict, and add a completed lifecycle state for parked one-off triggers in the UI. Remove the now-unused CronTriggerConfig/WebhookTriggerConfig models, the duplicate scheduler-state config builder, and dead WebhookAuthService wrappers; derive the concurrency-policy set from its enum. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Some long-lived databases were stamped past 20260728_000001 before the storage migrations were inserted into the chain, so Alembic reports them at head while the storage tables are absent. This idempotent, table-guarded migration reconciles the schema without disturbing databases that already created the tables through the normal path. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…harnesses Make the dedicated test database URL take precedence over DATABASE_URL so `cargo test` targets the ephemeral test database rather than whatever DATABASE_URL happens to point at, matching the Python conftest precedence. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…r trust Add a DistributedRateLimiter that counts requests in Redis (atomic INCR + first-hit EXPIRE) with an in-memory fallback when Redis is unavailable, so limits hold across API replicas instead of per-process. Trust forwarded client-IP headers only when the direct peer falls within the configured TRUSTED_PROXY_CIDRS, closing the header-spoofing gap when TRUST_FORWARDED_HEADERS is enabled. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…ility The next_action shape no longer carries a `kind` discriminator; update the module doc comment to match the current location-neutral hint contract. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
… task-cancel hardening Replace hard deletion with a deleted_at soft-delete so run history survives and trigger names can be reused after removal. Partial unique indexes scope name uniqueness to live rows (per-project and global), and every read/scheduler path now filters deleted_at IS NULL. Deletion takes a row lock, refuses while the scheduler holds a fresh claim (TRIGGER_FIRE_IN_PROGRESS) or active runs exist (TRIGGER_HAS_ACTIVE_RUNS), and webhook/manual fire + agent submission re-lock the row so a concurrently deleted trigger cannot dispatch. Persisted last_payload snapshots are now bounded and redacted via a payload sanitizer. Task cancellation only relays to a sandbox the task actually owns and finalizes DB-only cancels with an owner-epoch guard, so cancels can't drift from runtime state. Scheduler advance/release calls carry expected_locked_by to avoid clobbering another worker's claim. The accompanying migration soft-deletes pre-existing duplicate global trigger names before adding the unique index. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…ger UI Wire the create/detail/list trigger surfaces and the create dialog to the soft-delete and active-run-conflict contract, with en/zh copy for the new states and helpers in the managed triggers client. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…mpt_template Mark the trigger system_prompt request field and service parameter as deprecated (kept for API compatibility); trigger-specific instructions belong in prompt_template while the agent's own system_prompt stays the base behavior. Documentation-only: no validation or runtime change. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…paths Consolidate the triplicated "SELECT ... FOR UPDATE on a live trigger + raise TRIGGER_NOT_FOUND" logic behind TriggerRuntimeGate.lock_stmt/trigger_not_found_error, and collapse the two byte-identical owner-match cancel branches in TaskCancellationService into a shared _finalize_owner_matched_cancel plus a single state-sync error factory. Rename scheduler-state _owns_claim to _owns_claim_or_release to surface its lock-releasing commit, fix a dict/list variable rebind in the payload sanitizer, and document why the trigger is re-locked after session creation. Pure refactor; behavior unchanged. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
create_secret only purged soft-deleted rows before insert, so an active (project_id, name) collision hit the partial unique index and surfaced as an unhandled IntegrityError → HTTP 500. Wrap the commit and convert the name-index violation into ResourceConflictError(SECRET_NAME_EXISTS) (409), re-raising other integrity errors unchanged. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.