Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,11 @@ cmd/odek/
subagent_tool.go delegate_tasks built-in tool (sub-agent spawning)
subagent_key.go FD-based API key handoff (parent → sub-agent, never via env)
browser_tool.go Built-in browser tool (HTTP fetch + headless navigation)
file_tool.go Built-in file tools (read_file, write_file, search_files, patch, batch_read, glob, file_info)
file_tool.go Built-in file tools (read_file, write_file, search_files, patch, glob, file_info)
external_ref.go --external-ref flag parsing (run + continue) → session.ExternalRef
perf_tools.go Performance/parallelism tools (batch_patch, parallel_shell, http_batch, math_eval, diff,
multi_grep, json_query, tree, checksum, head_tail, base64)
perf_tools.go Native utilities (math_eval, diff, json_query, tree, checksum, head_tail, base64)
http_request_tool.go Single-URL HTTP status/size checks with SSRF and redirect guards
native_tool_context.go Invocation-scoped contexts for independent native tool calls
mcp.go MCP server implementation (stdio transport)
mcp_approval.go Per-tool MCP server approval UI and persistence (key hashes limits/artifact_roots)
project_sandbox_approval.go Project-level sandbox config approval gate
Expand All @@ -62,7 +63,6 @@ cmd/odek/
memory_cmd.go `odek memory` command
cleanup.go `odek cleanup [--dry-run]` one-shot storage sweep + janitor wiring (telegram/serve/schedule daemon)
upgrade.go `odek upgrade [--check]` self-upgrade from GitHub Releases (checksums.txt SHA-256 verified)
parallel.go Parallelism helpers
toolctx.go Tool-call context plumbing
security_report_validation_test.go Regression bar for every documented mitigation
*_test.go 250+ unit + E2E tests covering all tools
Expand Down Expand Up @@ -109,15 +109,15 @@ docs/ Documentation (CLI, API, CONFIG, MCP, EXTENSIONS,
ReAct cycle: observe → think → act → repeat.
- LLM returns tool calls or a final answer.
- **Parallel tool execution** — independent tool calls run concurrently (`max_tool_parallel`, default: 4).
- **Batch approval gate** — multiple risky tools shown in one prompt. `classifyToolCall` classifies every command inside `parallel_shell`, every path inside `batch_patch`, and the `browser` tool; shows full commands; withholds blanket `SetTrustAll` when unclassifiable tools (incl. MCP tools, classified `unknown`) remain.
- **Batch approval gate** — multiple risky tools shown in one prompt. `classifyToolCall` classifies each individual `shell` command, each `patch`/`write_file` target, and the `browser` tool; shows full commands; withholds blanket `SetTrustAll` when unclassifiable tools (incl. MCP tools, classified `unknown`) remain.
- **Tool-failure recovery** — retry transient errors, skip permanent failures, continue without crashing. Stall detection: 3 consecutive identical tool calls inject a corrective hint + fire `tool_recovery` — a hint, never aborts the run.
- **Context-limit protection** — graduated trimming near the context window: old large tool results replaced with markers (4 most recent kept intact), then oldest turn groups dropped atomically (tool messages stay grouped with their parent). The protected head (system prompt, memory block, compaction digest, original task) is never dropped. Token estimator counts tool schemas + reasoning content; safety margin self-tightens when provider-reported tokens exceed estimates (`margin_calibrated` signal). `trimToSurvival` handles provider context-length errors. **Rolling compaction** (on by default; `compaction: false` / `ODEK_COMPACTION=false` / `--no-compaction`) sketches dropped groups extractively into a rolling digest immediately, then a thinking-off side call replaces that sketch with a model digest on a later iteration if it arrives.
- **Interaction modes** — engaging (narrated), enhance (persistent), verbose (raw), off.
- Max 90 iterations by default. On iteration-budget exhaustion the engine makes one final tool-less LLM call for a partial-progress summary (30s bound), returned marked `[Iteration budget reached — partial summary]`.
- **Post-response async processing** — episode extraction and extended-memory extraction run in background goroutines; `Agent.Close` drains them (~15s bound).
- **Per-turn session persistence** — `SetMessagesPersistCallback` fires after each completed step with a fresh snapshot; CLI/REPL/serve/Telegram wire it to `Store.SaveNoIndex` (atomic, skips remote vector indexing). Interrupted runs resume via `odek continue` from the last completed step; error paths persist partial history minus dangling tool calls. The final save per turn still updates the semantic index once.
- **Storage maintenance janitor** — `maintenance.Start` sweeps `~/.odek` (retention, log rotation, media sweep) inside `odek telegram`/`serve`/`schedule daemon`; `odek cleanup [--dry-run]` runs it on demand. Session files are trimmed at write time when they would exceed `MaxSessionFileBytes`. Operator-only config. See docs/MAINTENANCE.md.
- **Artifact-aware file search** — `search_files`/`multi_grep` skip `node_modules`, `vendor`, `.git`, `__pycache__`, `.venv`, etc.
- **Artifact-aware file search** — `search_files` skip `node_modules`, `vendor`, `.git`, `__pycache__`, `.venv`, etc.
- **Semantic session search** — `session_search` tool: go-vector RandomProjections + k-NN, two-tier (vector index → exhaustive fallback).
- **Background commands** — `bg_start`/`bg_list`/`bg_status`/`bg_output`/`bg_stop` tools over a process-scoped, session-keyed process manager (`internal/bgproc`): session-scoped job lifetime, bounded in-memory output rings, spawn-time danger classification parity with `shell`, group-signal stop (sandbox mode via the pidfile follow-up). Config: `background` section (docs/CONFIG.md).

Expand All @@ -129,7 +129,7 @@ ReAct cycle: observe → think → act → repeat.
- **Execution budgets** — `limits` config section + `--max-runtime/--max-tool-calls/--max-input-tokens/--max-output-tokens/--max-cost-usd` on `run`; typed `budget.Error` → CLI exit code 4; session persisted before return. Per-model prices via `limits.model_prices` with flat-pair fallback; cost enforcement only when cap + prices configured. `odek init --global` scaffolds the section (zeros = off). `GET /api/limits` on serve exposes limits + effective prices for cost rendering.

### Tools
All built-in tools with zero subprocess forks: batch_read, batch_patch, parallel_shell, http_batch, math_eval, diff, multi_grep, json_query, tree, checksum, head_tail, base64, transcribe, browser, read_file, write_file, search_files, patch, shell, delegate_tasks, session_search, config_view, list_tools.
Built-in tools include: http_request, math_eval, diff, json_query, tree, checksum, head_tail, base64, transcribe, browser, read_file, write_file, search_files, patch, shell, delegate_tasks, session_search, config_view, list_tools.

### Terminal Rendering (`internal/render/`)
Vertical space compression is baked into the render paths; blank lines removed from Iteration/FinalAnswer/Summary. Raw-mode cursor uses `\r\n` for cross-platform compatibility.
Expand All @@ -148,7 +148,7 @@ Layered prompt-injection / approval-fatigue defenses. The full per-mitigation li
- **Sub-agent caps** — `delegate_tasks` carries trust_level + max_risk enforced via the sub-agent's DangerousConfig; MCP tools withheld from untrusted sub-agents; API keys handed off via unlinked-tempfile FD, never env.
- **MCP hardening** — subprocess env sanitization (secret-pattern stripping), tool-name/description/inputSchema validation + injection scans, per-tool approval for every server (keys hash command/args/env + schema hash + description text + all four limit fields), per-server limits with absolute ceilings, artifact-ref fail-closed validation.
- **Config trust split** — `./odek.json` is untrusted: sensitive sections (provider, providers, llm, base_url, api_key, system, dangerous, memory, telegram, web_search, embedding, sessions, skills.dirs) ignored with warnings; sandbox knobs gated behind explicit operator approval (incl. implicit `Dockerfile.odek` builds, content-hash keyed); project limits may only lower global budgets, project prices rejected outright. Global config/secrets permission-checked; config files size-capped.
- **Serve / network surface** — per-instance CSRF token on `/ws` and all `/api/*`, loopback Host checks, local-origin requirement for mutations, per-session auth tokens + rate limiting, clickjacking headers, WS message-size caps. SSRF dial guard (DNS-rebinding-safe, internal-IP refusal, proxy refusal) on browser/http_batch/web_search.
- **Serve / network surface** — per-instance CSRF token on `/ws` and all `/api/*`, loopback Host checks, local-origin requirement for mutations, per-session auth tokens + rate limiting, clickjacking headers, WS message-size caps. SSRF dial guard (DNS-rebinding-safe, internal-IP refusal, proxy refusal) on browser/http_request/web_search.
- **Budgets, events, refs (v1.24.0)** — budget clamp merge (see above); event stream carries SHA-256 arg hashes + sizes only (never raw args), redact applied, JSONL sink 0600/no-symlink/fsync-per-event, drop-on-full dispatch; external refs validated and never dereferenced.
- **Resource bounds** — pervasive size caps (shell output 1 MiB/stream, perf-tool files 10 MiB, session files 32 MiB, skill files 1 MiB, browser snapshots/history/elements, tree width, search results, write_file content, patch expansion) to keep hostile input from OOMing the process.
- **Telegram** — chat-scoped sessions/plans/media, callback binding to originating user, outbound media allowlist + approval, secret-subtree rejection, singleton flock, 0600 logs.
Expand Down
10 changes: 8 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ odek is not a framework. It's a **runtime** — the smallest possible surface ar
Every session can run in an isolated Docker container: no network, no host mounts beyond the working directory, zero capabilities, destroyed on exit. Sandboxing is **on by default** for `odek run`, `odek repl`, and `odek serve`; opt out with `--no-sandbox` / `ODEK_NO_SANDBOX=1` (unsandboxed runs warn loudly, and `ODEK_REQUIRE_SANDBOX=1` makes them fatal). `--ctx` files are auto-injected into the container at `/workspace/`. Full security model in [docs/SANDBOXING.md](docs/SANDBOXING.md).

### 🛡️ Prompt-Injection-Aware
External content the agent ingests (`browser`, `read_file`, `shell`, `search_files`, `multi_grep`, `transcribe`, `vision`, `web_search`, `session_search`, MCP tools) is wrapped in per-call nonce'd `<untrusted_content>` boundaries so the model can distinguish data from instructions. Redirect hops are re-classified (`browser`/`http_batch`), MCP tool descriptions are scanned for injection at registration, and the MCP error channel is wrapped too. The danger classifier resists common shell-evasion tricks (`$()`/backtick substitution, `$IFS`, brace expansion, `command`/`env` wrappers, `\rm`, basenamed absolute paths, and more). Approvers engage friction mode after 3 same-class approvals in 60 s. Memory episodes from tainted sessions are stored but never auto-replayed. Imported and project skills track provenance — untrusted ones stay excluded from trigger matching until explicit `odek skill promote --force`. `odek audit <session-id>` surfaces every ingest + per-turn divergence heuristic. Full threat model in [docs/SECURITY.md](docs/SECURITY.md).
External content the agent ingests (`browser`, `read_file`, `shell`, `search_files`, `transcribe`, `vision`, `web_search`, `session_search`, MCP tools) is wrapped in per-call nonce'd `<untrusted_content>` boundaries so the model can distinguish data from instructions. Redirect hops are re-classified (`browser`/`http_request`), MCP tool descriptions are scanned for injection at registration, and the MCP error channel is wrapped too. The danger classifier resists common shell-evasion tricks (`$()`/backtick substitution, `$IFS`, brace expansion, `command`/`env` wrappers, `\rm`, basenamed absolute paths, and more). Approvers engage friction mode after 3 same-class approvals in 60 s. Memory episodes from tainted sessions are stored but never auto-replayed. Imported and project skills track provenance — untrusted ones stay excluded from trigger matching until explicit `odek skill promote --force`. `odek audit <session-id>` surfaces every ingest + per-turn divergence heuristic. Full threat model in [docs/SECURITY.md](docs/SECURITY.md).

### 🧩 Sub-Agent Delegation
Parallel OS-process sub-agents via `delegate_tasks`. True isolation — each sub-agent is a fresh `odek subagent` process with its own config, tools, and termination timeout. Up to 8 concurrent workers. Operator-defined **capability profiles** (top-level `profiles` config) override a sub-agent's permissions by name and fail closed on unknown names — a curated starter set of 21 task profiles ships in [`profiles.template.json`](profiles.template.json). See [docs/SUBAGENTS.md](docs/SUBAGENTS.md) and [docs/SECURITY.md](docs/SECURITY.md).
Expand Down Expand Up @@ -92,7 +92,7 @@ Attach files to any prompt with `--ctx` / `-c` (CLI), `@filename` inline referen
Built-in `read_file`, `write_file`, `search_files`, `patch`, `shell`, and `browser` tools. All gated by a unified security layer (`dangerous` config) — classify operations as `allow` / `deny` / `prompt` per risk class. No third-party dependencies. [docs/SECURITY.md](docs/SECURITY.md)

### 🌐 Local Web Search
`web_search` queries a **self-hosted [SearXNG](https://docs.searxng.org/) metasearch instance** — no cloud search API, no keys. Returns ranked results (title, url, snippet) the agent then fetches with `browser` / `http_batch`; results are wrapped as untrusted content and gated as `network_egress`. The Docker Compose setup runs a SearXNG sidecar and enables it out of the box; standalone installs point `web_search.base_url` at any SearXNG instance. [docs/CHEATSHEET.md](docs/CHEATSHEET.md#web-search)
`web_search` queries a **self-hosted [SearXNG](https://docs.searxng.org/) metasearch instance** — no cloud search API, no keys. Returns ranked results (title, url, snippet) the agent then fetches with `browser`; `http_request` checks status and size only; results are wrapped as untrusted content and gated as `network_egress`. The Docker Compose setup runs a SearXNG sidecar and enables it out of the box; standalone installs point `web_search.base_url` at any SearXNG instance. [docs/CHEATSHEET.md](docs/CHEATSHEET.md#web-search)

---

Expand Down Expand Up @@ -181,6 +181,12 @@ odek run "@README.md what does this project do?"

---


Independent native tool calls are scheduled by the loop under `max_tool_parallel`
(default 4). Batch execution tools have been retired; see the
[migration guide](docs/MIGRATION.md#batch-execution-tool-retirement) for replacements
and configuration changes. `delegate_tasks` and background job tools remain.

## Docs

| Doc | Covers |
Expand Down
16 changes: 16 additions & 0 deletions cmd/odek/approval_test_helpers_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
package main

import (
"github.com/BackendStack21/odek/internal/danger"
)

// promptLocalWriteConfig returns a DangerousConfig that treats local file
// writes as Prompt, so simple commands like `touch` can exercise the approval
// path deterministically without requiring sudo or network access.
func promptLocalWriteConfig() danger.DangerousConfig {
return danger.DangerousConfig{
Classes: map[danger.RiskClass]danger.Action{
danger.LocalWrite: danger.Prompt,
},
}
}
131 changes: 0 additions & 131 deletions cmd/odek/batch_patch_audit_test.go

This file was deleted.

Loading
Loading