Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/shadow-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ on:
inputs:
shadow_ref:
type: string
default: v0
default: v1.8
description: "git ref of sudsali/shadow to clone (tag, branch, or sha)"
pr_number:
type: string
Expand Down Expand Up @@ -155,7 +155,7 @@ jobs:
EVENT_AFTER: ${{ github.event.pull_request.head.sha || github.event.after }}
KB_S3_BUCKET: ${{ secrets.KB_S3_BUCKET }}
KB_S3_KEY: ${{ secrets.KB_S3_KEY }}
BEDROCK_MODEL_ID: ${{ secrets.BEDROCK_MODEL_ID || 'us.anthropic.claude-opus-4-7' }}
BEDROCK_MODEL_ID: ${{ secrets.BEDROCK_MODEL_ID || 'us.anthropic.claude-opus-4-8' }}
BEDROCK_REPORTER_MODEL_ID: ${{ secrets.BEDROCK_REPORTER_MODEL_ID || 'us.anthropic.claude-haiku-4-5-20251001-v1:0' }}
BEDROCK_CRITIC_MODEL_ID: ${{ secrets.BEDROCK_CRITIC_MODEL_ID }}
BEDROCK_ISSUE_MODEL_ID: ${{ secrets.BEDROCK_ISSUE_MODEL_ID }}
Expand Down
39 changes: 20 additions & 19 deletions README.md

Large diffs are not rendered by default.

10 changes: 6 additions & 4 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The bot's privileged operations are:
inputs include the diff, codebase tree, and existing PR/issue comments.
2. **Posting on GitHub as the workflow's bot identity** — inline review
comments, labels, optional Slack pings.
3. **Calling Bedrock from your AWS account** — Anthropic Opus 4.7 + Haiku 4.5
3. **Calling Bedrock from your AWS account** — Anthropic Opus 4.8 + Haiku 4.5
inferences, scoped to your IAM role's permissions.

### Adversaries
Expand All @@ -25,9 +25,11 @@ The bot's privileged operations are:
workflow file, or Shadow source.
- **Existing-repo collaborator.** Same as above plus can author issue/comment
content under any GitHub login the bot is configured to recognize.
- **Compromised upstream `sudsali/shadow`.** A malicious commit to the moving
`v0` tag would propagate to every adopter pinning `@v0` on their next workflow
run. Adopters who SHA-pin (`@<40-char SHA>`) are insulated.
- **Compromised upstream `sudsali/shadow`.** A `v1.x` release tag is immutable
by convention, but a tag *can* be force-repointed at the source repo; if that
happened, adopters pinning that tag (`@v1.8`) would pick up the malicious
revision on their next workflow run. Adopters who SHA-pin (`@<40-char SHA>`)
are insulated — a SHA cannot be repointed.
- **Compromised Anthropic / AWS Bedrock.** Out of scope; same trust boundary
as the adopter's other Bedrock usage.

Expand Down
12 changes: 6 additions & 6 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ branding:
# and forwards to the reusable workflow; all real work happens there.
#
# Adopters can use either shape:
# uses: sudsali/shadow@v0
# uses: sudsali/shadow@v1.8
# with:
# aws_role_arn: ${{ secrets.AWS_ROLE_ARN }}
# or directly:
# uses: sudsali/shadow/.github/workflows/shadow-review.yml@v0
# uses: sudsali/shadow/.github/workflows/shadow-review.yml@v1.8
# secrets:
# AWS_ROLE_ARN: ${{ secrets.AWS_ROLE_ARN }}
#
Expand Down Expand Up @@ -60,9 +60,9 @@ inputs:
required: false
default: "false"
shadow_ref:
description: Pin to a Shadow release (`v0` or a 40-char SHA).
description: Pin to a Shadow release (a `v1.x` tag or a 40-char SHA).
required: false
default: "v0"
default: "v1.8"

outputs:
shadow_result:
Expand All @@ -79,13 +79,13 @@ outputs:
#
# jobs:
# shadow:
# uses: sudsali/shadow/.github/workflows/shadow-review.yml@v0
# uses: sudsali/shadow/.github/workflows/shadow-review.yml@v1.8
#
# When GitHub Actions adds composite-to-reusable-workflow chaining
# (https://github.com/orgs/community/discussions/56086), this file flips
# from stub to real wrapper. Until then, this exists so:
# 1. Marketplace search finds Shadow
# 2. `uses: sudsali/shadow@v0` doesn't 404
# 2. `uses: sudsali/shadow@v1.8` doesn't 404
# 3. The branding badge renders on PRs
runs:
using: "composite"
Expand Down
10 changes: 6 additions & 4 deletions examples/caller-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ on:
type: boolean
default: true
shadow_ref:
default: v0
description: "Shadow ref to pin to: v0 (moving tag), or a 40-char commit SHA"
default: v1.8
description: "Shadow ref to pin to: a v1.x release tag, or a 40-char commit SHA"

permissions:
contents: read
Expand All @@ -43,11 +43,13 @@ permissions:
# disable both jobs without deleting this workflow file.
jobs:
shadow:
uses: sudsali/shadow/.github/workflows/shadow-review.yml@v0
uses: sudsali/shadow/.github/workflows/shadow-review.yml@v1.8
with:
pr_number: ${{ inputs.pr_number }}
dry_run: ${{ inputs.dry_run && 'true' || 'false' }}
shadow_ref: ${{ inputs.shadow_ref || 'v0' }}
# Keep this in sync with the `uses:` pin above — GitHub forbids
# expressions in `uses:`, so the two can't share a variable.
shadow_ref: ${{ inputs.shadow_ref || 'v1.8' }}
# Cost levers — uncomment to override the defaults (10/8). Lower
# numbers cap Bedrock spend at the cost of review depth.
# investigator_max_tool_calls: "5"
Expand Down
4 changes: 2 additions & 2 deletions examples/shadow.example.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ bot:
# Env vars (BEDROCK_MODEL_ID, BEDROCK_REPORTER_MODEL_ID, BEDROCK_CRITIC_MODEL_ID)
# take precedence over yaml.
models:
investigator: us.anthropic.claude-opus-4-7
critic: us.anthropic.claude-opus-4-7
investigator: us.anthropic.claude-opus-4-8
critic: us.anthropic.claude-opus-4-8
reporter: us.anthropic.claude-haiku-4-5-20251001-v1:0 # PR-review JSON formatting
# issue: <model> # issue/followup answers; defaults to `reporter`. Must
# # support structured output (Haiku 4.5 or Sonnet 4.6).
4 changes: 2 additions & 2 deletions examples/shadow.java.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ bot:
escalate_label: needs-human

models:
investigator: us.anthropic.claude-opus-4-7
critic: us.anthropic.claude-opus-4-7
investigator: us.anthropic.claude-opus-4-8
critic: us.anthropic.claude-opus-4-8
reporter: us.anthropic.claude-haiku-4-5-20251001-v1:0
10 changes: 6 additions & 4 deletions infrastructure/shadow-iam-stack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Metadata:
GitHubOrg: { default: "Your GitHub org or user (e.g., awslabs, sudsali)" }
GitHubRepo: { default: "Your repo name (use * for org-wide; audit first)" }
ShadowSourceRepo: { default: "Shadow source repo (sudsali/shadow upstream, or your fork)" }
ShadowWorkflowRef: { default: "Shadow workflow ref pin (use * for v0; SHA for prod)" }
ShadowWorkflowRef: { default: "Shadow workflow ref pin (use * for quick start; tag or SHA for prod)" }
BedrockRegion: { default: "Bedrock region (us-east-1, us-west-2, us-east-2)" }
MonthlyBudgetLimit: { default: "Monthly Bedrock spend limit in USD (0 = disabled)" }
BudgetEmailAddress: { default: "Email for budget threshold notifications" }
Expand Down Expand Up @@ -65,9 +65,11 @@ Parameters:
Description: >-
`job_workflow_ref` pin for Shadow's reusable workflow. Use `*` for
a quick start (any version of the workflow can assume this role).
For production, pin to a 40-char SHA you've audited:
`refs/tags/v0` or `<commit-sha>`. The trailing wildcard lets the
same role work across Shadow patch releases until you re-pin.
For production, pin to a specific release tag or an audited 40-char
SHA: `refs/tags/v1.8` or `<commit-sha>`. Avoid a family wildcard like
`refs/tags/v1.*` for the trust pin — it auto-trusts every FUTURE v1.x
tag, including one an attacker who compromised upstream could push, so
it defeats the point of pinning. A SHA is strongest (immutable).
Default: "*"
MinLength: 1
MaxLength: 200
Expand Down
6 changes: 3 additions & 3 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Version-pinned for reproducibility. Production hardening (post-v0):
# regenerate with `pip-compile --generate-hashes` and call `pip install
# --require-hashes` in the workflow to defend against PyPI compromise.
# Version-pinned for reproducibility. Not hash-pinned yet: to defend against
# PyPI compromise, regenerate with `pip-compile --generate-hashes` and call
# `pip install --require-hashes` in the workflow.
requests==2.33.1
boto3==1.42.94
PyYAML==6.0.2
2 changes: 1 addition & 1 deletion src/scripts/shadow/bedrock_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ def invoke(self, system_prompt, user_prompt, max_tokens=4096,
write premium with zero reuse. Issue/followup callers pass True
since their prefix is KB + codebase_map, stable across calls.
- model_id: override the default model for this call. Issue triage
uses Haiku because Opus 4.7 doesn't accept outputConfig.textFormat
uses Haiku because Opus (4.7/4.8) doesn't accept outputConfig.textFormat
(structured output) over Bedrock today; Haiku does.
"""
target_model = model_id or self._model_id
Expand Down
6 changes: 3 additions & 3 deletions src/scripts/shadow/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

# Built-in default model. Module-level so the cost-pricing test can assert
# the pricing table covers it without re-encoding the literal in two places.
_DEFAULT_MODEL = "us.anthropic.claude-opus-4-7"
_DEFAULT_MODEL = "us.anthropic.claude-opus-4-8"
# Reporter built-in default. Haiku because the Reporter is JSON-formatting
# only — Opus's reasoning depth is wasted there, and Haiku is the model
# documented as the default in the README.
Expand Down Expand Up @@ -40,8 +40,8 @@ def __init__(self):
_DEFAULT_MODEL,
), _DEFAULT_MODEL)
# Reporter is JSON-formatting only; Haiku handles structured output
# well enough AND is the model the issue path REQUIRES (Opus 4.7
# rejects outputConfig.textFormat over Bedrock today). Adopters who
# well enough AND is the model the issue path REQUIRES (Opus 4.7/4.8
# reject outputConfig.textFormat over Bedrock today). Adopters who
# override BEDROCK_MODEL_ID (Investigator) without setting Reporter
# silently get Haiku here — log loud at INFO so the asymmetry is
# visible in workflow logs rather than discovered via "why does my
Expand Down
10 changes: 6 additions & 4 deletions src/scripts/shadow/doctor.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,12 +117,14 @@ def _check_bedrock_access(args, result):
# which model families reject sampling params.
from .bedrock_client import _build_inference_config
from . import shadow_config
from .config import _DEFAULT_MODEL, _DEFAULT_REPORTER_MODEL
# Validate the models the adopter actually configured, not hardcoded ones —
# otherwise doctor greens an install whose real model (e.g. an Opus 4.8
# otherwise doctor greens an install whose real model (e.g. a Sonnet 4.6
# override) is unreachable. Resolve exactly as Config does: env > .shadow.yml
# > default, so a yaml-only override is validated too.
_default_opus = "us.anthropic.claude-opus-4-7"
_default_haiku = "us.anthropic.claude-haiku-4-5-20251001-v1:0"
# > default, so a yaml-only override is validated too. Defaults come from
# config so a default-model bump can't drift doctor out of sync.
_default_opus = _DEFAULT_MODEL
_default_haiku = _DEFAULT_REPORTER_MODEL
yml = shadow_config.load(args.repo_root or os.getenv("SHADOW_REPO_ROOT", "."))
investigator = shadow_config.env_or(
"BEDROCK_MODEL_ID", shadow_config.get(yml, "models", "investigator"), _default_opus)
Expand Down
21 changes: 12 additions & 9 deletions src/scripts/shadow/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
import time
import uuid

from .config import Config
from .config import Config, _DEFAULT_MODEL, _DEFAULT_REPORTER_MODEL
from .bedrock_client import BedrockClient
from .github_client import GitHubClient
from .knowledge_base import KnowledgeBase
Expand Down Expand Up @@ -170,7 +170,7 @@ def analyze():

# The model id stamped on this item's artifacts — and interpolated into the
# user-facing "Generated by AI (model: …)" footer act() posts. The
# issue/followup surface invokes the reporter model (Haiku; Opus 4.7 rejects
# issue/followup surface invokes the reporter model (Haiku; Opus 4.7/4.8 reject
# structured output over Bedrock), the PR pipeline invokes the investigator
# model. Name the model that actually runs on THIS surface so the footer is
# truthful. Computed once here so the many early-exit artifacts below can't
Expand Down Expand Up @@ -1837,7 +1837,7 @@ def _finalize_metrics(metrics, inv, crit):
# Bedrock per-1M-token list prices for the cross-region inference profiles
# Shadow defaults to. Adopters who override BEDROCK_*_MODEL_ID get cost
# fallback to the default so the line is still informative — under-counts
# only when the override is more expensive than Opus 4.7.
# only when the override is more expensive than Opus 4.8.
_MODEL_PRICING_PER_M_TOKENS = {
"us.anthropic.claude-opus-4-7": {"input": 5.00, "output": 25.00},
"us.anthropic.claude-opus-4-8": {"input": 5.00, "output": 25.00},
Expand All @@ -1849,8 +1849,8 @@ def _finalize_metrics(metrics, inv, crit):
def _stage_cost(stage_metrics):
if not isinstance(stage_metrics, dict) or stage_metrics.get("skipped"):
return 0.0
model_id = stage_metrics.get("model_id") or "us.anthropic.claude-opus-4-7"
pricing = _MODEL_PRICING_PER_M_TOKENS.get(model_id) or _MODEL_PRICING_PER_M_TOKENS["us.anthropic.claude-opus-4-7"]
model_id = stage_metrics.get("model_id") or _DEFAULT_MODEL
pricing = _MODEL_PRICING_PER_M_TOKENS.get(model_id) or _MODEL_PRICING_PER_M_TOKENS[_DEFAULT_MODEL]
in_tok = stage_metrics.get("input_tokens", 0) or 0
out_tok = stage_metrics.get("output_tokens", 0) or 0
cache_read = stage_metrics.get("cache_read_tokens", 0) or 0
Expand Down Expand Up @@ -2032,10 +2032,13 @@ def _build_provenance_from_env():
in early-exit artifacts while runtime actually ran Haiku. (yaml-only model
overrides are not visible here — this path has no Config; those artifacts are
the cheap SKIP/ESCALATE ones, and the full path uses _build_provenance(cfg).)"""
_opus = "us.anthropic.claude-opus-4-7"
_haiku = "us.anthropic.claude-haiku-4-5-20251001-v1:0"
investigator = os.getenv("BEDROCK_MODEL_ID", _opus)
reporter = (os.getenv("BEDROCK_REPORTER_MODEL_ID") or "").strip() or _haiku
# Mirror Config's defaults from the single source of truth so a future
# default-model bump can't drift this early-exit path out of sync. Use the
# `(getenv or "").strip() or default` idiom on every line so an empty or
# whitespace env value falls back to the default, matching Config.env_or
# (a bare `getenv(name, default)` would stamp "" when the var is set-empty).
investigator = (os.getenv("BEDROCK_MODEL_ID") or "").strip() or _DEFAULT_MODEL
reporter = (os.getenv("BEDROCK_REPORTER_MODEL_ID") or "").strip() or _DEFAULT_REPORTER_MODEL
return {
"schema_version": 1,
"shadow_ref": os.getenv("SHADOW_BOT_REF", "").strip() or "unknown",
Expand Down
22 changes: 22 additions & 0 deletions tests/integration/test_config_init.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,28 @@ def _clear_model_env(monkeypatch):
monkeypatch.delenv(v, raising=False)


def test_defaults_resolve_to_builtin_models(monkeypatch):
"""A bare Config() (no env, no yaml) must resolve the reasoning stages to
_DEFAULT_MODEL and the reporter/issue stages to _DEFAULT_REPORTER_MODEL.
Asserts against the constants (not literals) so it survives a deliberate
default bump, but still catches a silent wiring regression — a stale
literal on the env_or line, _scrub_model_id mangling a clean default, or
env_or's 3rd-arg being dropped — that test_provenance_from_env (which
guards main.py's separate os.getenv path) would not."""
_set_required_env(monkeypatch)
_clear_model_env(monkeypatch)
with tempfile.TemporaryDirectory() as tmp:
monkeypatch.setenv("SHADOW_REPO_ROOT", tmp)
from shadow.config import (
Config, _DEFAULT_MODEL, _DEFAULT_REPORTER_MODEL,
)
cfg = Config()
assert cfg.bedrock_model_id == _DEFAULT_MODEL
assert cfg.critic_model_id == _DEFAULT_MODEL # critic follows investigator
assert cfg.reporter_model_id == _DEFAULT_REPORTER_MODEL
assert cfg.issue_model_id == _DEFAULT_REPORTER_MODEL # issue follows reporter


def test_yaml_models_block_overrides_per_stage(monkeypatch):
"""models.investigator/critic/reporter in .shadow.yml should override
the built-in defaults — adopters tune cost/quality without env vars."""
Expand Down
3 changes: 2 additions & 1 deletion tests/unit/test_cost.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@ def test_cache_writes_are_one_point_two_five_input_price():

def test_unknown_model_falls_back_to_opus_pricing():
"""Adopter who overrides BEDROCK_MODEL_ID to something unrecognized still
gets a cost number — pricing table fallback to Opus 4.7."""
gets a cost number — pricing table fallback to the default model (Opus,
$5/1M input)."""
stage = {
"skipped": False, "model_id": "anthropic.claude-9000-not-yet",
"input_tokens": 1_000_000, "output_tokens": 0,
Expand Down
18 changes: 17 additions & 1 deletion tests/unit/test_provenance_from_env.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

from shadow import main as m

_OPUS = "us.anthropic.claude-opus-4-7"
_OPUS = "us.anthropic.claude-opus-4-8"
_HAIKU = "us.anthropic.claude-haiku-4-5-20251001-v1:0"
_MODEL_ENVS = ("BEDROCK_MODEL_ID", "BEDROCK_REPORTER_MODEL_ID",
"BEDROCK_CRITIC_MODEL_ID", "BEDROCK_ISSUE_MODEL_ID")
Expand Down Expand Up @@ -45,6 +45,22 @@ def test_opus_investigator_does_not_drag_reporter_or_issue_to_opus(monkeypatch):
assert mo["issue"] == _HAIKU


@pytest.mark.parametrize("empty_value", ["", " "])
def test_empty_or_whitespace_env_falls_back_to_defaults(monkeypatch, empty_value):
# A set-but-empty/whitespace env var (e.g. `${{ secrets.X }}` where X is
# unset -> GitHub passes "") must fall back to the default on EVERY stage,
# matching Config.env_or. Guards the investigator line specifically: a bare
# `getenv(name, default)` would stamp "" here while runtime resolves the
# real default, mislabeling the audit provenance.
for var in _MODEL_ENVS:
monkeypatch.setenv(var, empty_value)
mo = _models()
assert mo["investigator"] == _OPUS
assert mo["critic"] == _OPUS
assert mo["reporter"] == _HAIKU
assert mo["issue"] == _HAIKU


def test_issue_follows_reporter_env(monkeypatch):
monkeypatch.setenv("BEDROCK_REPORTER_MODEL_ID", "us.anthropic.claude-sonnet-4-6")
assert _models()["issue"] == "us.anthropic.claude-sonnet-4-6"
Expand Down
Loading