Skip to content

Add portable plugin validation and harden Reddit gates - #1

Merged
PalmPalm7 merged 13 commits into
mainfrom
agent/portable-plugin-and-reddit-gates
Aug 7, 2026
Merged

PalmPalm7 merged 13 commits into
mainfrom
agent/portable-plugin-and-reddit-gates

Conversation

@PalmPalm7

@PalmPalm7 PalmPalm7 commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

What changed

  • add the root Agent Plugins v1 plugin.json
  • validate the portable manifest, Agent Skills, and package-path containment in CI
  • reject the legacy .codex-plugin layout and require every tracked SKILL.md to live exactly one directory below skills/
  • validate both source skills and every copy produced by the real installer with the same pinned official Agent Skills validator
  • pin GitHub Actions, the official Agent Skills validator commit, the schema digest, and the real skills installer version
  • make requirements distinguish missing subreddits from partial/rate-limited endpoint failures
  • return full rule descriptions, endpoint status, and rate-limit reset data
  • reduce scout candidate overfetch so it does not exhaust the read budget before the required posting gate
  • add the missing reddit-voice README entry and a reproducible read-only tester path
  • replace an over-broad crawler claim with the evidence-bounded meaning of is_robot_indexable, plus a corrected proof card
  • add a structured reddit-scout test-report form that separates completed runs from failed attempts and enforces privacy/write-safety confirmations

Why

The package needed to conform to the portable Agent Plugins v1 specification. A live GTM run also exposed two concrete failures:

  1. after scout, requirements ClaudeCode reported subreddit_not_found even though the actual Reddit responses were HTTP 429; that could encourage an agent to misread the destination or bypass the rule gate
  2. the campaign invited people to try reddit-voice, but the repository README did not list or explain that skill

The review also found that is_robot_indexable: false proves Reddit marked the post ineligible for robot indexing, not that every possible search or AI system is technically unable to access it. The new wording stays within the measured evidence.

Impact

Installed copies remain self-contained. Operators now get an explicit partial/full gate, current rule text, and rate-limit diagnostics instead of a false not-found result. Scout examines only the requested bounded candidate set. Testers can install the two read-only skills directly, submit one concrete rule match/mismatch without exposing session data, and the visibility proof no longer overstates what the field proves.

The compliance guard now checks every tracked package path after filesystem resolution, so a symlink cannot escape the plugin root. It rejects legacy manifests and nested SKILL.md files that portable clients would not discover as direct-child skills. It installs, validates, and exercises all five direct-child skills instead of sampling only reddit-post.

Validation

  • Agent Plugins v1 official closed schema with pinned SHA-256: pass
  • pinned official Agent Skills validator on source skills: 5/5 pass
  • pinned official Agent Skills validator on real installed copies: 5/5 pass
  • local Skill Creator validator: 5/5 pass
  • immutable GitHub Action commits and skills@1.5.22: pinned
  • tracked package-path containment: pass; escaping-symlink negative test rejected as expected
  • .codex-plugin and nested tracked SKILL.md negative tests: rejected as expected
  • actionlint v1.7.12: pass
  • node --check for canonical and installed CLI copies: pass
  • sync idempotence and git diff --check: pass
  • real npx skills@1.5.22 add smoke: all 5 installed skills contain their own CLI/reference and expose help
  • live read-only requirements ClaudeCode: all four endpoints HTTP 200, partial: false, post_gate_complete: true
  • proof-card text reviewed against the actual is_robot_indexable signal
  • tester issue form YAML parse, attribution fields, and two required safety checks: pass
  • live Weekly Showcase comment was dry-run first and verified publicly visible after posting

Scout topical-fit regression fix

The search-share component previously treated incidental keyword-bearing posts as proof that an entire community was relevant. On the exact read-only TikTok developer-tool query, that ranked BestofRedditorUpdates first and recommended unrelated gaming, World of Warcraft, celebrity, and Black Friday communities.

  • add an independent, auditable topical-fit gate based on display name, title, and public description
  • keep search examples as supporting-only evidence; they can never open the gate
  • preserve the legacy postable count as non-banned/rule-permitted, and add recommended_count, not_recommended, and per-result recommended fields for campaign selection
  • retain promotion-ban verdict precedence, add deterministic offline fixtures, and make explicit CLI help exit successfully

Additional validation:

  • offline ranking regression: 3/3 pass across 10 false-positive and developer-community fixtures
  • exact live read-only query: postable remains 8; recommended_count is 2; six obvious off-topic communities now score 0 and report NOT RECOMMENDED; both promotion bans remain DO NOT POST
  • known relevant live read-only query Claude Code agent skills: 6/8 communities recommended, confirming the conservative gate is not all-zero
  • official Agent Plugins v1 schema and pinned Agent Skills validator: source 5/5 pass
  • real skills@1.5.22 install: installed 5/5 self-contained copies validate, parse, contain the topical-fit module, and expose help

Telemetry integrity and independent evidence

  • set job-level DO_NOT_TRACK=1 so maintainer and CI installer smoke tests cannot seed skills.sh telemetry
  • expand the tester-report funnel to all five Reddit skills, with exact named completion states and fail-closed write/network boundaries
  • add exit-code and sanitized-stderr evidence fields, and synchronize the scout form with the topical-mismatch verdict
  • move reddit-post dry-run handling before flair resolution and post-requirements lookup, so submit without --yes is fully local
  • expose RDTX_STATE_DIR for isolated deterministic tests and add an offline audit-log regression

Additional validation:

  • offline rdtx regressions: 4/4 pass
  • actionlint v1.7.7, issue-form semantic gates, CLI copy synchronization, node syntax, and git diff check: pass
  • independent local single-skill install with DO_NOT_TRACK=1: only reddit-post installed
  • protected forward test with CDP port 1 and a nonexistent browser executable: exit 0, stderr empty, dry_run=true, both remote checks not_run
  • isolated audit: exactly one executed=false / reason=dry-run record; default audit unchanged
  • no CDP connection, Reddit request, Reddit write, or GitHub tester issue was made during the forward test

@PalmPalm7
PalmPalm7 marked this pull request as ready for review August 7, 2026 06:55
@PalmPalm7
PalmPalm7 merged commit 81193b5 into main Aug 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant