Skip to content

Add program metadata, asset value tracking, and Intigriti reward-grid parser#95

Open
psyray wants to merge 6 commits into
sw33tLie:mainfrom
psyray:feat/add-program-metadatas
Open

Add program metadata, asset value tracking, and Intigriti reward-grid parser#95
psyray wants to merge 6 commits into
sw33tLie:mainfrom
psyray:feat/add-program-metadatas

Conversation

@psyray

@psyray psyray commented Jul 12, 2026

Copy link
Copy Markdown

Summary

Introduces a program metadata system that captures rich program-level information (rewards, rules, qualifying vulnerabilities, reports stats, testing instructions, etc.) for all supported platforms, tracks per-asset value severity across platforms, and adds a markdown reward-table parser for Intigriti. The web UI gains a metadata section on program pages and new filters on the scope table.

fix #81

Changes

Program metadata system

  • New scope.ProgramMetadata and scope.RewardGrid types (pkg/scope/metadata.go) modeling rewards, rules, qualifying/non-qualifying vulnerabilities, testing instructions, account setup, and program stats. Pointer fields distinguish "unset" from "zero".
  • New program_metadata Postgres table (pkg/storage/storage.go) with UpsertProgramMetadata, GetProgramMetadata, GetProgramMetadataByPlatformHandle (pkg/storage/metadata.go).
  • polling.processOneProgram now upserts metadata after scope entries (non-fatal on failure).
  • New bbscope db program <platform>/<handle> subcommand (cmd/db.go) to print stored metadata from the CLI.
  • Each platform poller now produces metadata:
    • HackerOne: limited (title, type, bounty/VDP flags) — public API exposes nothing else.
    • Bugcrowd: full metadata including per-scope-group reward grids (P1–P5 → critical/info), rules (HTML), safe harbor, test-account flag, account access section parsed from brief HTML.
    • Intigriti: metadata from researcher API + bounty info from listing endpoint; testing requirements (User-Agent, request header, automated tooling limit, intigritiMe); scope count from domains.content. Also includes a fallback to the newer assetsCollection shape when the legacy domains.content array is empty.
    • YesWeHack: full metadata including reward grids per asset value (very_low/low/default/medium/high/critical), reports stats (count, avg reward, avg first response), VPN requirements, qualifying/non-qualifying vulnerabilities, account access, suggested User-Agent.
    • Immunefi: minimal metadata (title, type, flags).

Intigriti markdown reward-grid parser

  • Since the Intigriti researcher API does not expose structured bounty tables, parseIntigritiMarkdownRewardGrids scans the rules markdown for tables that look like reward grids (severity rows × asset category columns with currency amounts) and parses them into scope.RewardGrid entries — one per asset category column.
  • Tables without currency markers (e.g. CVSS score ranges, validation-time SLAs) are skipped via amountRegex, which requires a currency symbol ($//£) or code (USD/EUR/GBP).
  • Handles Up to $X (Max only), $X - $Y (both), $X (min=max), and From $X (Min only) cell syntaxes.

Asset value tracking

  • New asset_value column on targets_raw (with idempotent ADD COLUMN IF NOT EXISTS migration) and AssetValue field on ScopeElement / TargetItem / UpsertEntry / Entry / Change.
  • Platform pollers populate normalized asset values (critical, high, medium, low, very_low) from each platform's tier/P1-max/bounty-tier semantics:
    • Bugcrowd: derived from P1 max bounty per scope group.
    • Intigriti: from tier.id (legacy) or bountyTierId (assetsCollection).
    • YesWeHack: from the asset_value field in the program response.
  • Storage upsert detects asset-value changes as updates; listing/detail queries return asset value; program detail page renders a colored severity badge per asset, with targets sorted by severity within in-scope/out-of-scope groups.

Web UI (website/)

  • Program detail page now renders a "Program Information" section with rewards grid table, reports stats, testing instructions, qualifying/non-qualifying vulnerabilities (collapsible), account access, and a collapsible Rules section.
  • New dependency-free markdown renderer (website/pkg/core/markdown.go) for safely rendering platform rules as HTML — supports headings, lists, GFM tables, code blocks, blockquotes, inline bold/italic/code/links. Input is HTML-escaped up-front; only generated tags appear in output. Links with javascript:/data: schemes are neutralized.
  • Conservative HTML sanitizer (sanitizeRulesHTML) for HTML-formatted rules: strips <script>/<style>/<iframe>/<object>/<embed>, on*= event handlers, and javascript: URLs.
  • Scope table gains Rewards and Reports columns plus new sort keys (bounty_reward_max, reports_count), and three new filters: min reward, currency, and "has reports". URL state is preserved for the new filters.
  • Program listings, sitemap slugs, and updates API response now include the metadata title (falling back to handle) for display.
  • program_metadata.title is joined into the programs index and updates feed via a LEFT JOIN on program_metadata.
  • Bugcrowd handles are sorted/displayed with the /engagements/ prefix stripped for consistency.

Other

  • AGENTS.md added with repository guidance for AI agents.
  • Tests added for the Bugcrowd, Intigriti, YesWeHack, and HackerOne metadata extractors and the Intigriti markdown reward-grid parser. Fixtures use inlined JSON; no real APIs or DBs are hit.
  • polling.go field alignment cleaned up via gofmt.

Validation

  • Run go vet ./... and go test ./... from the repo root before merge.
  • Database migrations are idempotent (ADD COLUMN IF NOT EXISTS, CREATE TABLE IF NOT EXISTS).
  • New unit tests cover metadata extraction and the markdown reward-grid parser without hitting real APIs or DBs.

Notes for reviewers

  • The Intigriti markdown reward-grid parser is intentionally conservative: tables without currency markers are skipped. See parseIntigritiMarkdownRewardGrids and its tests for the supported syntaxes.
  • The web markdown renderer is deliberately minimal (CommonMark subset) to avoid pulling in a new top-level dependency; exotic syntax degrades to a paragraph rather than failing.
  • No new top-level Go dependencies were added.

psyray added 6 commits July 11, 2026 03:27
Introduce a new ProgramMetadata struct to capture program-level
details (rewards, rules, testing instructions, stats) for each
platform. This data is stored in a new `program_metadata` SQLite
table and queryable via `bbscope db program`.

- Add metadata extraction logic for Bugcrowd, HackerOne, Immunefi,
  Intigriti, and YesWeHack pollers.
- Implement upsert and retrieval methods in the storage package.
- Update platform documentation with details on captured metadata.
- Add tests for Bugcrowd, Intigriti, and YesWeHack metadata parsers.
Normalize asset value for Bugcrowd, Intigriti, and YesWeHack programs.
The value is stored in the database and exposed via the API. Add UI
dropdowns to filter programs by minimum reward, currency, and report
count.
Add AGENTS.md with AI agent guidelines
The researcher API omits structured bounty tables, so parse them
from the markdown rules payload. This unifies rendering across
platforms and adds a minimal markdown renderer for the web UI.
@sw33tLie

Copy link
Copy Markdown
Owner

Hey! Thanks for this!

I need some time to review it as it's a big one, but will do!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Suggestion: adding option to retreive bounty table

2 participants