[ctron]

Depends on #2775 (utoipa 6 upgrade). Merge #2775 first.

Summary

An evidence-based correlation engine (ADR 00023): ingestion stores what documents state,
extractors turn those statements into evidence (component × vulnerability, with match
dimension, confidence and provenance), and verdicts are resolved from the evidence.
This covers stages 1–3 of the ADR roadmap, plus what real-world data required on top.

Correlation (modules/correlation)

• correlation_evidence table, extraction worker, and verdict resolution (Fixed > NotAffected >
  Affected > UnderInvestigation).
• Extractors:
  • Digest and product identifiers (SKU, model number, serial number);
  • PURL: version ranges per PURL type, CSAF vls ranges, RPM release streams (incl. RHEL
    mainline minors, EUS/AUS/E4S as separate products), SUSE codestreams;
  • CPE: version ranges and extended attributes.
• Product context: evidence is scoped to the product a statement is made for (from describing
  CPEs, ancestors, and RPM PURLs), so statements of other products/streams don't leak.
• Correlation options per publisher, e.g. versionless PURLs, or fixed_is_first_fixed (Red Hat
  states fixed only for the fixing build; enabled for Red Hat by default).
• API: GET /v3/correlation/sbom/{id}, POST /v3/correlation/verdict-counts,
  GET /v3/correlation/query, GET /v3/correlation/advisory/{id}/statement,
  CRUD on /v3/correlation/option.

Ingestion

• CSAF: all product statuses (incl. recommended, first_fixed, first_affected,
  last_affected, under_investigation), version ranges as stated, product context of
  statements, products defined outside of branches.
• Red Hat's "fixed in X ⇒ affected before X" is no longer derived at ingestion (now a correlation
  rule). The fundamental SBOM, PURL and vulnerability APIs still report those "affected"
  statuses: as a bridge until they use the correlation verdicts, they read the views
  purl_status_effective / version_range_effective (m0002560), which add them at query time
  for publishers enabling fixed_is_first_fixed (nothing inferred is stored).
• Compressed uploads from the UI and URLs.

UI (ui/, Yew/PatternFly)

SBOM list with verdict counts, correlation results with evidence and product context, advisory
viewer with stored statements, management of correlation options.

Other

• Migrations m0002380 … m0002560. Advisories ingested before m0002510/m0002540 need to be
  re-ingested for product contexts and the additional CSAF statuses.
• deny.toml exceptions for the correlation UI (wasm, not shipped with the server).
  Policy changes — please confirm:
  • MPL-2.0 for progenitor-client only (via trustify-client);
  • the git source ctron/trustify-clients (trustify-client with derived PartialEq, until
    feat: derive PartialEq on all generated API types rh-jfuller/trustify-clients#10 is released);
  • unmaintained bincode, paste, proc-macro-error (via yew and monaco).
• The supply chain audit still fails on RUSTSEC-2024-0449 (unsafe-libyaml via serde_yaml_ng),
  which affects main as well and is left to be handled there.
• Known gaps and follow-ups are tracked in TODO.md.

Review

This PR is large (~23k lines of code; ~118k lines are test data under etc/test-data). It could
be split into ingestion changes / correlation engine / UI if that helps reviewing.

Test plan

• cargo test -p trustify-module-correlation: scenario tests (etc/test-data/scenarios, each
  run in both ingestion orders) against real advisories (Red Hat, SUSE, OSV, CVE, CERT@VDE), incl.
  S24, a syft SBOM of a UBI 10 image
• cargo xtask precommit (includes the UI crate)
• CI green

Try it out

1. Start the database, from the repository root:

   podman compose -f etc/deploy/compose/compose.yaml up

2. Start the server (with the embedded OIDC server), from the repository root:

   cargo run -p trustify-trustd -- api --devmode --embedded-oidc

3. Start the UI, a Yew/PatternFly app in ui/ (needs rustup target add wasm32-unknown-unknown,
   cargo install trunk, and Node.js; see ui/README.md):

   cd ui
   npm ci
   trunk serve --port 8081

   and open http://localhost:8081.

🤖 Generated with Claude Code

[sourcery-ai]

Reviewer's Guide

This PR establishes an evidence-first correlation foundation: it persists extractor-produced component/vulnerability links, resolves them into precedence-based verdicts, exposes those results through a new SBOM correlation endpoint, documents the architecture, and wires in initial service and endpoint tests.

Sequence diagram for SBOM correlation verdict retrieval

sequenceDiagram
    participant Client
    participant API as CorrelationEndpoint
    participant Service as CorrelationService
    participant DB as ReadOnlyDatabase

    Client->>API: GET /v3/correlation/sbom/{id}
    API->>DB: Query correlation_evidence for sbom_id
    DB-->>Service: Evidence rows
    Service->>DB: Load nodes, checksums, advisories, vulnerabilities
    DB-->>Service: Correlation metadata
    Service->>Service: correlate_sbom(sbom_id, connection)
    Service-->>API: CorrelationResult with verdicts and evidence
    API-->>Client: 200 JSON response

Loading

Entity relationship diagram for correlation evidence storage

erDiagram
    SBOM ||--o{ CORRELATION_EVIDENCE : contains
    ADVISORY ||--o{ CORRELATION_EVIDENCE : provides
    CORRELATION_EVIDENCE {
        uuid id PK
        uuid sbom_id FK
        text node_id
        uuid advisory_id FK
        text vulnerability_id
        assertion_status status
        match_dimension match_dimension
        float confidence
        text extractor
    }

Loading

File-Level Changes

Change	Details	Files
Defines the persisted evidence model and database schema for component-to-vulnerability correlation.	Adds the correlation evidence entity with assertion status and match-dimension enums. Adds a migration creating the evidence table, PostgreSQL enums, indexes, foreign keys, and extractor-scoped uniqueness. Records the correlation architecture and staged extractor roadmap in ADR 00022.	docs/adrs/00022-correlation-engine.md entity/src/correlation_evidence.rs entity/src/lib.rs migration/src/lib.rs migration/src/m0002380_create_correlation_evidence.rs
Introduces a correlation service that converts stored evidence into API-ready verdicts.	Loads evidence and related SBOM nodes, checksums, advisories, and vulnerabilities. Groups evidence by component and vulnerability and resolves statuses using Fixed > NotAffected > Affected > UnderInvestigation precedence. Returns evidence provenance and component/vulnerability metadata, with empty verdicts when no evidence exists.	modules/correlation/src/service/mod.rs modules/correlation/src/model/mod.rs modules/correlation/src/error.rs
Exposes the evidence-backed correlation result through the server API and OpenAPI contract.	Adds GET /v3/correlation/sbom/{id} and wires the correlation module into the server profile. Defines serializable response models for verdicts, evidence, components, vulnerabilities, statuses, and match dimensions. Updates the checked-in OpenAPI specification with the endpoint and schemas.	modules/correlation/src/endpoints/mod.rs modules/correlation/src/endpoints/test.rs modules/correlation/src/lib.rs server/src/profile/api.rs server/Cargo.toml openapi.yaml
Adds initial integration coverage for empty and evidence-backed SBOM correlation flows.	Verifies service and endpoint behavior for an SBOM with no evidence. Ingests sample SBOM/advisory data, inserts digest evidence, and verifies the resolved affected verdict and provenance.	modules/correlation/src/service/test.rs modules/correlation/src/endpoints/test.rs
Registers the new correlation crate as a workspace and server dependency.	Adds the module to the workspace members and shared dependency declarations. Adds the module's runtime and test dependencies.	Cargo.toml Cargo.lock modules/correlation/Cargo.toml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Sorry, we are unable to review this pull request

The GitHub API does not allow us to fetch diffs exceeding 300 files, and this pull request has 418

[rh-jfuller]

there is now a m0002390_create_crypto_remediation

[ctron]

Fixed in feadf62. The PR migrations are now numbered sequentially from m0002410 through m0002590, after upstream's crypto remediation and base PURL name index migrations. Updated the SQL include paths, tests, and documentation references as well. Formatting and compilation checks for the migration and correlation test targets passed.

[rh-jfuller]

there is now a m0002390_create_crypto_remediation

[ctron]

Fixed in feadf62: the correlation evidence migration is now m0002410_create_correlation_evidence, and the remaining PR migrations follow sequentially through m0002590. Upstream's existing migration names are preserved.

[rh-jfuller]

I tihnk auth runs after download ... require_permission() is only called once format has been detected from fetched bytes so any authed user can make server issue requests

[ctron]

Authentication already runs in the /api middleware before the handler, but you are right that the format-specific authorization check happened after downloading. Fixed in 2803f79 by adding Require before the handler runs. It requires at least one of create.sbom, create.advisory, create.weakness, or create.exploit before any outbound request. After downloading and detecting the format, we still require that format's specific permission before ingestion. Regression tests verify that anonymous users and users with no upload permission (including unrelated create permissions) trigger zero outbound requests, and that a different upload permission does not authorize ingestion. All 12 upload tests pass.

[rh-jfuller]

I think this migration is 'one way' ...

[rh-jfuller]

I found all the sql to have no kind of injection paths ;) which is good ... I provided a few comments, otherwise this PR is good enough to go into main ... I would rather we get it in before it grows any bigger !