Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ All modules live under `traust_core.v1` (see [Versioning](#versioning)).
| **Validation report REST** | `rest.validation` | `POST /v1/validation-records` → `services.validation.record_order`; live leased `run_order_id`, exact report text, `Provenance`, native `ValidationRecordInput` bindings → `ValidationRecordResult` | source-label lookups, route-owned authorization or verdict rules, human decision commands |
| **Reporting** | `services.reporting`, `providers.report_files` | `ctx.reporting` (the same instance as `ctx.services().reporting`; it never opens the record store, so file-only callers use it) over a report document the caller holds, nothing stored: `validate(payload, XArtifact)` → contract `Issue`s; `validate_reports(path, strict=, schema=)` → a `ReportValidation` (errors, warnings) per report file at `path` (a directory sweeps its .json files; the schema comes from each file name unless a contract schema is named): the contract schema, then the family's content checks (`services.reporting.checks`: counts against findings, canonical ids, `metadata.repository`, dispositions, review surfaces, OWASP threat ratings) and, with `strict`, the audit-report warnings; `lint(doc, files=None)` runs every core `Lint` registered for the type; `render(doc, RenderFormat.MARKDOWN \| HTML)` (triage in both; security reports and threat models as Markdown: `services.reporting.report_markdown`, `services.reporting.threat_model`); `lint_threat_models(paths, strict=)` lints THREAT_MODEL.md prose (`threat_model.lint_threat_model`, OWASP cells via `services.reporting.threat_rating`; ATT&CK and ATLAS ids against the registered catalogs, which the pack always pins; without them the door raises naming the register doors); `lost_sections(existing, rendered)` (the render guard); `sarif_reports(root)` is the SARIF sweep's report selection; `sarif(doc)` for report and cloud-config audits (`SarifExport.report`, `locations.sarif_tool_uri` as `informationUri`). `ctx.services().redaction.redact(payload)` scrubs secrets (`RedactionPolicy`; `scan(text)` finds them without redacting) | `traust_engine.reporting` (`validate`, `lint`, `render`, `sarif`), `_util/redact.py` |
| **Rendering** | `rendering` | `Report` → `MarkdownRenderer` → `publish()`; `markdown_table()`; `markdown_text()` escapes a structured cell as plain text (no HTML, links or inline formatting) | 18 copies of `render_md` |
| **Detection-quality and compliance renderers** | `rendering.benchmark`, `rendering.compliance_precision`, `rendering.compliance` | `benchmark_markdown(run)` / `scorecards_markdown(cards)` and `compliance_precision_markdown(calibration)` render typed results with untrusted cells escaped; `rendering.compliance.assessment_report(document)` → `ComplianceService.render_assessment(document)` Markdown bytes | consumer-owned report renderers |
| **Detection-quality and compliance renderers** | `rendering.benchmark`, `rendering.compliance` | `benchmark_markdown(run)` / `scorecards_markdown(cards)` render typed benchmark results; `rendering.compliance.assessment_report(document)` → `ComplianceService.render_assessment(document)` Markdown bytes | consumer-owned report renderers |
| **Compliance field precision** | `models.insight`, `services.insight` | `insight.publish(ViewName.COMPLIANCE_PRECISION, ViewFilter(framework=…, precision_gate=…))`: native calibration under `calibration`, with shared Markdown/HTML and a packaged view schema | standalone Markdown helpers or consumer-owned precision calculations |
| **Secret detection** | `models.redaction` | `SecretDetector.publication_rules()` / `evidence_rules()` yield ordered `SecretRule`s whose `spans(text)` return exact `SecretSpan(tag, start, end)` offsets; `RedactionPolicy` publishes through the same catalog, so it is the one owner of secret patterns | a consumer-owned credential regex table |
| **Verification worklists** | `services.remediation`, `models.remediation`, `services.findings` | `verification.sweep(repository_ids, include_branches=False)` keeps repository/ref identity, branch head source and line-specific counts; `sweep_markdown(sweep)` renders the same queue; `findings.on_ref(repository_ref_id)` folds resolution on that ref's line | a second worklist store |
| **Historical report signatures** | `services.reporting`, `services.ledger`, `models.ledger` | `validate_reports(..., signing_pubkey=Path(...))` requires a historical layer document's Merkle tree and public-key signature through `LedgerService.verify_document`; `ReportValidation.integrity` holds the typed `DocumentVerification` | trusting an unsigned layer |
Expand Down Expand Up @@ -97,6 +98,21 @@ The remaining correctness changes are:
`GET /v1/insights/critical-misses?window_start=…&window_end=…`; both boundaries are required
together. View-schema validation exposes a recursive JSON type without a model-layer
dependency in the contracts facade.
- `ViewName.COMPLIANCE_PRECISION` wraps the existing
`DetectionQualityService.calibrate_compliance` result under `calibration`. Its packaged
view schema reuses the canonical compliance-assessment framework vocabulary through
the local artifact-schema registry, without network resolution or persisted artifacts.
`ViewFilter.framework` is required; optional `precision_gate` is finite and between zero
and one, defaulting to the producer's existing gate. Only this view consumes those
parameters; unsupported scope, historical and time-window filters are refused.
Implicit refresh skips it without a framework and selects only compatible views when
calibration parameters are supplied. The existing REST registry accepts `framework`
and `precision_gate` query parameters.
Counts retain the producer's distinct-document digest deduplication, automatic
not-satisfied firings and attributed human dismissals. Precision at or below the gate
flags a check for review, never automatic demotion. No assessments or no firings is
unmeasured, not zero or perfect precision; field precision is not evaluator calibration
or coverage. Publishing changes neither database rows nor object bytes.


## Repository checkouts
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ dependencies = [
"tree-sitter-java==0.23.5",
"tree-sitter-rust==0.24.2",
"tree-sitter-ruby==0.23.1",
"traust-contracts @ git+https://github.com/traust-security/traust-contracts.git@620b83849a32dd3cc48980944286411bd485e9fa",
"traust-contracts @ git+https://github.com/traust-security/traust-contracts.git@a1648d7585b2c2350e3e0168e79e17a517073678",
"traust-ledger @ git+https://github.com/traust-security/traust-ledger.git@6a5a3eef950d7657e0107dfa95bffd5829cc4fe4"
]

Expand Down
2 changes: 2 additions & 0 deletions src/traust_core/v1/context/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,7 @@
AttackCoverageView,
CensusView,
ComplianceDashboardView,
CompliancePrecisionView,
ContainerFindingsView,
CoverageView,
CriticalMissesView,
Expand Down Expand Up @@ -1096,6 +1097,7 @@ def _wire(self, uow: SqlTraustUnitOfWork, policy: PortfolioPolicy) -> Services:
ContainerFindingsView(findings, portfolio, assessment, rating),
InsecurePatternsView(findings, portfolio),
CriticalMissesView(detection_quality),
CompliancePrecisionView(detection_quality),
trend,
ProgramView(spend, ops, self.assets.names),
CoverageView(assessment, portfolio, self.clock),
Expand Down
4 changes: 3 additions & 1 deletion src/traust_core/v1/contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,9 @@ def view_schema(name: str) -> dict[str, JsonValue]:

@cache
def _view_validator(name: str) -> Draft202012Validator:
return Draft202012Validator(view_schema(name), format_checker=FormatChecker())
return Draft202012Validator(
view_schema(name), registry=_registry(), format_checker=FormatChecker()
)


def validate_view(name: str, document: JsonValue) -> None:
Expand Down
10 changes: 9 additions & 1 deletion src/traust_core/v1/models/insight.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
from traust_core.v1.models.common import TimeBucket, TimeWindow
from traust_core.v1.models.compliance import ComplianceRollup, DocVariance
from traust_core.v1.models.dependencies import ExposureRollup
from traust_core.v1.models.detection_quality import CriticalMiss
from traust_core.v1.models.detection_quality import ComplianceCalibration, CriticalMiss
from traust_core.v1.models.findings import (
Decision,
FindingRollup,
Expand All @@ -35,6 +35,7 @@
from traust_core.v1.models.pqc import PqcReadiness, PqcRollup, PqcSite, PqcVendorClaim
from traust_core.v1.models.validation import ValidationRollup
from traust_core.v1.models.vocabulary import (
ComplianceFramework,
IsolationDimension,
OwnershipClass,
RemediationStatus,
Expand All @@ -61,6 +62,7 @@ class ViewName(StrEnum):
PQC_PORTFOLIO = "pqc-portfolio"
PQC_VENDORS = "pqc-vendors"
COMPLIANCE = "compliance"
COMPLIANCE_PRECISION = "compliance-precision"
PROGRAM = "program"
EXECUTIVE_SUMMARY = "executive-summary"
REVIEW_CARD = "review-card"
Expand All @@ -86,6 +88,10 @@ class CriticalMisses(ViewModel):
misses: tuple[CriticalMiss, ...]


class CompliancePrecision(ViewModel):
calibration: ComplianceCalibration


class ExecutivePanel(StrEnum):
ATTACK = "attack"
PQC = "pqc"
Expand All @@ -112,6 +118,8 @@ class ViewFilter(Model):
report_stem: str | None = None
compare_as_of: datetime | None = None
first_party_divisor: Decimal | None = Field(default=None, gt=0, allow_inf_nan=False)
framework: ComplianceFramework | None = None
precision_gate: float | None = Field(default=None, ge=0, le=1, allow_inf_nan=False)


class SeverityCounts(Model):
Expand Down
39 changes: 0 additions & 39 deletions src/traust_core/v1/rendering/compliance_precision.py

This file was deleted.

5 changes: 5 additions & 0 deletions src/traust_core/v1/rest/insight/insights.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from traust_core.v1.errors import ValidationError
from traust_core.v1.models.common import TimeBucket, TimeWindow
from traust_core.v1.models.insight import ViewFilter, ViewModel, ViewName
from traust_core.v1.models.vocabulary import ComplianceFramework
from traust_core.v1.rest.common.aware_datetime import UtcDatetime
from traust_core.v1.rest.common.deps import Caller
from traust_core.v1.rest.common.problems import PROBLEMS
Expand All @@ -34,6 +35,8 @@ def get_insight(
category: Labels = None,
window_start: UtcDatetime | None = None,
window_end: UtcDatetime | None = None,
framework: ComplianceFramework | None = None,
precision_gate: Annotated[float | None, Query(ge=0, le=1, allow_inf_nan=False)] = None,
) -> SerializeAsAny[ViewModel]:
"""One insight (a dashboard, rollup or export such as SARIF) for a scope at `as_of`, built on
demand; one route for every insight, so a new one needs no route or SDK release."""
Expand All @@ -52,5 +55,7 @@ def get_insight(
),
artifact_id=artifact_id,
repository_id=repository_id,
framework=framework,
precision_gate=precision_gate,
)
return insight.build(name, filter)
105 changes: 104 additions & 1 deletion src/traust_core/v1/services/insight.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
malicious_id,
package_name,
)
from traust_core.v1.models.detection_quality import MissReason
from traust_core.v1.models.detection_quality import COMPLIANCE_PRECISION_GATE, MissReason
from traust_core.v1.models.findings import (
FindingClock,
FindingDimension,
Expand All @@ -66,6 +66,7 @@
CensusLayer,
CensusProduct,
ComplianceDashboard,
CompliancePrecision,
ContainerFinding,
ContainerFindings,
ContainerReport,
Expand Down Expand Up @@ -283,6 +284,10 @@ def refresh(self, views: Sequence[ViewName] | None, filter: ViewFilter) -> Refre
for n, v in self._views.items()
if not getattr(v, "per_item", False)
and (filter.window is None or _accepts_window(v))
and (
n == ViewName.COMPLIANCE_PRECISION
or (filter.framework is None and filter.precision_gate is None)
)
and ((supports := getattr(v, "supports_filter", None)) is None or supports(filter))
]
if views is None
Expand Down Expand Up @@ -314,11 +319,109 @@ def _view(self, name: ViewName, filter: ViewFilter | None = None) -> View:
raise ValidationError(
f"view {name.value!r} is not built; built views: {known}"
) from None
if (
filter is not None
and name != ViewName.COMPLIANCE_PRECISION
and (filter.framework is not None or filter.precision_gate is not None)
):
raise ValidationError(
"framework and precision_gate are supported only by compliance-precision"
)
if filter is not None and filter.window is not None and not _accepts_window(found):
raise ValidationError(f"{name.value} does not support a window")
return found


class CompliancePrecisionView:
name = ViewName.COMPLIANCE_PRECISION

def __init__(self, detection_quality: DetectionQualityService) -> None:
self._detection_quality = detection_quality

@staticmethod
def supports_filter(filter: ViewFilter) -> bool:
return filter.framework is not None and not filter.model_dump(
exclude={"framework", "precision_gate"}, exclude_defaults=True
)

def build(self, filter: ViewFilter) -> CompliancePrecision:
if filter.framework is None:
raise ValidationError("compliance-precision requires a framework")
if not self.supports_filter(filter):
unsupported = filter.model_dump(
exclude={"framework", "precision_gate"}, exclude_defaults=True
)
raise ValidationError(
"compliance-precision supports only framework and precision_gate; "
"unsupported filters: " + ", ".join(sorted(unsupported))
)
calibration = self._detection_quality.calibrate_compliance(
filter.framework,
gate=(
COMPLIANCE_PRECISION_GATE
if filter.precision_gate is None
else filter.precision_gate
),
)
model = CompliancePrecision(calibration=calibration)
document: dict[str, contracts.JsonValue] = model.model_dump(mode="json")
contracts.validate_view(self.name, document)
return model

def report(self, model: CompliancePrecision, generated_at: datetime) -> Report:
calibration = model.calibration
population = Section(
heading="Population",
body="Distinct recorded compliance documents for the selected framework, counted "
"once per digest. A firing is an automatic not-satisfied check or an attributed "
"human override reversing that verdict; the latter is a dismissal. This is "
"field precision, not evaluator idempotence, seeded efficacy or clean coverage.",
table=Table(
columns=("Framework", "Distinct assessments", "Precision gate (inclusive)"),
rows=(
(
calibration.framework.value,
str(calibration.assessments),
str(calibration.gate),
),
),
),
)
if not calibration.assessments:
results = Section(
heading="Results",
body="No assessments recorded for this framework; field precision is unmeasured.",
)
elif not calibration.checks:
results = Section(
heading="Results",
body="No eligible automatic check firings; field precision is unmeasured.",
)
else:
results = Section(
heading="Check precision",
body="Flagged checks require human demotion review, not automatic changes.",
table=Table(
columns=("Check", "Fired", "Dismissed", "Precision", "Demotion review"),
rows=tuple(
(
check.check_id,
str(check.fired),
str(check.dismissed),
"—" if check.precision is None else str(check.precision),
"flagged" if check.flagged else "not flagged",
)
for check in calibration.checks
),
),
)
return Report(
title="Compliance field precision",
generated_at=generated_at,
sections=(population, results),
)


class CriticalMissesView:
name = ViewName.CRITICAL_MISSES
windowed = True
Expand Down
44 changes: 0 additions & 44 deletions tests/v1/rendering/test_compliance_precision.py

This file was deleted.

Loading
Loading