Skip to content

test: cover ai-evals API drift and CCM anomaly v2 wiring#552

Draft
cursor[bot] wants to merge 1 commit into
mainfrom
cursor/missing-test-coverage-7cf1
Draft

test: cover ai-evals API drift and CCM anomaly v2 wiring#552
cursor[bot] wants to merge 1 commit into
mainfrom
cursor/missing-test-coverage-7cf1

Conversation

@cursor

@cursor cursor Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Description

Adds regression tests for two recently merged production changes that shipped without dispatch/schema contract coverage.

Risky behavior now covered

AI Evals (#529) — prevents reintroduction of control-plane API drift:

  • eval_metric create requires and forwards dimension (422 without it)
  • eval_annotation create/update expose and preserve thumbs_up (including false)
  • online_eval evaluate uses metric_set_id + judge_llm_connector_ref instead of deprecated metric_ids

CCM anomalies (#533) — guards v2 API migration:

  • anomalyListExtract projection shape
  • cost_anomaly v2 list body builder: defaults, explicit time range vs time_filter, zero thresholds (!= null), ordering/search
  • cost_anomaly_drilldown path routing (details vs time-series vs sub-item list) and skipCompact round-trip

Test files added/updated

  • tests/registry/ai-evals.test.ts — control-plane drift + online_eval dispatch tests
  • tests/registry/ccm-anomaly.test.ts — new v2 anomaly dispatch and drill-down tests
  • tests/registry/ccm-extractors.test.tsanomalyListExtract unit tests

Why these tests materially reduce regression risk

Both fixes changed request body shapes and API paths that agents rely on at runtime. Schema-only checks would miss dispatch bugs (e.g. dropping thumbs_up: false, reverting to metric_ids, or using truthy checks that drop min_amount: 0). These tests assert the exact HTTP path/body the registry emits.

Type of Change

  • Bug fix
  • New feature
  • Refactor
  • Documentation
  • Other — test coverage

Checklist

  • pnpm test passes (67 tests in touched files)
  • No production code changes
Open in Web View Automation 

Add regression tests for recent production fixes that lacked dispatch
and schema contract coverage:

- AI Evals (#529): dimension on metric create, thumbs_up on annotations,
  online_eval metric_set_id migration away from deprecated metric_ids
- CCM anomalies (#533): anomalyListExtract, v2 list body builder
  (zero thresholds, time filters, ordering), drill-down path routing

Co-authored-by: Rohan Gupta <thisrohangupta@users.noreply.github.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants