[hayssams]

Problem

On a data branch (SL_DATA_BRANCH), native BigQuery transforms and native BigQuery loads write to <branch>.<table>, but their expectations still queried domain.table, i.e. the production table. A branch run could therefore pass its expectations on production data and never check what it actually wrote.

Fix

• Transforms (AutoTask, BigQueryAutoTask): new overridable expectationTarget (database, domain, table), used by runExpectations and runAndSinkExpectations. It defaults to the task's target; BigQueryAutoTask returns its branch target when a branch is active.
• Loads (IngestionExpectations): when the native BigQuery loader runs on a branch, expectations check <project>.<branch>.<table>. Spark loads never branch, so they keep checking the loaded table. The target resolution is shared with BigQueryNativeLoader via BigQueryNativeLoader.originalTableId.
• IngestionExpectations.expectationJob(handler) is split out of runExpectationsWithHandler so tests can build the job without sinking reports.

Expectation reports on a branch now record the branch as their domain, which tells branch runs apart from production runs in the expectations audit table.

Tests

• BigQueryBranchExpectationsSpec (transform): branch run checks prj.sl_branch.order_summary, non-branch run checks prj.kpi.order_summary.
• BigQueryBranchLoadExpectationsSpec (load): native load on a branch checks prj.sl_branch.orders; native load without a branch and Spark load on a branch check sales.orders.
• Both specs record the SQL handed to the assertion handler, no BigQuery access needed. Both failed before the fix on the branch cases only.
• testOnly ai.starlake.job.ingest.* ai.starlake.job.transform.*: 101 passed. scalafmtCheckAll clean.
• Not run: the live BigQueryBranchSpec (SL_REMOTE_TEST).

Suggested changelog entry (1.8.14)

Bug fix:

• Expectations on a BigQuery data branch check the branch copy: with SL_DATA_BRANCH set, native BigQuery transforms and loads wrote to the branch dataset but ran their expectations against the production table. They now check the table the job actually wrote.

Follow-up

starlake-ai/starlake-docs#45 lists this as a limitation; update that line once this ships.

🤖 Generated with Claude Code