Repository navigation
Conversation
Sashiko review — v2Commit 2/9 —
|
|
FYI @alistair23 |
Describe a review step that judges a patch against the history of the code it changes. A deterministic collector reads the commits that wrote the modified and deleted lines, the commits that introduced identifiers the patch removes, and the commits that later fixed or reverted them, plus the lore threads where those commits were reviewed, using a per-project profile for which paths count as code and where reviews happen. QEMU runs the history stage on every review. Its concerns must name the historical commits and the historical fact they rely on; the commits are grounded in the collected history and carried through verification to the report. Post-verification also stops repeating findings verification already validated. Signed-off-by: Chao Liu <chao.liu@processmission.com>
Add code_history, which reads, as of the parent of the target commit, the git history of the code a patch changes: the commits that last wrote the modified and deleted lines, the commits that introduced identifiers the patch removes, and the commits that later fixed or reverted any of them. The commits are ranked by how directly they explain the changed code and rendered with their messages and diffs in the modified files, within fixed size, command count, and time bounds. Series commits are excluded and only object names reach git as revisions. With the server feature, it also fetches the review threads of the most relevant commits through the existing lore client, by Message-ID or Link trailer and by stable patch ID, which also finds earlier versions. It keeps postings, cover letters, and reply prose about each commit, parses mail off the async runtime, caches a thread only when every request behind it succeeded, and escapes tag-shaped text in commits and mail before it reaches a prompt. Which paths count as code and where reviews happen come from a per-project HistoryProfile. Only QEMU has one so far. Signed-off-by: Chao Liu <chao.liu@processmission.com>
Add a history analysis stage that runs on every QEMU review, beside goal, implementation, and execution-flow. Before the review, the worker collects the code history of the target commit, and the stage judges the change against it as a long-term maintainer would: does it remove a safeguard whose reason still holds, repeat a reverted approach, leave earlier occurrences of a fixed defect behind, or break a compatibility or specification constraint recorded in history? The stage is left out of the run when nothing was collected, and in cheap mode it runs beside the combined stage. Every history concern must name a history category, the historical fact it relies on, and the historical commits it cites. The cited commits are grounded in the collected history, taking SHAs and subjects from git and flagging commits the history did not show, so verification can route those to post-verification, which reads each cited commit. The grounded commits are carried onto inconclusive items, findings, and pre-existing findings from their source concerns only, and the report names them. Concerns about Fixes trailers, commit messages, comments, tests, or style stay with the other stages. The QEMU false-positive guide gains the history-specific cases. Signed-off-by: Chao Liu <chao.liu@processmission.com>
Post-verification batches run after verification and see neither each other nor the findings verification validated directly, so a batch could confirm, under a new ID, a defect already reported; on a real RISC-V commit one vmstate defect was reported three times. List the validated findings, with their pre-existing status, in the post-verification prompt and have it dismiss a candidate with the same root cause and status and report only the mechanism it was asked to verify. Have verification classify each root cause once, never into both findings and inconclusive items. Signed-off-by: Chao Liu <chao.liu@processmission.com>
8b1b2a5 to
72dff34
Compare
|
Thanks. All findings of the v2 review are valid and fixed |
Sashiko review — v3Commit 2/4 —
|
The patch ID path must tell a cut diff from a complete one, and run_git drops the cap flag. Signed-off-by: Chao Liu <chao.liu@processmission.com>
A capped capture is cut at the last newline below MAX_GIT_STDOUT, so a length check misses it and hashes a partial diff into a patch ID that matches nothing or the wrong thread. Signed-off-by: Chao Liu <chao.liu@processmission.com>
The negative case alone would stay green if commit_patch_id returned None for every diff. Signed-off-by: Chao Liu <chao.liu@processmission.com>
Sashiko review — v4Commit 2/7 —
|
Hunks and identifiers from docs, tests, and build files used up the blame and probe budgets. A diff without file names made git grep, git log -S, and git show run without pathspecs. Signed-off-by: Chao Liu <chao.liu@processmission.com>
|
The v4 findings are fixed in c634172. The collector filters the diff by HistoryProfile::is_code_path before extracting blame targets and removed identifiers, so skipped paths no longer consume the blame hunk or identifier probe budgets. The pathspec list is seeded from the blame target pre-image paths when a diff has no git headers, so git grep, git log -S, and git show no longer run without pathspecs. Tests cover git-header filtering, headerless scoping, and leading headerless sections. |
Sashiko review — v5✓ No issues found across 8 commits. |
|
This is a cool idea! Reading the test results though, does this actually catch any issues that wouldn't otherwise be detected? |
I'm testing more. |
SummaryI ran this branch against a QEMU series that the baseline had already reviewed, with the same command and settings, and compared the findings. Tested series
Baseline: sashiko 0.9.1. This PR: head The Test results
Findings from the history stage
Patch 1/26, the strongest result. Patch 27, history-only. Patch 29, severity raised. Patch 28, evidence added. |
Summary
I added a more global review step to the QEMU review: it retrieves the
history and past review discussions of the source code a patch touches,
and uses them to judge whether the patch is sound.
The new
historystage runs on every QEMU review, alongside goal,implementation, and execution-flow. Before the review,
src/workflows/code_history.rscollects the git history of the modifiedcode (the commits that wrote the changed lines and the commits that later
fixed or reverted them) and, with the
serverfeature, the lore threadswhere those commits were reviewed. The stage checks whether the patch
removes a safeguard, repeats a reverted change, or breaks a constraint
recorded in that history. Each history concern must cite the historical
commits it relies on.
The collector picks paths and mail sources from a per-project profile;
only QEMU has one for now. The series also stops QEMU post-verification
from reporting a finding that verification already validated.
Design:
designs/DESIGN_CODE_HISTORY_REVIEW.md.Test results
Six QEMU RISC-V commits that were later fixed or reverted, each reviewed
three times with and without the history stage, in a repository that
does not contain the later fix.
Detection is the same. With history, the MPV/GVA regression is rated
High in every run instead of Medium, one root cause is reported once
instead of being split, and findings name the historical commits they
rest on.