test(agent): pin image-space coord-click bucketing under screenshot downscaling - #2821
Conversation
…ownscaling Review feedback on webbrain-one#2813 showed the screenshot click scale is always >= 1 (cssW / shrunk.width, set only when a resize happened), so converting image-pixel taps to CSS pixels magnifies 2-4px nudges out of the fixed 5px bucket. The detector must stay in image space — the space the model reasons in. Pins both sides: the image-space taps accumulate to a nudge, and the same taps converted at a realistic scale spread across buckets.
|
@alectimison-maker is attempting to deploy a commit to the esokullu's projects Team on Vercel. A member of the Team first needs to authorize it. |
webbrain-one
left a comment
webbrain-one
left a comment
There was a problem hiding this comment.
The added test does not yet protect the advertised coordinate-space invariant. Please exercise the Agent click path that owns screenshot scaling.
| assert.equal(d._checkCoordClickLoop(1, 115, 200).kind, 'none'); | ||
| }); | ||
|
|
||
| test('coord click: buckets in image space even when a downscale factor is stored', () => { |
There was a problem hiding this comment.
[P2] Exercise the scaled-click call path in this regression test
The test feeds already-image-space values directly to LoopDetector and separately computes CSS buckets; it never stores a screenshot scale or invokes the Agent path that converts and dispatches click coordinates. A regression that forwards converted CSS coordinates to _checkCoordClickLoop would still pass. Exercise the actual click path with from_screenshot: true and a stored scale, or spy on the detector and assert that the original image-space arguments reach it.
3 participants
Summary
Follow-up to #2813 (closed after review). Adds one test pinning the intended behavior:
_checkCoordClickLoopbuckets image-space coordinates — the space the model actually reasons in — because the screenshot click scale is always ≥ 1 (cssW / shrunk.width, set only when a resize happened), so converting to CSS pixels would magnify 2-4px image nudges out of the fixed 5px bucket and let a dead-button loop burn the whole step budget.The test asserts both sides: image-space taps within a 5px bucket accumulate to a nudge on the 5th repeat, and the same taps converted at a realistic scale (2.449) spread across multiple CSS buckets.
Testing
node test/run.js— 1772 passed, 0 failed