You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 17a74a6
Browse filesBrowse the repository at this point in the historyBrowse files
docs: stop teaching two commands that do not work (#257)
* docs: 2.0.0 release readiness
Audit of everything a 2.0.0 final would ship, and the fixes that did not
need lib/ changes.
CHANGELOG
- A v2.0.0 entry written for someone upgrading from 1.15.1, not a diff of
the betas. What to change (the version), what they will see (exact
warning text), the five things that can actually break, and what 2.1
removes. Every claim verified against a real install; the beta sections
stay as history.
Version consistency
- README, docs/UPGRADING.md: no more "beta"/"alpha"/"experiment" framing
and no beta pins. Gemfile examples say `~> 2.0`.
- Gem name: `capybara-screenshot-diff` is the one we tell people to
install; `snap_diff-capybara` is a reserved identical mirror. Stated
once in the README with the dual-install consequence, applied
everywhere else.
- Stale "3.0" references in the Rakefile and docs/architecture.md are now
2.1 (#247 fixed the user docs and missed these).
Corrections to claims that were not true
- docs/drivers.md promised that everything 2.1 removes "warns once per
process naming 2.1". `driver: :auto` is silent whenever ruby-vips is
present, and the `driver:` setting itself never warns at all even
though 2.1 deletes it (`NoMethodError`). Both are now written down as
silent, in drivers.md and UPGRADING.md, since a note is the only notice
they can get.
- README called ruby-vips "Optional". With neither ruby-vips nor
chunky_png installed, comparisons raise
`Wrong adapter nil. Available adapters: []`. Says so now.
- Setup examples no longer teach `driver: :vips`, a line users have to
delete for 2.1.
Gem hygiene
- gemspec: summary/description that describe what the gem does, the
rubygems metadata links (source, changelog, bug tracker, docs), and
docs/docker-testing.md dropped from the package (it documents
bin/dtest, which is not packaged). Dead bindir/executables removed --
the allow-list never matched exe/.
- README's links to CONTRIBUTING.md and docker-testing.md are absolute,
so they resolve from inside the gem too.
- test/unit/gemspec_packaging_test.rb pins the packaged file list: both
Bundler.require entry files present (this broke twice), consumer docs
in, contributor docs and build files out, capybara the only runtime
dependency. Verified it fails when an entry file is unpackaged.
- *.gem is gitignored.
Release process
- The GitHub Release body linked to blob/main on a repo whose default
branch is master -- 404 on every release so far. Links to the tag now.
- docs/RELEASE_PREP.md was a stale v1.15.1 checklist. It is now a runbook
for how releases actually happen: what the workflow does step by step,
the trusted-publisher prerequisite for BOTH gem names, prereleases,
post-release verification, and what to do when a run fails halfway.
- CONTRIBUTING.md pointed at the wrong version.rb and recommended
`rake release`, which publishes only one of the two gem names.
Verified with real installs on ruby 4.0.6: 1.15.1 -> this master via
path:, a canonical-names setup, `Bundler.require` under each gem name
from the built .gem, and the dual-install guard with both gems installed.
No lib/ changes. Version not bumped.
* docs: SnapDiff::Error is the base class for errors the gem defines, not every error it raises
docs/snapdiff.md's object map said "Base class for every error this gem
raises". It is not: a missing image backend raises a bare RuntimeError
("Wrong adapter nil. Available adapters: []", reproduced on a bundle with
neither ruby-vips nor chunky_png) and StableScreenshoter raises
ArgumentError. Verified the four defined errors -- ExpectationNotMet,
UnstableImage, WindowSizeMismatchError, DualInstallError -- do all inherit
SnapDiff::Error, so the useful half of the promise holds and is now the
one being made.
* docs: tag protection, not branch protection, is what gates the release tag push
Branch protection rules do not govern tag pushes; tag protection rules
(or rulesets) do. The runbook prerequisite now names the right control.
* docs: the per-screenshot driver: override dies quietly, the config setting dies loudly
Both were lumped together as "raises NoMethodError on 2.1". Only the config
setting does. Per-screenshot options are a free-form hash, so on 2.1
`screenshot "index", driver: :vips` is inert and nothing tells you the line
is dead -- #249's own upgrade note spells out the split. Grep-for-it advice
added, since that is the only signal a user gets.
* docs: pin versions, and stop telling people to delete baselines
Second pass, from customer-persona findings. Each verified here before
acting; two of the four reported items turned out to be artifacts of the
PUBLISHED beta3 rather than of master, and are handled as such.
Baselines (the oldest bug in the tracker: #5 and #6 in 2018, #133 in 2024)
- README told users to "delete the baseline and re-run" in two places.
It cannot work. `Vcs.checkout_vcs` (lib/snap_diff/vcs.rb:24) resolves
every baseline with `git show HEAD:<path>`, and
`ScreenshotMatcher#check_base_screenshot` calls it before
`need_to_compare?` tests `base_path.exist?` -- so a committed baseline is
fetched from HEAD no matter what the working tree says, and `rm` changes
nothing.
- New first-class "Accepting an intentional change" section: the mechanism,
the commit that actually accepts it, and the surprising part -- staging is
not enough, so no local run goes green until you commit. The FAQ answer
now says the same thing instead of the opposite.
- Deliberately does NOT document RECORD_SCREENSHOTS. It is printed by our
own error message (screenshot_matcher.rb:73) but read nowhere in lib/;
a separate lane is implementing it, and it should be documented once it
works, not before.
Version pinning
- `gem "snap_diff-capybara"` unpinned installs 0.0.1 -- a placeholder whose
entire payload is one README and zero Ruby files (verified by fetching and
unpacking it), so the user gets an immediate LoadError. And unpinned
`gem "capybara-screenshot-diff"` resolves to 1.15.1, not to the 2.0 the
surrounding prose is selling. Every install instruction now pins, and the
README says plainly that the mirror name is not the one to reach for.
CHANGELOG, all verified
- Failure messages leaked a libvips pointer struct via the comparison
metadata; `to_h` excludes `diff_mask` since #234, which landed after the
beta3 tag, so 2.0.0 final is the fix.
- Known limitation: fork-parallel runs write no HTML report. Workers
accumulate assertions per process; the report is written from
`Minitest.after_run` in the parent (integrations/minitest.rb:69), which
never sees them. Artifacts and pass/fail are unaffected.
- A note for anyone sitting on a prerelease: beta3's deprecation channel was
incomplete, so its silence is not evidence of being migrated.
Constants
- `Capybara::Screenshot::Os` -> `SnapDiff::Os` was in no rename table.
Gemspec
- rubygems_mfa_required. The four URI fields were added in the first commit.
* docs: stop teaching two commands that do not work
Two customer personas independently followed the docs and got a green bar
on a page they had deliberately broken.
`rake test` does not run `test/system/` in a Rails app. The Quick Start
told users to run it, so step 1 produced `0 runs` and no baselines --
which reads as a pass. The example is a Rails system test; the command
now matches it, with a callout, because "0 runs" is the single easiest
way to believe visual testing is working when nothing is running.
`RECORD_SCREENSHOTS=1` appeared in three user-facing docs for a feature
that has never existed in `lib/` -- it is this repository's own test-suite
convention, read by `test/test_helper.rb`. The user-facing copies are
replaced with the flow that actually works: run the suite, which rewrites
every changed baseline in place, then `git add` and commit. The
contributor page keeps it and now says plainly that it is not a library
feature.
* docs: make the Quick Start something a new user can actually run
A reviewer built a stock Rails 8.1 app, ran the documented Quick Start
end to end, and none of it worked. Fixes, each verified against a scratch
app or against lib/:
- `gem "capybara-screenshot-diff", "~> 2.0"` does not resolve. rubygems
has 1.15.1 and 2.0.0.alpha1..beta3 and no final 2.x, and Bundler never
picks a prerelease from a plain requirement, so `bundle install` fails
with `Could not find gem 'capybara-screenshot-diff (~> 2.0)'`. All five
install snippets now pin `2.0.0.beta3` and say why. RELEASE_PREP gains
the step that swaps them back to `~> 2.0` as part of the 2.0.0 push,
so the good pin lands with the release rather than before it.
- The Quick Start taught the API 2.1 deletes, silently. `require
"capybara_screenshot_diff/minitest"` + `include
CapybaraScreenshotDiff::Minitest::Assertions` print nothing: they are
eager aliases, so `const_missing` never fires. The Quick Start now
starts on canonical `SnapDiff`, and README/CHANGELOG/snapdiff.md say
which doors actually warn (config accessors, `include`,
`default_options`, `const_missing`) and which cannot.
- The CI "Record new baselines" job could not record a new baseline.
`check_base_screenshot` runs before `capture_screenshot` and
`fail_if_new` is true whenever `ENV["CI"]` is set, so a new screenshot
raises before anything is written and the commit step finds nothing.
The job now clears `CI` for that step.
- `bundle exec rake test` / `rails test` swept out of the three CI
workflows, "The Short Version", and the historical upgrade sections:
in a Rails app they skip `test/system/` and report `0 runs`.
- The delete-the-baselines block in UPGRADING replaced with the commit
workflow the README documents.
- `docs/drivers.md` told you to delete the `driver:` setting on one
screen and to add it on another; same for configuration.md,
migration-guide.md.
- `rescue SnapDiff::Error` does not catch a failed assertion under the
framework integrations -- Minitest converts it to `Minitest::Assertion`
and RSpec to `ExpectationNotMetError`. Said so.
- "2.0 will not rewrite a baseline you already committed" contradicted
the README and reality: a failing run does rewrite the baseline path.
Reworded to what is meant (no re-encoding) plus what actually happens.
Also, all verified in a scratch app: the `git add
test/fixtures/screenshots/` path was never the default (`doc/screenshots`
is); the example failure output showed a `max_color_distance` key the
vips path never emits; the artifact table listed three of five files;
`application_system_test_case.rb` omitted both `require "test_helper"`
and `driven_by` (without the latter the same page captures at 2800x1610
instead of 1400x1257); `homepage_test.rb` omitted `require
"application_system_test_case"` and raised NameError as printed; and
`DEBUG=1` never had anything to do with keeping `.diff.png` files.
Docs only. rake test:unit 610/0, standardrb clean.
* docs: three residuals from the verifier pass
- bug_report template told reporters to run `rake test`, which runs zero
system tests in a Rails app -- the same trap this branch exists to remove.
- drivers.md said an unknown per-screenshot `driver:` key is "simply inert".
It is validated and raises; only the deprecation warning is absent.
- drivers.md said `:auto` and `:chunky_png` each warn once per process.
`:auto` is silent when ruby-vips resolves; only `:chunky_png` warns.
4.**Stubbing `Capybara::Screenshot::Diff::AVAILABLE_DRIVERS` no longer works.** It
97
+
is an eager alias; the gem reads `SnapDiff::Drivers::AVAILABLE_DRIVERS`. Stubbing
98
+
the legacy name rebinds the alias only, so the test passes for the wrong reason.
99
+
5.**`SnapDiff::Config::MAPPING` is gone**, split into `SnapDiff::Config::SETTINGS`
100
+
(setting names) and the `@api private``SnapDiff::LegacyShims::CONFIG_MAPPING`.
101
+
102
+
### Act now: what 2.1 removes
103
+
104
+
2.1 deletes the v1 namespace trees, the ChunkyPNG driver, `shift_distance_limit`,
105
+
and the whole driver abstraction — **libvips becomes the only backend**. Removing
106
+
public API in a minor is a deliberate departure from strict semver; publishing the
107
+
contract one release ahead is the mitigation. 2.0 warns once per process for each of:
108
+
109
+
| You will hear about it when you… | Do this in 2.0 |
110
+
|---|---|
111
+
| select `driver: :chunky_png`| add `gem "ruby-vips"` and drop the option |
112
+
| run `driver: :auto`**without ruby-vips** — nothing in your setup says `chunky_png`, so this warning is the only sign 2.1 will break the process | install libvips + `ruby-vips`|
113
+
| set `shift_distance_limit`| use `median_filter_window_size`, `tolerance` or `color_distance_limit`|
114
+
| read `SnapDiff::Drivers.loaded` / `.available`| require `ruby-vips` instead of branching on a detected list |
115
+
|`include SnapDiff::Driver` in your own driver | nothing — custom drivers have no migration path |
116
+
117
+
Two removals 2.0 cannot warn about, so they are written down instead: **`driver:`
118
+
as a setting goes away entirely** — `SnapDiff.config.driver = :vips` and
119
+
`Capybara::Screenshot::Diff.driver = :vips` raise `NoMethodError` on 2.1, and the
120
+
per-screenshot `screenshot "x", driver: :vips` is silently ignored there. Delete
121
+
both; one backend needs no selection. The legacy `LOADED_DRIVERS` /
122
+
`AVAILABLE_DRIVERS` constants are also plain aliases with nothing to hook.
123
+
124
+
### Added
125
+
-**`SnapDiff` is the canonical namespace** — the implementation lives in
126
+
`lib/snap_diff/`. Configuration (`SnapDiff::Config`, one storage behind every
127
+
settings surface), errors (`SnapDiff::Error` and friends), `SnapDiff::Region`,
0 commit comments