Give store paths a type in the Perl, and strip the store dir at the edges - #1872
Open
Ericson2314 wants to merge 1 commit into
Open
Give store paths a type in the Perl, and strip the store dir at the edges#1872Ericson2314 wants to merge 1 commit into
Ericson2314 wants to merge 1 commit into
Conversation
…dges
Nix and the Rust parts of Hydra previously switched from using raw
strings to a dedicated `StorePath` type. Now the Perl part makes the
same switch.
This change does make the Perl code longer, but I think the clarity is
well worth the verbosity.
Misc implementation notes:
- A new Perl "type" (blessed ref): `Nix::StorePath` is a store path base
name: a bare `<hash>-<name>`, no store directory, just like in Rust
and C++.
- The bindings speak it in both directions via an XS typemap, so no
`char *` store paths remain, and no
`nix::parseStorePath`/`nix::printStorePath` calls.
- The database still holds full paths, so the conversion happens where a
row is read or written, via `Hydra::Component::InflateStorePath`.
(`hydra-queue-runner` also converts at the SQL boundary.) This is done
via new `Hydra::Schema::_storeDir` state (with a getter/setter
wrapper).
- There are new `printStorePath` and `parseStorePath` functions in
*Perl*, because we don't want observable behavior like the web UI and
JSON API to change. Now, with `StorePath` sans store dir everywhere
internally, we have to be careful to use those functions in all the
boundary code.
- The JSON API keeps emitting full paths without extra work:
`Hydra::Component::ToJSON` reads columns with `get_column`, which is
the raw database value and so bypasses inflation entirely. Noted there
to prevent accidental breakage.
- `BuildProducts.path` is not a store path but a path underneath one, so
it inflates to a `Hydra::RelativeStorePath` pair with an accessor for
each half. The column wants to be two columns, and a migration to
make it two would not touch a caller.
- Two `DBIx::Class` details to note:
- deflation only runs on references, so StorePath being a blessed ref
and not a string isn't merely cosmetic but load-bearing.
- `search`/`find` conditions are never deflated, so a query against
one of these columns needs to print by hand.
- `$MACHINE_LOCAL_STORE` becomes a `machineLocalStore()` thunk, opened
on first use rather than while `Hydra::Helper::Nix` is being
"compiled". This is better in general because connecting to a daemon is
such a non-trivial side effect. And it is better in particular because
`Hydra::Schema::storeDir` reaches for `Hydra::Helper::Nix` itself, so
eagerly opening would mean that merely asking where the store *is*
opens one.
- For the tests, with our non-default store directories, it is extra
important that we do not prematurely open either the wrong store, or
the right store before the directories it needs exist. So the harness
uses `Hydra::Schema::storeDir` in setter mode to say which store the
columns belong to, and loading the schema then has no side effects at
all. (The test stores will then only be opened later, when they were
before, and when they are actually needed.)
Some things this turned up on the way, none of them new:
- `common.tt` stripped the store directory with `substr(11)`, which
would have eaten eleven characters of the hash the moment the value
stopped being a full path.
- `NixManifest.pm` and `S3Backup.pm` called `computeFSClosure` and
`queryPathInfo` as free functions, which `@EXPORT` never provided;
those paths could not have run.
- `Root.pm`'s `log` route rebuilt a full path from a URL segment with
a hardcoded `/nix/store`. There is now no hardcoded store directory
left in the Perl at all.
- `resolved.t` derived a store directory with `dirname` because it had
no better way to reach one; it now writes store paths and lets
deflation supply it. `content-addressed/basic.t` asked in a comment
for a way to query Nix for its store dir, and no longer needs one.
- `Hydra::Model::DB` computes `connect_info` in a `->config` call at the
top level, so merely `use`-ing anything that reaches it pins the
database before a test has set `$HYDRA_DATABASE_URL`. A test that
wanted `printStorePath` and reached for `Hydra::Helper::Nix` to get it
was enough to point the whole Catalyst app at the developer's own
database. Nothing here fixes that; the tests just take care to load
such modules at runtime, with the environment already in scope.
Assisted-by: Claude Code (Opus 5)
Ericson2314
force-pushed
the
perl-store-path
branch
from
August 25, 2026 14:42
47474a0 to
c6b6d41
Compare
dasJ
reviewed
Aug 25, 2026
dasJ
left a comment
Member
There was a problem hiding this comment.
From what I have seen (skimmed most of it, skipped the tests), I agree with the concept and what I have read looks good enough. After a rough sanity check, we might just be able to roll this without having to implement tests for every single place this touches
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nix and the Rust parts of Hydra previously switched from using raw strings to a dedicated
StorePathtype. Now the Perl part makes the same switch.This change does make the Perl code longer, but I think the clarity is well worth the verbosity.
Misc implementation notes:
A new Perl "type" (blessed ref):
Nix::StorePathis a store path base name: a bare<hash>-<name>, no store directory, just like in Rust and C++.The bindings speak it in both directions via an XS typemap, so no
char *store paths remain, and nonix::parseStorePath/nix::printStorePathcalls.The database still holds full paths, so the conversion happens where a row is read or written, via
Hydra::Component::InflateStorePath. (hydra-queue-runneralso converts at the SQL boundary.) This is done via newHydra::Schema::_storeDirstate (with a getter/setter wrapper).There are new
printStorePathandparseStorePathfunctions in Perl, because we don't want observable behavior like the web UI and JSON API to change. Now, withStorePathsans store dir everywhere internally, we have to be careful to use those functions in all the boundary code.The JSON API keeps emitting full paths without extra work:
Hydra::Component::ToJSONreads columns withget_column, which is the raw database value and so bypasses inflation entirely. Noted there to prevent accidental breakage.BuildProducts.pathis not a store path but a path underneath one, so it inflates to aHydra::RelativeStorePathpair with an accessor for each half. The column wants to be two columns, and a migration to make it two would not touch a caller.Two
DBIx::Classdetails to note:deflation only runs on references, so StorePath being a blessed ref and not a string isn't merely cosmetic but load-bearing.
search/findconditions are never deflated, so a query against one of these columns needs to print by hand.$MACHINE_LOCAL_STOREbecomes amachineLocalStore()thunk, opened on first use rather than whileHydra::Helper::Nixis being "compiled". This is better in general because connecting to a daemon is such a non-trivial side effect. And it is better in particular becauseHydra::Schema::storeDirreaches forHydra::Helper::Nixitself, so eagerly opening would mean that merely asking where the store is opens one.For the tests, with our non-default store directories, it is extra important that we do not prematurely open either the wrong store, or the right store before the directories it needs exist. So the harness uses
Hydra::Schema::storeDirin setter mode to say which store the columns belong to, and loading the schema then has no side effects at all. (The test stores will then only be opened later, when they were before, and when they are actually needed.)Some things this turned up on the way, none of them new:
common.ttstripped the store directory withsubstr(11), which would have eaten eleven characters of the hash the moment the value stopped being a full path.NixManifest.pmandS3Backup.pmcalledcomputeFSClosureandqueryPathInfoas free functions, which@EXPORTnever provided; those paths could not have run.Root.pm'slogroute rebuilt a full path from a URL segment with a hardcoded/nix/store. There is now no hardcoded store directory left in the Perl at all.resolved.tderived a store directory withdirnamebecause it had no better way to reach one; it now writes store paths and lets deflation supply it.content-addressed/basic.tasked in a comment for a way to query Nix for its store dir, and no longer needs one.Hydra::Model::DBcomputesconnect_infoin a->configcall at the top level, so merelyuse-ing anything that reaches it pins the database before a test has set$HYDRA_DATABASE_URL. A test that wantedprintStorePathand reached forHydra::Helper::Nixto get it was enough to point the whole Catalyst app at the developer's own database. Nothing here fixes that; the tests just take care to load such modules at runtime, with the environment already in scope.