fix(deps): update rust crate rand to 0.10.1 [security]#33
Closed
bootc-bot[bot] wants to merge 477 commits into
Closed
fix(deps): update rust crate rand to 0.10.1 [security]#33bootc-bot[bot] wants to merge 477 commits into
bootc-bot[bot] wants to merge 477 commits into
Conversation
This is a very old assert added during development and probably never intended to remain. Sometimes containers will be in non-ustar tar files and that's OK. In fact, the Fedora bootc image is one such example. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
So far we've been testing with local podman builds, which don't produce these by default, but we need to support these, as they appear in some bootc images. It's a bit tricky because our root directory isn't an Inode, so add a special path. While we're at it, add some error context to the path where this was failing before. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Instead of having a large and growing list of functions which perform
any number of possibly desired image flows:
- from:
- OCI container
- filesystem
- with or without root directory stat
- selinux relabel (yes/no)
- output:
- create an image
- compute an image checksum
- print a dumpfile
Add a function for each source (oci, fs) for creating a FileSystem and
define some new high-level transformations and operations on the
FileSystem object itself.
These operations sort of depend on everything whereas the rest of the
code in tree.rs depends on almost nothing, so it feels a bit weird to
include them in that file. I tried some other approaches here:
- define a bunch of functions that take a FileSystem as their first
argument and operate on it. This had bad ergonomics because they
essentially class methods, and it wasn't able to invoke them as such.
- move FileSystem into a separate file and define the methods there.
This was strange because lower-level parts of the code still
had to create a FileSystem object, so they ended up depending on the
higher-level file again.
- define a "FileSystemOps" helper trait which is implemented only for
FileSystem: this was probably the cleanest approach and had the
advantage of requiring the trait to be in scope in order to perform
the high-level operations. The only reason I dismissed this one was
because it was pointlessly complicated and required writing the
method signatures twice.
In the end we just make use of the fact that it's possible to have
multiple `impl` blocks for a given `struct` as long as they're in the
same crate. The "trait" approach might be a bit cleaner, but this works
as well.
Adjust various API users to make use of the new operations.
Overhaul the `cfsctl` commandline to be more consistent about the
offered operations and the flags that get passed to them. Update the
examples as appropriate.
Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Rename a couple of things for consistency and clarity. Add a dirfd parameter to the read-from-directory code. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
We'll want to use this for non-UKI cases soon, as well. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Fix a longstanding TODO in composefs-setup-root for properly dealing with quoting in cmdline arguments by adding a helper function in a separate file (we'll use that elsewhere soon, as well). Also: start using String a bit more. I think I've been avoiding it as inefficient but now that I understand more about how it works (particularly how the length and indices are in byte offsets, not character offsets), I have a lot less reason to avoid it. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
We're going to want to use this soon in order to verify the correctness of the composefs= cmdline parameter when installing images. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Drop the hidden "meta" layer approach where we try to preserve the image without the boot resources and move to a model where we explicitly delete the /boot directory if it exists. This makes creating images a lot cleaner and more natural, shown by the cleanup in the examples. At the same time we also add support for finding resources from the locations that they're present in bootc images: /usr/lib/modules in versioned subdirectories with vmlinuz and initramfs.img. Add some example "simple" writer code that can write the boot resources to a given directory. We deal with the /usr/lib/modules cases by converting them into Type #1 boot entries and writing them as such. Other consumers of this API might want to do other things, such as producing grub entries, etc. Overhaul the cfsctl oci prepare-boot command to use the new stuff. Fixes #35 Fixes #21 Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Move the printing of SHA256 and verity from the library to cfsctl binary. Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
This is like https://docs.rs/cap-std-ext/latest/cap_std_ext/dirext/trait.CapStdExtDirExt.html#tymethod.open_dir_optional etc. The rationale here is that it's really common for filesystem operations to want to handle "not found" and that maps most cleanly to `Option`. We only have one use right now but there could easily be more. Signed-off-by: Colin Walters <walters@verbum.org>
Co-authored-by: Gemini Code Assist (The code didn't compile at first but the changes were minor, some of the unit tests were plausible but didn't actually pass not due to bugs in the code, so I just deleted them) Signed-off-by: Colin Walters <walters@verbum.org>
Instead of parsing ostree's toml config, we will have and parse composefs-rs config located at `/usr/lib/composefs/setup-root-conf.toml` Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Newer versions of dracut throw thousands of errors when trying to copy files into the initramfs in some container setups, and in some cases of symlinks can fail to include files entirely. This has led to a long-lingering problem that makes it difficult to build images locally (despite them working in CI). Here's the workaround: DRACUT_NO_XATTR=1. Thanks to Dick Marinus for that. See https://discussion.fedoraproject.org/t/146603/3 This lets us finally increase our Fedora version in the examples to 42. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Split into a few separate crates:
- libraries:
- composefs
- composefs-oci
- composefs-boot
- binaries:
- cfsctl
- composefs-setup-root
- erofs-debug
Move our lint config (which only forbids missing debug impls) to the
workspace level and have all crates inherit from that.
Add a new workflow for testing that we can `cargo package` everything.
We need a nightly cargo in order to do this with workspaces containing
inter-dependent crates: rust-lang/cargo#13947
Make 'oci' an optional feature of cfsctl, but enable it by default.
Adjust our rawhide bls example (which included --no-default-features) to
*not* disable that.
This is not a huge improvement in terms of compile speed, and it has
some drawbacks (like 'cargo run' no longer defaulting to cfsctl) but it
seems like the right step at this point. I want to start to add some
more experimental code without making it part of the main crate.
Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Recent changes in the 6.15-rc kernel series have broken composefs-setup-root when running without using the newest features of the mount API (ie: with the pre-6.15 feature left enabled, which is the default). Add a case to the examples for testing that. Once it gets working again it would be useful to make sure it continues to work. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
We need to add a couple of exceptions in order to make ioctls. Hopefully we can remove those and set this to forbid, but for now it's a pretty strong statement going forward. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
This is the suggested config from samply. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Copy the config from bootc. This results in an impressive (> 50%) amount of savings. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
We need these for the (soon to arrive) FUSE backend and for mutating the filesystem tree for flatpak images. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
This returns the same type of FsVerityHashValue as is used by the Repository that it's passed. Update the signature to reflect that. Fixes #129 Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
This allows us to mount a FUSE filesystem from the in-memory filesystem tree. That's useful because using erofs requires CAP_SYS_ADMIN in the root namespace, which we often won't have. This is not particularly complete. It's a sort of minimal-effort first implementation, but already basically works. It's also not particularly efficient. In particular, it doesn't use splice or passthrough fds. Passthrough fds also currently require root (although the FUSE developers are working on easing that restriction) and although the Rust binding layer for FUSE recently gained passthrough fd support, it hasn't been related yet. Splice support is also missing from the Rust binding layer. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Grub needs absolute paths to initrd and vmlinuz if we do not have `/boot` in a boot partition, which we do not in bootc. Add param `boot_subdir` which acts like a subdirectory in the boot directory in case the boot partition is mounted in another directory. Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Reviving the more sane part of #120. We should provide a clearer error variant here instead of just passing ETXTBSY through to the caller. Signed-off-by: John Eckersberg <jeckersb@redhat.com>
We're leaking fds into the sandbox in flatpak-rs. Let's fix these obvious ones. Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
It turns out that the information contained in splitstreams to assist
with garbage collection (ie: the list of things that we mustn't discard)
is exactly the required information for downloading (ie: the list of
things that we must acquire).
Use this fact to add support for fetching repository content from HTTP
servers. We only download the objects that are actually required, so
incremental pulls are very fast.
This works with just about any HTTP server, so you can do something like
python -m http.server -d ~/.var/lib/composefs
and download from that. With a fast enough web server on localhost,
pulling a complete image into an empty repository takes about as long as
pulling an `oci:` directory via skopeo with `cfsctl oci pull`.
In practice, this is intended to be used with a webserver which supports
static compression and pre-compressed objects stored on the server. In
particular, zstd support is enabled in the `reqwest` crate for this
reason, and it's working with something like:
find repo/objects/ -type f -name '*[0-9a-f]' -exec zstd -19 -v '{}' +
static-web-server -p 8888 --compression-static -d repo
There's also an included s3-uploader.py in the examples/ directory which
will upload a repository to an S3 bucket, with zstd compression.
Signed-off-by: Allison Karlitskaya <allison.karlitskaya@redhat.com>
Describe the current OCI storage model: naming conventions for manifest/config/layer/blob splitstreams, how tags map to refs under streams/refs/oci/, the named_ref chains (manifest→config+layers, config→layers), and how the GC walks from tags to objects. Also notes the current gap: EROFS images derived from OCI content are not referenced by any splitstream, so their lifecycle must be managed separately. Assisted-by: OpenCode (Claude claude-opus-4-6) Signed-off-by: Colin Walters <walters@verbum.org>
Add test utilities for creating multi-layer OCI images from composefs dumpfile strings. This uses the real dumpfile format parsed by dumpfile_to_filesystem(), then walks the resulting FileSystem tree to emit tar bytes for import_layer(). Two convenience builders with versioned boot content: - create_base_image: 5-layer busybox-like app image - create_bootable_image(version): 20-layer bootable OS with kernel and UKI v1 and v2 share userspace layers (busybox, libs, systemd, configs) but differ in kernel version (6.1.0 vs 6.2.0), initramfs, modules, and UKI. When both are pulled into the same repo the shared layers deduplicate, exercising GC correctness with content referenced by multiple images. Prep for adding boot image management API. Assisted-by: OpenCode (Claude claude-opus-4-6) Signed-off-by: Colin Walters <walters@verbum.org>
When pulling an OCI container image, automatically generate the composefs EROFS image and reference it from the config splitstream. This creates a clean GC chain: tag → manifest → config → EROFS image, so all derived artifacts stay alive as long as the tag exists. For bootable images, a second "boot" EROFS variant is generated with transform_for_boot applied (clearing /boot to avoid circular UKI refs). The old seal/mount mechanism is replaced by a simpler Mount subcommand that looks up the EROFS ref from the config splitstream. OpenConfig struct replaces the (config, layer_map) tuple for richer open_config returns. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
When the background object-storage task failed (e.g. EACCES on a read-only repo), stream_large_file() would break out of its content loop without consuming the remaining file bytes from the tar stream. The outer parser then tried to interpret those leftover content bytes as the next tar header, producing misleading 'invalid octal field' or 'checksum mismatch' errors. Both tx.send() failure paths now drop the sender, await the background handle to extract the actual error, and return it immediately. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Pulling into a read-only repository previously failed deep inside the tar splitting pipeline with confusing errors like 'invalid octal field' because background object-storage failures (EACCES) weren't propagated; leftover content bytes were misinterpreted as tar headers. Fix the error propagation in stream_large_file() so the real error surfaces, and add a faccessat(W_OK) pre-flight check via a new zero-sized WritableRepo proof token. The token is threaded through SplitStreamWriter and SplitStreamBuilder so that per-object writes skip redundant faccessat calls. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
We didn't have good converage of this before at the unit testing level. This builds on top of our prior dumpfile based test fixture. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Signed-off-by: bootc-bot[bot] <225049296+bootc-bot[bot]@users.noreply.github.com>
Refactor the SELinux labeling code to decouple policy loading from the composefs repository. The core change is splitting Policy::build into a generic Policy::build_from that accepts a file-open callback, and factoring the overall flow into build_policy + apply_policy phases. This enables a new selabel_from_dir() entry point that reads SELinux policy files directly from an on-disk filesystem via a directory fd, rather than requiring file objects to be stored in a repository first. Prep for allowing composefs digest computation without allocating tempfiles in the repository (needed on filesystems like fuse-overlayfs that don't support O_TMPFILE). Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Add a BootOps method that applies boot filesystem transformations (emptying /boot and /sysroot, SELinux relabeling) using a directory fd for reading file content instead of a composefs repository. Prep for allowing composefs digest computation without allocating tempfiles in the repository. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
The bare "Invalid string length" from hex::decode_to_slice gave no clue about what went wrong. Include the actual vs expected hex length and the algorithm name so a mismatch (e.g. SHA-512 hash in a SHA-256 repo) is immediately obvious. Also surface the UKI .cmdline contents when write_t2_simple fails. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
For the `cfsctl oci compute-id` case we were making full copies of the objects in a temporary repository - a completely unnecessary performance hit. Further that path expects `O_TMPFILE` which isn't supported everywhere. (In theory we should support non-O_TMPFILE filesystems too) Fixes: bootc-dev/bootc#1977 Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Add the ability to convert a FileSystem<T> to FileSystem<U> by mapping the regular file content type. This preserves Rc hardlink sharing: the mapping function is called exactly once per unique leaf, and all references to the same leaf produce references to the same mapped leaf. Prep for the async filesystem import, which scans the directory tree into a FileSystem<PendingFile> and then converts to the final FileSystem<RegularFile<ObjectID>> after parallel verity computation. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Previously, we had a lot of synchronous code for interacting with the local filesystem, but the tar processing used by OCI was async. This created a need to do "the same thing" in two different ways. But importing from a local filesystem is equally amenable to being async! Replacing that with a model where we read the filesystem metadata synchronously, but defer fsverity computation and object import to worker threads results in *dramatic* speedup for large filesystem trees. Computing the composefs digest of the 40G `target/` directory I have locally is 1m20s before this patch, and 8s after (32 cores, so a lot more CPU time used of course). Three optimized paths depending on context: - Secure repo: std::io::copy (uses copy_file_range for reflinks on CoW filesystems) then kernel fsverity enable + measure - Insecure repo: tee through FsVerityHasher while copying to tmpfile, computing the digest in a single pass - No repo: incremental FsVerityHasher from fd, one block at a time Also: the composefs-http ensure_object call is migrated to ensure_object_async which it should have been using in the first place! This is a clear advantage of having one way to do it. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
While some of our tests pull large full images, it's handy to have a fully reproducible self-contained, but also representative image. Extend our synthetic one with some large files - this is prep for testing the reflink/hardlink support for containers-storage imports. Add a security. xattr etc. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Remove unused Mode and mkdirat imports from splitstream test module, left behind by the edition 2024 migration. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Three improvements to the bootc revdep testing workflow: - Reject uncommitted changes in composefs-rs so tests always run against a real commit - Embed the composefs-rs commit hash in a Cargo.toml comment so podman's build cache is invalidated when the source changes (path deps don't record a revision in Cargo.lock) - Always re-run cargo update cfsctl on each invocation Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Add a new cstorage crate that provides read-only access to containers/storage (the backend used by Podman, Buildah, and CRI-O). This enables composefs to directly read OCI layers from local container stores without re-downloading them. The crate supports: - Overlay storage driver with layer chain resolution - Additional image stores (read-only stores) - Rootless storage via user namespace helpers - tar-split based layer reconstruction - JSON-RPC over Unix socket for user namespace operations - Container store locking (shared/exclusive) Also adds jsonrpc-fdpass as a workspace dependency for the Unix socket IPC used by the user namespace helper. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Add ObjectStoreMethod::Reflinked/Hardlinked variants and a new ImportContext that caches per-(src_dev, dst_dev) reflink support across bulk import operations. ensure_object_from_file() tries FICLONE first, falls back to hardlink (linking the source file directly into the objects directory after enabling fs-verity on it), and finally falls back to a regular data copy. This avoids data copying when importing from containers-storage on filesystems that support reflinks (btrfs, XFS) or even on ext4 via hardlinks. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Extend ImportStats with reflink/hardlink counters and byte totals, plus layer-level tracking (layers, layers_already_present). The Display impl now shows a detailed breakdown when zero-copy methods were used while preserving the existing compact format for copy-only imports. Add PullOptions struct to the pull() signature, preparing for the containers-storage import path which needs extra knobs (zerocopy mode, explicit storage root, additional image stores). Visibility changes (pub(crate) on helpers, pub on ContentAndVerity) prepare for the cstor module to reuse these internals. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Add four new integration tests exercising existing cfsctl functionality through the CLI: - test_oci_tag_and_untag: multi-tag and selective untag workflow - test_oci_gc_removes_untagged: verifies GC collects untagged images - test_layer_tar_roundtrip: imports a layer and verifies tar extraction - test_compute_image_id: deterministic fs-verity image ID computation Also fix create_oci_layout to include a runtime config (ConfigBuilder) which is required for the seal/compute-id operations. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Add podman, skopeo, and xfsprogs to test dependencies for the containers-storage integration tests coming next. Disambiguate the integration test binary name in Justfile cargo commands since the integration-tests crate will have multiple binaries (the main test runner and a cleanup helper). Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
Implement the containers-storage import path (cstor module) which can import OCI images directly from podman/buildah storage without going through skopeo, using reflinks or hardlinks to avoid data copies when the composefs repo is on the same filesystem. Assisted-by: OpenCode (Claude Opus 4) Signed-off-by: Colin Walters <walters@verbum.org>
The nextest config (.config/nextest.toml) was previously dead code — tests were run via the libtest-mimic binary directly. Switch the Justfile targets and CI workflow to prefer nextest when available, so the parallelism controls (threads-required = 2 for privileged VM tests) actually take effect and prevent OOM kills on 16 GB CI runners. Also fix the nextest filter syntax: ~^foo never matches because nextest's ~ operator treats ^ as a literal character. Use /regex/ syntax instead. Assisted-by: OpenCode (Claude Opus 4)
Signed-off-by: bootc-bot[bot] <225049296+bootc-bot[bot]@users.noreply.github.com>
…erability fix(deps): update rust crate fuser to 0.16.0 [security]
Signed-off-by: bootc-bot[bot] <225049296+bootc-bot[bot]@users.noreply.github.com>
Contributor
Author
Renovate Ignore NotificationBecause you closed this PR without merging, Renovate will ignore this update ( If you accidentally closed this PR, or if you changed your mind: rename this PR to get a fresh replacement PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
0.10.0→0.10.10.10.0→0.10.1Rand is unsound with a custom logger using rand::rng()
GHSA-cq8v-f236-94qc
More information
Details
It has been reported (by @lopopolo) that the
randlibrary is unsound (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:logandthread_rngfeatures are enabledrand::rng()(previouslyrand::thread_rng()) and calls anyTryRng(previouslyRngCore) methods onThreadRngThreadRng(attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)getrandomcrate) is unable to provide a new seedTryRng(previouslyRngCore) methods forThreadRnguseunsafecode to cast*mut BlockRng<ReseedingCore>to&mut BlockRng<ReseedingCore>. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of aliased mutable references is Undefined Behaviour, the behaviour of optimized builds is hard to predict.Severity
Low
References
This data is provided by the GitHub Advisory Database (CC-BY 4.0).
Release Notes
rust-random/rand (rand)
v0.10.1Compare Source
This release includes a fix for a soundness bug; see #1763.
Changes
make_rngand add#[track_caller](#1761)log(#1763)Configuration
📅 Schedule: (in timezone UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about these updates again.
This PR has been generated by Mend Renovate.