Conversation
The default dataset root (HF_LEROBOT_HOME) resolves under /root when the process runs as root, so the system-directory guard rejected every video path and the script's default invocation could not complete.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thanks for shipping the SO-100 conversion script — this turned up while running it as root in a container.
Fixes #
Changes proposed in this pull request:
/rootfrom thesystem_dirsblocklist in_validate_video_pathsWhat happens
_validate_video_pathslists/rootalongside genuine system directories:/rootis root's home directory.--rootdefaults toNone, in which case L491 falls back toHF_LEROBOT_HOME / repo_id, which for a root user with noHF_HOMEoverride is/root/.cache/huggingface/lerobot/.... So the script's default invocation rejects every video it is asked to convert.Measured on
51d4c89, running as root,HF_HOMEand friends unset:Metadata and parquet conversion complete first; it dies at "Converting concatenated MP4 files back to per-episode videos".
The same thing happens on the README path when the checkout is under root's home —
examples/SO100/README.mdline 10 suggestsexport GR00T_REPO=~/Isaac-GR00T, which for root makes the relative--root examples/SO100/finish_sandwich_lerobotresolve to/root/Isaac-GR00T/.... I have not run that variant; the traceback above is the one I measured.Running as root is the default in a container, and #653 reports that the repo's own Dockerfile has no
USERdirective.Why remove the entry rather than make it conditional
I first tried exempting
/rootwhen it is the invoking user's home, and that predicate is wrong:Path.home()resolves$HOMEbefore the passwd entry, soHOME=/rootwith a non-root user drops the guard, whilesudowithHOME=/home/alicekeeps blocking the case this is meant to fix. Keying onos.geteuid()instead would introduce a mechanism this repo uses nowhere. A blocklist entry whose effect depends on an environment variable seemed worse than either constant, so this PR just removes it. The other five entries are untouched.One thing I'd flag rather than change
These paths are not purely operator-supplied. L506 derives
video_keysfrominfo["features"]of a dataset fetched bysnapshot_download, and those strings are interpolated into the source and destination paths, so a hostilerepo-idinfluences where the script reads and writes. The comment at L282 says "Check for path traversal attempts in the original paths", but the code under it checks null bytes and control characters only.A containment check — asserting
src_resolvedis underrootanddst_resolvedundernew_root— would cover that properly and make the wholesystem_dirslist redundant,/rootincluded. That is a bigger change to your security posture than I'd want to make uninvited, so I've left it out of this PR; happy to open it separately if you'd like it.Verification
Same command, same environment, before and after on this branch:
51d4c89rc=1,ValueErroraboverc=0, both camera streams convertedtests/examples/test_so100.py::test_so100_readme_workflow_executes_via_subprocessalso passes end to end with the change (18m12s, one H100).ruff checkandruff format --checkclean.Before submitting
CONTRIBUTINGdocs — that anchor 404s for me;CONTRIBUTING.mdcurrently has# Contributionsand## Supportonly, so I've followed its "submit a pull request" line. Tell me if there's a step I've missed./rootis absent, so it doesn't get re-added.tests/examples/test_so100.pycovers it end to end but needs a GPU. A focused unit test would have to live underscripts/lerobot_conversion, which has no test module today; say the word and I'll add one there.