feat(cli,frontend): preserve original under data/raw/ whenever output is transformed

[Mandyx22]

Summary

Broadens raw-original preservation beyond JSON. Previously both the CLI (processFile) and the web wizard (runGenerate) kept the untouched input under data/raw/ only for JSON/JSONL (CSV was assumed to have no separate raw form). This changes the trigger to "the input was transformed" — i.e. preserve the original whenever the Psych-DS output is not a verbatim, same-named copy of the input:

transformed = isJsonDataExt(ext) || !csvVerbatimEligible || mainName !== inputName

Newly covered for CSV inputs:

• Renamed — a clean CSV whose content is unchanged but whose filename was changed to a Psych-DS-compliant name (e.g. mydata.csv → subject-x_data.csv). The original is kept under its old name. (Near-universal for real OSF data — the high-impact case.)
• Re-serialised — a CSV that only parsed thanks to quote relaxation (#132) and is therefore rewritten, so the bytes change. The malformed original is preserved.

A clean CSV written byte-for-byte under its own already-compliant name is the only no-raw case, so nothing is duplicated. The existing flat-dir disambiguation and root .psychds-ignore (so the validator skips data/raw/) apply to these preserved CSVs too.

Note

Stacked on #132 (fix/csv-relax-quotes) — base this PR against that branch. This intentionally changes the output layout of currently-working datasets (renamed CSVs now grow a data/raw/ dir + .psychds-ignore), which is why it's a separate PR from the tight #132 bug fix.

Implementation

• CLI (packages/cli/src/data.ts): capture mainOutputName from whichever write path ran (planned → planned.mainName; non-planned → the built main file), move the raw-write block to after the output is written, and replace the isJsonDataExt gate with the transformed check.
• Frontend (packages/frontend/src/pages/DataUpload.tsx): hoist csvVerbatimEligible out of the CSV branch and compare the built main filename against file.name.

BOM-only changes are deliberately not treated as transformed — the preserved content is already BOM-stripped, so preserving it would be pointless.

Testing

• 3 CLI tests (data.test.ts): renamed CSV preserved; already-compliant verbatim CSV gets no data/raw/; re-serialised CSV preserved even with an unchanged name.
• 3 frontend component tests (DataUpload.test.tsx): drive the real component and inspect convertedStore.paths().
• Full suite: 664 passing.
• Manually verified end-to-end on the real OSF phxq4 dataset (the unquoted-stimulus-HTML files that previously failed to parse): all files read, re-serialised output in data/, malformed originals preserved in data/raw/, .psychds-ignore written. Also confirmed clean DataPipe CSVs: renamed → preserved; already-compliant → not preserved.

🤖 Generated with Claude Code

[changeset-bot]

🦋 Changeset detected

Latest commit: 36e6d15

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@jspsych/metadata	Patch
@jspsych/metadata-cli	Patch
frontend	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[jodeleeuw]

Maintainer update: @jodeleeuw approved this PR's behavior as designed — preserving the original under data/raw/ for any transform, including rename-only. Rationale: filename-keyed mappings (analysis scripts, lab software referencing original filenames) are unrecoverable after a rename, and that provenance is worth the storage for this project.

Housekeeping since your last push (all on this branch):

• fix(metadata,cli,frontend): ingest jsPsych CSVs with unescaped quotes #132 merged to main (including the quote+comma repair fallback), and main was merged in here; the PR now targets main directly. (GitHub auto-closed this PR when the old base branch was deleted — reopened and retargeted.)
• Docs updated across docs/, website/docs/, and the package READMEs — 12 pages still described the old JSON-only rule (some explicitly said clean CSVs are never duplicated); they now describe the transformed trigger with the verbatim same-named clean CSV as the only no-raw case.
• Added the missing test for the interactive rename-plan (planned) path — the code path real CLI users hit when any file needs renaming — asserting a renamed CSV's original lands in data/raw/.
• Small test-hygiene fix: the mockIsValid implementation overrides are now restored after each test (clearAllMocks doesn't reset implementations).

CI is green (including the new typecheck/lint gates and the Node 22/24 matrix). Thanks for the careful PR split and the transparent flagging of the storage tradeoff — it made the policy decision easy to review.

🤖 Generated with Claude Code