Repository navigation
aligner: mates that share a file name keep their own converted temp files (#1265) - #1270
Merged
Merged
Conversation
…iles (#1265) The paired-end converted temp files were named <prefix.>?<file name><kind>.<ext> with no mate number, so when R1 and R2 have the same file name in different directories (-1 a/r.fq -2 b/r.fq) and both mates are converted to the same kind, the second conversion overwrote the first. That is every non-directional run that writes the converted reads as files: HISAT2 (always files, also with --multicore, which is one -p N instance), Bowtie 2 with --no_stream_converted or when no named pipe can be made in --temp_dir, and the three --combined_index --non_directional paired-end models; the single-pass model had both writers open on one path. With real Bowtie 2 the faithful run exited 1 ("Chromosome number extraction failed for *"); the combined models died on a read-name desync. Perl names these files the same way. ConvertOptions::for_pair sets mate_in_name when the pair's file names are equal (ASCII case-insensitive, since case-only names collide on macOS), and the PE converters then name the files <file name>_<mate><kind>, e.g. r.fq_1_C_to_T.fastq, r.fq_1_CT_GA_tagged.fastq. convert_pe_pair applies it for every convert_pe_kind call site; the tagged model applies it inline. Pairs with different file names keep today's names, so perl-oracle runs and stderr are unchanged; same-named directional/pbat pairs, which worked, also get the infix. converted_name is unchanged, so streamed FIFO names do not change. reject_shared_converted_path stops a run whose pair would still share a converted path. process_pe_chunk prints its "Created ... converted version" lines after the pair's conversions instead of between them. With real Bowtie 2 on 400 simulated non-directional pairs, the same-name --no_stream_converted run now gives the same BAM records as the distinct-name run and leaves no temp files. Fixes #1265
The single-pass combined model checks its two tagged temp paths before opening
either writer, so a shared path stops the run before anything is written; the
separate check after conversion in process_pe_chunk_combined_nondir_tagged is
gone. The equal-path check is now convert::reject_shared_temp_path: it
compares ignoring ASCII case, matching for_pair, and names the pair ("the
converted reads of <r1> and <r2> share the temp file <path>").
tests: the faithful same-name test logs each aligner input's first header and
sequence and checks that each mate is its own reads, converted.
CHANGELOG links #1270 and says what users saw. One overlong doc line wrapped.
FelixKrueger
added a commit
that referenced
this pull request
Nov 21, 2026
wiki: paired-end temp names when R1 and R2 share a file name (#1270)
FelixKrueger
added a commit
that referenced
this pull request
Nov 21, 2026
…site) (#1274) From the 3.4.0 pre-release audit (plans/11212026_release-3.4.0/): - CHANGELOG: #1270 entry describes R2 overwriting R1's converted reads; #1264 names the UMI modes it needs; #1272 says streaming is the default since 3.2.0 and the hang is under CPU contention. - rust/README.md: Milestones lines for #1264, #1266, #1270 and a 3.3.0 backfill for #1261; the #1272 line links its PR; the header no longer points at the closed project board. - Docs site: dedup --umi, --bclconvert and --parallel entries; the regular-file rule applies only above --umi_mismatches 0; extractor inputs are BAM/SAM/CRAM and colliding output names stop the run; the aligner's mate-2 --un/--ambiguous check; a note on the 3.2.0/3.3.0 streamed-run hang; a typo. - THIRD-PARTY-NOTICES and a Dockerfile comment drop stale beta/GA text.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Paired-end runs whose R1 and R2 files have the same name in different folders (
-1 a/r.fq -2 b/r.fq) no longer overwrite their converted temp files. Those files now carry the mate number (r.fq_1_C_to_T.fastq) when the two file names match; every other run keeps its temp names, and output is unchanged.Fixes #1265
Details (AI-assisted)
--multicore), Bowtie 2 with--no_stream_convertedor when--temp_dircan't hold named pipes, and all three--combined_index --non_directionalpaired-end models. Streamed Bowtie 2, Bowtie 2--multicore, single-end and 5-Base were never affected. Perl names these files the same way.--no_stream_convertedrun used to exit 1 (Chromosome number extraction failed for *); it now gives the same BAM records as the distinct-name run and leaves no temp files.dev; removing the fix from either call path makes them fail via the new check. fmt, clippy-D warningsandcargo test -p bismarkpass locally with and withoutbinseq-input; HISAT2 was not run (not installed locally).wiki/temp-file paragraph will follow after merge.