Skip to content

fix: roll active segment before retention - #4781

Open
ParthibanRajasekaran wants to merge 7 commits into
apache:masterfrom
ParthibanRajasekaran:fix/idle-partition-segment-roll
Open

ParthibanRajasekaran wants to merge 7 commits into
apache:masterfrom
ParthibanRajasekaran:fix/idle-partition-segment-roll

Conversation

@ParthibanRajasekaran

Copy link
Copy Markdown
Contributor

Which issue does this PR address?

Closes #4650

Rationale

Idle partitions can retain expired or over-budget data because their active segment is never sealed. Retention cannot remove the active segment, so cleanup remains blocked until another write triggers rotation.

What changed?

The cleaner now asks the partition primary to replicate a RollSegment operation when a non-empty active segment would become eligible for retention after sealing. The operation is applied at the committed log position and rotates the segment consistently across replicas before the existing cleanup pass runs. Empty active segments are not rolled.

Local Execution

  • Passed cargo fmt --all
  • Passed cargo check -p partitions -p server
  • Passed cargo test -p partitions --lib (731 passed, 3 ignored)
  • Passed cargo test -p server segment_cleaner --lib (3 passed)
  • Pre-commit hooks not run: prek was not installed

@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

If AI assistance was used to create this PR, you are required to run the team-review skill before opening it.

Slash commands (own line, regular comment) move it around the queue:

  • /ready - back to S-waiting-on-review after addressing feedback
  • /author - flip to S-waiting-on-author while you finish changes
  • /request-review @user-or-team - request a reviewer
  • /pin - exempt the PR from the stale bot, /unpin to undo

See CONTRIBUTING.md for details.

@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Nov 8, 2026
@ParthibanRajasekaran ParthibanRajasekaran changed the title Fix/idle partition segment roll fix: roll active segment before retention Nov 8, 2026
@codecov

codecov Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.03175% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.87%. Comparing base (689436f) to head (287a16b).

Files with missing lines Patch % Lines
core/partitions/src/iggy_partition.rs 91.37% 3 Missing and 2 partials ⚠️

❌ Your patch check has failed because the patch coverage (15.92%) is below the target coverage (50.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #4781      +/-   ##
============================================
- Coverage     90.89%   90.87%   -0.03%     
  Complexity     3878     3878              
============================================
  Files          1564     1564              
  Lines        407450   407552     +102     
  Branches     341308   341411     +103     
============================================
- Hits         370368   370366       -2     
- Misses        29706    29782      +76     
- Partials       7376     7404      +28     
Components Coverage Δ
Rust Core 92.34% <95.83%> (-0.03%) ⬇️
Java SDK 79.39% <ø> (+<0.01%) ⬆️
C# SDK 78.17% <100.00%> (-0.22%) ⬇️
Python SDK 86.52% <ø> (ø)
PHP SDK 92.23% <ø> (ø)
Node SDK 96.73% <100.00%> (-0.02%) ⬇️
Go SDK 71.87% <100.00%> (+0.06%) ⬆️
C++ SDK 86.84% <ø> (ø)
Files with missing lines Coverage Δ
core/binary_protocol/src/consensus/operation.rs 97.86% <100.00%> (+0.04%) ⬆️
core/binary_protocol/src/dispatch.rs 89.58% <100.00%> (ø)
core/consensus/src/observability.rs 51.34% <100.00%> (+0.37%) ⬆️
core/metadata/src/stm/authz.rs 94.13% <100.00%> (ø)
core/server/src/dispatch/authz.rs 98.40% <ø> (ø)
core/server/src/segment_cleaner.rs 91.08% <100.00%> (+3.69%) ⬆️
foreign/csharp/Iggy_SDK/Vsr/VsrOperation.cs 96.52% <100.00%> (ø)
foreign/go/internal/vsr/operation.go 100.00% <100.00%> (ø)
foreign/node/src/wire/vsr/operation.ts 100.00% <100.00%> (ø)
core/partitions/src/iggy_partition.rs 94.82% <91.37%> (-0.03%) ⬇️

... and 46 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review PR is waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

An idle partition's active segment never seals, so its data never expires

1 participant