Skip to content

KAFKA-21140: Add randomized testbed for streams group task assignors - #23556

Open
gabriellefu wants to merge 4 commits into
apache:trunkfrom
gabriellefu:common_test
Open

gabriellefu wants to merge 4 commits into
apache:trunkfrom
gabriellefu:common_test

Conversation

@gabriellefu

Copy link
Copy Markdown
Contributor
  • TaskAssignorTestbed — generates a random topology and group from a seed
    (two profiles: SMALL for logical corners, LARGE for production shape), runs the
    assignor until the assignment is a fixed point, then applies random rebalance
    events (process add/drop/restart/retag, member add/drop, standby count change)
    and converges again. A failure prints the seed and the full history; replay a
    single scenario with STREAMS_ASSIGNOR_FUZZ_SEED=<seed> or shift the whole run
    with STREAMS_ASSIGNOR_FUZZ_BASE_SEED=<seed> (system properties work from an IDE).
  • AssignmentInvariants — properties every assignor must satisfy, asserted on
    every assignment: output covers exactly the group members, every task is active
    on exactly one member, standbys only on stateful tasks, at most
    numStandbyReplicas standbys per task, no process holds two copies of a task.
  • AssignmentMetrics — grades an assignment without asserting: active/total
    spread per member, load spread per process, process/member stickiness, task
    moves, rack diversity per tag. The summary over a run is printed so two versions
    of an assignor can be compared.
  • StickyTaskAssignorFuzzTest — runs StickyTaskAssignor through the testbed
    and adds the checks its algorithm guarantees: no member above
    ceil(activeTasks / members), every stateful task gets exactly
    min(numStandbyReplicas, processes - 1) standbys, and rack-aware tags do not
    change the active assignment.

@github-actions github-actions Bot added tests Test fixes (including flaky tests) group-coordinator triage PRs from the community labels Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

group-coordinator tests Test fixes (including flaky tests) triage PRs from the community

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant