Repository navigation
[stress-tests] relax event limits in otns-performance test - #836
Merged
Merged
Conversation
The `otns-performance` stress test simulates a 32-node router network for 3600 seconds of simulation time under a fixed PRNG seed (48392). Its primary pass criterion is execution time (<= 60s). The test also enforces hardcoded upper bounds on `AlarmEvents` (900k) and `RadioEvents` (680k). These limits were tuned with very narrow headroom (~5%) against a specific baseline. Consequently, any subtle PRNG stream shift in the OpenThread stack during node initialization or runtime (e.g., slight variations in MLE advertisement or backoff jitters over a 1-hour simulation) can produce ~3–5% fluctuations in event totals, tripping these thresholds and falsely failing CI even when simulation performance easily passes (e.g. 35s execution time). This commit relaxes the event count thresholds to provide adequate headroom (~15–20% margin) while continuing to catch runaway event loops or true performance regressions: - Increases `AlarmEvents` threshold to 1,100,000 (from 900,000). - Increases `RadioEvents` threshold to 800,000 (from 680,000).
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request increases the threshold limits for AlarmEvents and RadioEvents in the OTNS performance stress test. The review feedback suggests defining these threshold values as variables or constants and using Python's numeric underscores (e.g., 1_100_000) to eliminate duplication between the comparison logic and the error messages, thereby improving readability and maintainability.
EskoDijk
approved these changes
Sep 17, 2026
jwhui
approved these changes
Sep 17, 2026
EskoDijk
pushed a commit
to EskoDijk/ot-ns
that referenced
this pull request
Sep 30, 2026
…ead#836) The `otns-performance` stress test simulates a 32-node router network for 3600 seconds of simulation time under a fixed PRNG seed (48392). Its primary pass criterion is execution time (<= 60s). The test also enforces hardcoded upper bounds on `AlarmEvents` (900k) and `RadioEvents` (680k). These limits were tuned with very narrow headroom (~5%) against a specific baseline. Consequently, any subtle PRNG stream shift in the OpenThread stack during node initialization or runtime (e.g., slight variations in MLE advertisement or backoff jitters over a 1-hour simulation) can produce ~3–5% fluctuations in event totals, tripping these thresholds and falsely failing CI even when simulation performance easily passes (e.g. 35s execution time). This commit relaxes the event count thresholds to provide adequate headroom (~15–20% margin) while continuing to catch runaway event loops or true performance regressions: - Increases `AlarmEvents` threshold to 1,100,000 (from 900,000). - Increases `RadioEvents` threshold to 800,000 (from 680,000).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
otns-performancestress test simulates a 32-node router network for 3600 seconds of simulation time under a fixed PRNG seed (48392). Its primary pass criterion is execution time (<= 60s).The test also enforces hardcoded upper bounds on
AlarmEvents(900k) andRadioEvents(680k). These limits were tuned with very narrow headroom (~5%) against a specific baseline. Consequently, any subtle PRNG stream shift in the OpenThread stack during node initialization or runtime (e.g., slight variations in MLE advertisement or backoff jitters over a 1-hour simulation) can produce ~3–5% fluctuations in event totals, tripping these thresholds and falsely failing CI even when simulation performance easily passes (e.g. 35s execution time).This commit relaxes the event count thresholds to provide adequate headroom (~15–20% margin) while continuing to catch runaway event loops or true performance regressions:
AlarmEventsthreshold to 1,100,000 (from 900,000).RadioEventsthreshold to 800,000 (from 680,000).Related to
otns-performanceis fragile to minor PRNG stream shifts due to overly tight event limits openthread#13617