Skip to content

DNM: microbench seastar urgent value/coroutine returns - #31613

Draft
travisdowns wants to merge 1 commit into
redpanda-data:devfrom
travisdowns:td-urgent-sched-microbench
Draft

DNM: microbench seastar urgent value/coroutine returns#31613
travisdowns wants to merge 1 commit into
redpanda-data:devfrom
travisdowns:td-urgent-sched-microbench

Conversation

@travisdowns

Copy link
Copy Markdown
Member

Do not merge. This exists only to get a /microbench comparison against the baseline.

Pins seastar at travisdowns/seastar@2e6c9a1 (branch td-urgent-sched, cut from v26.3.x), which adds set_value_urgent() and uses it from:

  • the value and void branches of futurize<T>::satisfy_with_result_of()
  • coroutine_traits_base<T>::promise_type::return_value() / return_void()

Today only the third branch of satisfy_with_result_of() - the one where the continuation returns an already-ready future - resolves the next promise urgently, via forward_to() -> set_urgent_state(), which inserts the waiting task at the front of the reactor's task queue. The value and void branches call set_value(), which appends it to the back, behind everything queued while the chain was running. Coroutines always take the appending path, since return_value()/return_void() go through set_value(); with final_suspend() being suspend_never, each level of a nested co_await chain resumes its caller from its own tail-inserted task.

Note this changes the order in which queued tasks run. Seastar's futures_test and coroutines_test pass in dev, release and sanitize modes, and both files gain a test that asserts the new ordering.

The point of the microbench run is that the change is otherwise unmeasured.

Pins seastar at travisdowns/seastar@2e6c9a1, which makes a continuation that
returns a value - and a coroutine return - resume its waiter from the front of
the reactor's task queue rather than the back. Not for merge; this exists to get
a microbench comparison against the baseline.
@travisdowns

Copy link
Copy Markdown
Member Author

/microbench

@vbotbuildovich

Copy link
Copy Markdown
Collaborator

Performance change detected in https://buildkite.com/redpanda/redpanda/builds/88870#01a015bb-a728-4f25-a135-ca2989c25652:

Performance changes detected in 59 tests
coro_rpbench.coro_bench.co_await_ready_collect: inst -> +0.24pct
cloud_storage_rpbench.cstore_bench.column_store_append_with_intervening_lookup_10k_1k: inst -> -0.07pct
cloud_storage_rpbench.cstore_bench.cs_iteration_recompute_end_test_1000: inst -> +0.19pct
cloud_storage_rpbench.cstore_bench.cs_iteration_recompute_end_test_10000: inst -> +0.19pct
async_algorithm_rpbench.algo_bench.maybe_yield_loop_small: inst -> +1.54pct
async_algorithm_rpbench.algo_bench.std_for_each_small: allocs -> 0.00pct
async_algorithm_rpbench.algo_bench.std_for_each_small: inst -> +0.17pct
async_algorithm_rpbench.algo_bench.sync_std_for_each_small: allocs -> +infpct
heartbeat_bench_rpbench.fixture.test_new_hb_request_full: allocs -> -0.54pct
kafka_fetch_plan_rpbench.fetch_plan.t1p100_no_auth_ids: inst -> +0.58pct
kafka_fetch_plan_rpbench.fetch_plan.t1p100_yes_auth_ids: inst -> +0.54pct
role_store_bench_rpbench.role_store_bench.get_member_roles: inst -> -0.03pct
role_store_bench_rpbench.role_store_bench.get_member_roles_bare_query: inst -> -0.03pct
role_store_bench_rpbench.role_store_bench.get_member_roles_bare_query_mixed: inst -> +0.02pct
role_store_bench_rpbench.role_store_bench.get_member_roles_mixed: inst -> +0.02pct
role_store_bench_rpbench.role_store_bench.range_query_bare_query_mixed: inst -> +0.01pct
role_store_bench_rpbench.role_store_bench.user_range_query: inst -> -0.03pct
role_store_bench_rpbench.role_store_bench.user_range_query_bare_query: inst -> -0.03pct
partition_balancer_rpbench.partition_balancer_planner_fixture.counts_rebalancing: inst -> +0.01pct
record_multiplexer_rpbench.record_multiplexer_bench_fixture.protobuf_nested_10_levels_small_zstd: inst -> -0.02pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_KiB_dispatched: allocs -> +0.12pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_KiB_produced: allocs -> -0.18pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_dispatched: allocs -> +0.09pct
kafka_produce_partition_rpbench.produce_partition_fixture.1_produced: allocs -> -0.18pct
kafka_produce_partition_rpbench.produce_partition_fixture.4_KiB_dispatched: allocs -> +0.12pct
kafka_produce_partition_rpbench.produce_partition_fixture.4_KiB_produced: allocs -> -0.21pct
kafka_produce_partition_rpbench.produce_partition_fixture.8_KiB_dispatched: allocs -> +0.17pct
raft_replicate_rpbench.raft_benchmark_fixture.leader_ack_replicate_16B: allocs -> +28.94pct
raft_replicate_rpbench.raft_benchmark_fixture.leader_ack_replicate_16B: inst -> +22.17pct
raft_replicate_rpbench.raft_benchmark_fixture.quorum_ack_enqueue_16B: allocs -> +0.24pct
raft_replicate_rpbench.raft_benchmark_fixture.quorum_ack_enqueue_16B: inst -> +0.28pct
raft_replicate_rpbench.raft_benchmark_fixture.quorum_ack_replicate_16KiB: allocs -> -0.12pct
list_kafka_connections_rpbench.ListKafkaConnectionsTest.E2EOrderedJson100k: inst -> +0.09pct
list_kafka_connections_rpbench.ListKafkaConnectionsTest.E2EOrderedJson10k: inst -> +0.09pct
list_kafka_connections_rpbench.ListKafkaConnectionsTest.E2EUnorderedJson100k: inst -> +0.13pct
list_kafka_connections_rpbench.ListKafkaConnectionsTest.E2EUnorderedJson10k: inst -> +0.13pct
list_kafka_connections_rpbench.ListKafkaConnectionsTest.ToJson100k: inst -> +0.14pct
list_kafka_connections_rpbench.ListKafkaConnectionsTest.ToJson10k: inst -> +0.14pct
read_fanout_rpbench.read_fanout_bench.baseline: allocs -> +10.96pct
read_fanout_rpbench.read_fanout_bench.baseline: inst -> +1.68pct
read_fanout_rpbench.read_fanout_bench.baseline_noop: allocs -> +19.64pct
read_fanout_rpbench.read_fanout_bench.baseline_noop: inst -> +14.67pct
read_fanout_rpbench.read_fanout_bench.bypass: allocs -> +10.00pct
read_fanout_rpbench.read_fanout_bench.bypass: inst -> +1.40pct
read_fanout_rpbench.read_fanout_bench.serialized_noop: allocs -> +16.34pct
read_fanout_rpbench.read_fanout_bench.serialized_noop: inst -> +12.01pct
flush_task_rpbench.flush_task_fixture.flush_16mb: inst -> -2.29pct
read_merge_rpbench.read_merge_bench.baseline: inst -> +0.01pct
read_merge_rpbench.read_merge_bench.concurrent_same_object: allocs -> +5.61pct
read_merge_rpbench.read_merge_bench.concurrent_unique_objects: allocs -> +4.10pct
authorizer_bench_rpbench.authorizer_bench.describe_50k_prefix_acls: inst -> -0.52pct
authorizer_bench_rpbench.authorizer_bench.describe_literal_grants_10k: inst -> -1.95pct
group_offset_commit_rpbench.group_bench.cycle_1t_50p: inst -> -0.68pct
group_offset_commit_rpbench.group_bench.cycle_4t_64p: inst -> +0.06pct
group_offset_commit_rpbench.group_bench.prepare_1t_50p: inst -> -0.31pct
group_offset_commit_rpbench.group_bench.prepare_4t_64p: inst -> +0.04pct
future_util_rpbench.future_util_bench.ignore_shutdown_abort_requested: inst -> +0.11pct
future_util_rpbench.future_util_bench.ignore_shutdown_gate_closed: inst -> +0.11pct
future_util_rpbench.future_util_bench.ignore_shutdown_non_shutdown: inst -> +0.09pct

See https://redpandadata.atlassian.net/wiki/x/LQAqLg for docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants