Add an env var to schedule the batch worker ping by reeceyang · Pull Request #269 · convex/aggregate

reeceyang · 2026-09-08T20:50:54Z

Some people might be using the alpha version I cut with this (Changelog for 0.3.2-alpha.0 #270), so I'd like to merge this so they can upgrade to newer versions. Won't publicize it though, and eventually we can deprecate these options later

pkg-pr-new · 2026-09-08T20:51:29Z

npm i https://pkg.pr.new/@convex-dev/aggregate@269

commit: a31c1ef

cubic-dev-ai

No issues found across 5 files

_{Re-trigger cubic}

cubic-dev-ai

2 issues found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="README.md">

<violation number="1" location="README.md:810">
P2: When callers use `enqueueBatch`, this setting schedules one ping for the entire batch, not one scheduled function per queued operation. Describe the cost per non-empty enqueue call so scheduler-load guidance matches the implementation.</violation>
</file>

<file name="src/component/worker.ts">

<violation number="1" location="src/component/worker.ts:95">
P1: When the worker is in its final empty cycle, this scheduled ping can see `running` and no-op before the park commits. Since the enqueue no longer reads worker status, it cannot retry and schedule another ping, so the pending row can remain stranded; preserve the status dependency or make ping/parking coordination retry-safe.</violation>
</file>

_{Reply with feedback, questions, or to request a fix.

Re-trigger cubic}

cubic-dev-ai · 2026-09-08T21:10:17Z

    await ctx.db.insert("pendingOperations", entry);
  }
+  if (env.WORKER_SCHEDULE_PING === "true") {
+    await ctx.scheduler.runAfter(0, internal.worker.pingWorker, {});


P1: When the worker is in its final empty cycle, this scheduled ping can see running and no-op before the park commits. Since the enqueue no longer reads worker status, it cannot retry and schedule another ping, so the pending row can remain stranded; preserve the status dependency or make ping/parking coordination retry-safe.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/component/worker.ts, line 95: <comment>When the worker is in its final empty cycle, this scheduled ping can see `running` and no-op before the park commits. Since the enqueue no longer reads worker status, it cannot retry and schedule another ping, so the pending row can remain stranded; preserve the status dependency or make ping/parking coordination retry-safe.</comment> <file context> @@ -92,16 +92,7 @@ export async function enqueueOperations( - if (!status || status.kind === "idle") { - await ctx.scheduler.runAfter(0, internal.worker.pingWorker, {}); - } + await ctx.scheduler.runAfter(0, internal.worker.pingWorker, {}); return; } </file context>

Suggested change

await ctx.scheduler.runAfter(0, internal.worker.pingWorker, {});

const status = await ctx.runQuery(components.batchWorker.lib.status, {

name: OPS_WORKER_NAME,

});

if (!status || status.kind === "idle") {

await ctx.scheduler.runAfter(0, internal.worker.pingWorker, {});

}

In this case the worker cycle should retry, as it takes a read dependency on pendingUpdates if the snapshot query returned empty. When we enqueue operations, that will cause a conflict.

cubic-dev-ai · 2026-09-08T21:10:18Z

+
+Queuing a write then touches the worker component not at all — no read to
+conflict on and nothing written — and the ping runs in its own transaction
+afterwards. The cost is one scheduled function per queued write, whether or not


P2: When callers use enqueueBatch, this setting schedules one ping for the entire batch, not one scheduled function per queued operation. Describe the cost per non-empty enqueue call so scheduler-load guidance matches the implementation.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At README.md, line 810: <comment>When callers use `enqueueBatch`, this setting schedules one ping for the entire batch, not one scheduled function per queued operation. Describe the cost per non-empty enqueue call so scheduler-load guidance matches the implementation.</comment> <file context> @@ -805,12 +805,11 @@ app.use(aggregate, { -reach for this when it won't. +Queuing a write then touches the worker component not at all — no read to +conflict on and nothing written — and the ping runs in its own transaction +afterwards. The cost is one scheduled function per queued write, whether or not +the worker needed waking, so this trades contention for scheduler load. Prefer a +longer cooldown where that will do, and reach for this when it won't. </file context>

Suggested change

afterwards. The cost is one scheduled function per queued write, whether or not

afterwards. The cost is one scheduled function per non-empty enqueue call, whether or not

cubic-dev-ai

2 existing issues remain and no new issues found across 5 files

_{Requires human review: Auto-approval blocked by 2 unresolved issues from previous reviews.

Re-trigger cubic}

Queuing a write pings the batch worker from inside the queuing mutation, which reads the record holding the worker's run state and, when the worker is parked, writes it too. A burst arriving at a parked worker therefore has every mutation in it writing one record, and all but one losing the race. Under `WORKER_SCHEDULE_PING`, the ping is scheduled instead, so queuing a write touches the worker component not at all: no read to conflict on and nothing written. The scheduled ping runs after the mutation commits and does the same work it always did, so nothing can be stranded by racing the worker as it parks. Over a bursty workload this takes the reads made from queuing mutations from 31 to 0 and the writes from 6 to 0, leaving the same 12 writes in the worker's own transactions. Off by default: it costs one scheduled function per queued write, whether or not the worker needed waking, which is a poor trade for a deployment that can just poll for longer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

coderabbitai · 2026-10-21T17:27:08Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: convex/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Advanced
Run ID: b379ef59-b3f4-4c7c-991e-6a771617c8f0

📥 Commits

Reviewing files that changed from the base of the PR and between 243ccaf and d2a4416.

⛔ Files ignored due to path filters: 1

src/component/_generated/server.ts is excluded by !**/_generated/**

📒 Files selected for processing: 4

CHANGELOG.md
src/component/convex.config.ts
src/component/worker.test.ts
src/component/worker.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

The component adds the optional WORKER_SCHEDULE_PING setting. When it is "true", an enqueue with no pending entry schedules a worker ping; otherwise, the worker starts directly. Tests cover both paths and several enqueue and worker-parking cases. The changelog lists this setting and two other experimental worker options.

Sequence Diagram(s)

sequenceDiagram
  participant Enqueue as enqueueOperations
  participant Scheduler as Convex scheduler
  participant Ping as pingWorker
  participant Batch as pingBatchWorker
  Enqueue->>Scheduler: Schedule pingWorker
  Scheduler->>Ping: Run scheduled mutation
  Ping->>Batch: Ping batch worker

Suggested reviewers: ianmacartney

Priority: ⬇️ Low

Change: Feature

Merge Risk: ⚪ Minimal · up to d2a44

No actionable issue is established for the optional scheduled-ping behavior; the PR is mergeable after normal checks.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title clearly and concisely describes the main change: adding an environment variable to schedule batch-worker pings.
Description check		The description explains the purpose of the change and its intended support for users of an earlier alpha version. It is related to the changeset.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Commit to this branch
Create a new PR

🧪 Generate unit tests (beta)

Commit to this branch
Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

src/component/convex.config.ts

ESLint skipped: missing config or dependency (missing-dependency). The ESLint configuration references a package that is not available in the sandbox.

src/component/worker.test.ts

ESLint skipped: the matched ESLint configuration already failed (missing-dependency).

src/component/worker.ts

ESLint skipped: the matched ESLint configuration already failed (missing-dependency).

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

cubic-dev-ai Bot reviewed Sep 8, 2026

View reviewed changes

reeceyang force-pushed the schedule-ping-option branch from cab9b72 to 05abdfd Compare September 8, 2026 21:05

cubic-dev-ai Bot reviewed Sep 8, 2026

View reviewed changes

reeceyang mentioned this pull request Sep 10, 2026

Only ping the batch worker on the first enqueue in a transaction #272

Merged

reeceyang force-pushed the batch-worker-idle-tuning branch from be65f9e to 2b826d8 Compare October 20, 2026 23:21

reeceyang force-pushed the schedule-ping-option branch 2 times, most recently from c59f3a0 to f366aac Compare October 21, 2026 00:00

reeceyang force-pushed the batch-worker-idle-tuning branch from 2b826d8 to 54d147e Compare October 21, 2026 00:00

reeceyang force-pushed the schedule-ping-option branch from f366aac to 78abeb5 Compare October 21, 2026 00:01

reeceyang requested a review from ianmacartney October 21, 2026 00:09

ianmacartney approved these changes Oct 21, 2026

View reviewed changes

reeceyang force-pushed the schedule-ping-option branch from 78abeb5 to a31c1ef Compare October 21, 2026 00:33

reeceyang force-pushed the schedule-ping-option branch from a31c1ef to d2a4416 Compare October 21, 2026 17:23

reeceyang changed the base branch from batch-worker-idle-tuning to main October 21, 2026 17:24

reeceyang merged commit 24bc74b into main Oct 21, 2026
2 checks passed

Navigation Menu

Repository navigation

Add an env var to schedule the batch worker ping - #269

Add an env var to schedule the batch worker ping#269
reeceyang merged 1 commit into
mainfrom
schedule-ping-option

reeceyang commented Sep 8, 2026 •

edited

Loading

Uh oh!

pkg-pr-new Bot commented Sep 8, 2026 •

edited

Loading

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

cubic-dev-ai Bot Sep 8, 2026

Uh oh!

reeceyang Sep 8, 2026

Uh oh!

cubic-dev-ai Bot Sep 8, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

coderabbitai Bot commented Oct 21, 2026

Sequence Diagram(s)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-    await ctx.scheduler.runAfter(0, internal.worker.pingWorker, {});
+    const status = await ctx.runQuery(components.batchWorker.lib.status, {
+      name: OPS_WORKER_NAME,
+    });
+    if (!status || status.kind === "idle") {
+      await ctx.scheduler.runAfter(0, internal.worker.pingWorker, {});
+    }

	afterwards. The cost is one scheduled function per queued write, whether or not
	afterwards. The cost is one scheduled function per non-empty enqueue call, whether or not

Conversation

reeceyang commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pkg-pr-new Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

reeceyang Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot commented Oct 21, 2026

Sequence Diagram(s)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

reeceyang commented Sep 8, 2026 •

edited

Loading

pkg-pr-new Bot commented Sep 8, 2026 •

edited

Loading