Skip to content

perf: wake Loom imports from the worker webhook instead of polling - #3119

Closed
richiemcilroy wants to merge 1 commit into
fix/loom-restart-checksfrom
perf/loom-import-hooks
Closed

richiemcilroy wants to merge 1 commit into
fix/loom-restart-checksfrom
perf/loom-import-hooks

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Loom import workflows used to poll the database every 5 to 30 seconds until their media job finished. Each poll is a workflow step plus a replay on Vercel, so a ten minute import cost about 25 invocations, and a batch of 1,200 imports cost tens of thousands.

Now each processing attempt creates a workflow hook whose token goes into the worker's webhook URL. When the worker reports complete, error or cancelled, the progress route saves the state first and then resumes the hook. The workflow then reads the status once. A typical import now uses two status reads instead of dozens.

Safety nets:

  • If the signal is lost, the workflow still checks every 5 minutes, within the same 1 hour limit.
  • If the signal arrives before the saved state is visible, it falls back to the old short polling.
  • Tokens only resume hooks that belong to the same video. Unknown or disposed tokens are ignored.
  • Each hook is disposed when its attempt ends.
  • Runs already in flight stay on the deployment that started them.

Recording, instant and web uploads still use waitForVideoProcessing and are unchanged.

Stacked on #3118.

Tests: video-processing-status 15, media-server-progress 59, media-processing-workflows 27 pass. The 7 share-stage-identity failures also fail on main.

RetriggerView in GreptileConfidence Score: 4/5

The PR appears safe to merge, with a non-blocking gap in tests for the new signal helper.

Findings

  1. P2 Signal checks lack direct tests ▶
Fix with agent prompt
### Issue 1
apps/web/__tests__/unit/media-server-progress.test.ts:79-81
These tests replace `signalLoomImportFinished` entirely, so broken token checks or error branches would still pass. Add focused tests that mock `resumeHook` instead: missing and wrong-video tokens must not resume anything, matching tokens must pass the phase, and missing hooks and other errors must not escape. This would cover the promised behavior for stale worker callbacks.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Loom imports now wait for worker callbacks instead of frequent database checks.

  • Loom imports wake when their media worker finishes.

Reviews (1) · Last reviewed commit: "perf: wake Loom import workflows from th..." · Reviewed by Greptile

@vercel

vercel Bot commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
cap-status Skipped Skipped Oct 24, 2026 4:52pm UTC

@vercel
vercel Bot temporarily deployed to Preview – cap-status October 24, 2026 16:52 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai

Comment on lines +79 to +81
vi.mock("@/lib/loom-import-signal", () => ({
signalLoomImportFinished: mocks.signal,
}));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Signal checks lack direct tests

These tests replace signalLoomImportFinished entirely, so broken token checks or error branches would still pass. Add focused tests that mock resumeHook instead: missing and wrong-video tokens must not resume anything, matching tokens must pass the phase, and missing hooks and other errors must not escape. This would cover the promised behavior for stale worker callbacks.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/__tests__/unit/media-server-progress.test.ts
Line: 79-81

Comment:
**Signal checks lack direct tests**

These tests replace `signalLoomImportFinished` entirely, so broken token checks or error branches would still pass. Add focused tests that mock `resumeHook` instead: missing and wrong-video tokens must not resume anything, matching tokens must pass the phase, and missing hooks and other errors must not escape. This would cover the promised behavior for stale worker callbacks.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@richiemcilroy
richiemcilroy deleted the perf/loom-import-hooks branch October 24, 2026 17:06

This branch was successfully deployed

1 active and 1 inactive deployments
Preview – cap-web — ead7b39e Deployed Oct 24, 2026 by vercel[bot]
Preview – cap-status — ead7b39e Deployed Oct 24, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant