Add selective content masking to the timelapse editor (Updated with fixes) by httperry · Pull Request #198 · hackclub/lookout

httperry · 2026-09-16T16:27:58Z

Currently when recording timelapses, if sensitive info (API keys, .env files, private messages) appears on screen, the only option is to cut out that entire minute of footage. This PR adds selective masking directly to the editor so users can redact specific screen regions while preserving the rest of their recorded footage.

Summary

Adds an in-editor masking tool to redact sensitive on-screen areas, persists mask regions to the session, and applies a pixelation filter during video compilation.

Screen.Recording.2026-09-16.at.12.20.14.PM.mov

What's Added

Interactive canvas overlay (@lookout/react): Click-and-drag bounding box overlay on top of the video player to draw, reposition, and resize mask regions with 8-point handles.
Dedicated Mask tab: Keeps cut editing and mask editing separate so controls and timeline interactions don't conflict with each other.
Multi-track timeline lanes: Uses interval scheduling to automatically stack concurrent masks into separate lanes (supporting up to 3 parallel tracks), preventing overlapping masks from colliding.
Scene-aware handle snapping: Snaps mask start and end handles to shot transitions using a fast client-side frame difference check on a downscaled 48×48 canvas, eliminating extra server roundtrips.
Timeline minimap & zoom overview: Adds a translucent overview bar underneath the timeline with smooth scrubbing and dynamic duration formatting.
Video pixelation in compilation (@lookout/worker): Compiles masked videos through an ffmpeg filtergraph that burns pixelated boxes over the target coordinates and timestamps. Sessions without masks still take the instant lossless stream-copy path.
Session mask persistence (@lookout/server): Adds a masks column to the sessions table (migration 0032_session_masks.sql), returning saved masks in GET /units and validating them on PUT /cuts.
Shared data model & bounds validation (@lookout/shared): Centralizes the MaskRegion schema with bounds checking to ensure coordinates stay normalized (0..1) across the frontend, server, and worker.

Verification

Monorepo builds clean (tsc --noEmit).
All test suites pass:
- 162 React tests (canvas interactions, track allocation, snapping math)
- 32 Worker tests (pixelation filter execution, cutVideo bit-exactness, segment stitching)
- 30 Shared tests (coordinate bounds checking and interval intersection math)
Tested end-to-end locally: drawn masks save to Postgres, reload cleanly on refresh, and the worker compiles an MP4 with pixelated boxes in the specified coordinates.

… present

httperry · 2026-09-16T16:29:43Z

CDN URL for demo video: https://cdn.hackclub.com/01a0aa9e-a295-7cb1-9b9f-4518833d3f94/Screen%20Recording%202026-09-16%20at%2012.20.14%E2%80%AFPM.mov

Copilot

🟡 Changes recommended

One or more issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds selective screen-region masking to timelapse editing, persisting normalized mask regions and applying FFmpeg pixelation during compilation.

Changes:

Adds shared mask models, validation, persistence, and migration.
Adds React/desktop editor masking UI, timeline tracks, snapping, and API forwarding.
Adds worker-side mask conversion, FFmpeg processing, and tests.

File summaries

File	Description
packages/worker/test/maskVideo.test.ts	Updated as part of this pull request.
packages/worker/src/segments.ts	Updated as part of this pull request.
packages/worker/src/schema.ts	Updated as part of this pull request.
packages/worker/src/compile.ts	Updated as part of this pull request.
packages/shared/src/types.ts	Updated as part of this pull request.
packages/shared/src/masks.ts	Updated as part of this pull request.
packages/shared/src/masks.test.ts	Updated as part of this pull request.
packages/shared/src/index.ts	Updated as part of this pull request.
packages/shared/src/cuts.ts	Updated as part of this pull request.
packages/server/test/edits.integration.test.ts	Updated as part of this pull request.
packages/server/src/routes/sessions.ts	Updated as part of this pull request.
packages/server/src/lib/timeouts.ts	Updated as part of this pull request.
packages/server/src/db/schema.ts	Updated as part of this pull request.
packages/server/drizzle/meta/0032_snapshot.json	Updated as part of this pull request.
packages/server/drizzle/meta/_journal.json	Updated as part of this pull request.
packages/server/drizzle/0032_session_masks.sql	Updated as part of this pull request.
clients/react/tsup.config.ts	Updated as part of this pull request.
clients/react/src/index.ts	Updated as part of this pull request.
clients/react/src/hooks/webCodecsClipEngine.ts	Updated as part of this pull request.
clients/react/src/hooks/editorMath.ts	Updated as part of this pull request.
clients/react/src/hooks/editorMath.test.ts	Updated as part of this pull request.
clients/react/src/components/ResultView.tsx	Updated as part of this pull request.
clients/react/src/components/ResultView.test.tsx	Updated as part of this pull request.
clients/react/src/components/editorStyles.ts	Updated as part of this pull request.
clients/react/src/api/client.ts	Updated as part of this pull request.
clients/react/src/api/client.test.ts	Updated as part of this pull request.
clients/playground/src/App.tsx	Updated as part of this pull request.
clients/desktop/vitest.config.ts	Updated as part of this pull request.
clients/desktop/src/testSetup.ts	Updated as part of this pull request.
clients/desktop/src/components/EditorWindow.tsx	Updated as part of this pull request.
clients/desktop/src/components/EditorWindow.test.ts	Updated as part of this pull request.
clients/desktop/src/api/tauriClient.ts	Updated as part of this pull request.
clients/desktop/src-tauri/src/lib.rs	Updated as part of this pull request.
clients/desktop/src-tauri/crates/lookout-core/src/api.rs	Updated as part of this pull request.

Review details

Suppressed comments (4)

clients/desktop/src/components/EditorWindow.tsx:339

The close handler can run while TimelapseEditor is still loading. In that window masksRef.current is still its empty initial value, so this always sends masks: [] through the desktop client and clears any masks already saved for the session when the user closes immediately. Gate publishing on editor readiness or initialize the ref from the loaded session before allowing close.

            await client.setCuts(cutsRef.current, masksRef.current);

packages/shared/src/masks.ts:99

The normalized ID is retained for every non-empty id, but IDs are used as identity keys throughout the editor (React keys, selection, and the track-assignment map). A request containing two masks with the same ID is accepted and, after reload, those masks cannot be edited independently and one assignment overwrites the other. Reject duplicate IDs or generate a unique replacement during normalization.

      id: typeof id === "string" && id.trim().length > 0 ? id : `mask-${i}-${startMs}`,

packages/worker/test/maskVideo.test.ts:122

The pure timestamp-mapping tests in this file are nested under the FFmpeg-gated suite, so a machine without FFmpeg silently skips them even though they only exercise maskToVideoMask and do not invoke FFmpeg. Split those mapping tests into an ungated describe so out-of-range and wall-clock conversion behavior remains covered when only unit tests can run.
packages/worker/test/maskVideo.test.ts:140
Each loop starts a fresh testsrc and writes only its first frame, so every unit_i.jpg has the same pixels despite the comment. The assertions therefore cannot detect a mask applied to the wrong unit or active outside [startSec, endSec); an implementation that masks every frame could pass. Generate distinct per-unit frames and assert pre/post-interval frames remain unmodified.

Files reviewed: 34/35 changed files
Comments generated: 6
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

      return call<SetCutsResponse>("api_session_set_cuts", {
        ...(await session()),
        cuts,
+        masks: masks ?? [],


-        throw new Error("Invalid upload URL: must be HTTPS or a relative path.");
-      }
+        let isAllowed = false;
+        if (uploadUrl.startsWith("/") && !uploadUrl.startsWith("//")) {


+  return masks.map((m) => {
+    let startUnit = 0;
+    let endUnit = units.length;


+      typeof startSec === "number" &&
+      Number.isFinite(startSec) &&
+      typeof endSec === "number" &&
+      Number.isFinite(endSec) &&
+      endSec > startSec;


+  const videoMasks = masksToVideoMasks(masks, videoUnits);
+  const hasEffectiveMasks = videoMasks.length > 0;
+  const needsEdit = hasEffectiveCuts || hasEffectiveMasks;
+  const hasEdits = hasEffectiveCuts || (masks && masks.length > 0);


+        `[${currentIn}]split=2[m_base_${i}][m_crop_${i}]`,
+        `[m_crop_${i}]crop=${W}:${H}:${X}:${Y},scale=${downW}:${downH}:flags=neighbor,scale=${W}:${H}:flags=neighbor[m_pix_${i}]`,
+        `[m_base_${i}][m_pix_${i}]overlay=${X}:${Y}:enable='gte(t\\,${m.startSec - 0.005})*lte(t\\,${m.endSec + 0.005})'[${outTag}]`,


Lamparter · 2026-09-16T18:39:21Z

hey, this is a cool idea and from the video it looks like it was implemented very well. that said, this PR was not created in relation to a ready-to-build issue, and is therefore on the backlog for review for now

httperry added 10 commits September 16, 2026 09:32

define MaskRegion schema and interval bounds checking in shared

b30d16f

unit test mask validation and interval intersection logic

e9b8a99

add masks jsonb column to sessions in schema and migration 0032

465b3da

serialize masks in GET /units and validate them on PUT /cuts

7753c53

apply ffmpeg pixelation filtergraph during compilation when masks are…

dbf29d9

… present

add integration tests for worker video mask compilation

65ac109

implement timeline lane assignment algorithm and handle snapping

1009373

set hardwareAcceleration to prefer-hardware in WebCodecs clip engine

0e538a4

add interactive canvas overlay and timeline tracks to TimelapseEditor

aa5c78e

wire mask state into desktop EditorWindow and playground client

9618e19

Copilot AI lite review requested due to automatic review settings September 16, 2026 16:27

Copilot started reviewing on behalf of httperry September 16, 2026 16:28 View session

Copilot AI reviewed Sep 16, 2026

View reviewed changes

Navigation Menu

Add selective content masking to the timelapse editor (Updated with fixes) - #198

Add selective content masking to the timelapse editor (Updated with fixes)#198
httperry wants to merge 10 commits into
hackclub:mainfrom
httperry:feature/mask-tool-editor

httperry commented Sep 16, 2026

Uh oh!

httperry commented Sep 16, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Lamparter commented Sep 16, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

httperry commented Sep 16, 2026

Summary

What's Added

Verification

Uh oh!

httperry commented Sep 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (4)

Uh oh!

Lamparter commented Sep 16, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants