Repository navigation
feat: add LRU-aware cache warming for KeyValues - #3592
bblaszkow06 wants to merge 12 commits into
Conversation
| add :key_value_id, references(:key_values, on_delete: :delete_all), null: false | ||
| add :last_used_at, :utc_datetime_usec, null: false, default: fragment("now()") |
There was a problem hiding this comment.
hmm this also implies that if all 1 mil records are in use, then 1 mil records will also be created in this table. why would we want a separate table as compared to having these two fields directly on the :key_values table?
There was a problem hiding this comment.
I'm guessing that the separate table means that the table isn't tracked it isn't tracked as part of ContextCache since it is not included in the published tables, so bumping it would not trigger cache invalidations.
should make this clear in the KeyValuesUsage module
There was a problem hiding this comment.
should probably have some periodic truncation logic to keep the size of this table down
| @@ -0,0 +1,110 @@ | |||
| defmodule Logflare.KeyValues.UsageTracker do | |||
There was a problem hiding this comment.
instead of an ets-based tracker that gets routed through a genserver, a simpler and good enough solution would be periodically streamed from the KeyValues.Cache the Sources.recent_events_touch/1, and triggered with Oban via an Oban worker.
Since LRU cache warming is already limited to 500k results, we don't need 100% accuracy and only need a good-enough sort order during warming.
Furthermore, this would eat into memory, which the KeyValues would already be consuming quite a bit of
There was a problem hiding this comment.
we'll need to balance "correctness" for performance in this case. we should defer more accurate LRU tracking until the need arises, this would be too much overhead and overengineering for improved cache warming.
… KeyValues LRU Per reviewer feedback on PR Logflare#3592: drop the hot-path ETS/GenServer tracker in favour of periodically harvesting recently-written entries straight from KeyValues.Cache via Cachex.stream!, driven by an Oban cron worker every 15 min. - Remove UsageTracker (double-buffered ETS + GenServer), its bench scripts, and all related tests; remove it from ContextCache.Supervisor and test_helper.exs - Add Cache.touch_recent_usages/1: streams cache entries with modified >= cutoff, deduplicates accessor-path variants of the same (user_id, key), chunks DB writes with 500 ms sleeps — zero per-lookup overhead, good-enough sort order for warming - Add KeyValues.prune_usages/1: deletes key_value_usages rows older than 30 days to keep the table bounded (addresses reviewer comment on unbounded growth) - Add UsageTouchWorker (Oban, queue: default, max_attempts: 1) on */15 cron - Expand KeyValueUsage @moduledoc to make the logflare_pub exclusion explicit - Refactor tests: shorter names, combined cases where same input covers both paths Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… KeyValues LRU Per reviewer feedback on PR Logflare#3592: drop the hot-path ETS/GenServer tracker in favour of periodically harvesting recently-written entries straight from KeyValues.Cache via Cachex.stream!, driven by an Oban cron worker every 15 min.
12477ce to
5df497e
Compare
djwhitt
added
enhancement
Track key-value cache hits via a dual-buffer ETS UsageTracker GenServer that periodically upserts a `key_value_usages` table. On startup, CacheWarmer now warms only the top-N most-recently-used entries (ordered by last_used_at, falling back to updated_at) instead of streaming the full table.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… KeyValues LRU Per reviewer feedback on PR Logflare#3592: drop the hot-path ETS/GenServer tracker in favour of periodically harvesting recently-written entries straight from KeyValues.Cache via Cachex.stream!, driven by an Oban cron worker every 15 min.
caf320f to
9f7dc11
Compare
CacheWarmer records when the initial full warm completed; touch_recent_usages only considers cache entries written after that point, so the boot-time put_many of up to warm_limit rows no longer stamps every warmed key as recently used.
This branch has not been deployed
3 participants
Warm
KeyValues.Cachewith the most recently used entries instead of streaming the whole table.Usage tracking
key_value_usagestable (key_value_id,last_used_at). It is deliberately kept out oflogflare_pub, so bumping usage does not go throughCacheBusterand evict the entries being tracked.KeyValues.UsageTouchWorker(Oban cron, every 15 min, same pattern asRecentEventsTouchWorker)streams
{:lookup, …}entries written toKeyValues.Cachein the last 20 min and upsertslast_used_atin chunks of 1,000. No hot-path tracking and no extra in-memory state.Warming
CacheWarmer.warm_top_n/0replaceswarm_full/0: orders bylast_used_at DESC NULLS LAST, updated_at DESC, limited towarm_limit(default 500k, configurable).Accuracy trade-off
The signal is the Cachex
modifiedtimestamp (time of write, not read), so this is an approximationof LRU: it is good enough to order the warm set, not an exact access log.