perf: preserve payload maps during ingest cleaning

[djwhitt]

Summary

This is a safer replacement for #3765's LogEvent payload-cleaning optimization.

• keep the fused empty-value removal and BigQuery-safe key normalization traversal
• use the input map as the accumulator, returning it unchanged when no nested value or key changes
• delete, update, or rename only changed map entries
• retain the existing list rebuild, struct rebuild, normalized-key collision fallback, and non-binary-key behavior
• special-case large all-empty maps, where returning %{} after a scan is cheaper than deleting every key
• add a reproducible map/list mutation-density benchmark and focused compatibility coverage

The important difference from #3765 is sparse dirty behavior: rebuilding every map helped clean payloads but regressed representative dirty ones. Copy-on-write maps retain most of the clean-path gain while also improving sparse and list-heavy cases.

Benchmark methodology

Environment: Linux, 6 available cores, Elixir 1.19.5, Erlang/OTP 27.3.4.6, JIT enabled, Benchee parallelism 1.

The first commit (da3e3ca5) adds the benchmark without changing runtime code. Results compare that benchmark-only baseline with this PR's implementation commit.

Mutation-density benchmark

Command, run three times per revision as paired baseline/candidate samples; the table reports the median throughput and memory per scenario:

LF_BENCH_TIME=3 LF_BENCH_WARMUP=1 LF_BENCH_MEMORY_TIME=2 \
  ../bin/x mix run --no-start bench/log_event_cleaner.exs

Case	Baseline K ops/s	Candidate K ops/s	Throughput	Baseline KB	Candidate KB	Memory
clean 64-key map	155.04	717.82	+363.0%	21.81	7.06	-67.6%
sparse dirty 66-key map	146.97	563.57	+283.5%	21.20	7.97	-62.4%
half-empty 64-key map	226.99	291.83	+28.6%	13.98	9.87	-29.4%
all-empty 64-key map	2,427.61	2,688.36	+10.7%	1.03	1.03	0.0%
all-unsafe 64-key map	54.57	58.20	+6.7%	39.60	46.60	+17.7%
clean list of 64 maps	258.51	326.87	+26.4%	24.55	17.06	-30.5%
list with one nested change	257.53	329.82	+28.1%	24.60	17.25	-29.9%
half-empty list	805.08	908.42	+12.8%	6.64	5.16	-22.3%

The pathological all-unsafe map is the only measured memory regression. It renames every key, so copy-on-write creates intermediate maps; it remains 6.7% faster. Pre-scanning that case to select a rebuild was tested and was worse on both throughput and measured allocation.

End-to-end LogEvent.make/2

The existing profiling suites were run twice per revision in baseline→candidate and candidate→baseline order. Results below use each scenario's two-run median, then the geometric mean across scenarios.

MIX_ENV=test ../bin/x mix run test/profiling/log_event_make_type_detection_bench.exs
MIX_ENV=test ../bin/x mix run test/profiling/log_event_make_bench.exs

Suite	Scenarios	Throughput geomean	Memory geomean	Per-scenario throughput range
common logs/traces/metric	7	+61.0%	-42.2%	+19.6% to +128.2%
OTEL/edge transforms	12	+52.7%	-37.4%	+9.9% to +116.9%

The existing edge fixture includes a Cloudflare subtree larger than current production output, so the OTEL/edge suite should be read as allocation stress coverage rather than a production payload-frequency model. The seven-shape suite covers three logs, three traces, and one metric.

Validation

• ../bin/format --check-formatted lib/logflare/logs/ingest_transformers.ex test/logflare/logs/ingest/ingest_transformer_test.exs bench/log_event_cleaner.exs
• MIX_ENV=test ../bin/x mix compile --warnings-as-errors
• MIX_ENV=test ../bin/x mix credo --strict lib/logflare/logs/ingest_transformers.ex test/logflare/logs/ingest/ingest_transformer_test.exs bench/log_event_cleaner.exs
• MIX_ENV=test ../bin/test test/logflare/logs/ingest/ingest_transformer_test.exs — 1 property, 36 tests, 0 failures