[lets-order-some-fries]

Summary

cargo build --target wasm32-unknown-unknown fails on main with five errors.
This fixes them, removes the duplication that caused them, and adds a wasm32 CI
check so it does not silently rot again.

Fixes #77.

The two causes

Both are cfg branches that nothing ever built.

1. registry.rs — a field removed a year ago. The native and wasm32
loaders each carried their own full copy of the HarmonyEncoding struct
literal. d00ac3d ("make HarmonyEncoding usable concurrently", 2025-08-05)
removed the conversation_has_function_tools field and updated only the native
copy. The wasm32 copy has been setting a field that no longer exists, against an
AtomicBool that is no longer imported, ever since.

error[E0560]: struct `HarmonyEncoding` has no field named `conversation_has_function_tools`
error[E0433]: cannot find type `AtomicBool` in this scope
   --> src/registry.rs:119

2. public_encodings.rs — a function that was never written. The wasm32
Encoding::load calls load_encoding_from_bytes; only load_encoding_from_file
exists, and there is no filesystem to read on that target.

error[E0425]: cannot find function `load_encoding_from_bytes` in this scope
   --> src/tiktoken_ext/public_encodings.rs:220, :229, :231

Details

Re-syncing the two struct literals would fix the build and leave them free to
drift apart again, so instead:

• build_harmony_encoding holds the shared construction. The two loaders now
  differ only in how they obtain the tokenizer — blocking vs .await — so
  there is no second copy to fall behind.
• build_core_bpe does the same for the vocab side, shared by
  load_encoding_from_file and the new load_encoding_from_bytes.

load_encoding_from_bytes is #[cfg(target_arch = "wasm32")] because
public_encodings is a private module (mod public_encodings;), so an
unreachable pub fn trips dead_code under -D warnings on other targets.
Loading a vocabulary you already hold is useful off wasm32 too — an embedded or
air-gapped copy, cf. #101 — but exposing it there is an API decision, so I left
it to you rather than widening this PR.

The CI job. Without one this just breaks again: it broke in August 2025 and
no build noticed for over a year. It is a separate workflow file rather than
a job in CI.yml, because that file is generated ("This file is autogenerated
by maturin v1.8.7 — to update, run maturin generate-ci github") and a job
added there would be lost on the next regeneration.

How to Validate

rustup target add wasm32-unknown-unknown
cargo build --target wasm32-unknown-unknown   # fails on main with 5 errors

On this branch, observed locally (macOS, aarch64, rustc 1.98.1):

command	before	after
cargo check --target wasm32-unknown-unknown --lib (the CI job)	5 errors	exit 0
cargo test	31 passed	31 passed, 0 failed
cargo fmt --check	clean	clean
cargo clippy --all-targets -- -D warnings	2 errors	the same 2 errors

Those two clippy errors are pre-existing on main with this toolchain
(unneeded late initialization at src/encoding.rs:1413 and can be more succinctly written as a byte str at src/tiktoken.rs:312) — I verified by
stashing this change and re-running. They are untouched here; your CI presumably
pins an older toolchain where those lints do not fire. Say the word if you would
like them in a separate PR.

The wasm32 build still emits dead_code warnings for the file/cache helpers
that only apply off-wasm. The new job does not use -D warnings, so those
do not fail it; cleaning them up is a follow-up, not this PR.

🤖 Generated with Claude Code

[lets-order-some-fries]

Could a maintainer approve the two workflow runs here when convenient, CI and the wasm32 check this PR adds? Both have been waiting for approval since 2026-09-23, so neither has run yet.