Skip to content

Field-agnostic prover: extract provekit-field-bn254 + provekit-noir#455

Open
BornPsych wants to merge 13 commits into
worldfnd:v2from
BornPsych:ys/field-agnostic-prep
Open

Field-agnostic prover: extract provekit-field-bn254 + provekit-noir#455
BornPsych wants to merge 13 commits into
worldfnd:v2from
BornPsych:ys/field-agnostic-prep

Conversation

@BornPsych

@BornPsych BornPsych commented Jun 16, 2026

Copy link
Copy Markdown

Summary

Makes the proving spine (provekit-common, provekit-prover, provekit-verifier) field-agnostic and moves all bn254-specific code into a new provekit-field-bn254 crate. The Noir/mavros frontend moves into a new provekit-noir crate. bn254 output is unchanged from v2.

The field is now chosen by which crate a binary depends on, not by a Cargo feature. The spine names no concrete field; a binary links the one field crate it registers. Adding a field is a new crate plus a register() call.

This is the first of two PRs, both targeting v2. It has no Goldilocks code; the Goldilocks crate comes in the follow-up.

Why a runtime registry

The orphan rule blocks impl SomeTrait for ark_bn254::Fr from living in provekit-field-bn254 (it can only live in the trait's crate or in arkworks). So instead of a per-field trait impl, common exposes a FieldHashProvider registry, the same mechanism whir uses for its ENGINES/NTT registries. provekit_field_bn254::register() installs the provider and engines at startup, and the spine looks them up at runtime.

What changed

  • common: field-agnostic spine; new FieldHashProvider registry and DynFieldSponge; FieldElement derived from a single Identity<Fr> instantiation point; drops the ntt/poseidon2/skyscraper deps.
  • provekit-prover: generic WHIR engine, no field/acir/nargo deps.
  • provekit-verifier: field-agnostic; the caller registers the backend.
  • provekit-field-bn254 (new): Skyscraper/Poseidon2/NTT, the EC/bigint/witness-hint code, the FieldHashProvider impl, and register(). The bn254 public-input KATs live here.
  • provekit-noir (new): the Noir/mavros Prove impls, witness solving, and input glue.
  • Tooling (cli/ffi/bench/wasm) rewired to provekit-noir; standalone-verify paths and pk_init call register().

Verification

  • e2e compiler suite: 73/73 pass.
  • Field KATs, relocated unit tests, and MSM witness-solving tests pass.
  • cargo tree: common, prover, and verifier link no field crate.
  • bn254 is unchanged from v2: the R1CS-construction path has an empty diff against v2, and a regression test in provekit-bench pins basic-4's R1CS hash and proof-structure params.

WHIR ZK proofs use thread_rng blinding and are non-deterministic by design, so the regression test pins deterministic artifacts and checks proofs functionally rather than byte-comparing them.

Follow-ups (not blocking)

  • The Noir scheme structs still live in common, so common/prover keep the acir/mavros deps. This is the frontend axis (Noir is bn254-only) and is independent of the field work here.
  • The relocated bigint/EC/NTT code carries pre-existing clippy warnings (same as on v2); a focused cleanup is a separate task.

Introduce a ProofField trait carrying the per-field glue the WHIR spine
needs but whir's Embedding trait does not provide: engine/NTT registration,
field-native Merkle-hash engine ids, the public-input binding hashes, and
the field-native Fiat-Shamir sponge constructor. Implemented for the bn254
scalar field.

DynFieldSponge is an object-safe shim over spongefish's DuplexSpongeInterface
(whose methods return &mut Self, so it is not object-safe), letting the
runtime TranscriptSponge hold a field-native sponge behind a trait object
while staying field-agnostic.

Additive only: the trait is defined and implemented but not yet consumed,
so bn254 behaviour is unchanged.
Replace the direct skyscraper/poseidon2/ntt references in the spine with
ProofField calls so common no longer names the field crates outside the
trait impl:

- register_ntt() delegates to FieldElement::register_engines()
- HashConfig::engine_id and hash_field_elements dispatch the Skyscraper and
  Poseidon2 arms through ProofField (the private hash_skyscraper/hash_poseidon2
  helpers moved into the trait impl)
- TranscriptSponge holds the field-native sponge behind DynFieldSponge and
  builds it via ProofField::field_sponge, collapsing the Skyscraper/Poseidon2
  variants into one Field arm

Byte-identical: all public-input-binding KATs (skyscraper/poseidon2/sha256/
keccak/blake3) unchanged.
…iation

Replace the hardcoded `pub use ark_bn254::Fr as FieldElement` with a single
instantiation point:

  pub type ProvekitEmbedding = Identity<ark_bn254::Fr>;
  pub type FieldElement = <ProvekitEmbedding as Embedding>::Source;

At Identity the base and extension fields coincide, so FieldElement still
resolves to bn254::Fr and the spine is byte-identical. Choosing a field is now
a localized change here plus a ProofField impl, which keeps the follow-up
Goldilocks work purely additive.
…a a registered provider

Replace the in-common ProofField trait-impl with a runtime FieldHashProvider
registry: common now reaches the field-native Merkle engine ids, public-input
binding hashes, and Fiat-Shamir sponge through a registered provider, and names
no field crate (skyscraper/poseidon2/ntt) — verified via cargo tree.

- New provekit-field-bn254 crate: holds the moved skyscraper/poseidon2/ntt
  modules, implements FieldHashProvider for bn254, and exposes register() which
  installs the NTT + Merkle hash engines and the provider (same pattern as
  whir's ENGINES/NTT registries).
- common drops its ntt/poseidon2/skyscraper dependencies and the provekit_ntt
  feature; the orphan rule keeps the per-field impl out of common, so the
  registry is the seam that yields true crate-level field isolation (a bn254
  binary never links a goldilocks crate) and keeps the follow-up additive.
- Registration moves to the callers: prover/verifier register at entry, the
  r1cs-compiler frontend registers before reading the engine id at scheme
  construction, and the FFI registers in pk_init (all transitional until A6).
- Public-input binding KATs relocate to provekit-field-bn254 (they exercise the
  real registered provider; a dev-only dependency cycle would split common into
  two instances with separate registries).

bn254 stays byte-identical; full workspace builds, KATs and an end-to-end Noir
prove/verify pass.
Relocate the field-specific witness-hint computation from the prover into the
bn254 field crate: ec_arith (EC scalar mul / point add+double), bigint_mod
(256-bit modular arithmetic), and the witness solvers digits/limb_io/ram. Their
solver traits travel with them, so the per-field impls stay orphan-legal (trait
local to field/bn254).

The prover's per-builder solve() dispatch stays in the prover for now (it reads
the ACIR witness map + transcript) and calls the relocated computation through
provekit_field_bn254; it moves to the frontend in the prover split. ec_scalar_mul
is re-exported from the prover for the MSM witness-solving tests.

The whole solve() match moves intact (no per-variant split), so witnesses are
byte-identical by construction: the 16 MSM witness-solving tests, the field KATs,
and an end-to-end Noir prove/verify all pass; common stays field-free.
…r the backend

Drop the in-verify provekit_field_bn254::register() call and the verifier's
field-bn254 dependency, so the verifier crate no longer links any field crate
(cargo tree confirms). Registration becomes the caller's responsibility: the
prove path already registers (via the r1cs-compiler scheme construction), and
the standalone-verify entry points (cli verify, the verify benchmark, the wasm
verify endpoint) now call register() before verifying.

e2e prove/verify green.
provekit-prover is now a field- and Noir-agnostic WHIR proving engine: it keeps
only whir_r1cs.rs (commit / prove_noir / prove_mavros / sumcheck / blinding),
operating on R1CS + a solved witness + a WhirR1CSScheme, and no longer depends
on provekit-field-bn254, acir, or nargo (cargo tree confirms).

The new provekit-noir crate owns the Noir/mavros orchestration: the Prove trait
and its impls, the witness solving (solve_witness_vec + the per-builder solve()
match), CompressedR1CS/Layers, input_utils, and logging. It depends on the
prover engine + provekit-field-bn254 and registers the field backend at the
prove entry points.

All tooling (cli/ffi/bench/wasm + passport playground) now uses provekit-noir
for proving; the prover engine stays a pure dependency. bn254 byte-identical:
field KATs, the 16 MSM witness-solving tests, and end-to-end Noir prove/verify
all pass.

Residual: the engine still links mavros for prove_mavros (frontend axis, not
field) — extracting it is a later frontend-isolation follow-up.
Pin the byte-exact deterministic artifacts of a fixed circuit (basic-4) so the
field-agnostic refactor cannot silently change bn254 behaviour: the R1CS
Fiat-Shamir binding hash (SHA3-256 over the serialized R1CS) and the
proof-structure parameters (m, w1_size, m_0, num_challenges) that feed the
domain separator.

The goldens equal v2's values: the entire R1CS-construction path (common's
r1cs/sparse_matrix/interner/optimize + the r1cs-compiler noir_to_r1cs and the
WitnessBuilder schema) is byte-identical to upstream/v2 (git diff empty), so the
pinned hash is v2's. Public-input binding hashes are pinned by the KATs in
provekit-field-bn254; prove->verify ACCEPT and tamper->REJECT are covered by the
compiler cases and test_public_input_binding_exploit.
@vercel

vercel Bot commented Jun 16, 2026

Copy link
Copy Markdown

@BornPsych is attempting to deploy a commit to the World Foundation Team on Vercel.

A member of the Team first needs to authorize it.

@BornPsych BornPsych changed the title Field-agnostic spine: extract provekit-field-bn254 + provekit-noir (PR A, v2 prep) Field-agnostic prover: extract provekit-field-bn254 + provekit-noir Jun 16, 2026
- Shorten and de-narrate ~18 comments/doc-comments across the field-agnostic
  refactor (common, field-bn254, noir, prover, verifier, r1cs-compiler, bench):
  drop PR-narrative and rationale that belongs in the PR, cut filler.
- Fix broken intra-doc links so `cargo doc -D warnings` passes workspace-wide:
  the FieldElement type-alias method link, the renamed FieldHashProvider trait,
  and the poseidon2/sponge.rs links left dangling by the crate move.
- wasm: add default-features=false to the provekit-noir workspace dep so the
  wasm build's feature opt-out is honored (mirrors provekit-prover).
- CLAUDE.md: update the registration snippet to provekit_field_bn254::register().
Make MaybeHashAware pub and re-export Compression from the file module so
frontend crates (provekit-noir) can implement FileFormat/MaybeHashAware for
their own relocated types. Drop the now-unneeded private_bounds allows.

No behavior change; prep for moving the Noir frontend types out of common.
@shreyas-londhe

Copy link
Copy Markdown
Collaborator

Solid carve. Verified bn254 is byte-identical to v2: the boxed sponge forwards to the same DuplexSpongeInterface calls, and the public-input KATs were moved verbatim, not regenerated. Hash-via-registry is the right call too. whir dispatches its own Merkle hashes through ENGINES/EngineId, and hash config is runtime data, so a registry fits.

One architectural gap holds this back as a base for the Goldilocks work, plus a few panics to clean up.

Make the algebra generic over Embedding. Field-agnostic hashing isn't enough.

The spine still pins one concrete FieldElement (common/src/lib.rs: ProvekitEmbedding = Identity<Fr>, FieldElement = ::Source) and uses it for both committed data (coeffs/w1/w2 in interner.rs, r1cs.rs, sparse_matrix.rs) and challenges (verifier/src/whir_r1cs.rs). Goldilocks needs those to be different types: committed data in M::Source (base, Field64), challenges in M::Target (ext, Field64_3). A single alias can't carry both. So the flip to Basefield<Field64_3> re-threads the whole spine anyway, and the Identity-vs-Basefield differential test is impossible, since one global alias plus a one-slot provider can't hold two fields in one binary.

The fix is additive:

  • Keep the hash registry. whir does the same.
  • Make the math (sumcheck, covectors, sparse-matrix) generic over <M: Embedding>. Committed data becomes M::Source, challenges M::Target, base×ext products go through mixed_mul/mixed_dot. Instantiate Identity<Fr> for bn254; output stays byte-identical.
  • whir::Config<M: Embedding> is already generic, so this matches the dep. Our wrappers calling whir want the same M.

This is how the references layer it. whir: Config<M> for algebra, separate ENGINES registry for hashing. Plonky3: StarkGenericConfig bundle at the prove/verify entry, AIR and field math generic over F: Field underneath. None of arkworks, whir, or Plonky3 put a hash method on the field trait.

The earlier impl ProofField for Fr hit the orphan rule (E0117: both the trait and Fr are foreign to field/bn254), which is why the registry replaced it. Fine. If a per-field trait comes back later, impl it for a local marker (struct Bn254; impl ProofField for Bn254 { type Embedding = Identity<Fr>; }), same as arkworks FrConfig, Plonky3 BabyBearParameters, whir's own FConfig64. The math doesn't need that trait though. Generic over Embedding is leaner.

Panics in library paths. Fix before merge.

  • common/src/field/mod.rs: provider() is .expect(), so every Skyscraper/Poseidon2 path panics if register() wasn't called. Return a Result and propagate.
  • verifier/src/lib.rs: verify() dropped self-registration. A verify-only caller that skips register() panics. Tests survive only because the prover registers first in-process. Re-register or return Err.
  • field/bn254/src/lib.rs: field_sponge() is unreachable!() on non-native configs, but it's a public trait method. Narrow the input type or return Option.
  • common/src/field/mod.rs: register_field_hash_provider does let _ = OnceLock::set(...), silently dropping a conflicting second registration. Once a second field crate exists that's a silent wrong-field bug. debug_assert! or panic on conflict.

Scope. Ship this as the modularization plus hash registry. That part's good. Track the Embedding genericization as the next PR. That's the piece that carries base/ext.

Move NoirProofScheme/NoirSchemeData/NoirProof, Prover/NoirProver,
MavrosProver/MavrosSchemeData, Verifier, and NoirWitnessGenerator out of
the field-agnostic spine (provekit-common) into the bn254/Noir frontend
crate (provekit-noir), together with their FileFormat/MaybeHashAware impls
(new file_format.rs, native-only).

The five type files are git-renamed verbatim (only import paths retargeted),
so serde field/variant order is unchanged and the wire format is byte-identical
(bn254 deterministic-artifact gate stays green). NoirWitnessGenerator's fields
are now encapsulated behind a pub constructor; the r1cs-compiler builds it via
a free function (build_noir_witness_generator) instead of a struct literal.

Consumers (cli/ffi/wasm/bench/verifier/verifier-server/passport) repointed to
provekit_noir; r1cs-compiler/verifier/verifier-server gain a provekit-noir dep
(default-features off, types only). NoirElement/noir_to_native stay in common
for now (relocated to provekit-field-bn254 in the next commit).
… axis

Relocate the last Noir-toolchain pieces out of the field-agnostic spine so
provekit-common no longer depends on acir/noirc_abi/mavros-vm/mavros-artifacts:

- NoirElement alias + noir_to_native -> provekit-field-bn254 (new noir_element
  module; field-bn254 gains an acir dep). The acir<->native bridge only
  type-checks when the native field is bn254, so it belongs in the bn254 crate
  and is naturally excluded from non-bn254 builds.
- PrintAbi -> provekit-r1cs-compiler (git-renamed; sole consumer).
- convert_mavros_r1cs_to_provekit -> provekit-r1cs-compiler (new mavros_convert
  module; uses mavros-artifacts, only consumer). InternedFieldElement stays
  private to common (inferred, not named).
- Delete the orphan utils/file_io.rs (never wired into the module tree).
- Drop acir/noirc_abi/mavros-vm/mavros-artifacts from common/Cargo.toml.

Bridge consumers (noir, r1cs-compiler hot path, wasm, bench) repointed to
provekit_field_bn254. common/src now references the Noir toolchain only in a
code comment. Determinism gate + field KATs stay green; verifier/verifier-server
pull provekit-noir as types-only (no nargo/witness-generation).
Address review feedback on the field-agnostic seam:

- register_field_hash_provider: debug_assert on a second registration. Each
  field crate's register() is Once-guarded, so a failed OnceLock::set means a
  *different* field backend already registered — a silent wrong-field bug once
  more than one field crate exists. The set() still runs unconditionally; only
  the check is debug-gated (no release-build trap).
- ProvekitEmbedding doc: stop implying a field switch is just an alias flip.
  Identity has Source == Target, so the spine names one FieldElement for both
  committed data and challenges; an embedding with a distinct extension needs
  committed data in Embedding::Source and challenges in Embedding::Target,
  i.e. threading <M: Embedding> through the algebra. Deferred to the
  field-selection PR.

Doc/assert-only: no wire-format or behavior change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants