Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions fynd-core/src/solver.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1167,6 +1167,16 @@ impl Solver {
self.market_event_tx.subscribe()
}

/// Returns a clone of the [`MarketEvent`] broadcast sender.
///
/// A harness that writes synthetic components into [`MarketData`] must also announce them, or
/// workers never add the edge to their local graph. This is the same channel the Tycho feed
/// publishes real block updates on, so an injected event is indistinguishable from a real one.
#[cfg(feature = "experimental")]
pub fn market_event_sender(&self) -> broadcast::Sender<MarketEvent> {
self.market_event_tx.clone()
}

/// Submits a [`QuoteRequest`] to the worker pools and returns the best [`Quote`].
///
/// # Errors
Expand Down
103 changes: 71 additions & 32 deletions fynd-core/src/worker_pool_router/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,14 @@ use crate::{
SolveParams, SurplusInfo,
};

/// Minimum improvement over the public route, in basis points of the public route's output net of
/// gas, that an exclusive-access route must deliver before it is quoted.
///
/// The margin goes to the user: the commitment is raised by it, so the protocol only captures what
/// the route produces on top. A candidate that beats the public route by exactly this margin is
/// still quoted, with zero surplus.
const EXCLUSIVE_ROUTE_IMPROVEMENT_BPS: u64 = 1;

/// Which liquidity a solver pool (a group of workers) routes through, and therefore what its
/// candidates mean in a quote.
///
Expand Down Expand Up @@ -637,22 +645,27 @@ impl WorkerPoolRouter {
/// holds the candidates from `ExclusiveAccess`-scoped pools (routes that may use exclusive
/// components).
///
/// The user's net target is `required_net = public_net + margin`, where the margin is
/// `EXCLUSIVE_ROUTE_IMPROVEMENT_BPS` of `public_net`. A candidate whose net output falls below
/// `required_net` is skipped, so quoting an exclusive route always leaves the user better off than
/// the public market by at least that margin.
///
/// The committed amount is the larger of two lower bounds:
/// `max(public_amount_out, public_net + exclusive_gas)`. The first guarantees the quoted
/// `max(public_amount_out, required_net + exclusive_gas)`. The first guarantees the quoted
/// `amount_out` is never below the public market's; the second guarantees the user — who pays
/// the exclusive route's gas — nets at least what the public route would leave them.
/// the exclusive route's gas — nets at least `required_net`.
///
/// Which bound is larger depends on the gas comparison:
/// - `exclusive_gas > public_gas`: committed = `public_net + exclusive_gas`, which exceeds
/// - `exclusive_gas > public_gas`: committed = `required_net + exclusive_gas`, which exceeds
/// `public_amount_out`. The user receives more tokens than the public quote and nets exactly
/// `public_net`.
/// `required_net`.
/// - `exclusive_gas <= public_gas`: committed = `public_amount_out`. The user nets
/// `public_amount_out − exclusive_gas`, which exceeds `public_net` by the gas difference. A
/// commitment of `public_net + exclusive_gas` would still leave the user whole and capture more,
/// but it is below `public_amount_out` — quoting less than the public market is ruled out, so the
/// gas difference stays with the user.
/// commitment of `required_net + exclusive_gas` would still leave the user whole and capture
/// more, but it is below `public_amount_out` — quoting less than the public market is ruled out,
/// so the gas difference stays with the user.
///
/// If the best exclusive-access candidate beats the public reference net-of-gas and produces at
/// If the best exclusive-access candidate clears `required_net` and produces at
/// least the committed amount, this returns a new list whose head is the pinned surplus quote,
/// followed by every public candidate as price-guard fallbacks. The pinned quote is the winning
/// candidate with:
Expand All @@ -661,7 +674,8 @@ impl WorkerPoolRouter {
/// - an order-level [`SurplusInfo`] attached (observability).
///
/// Otherwise `public_ranked` is returned unchanged. Either way the user is never worse off than
/// the public market — neither in quoted `amount_out` nor net of gas.
/// the public market — neither in quoted `amount_out` nor net of gas. A candidate that clears
/// `required_net` exactly is quoted with zero surplus: the whole improvement goes to the user.
///
/// Per-leg attribution: the route's excess over the committed amount (`realized − committed`)
/// is deducted from the exclusive legs — each leg absorbs what it can, capped at its own output,
Expand Down Expand Up @@ -712,8 +726,9 @@ fn combine_with_surplus(
return public_ranked;
};

// The candidate route must beat the committed reference net-of-gas.
if exclusive_candidate.amount_out_net_gas() <= committed.amount_out_net_gas() {
// The candidate route must beat the public reference net-of-gas by the improvement margin.
let required_net_amount_out = with_improvement_margin(committed.amount_out_net_gas());
if exclusive_candidate.amount_out_net_gas() < &required_net_amount_out {
return public_ranked;
}

Expand All @@ -734,10 +749,11 @@ fn combine_with_surplus(

// The commitment is the larger of two lower bounds: the quoted amount_out is never below
// the public market's, and the user — who pays the exclusive route's gas — never nets less
// than the public route would leave them (public_net + gas). Together with the strict net
// check above, these gates guarantee the route covers the committed amount.
// than the public route would leave them plus the improvement margin (required_net + gas).
// Together with the net check above, these gates guarantee the route covers the committed
// amount.
let committed_amount_out =
(committed.amount_out_net_gas() + &exclusive_gas_cost).max(public_amount_out.clone());
(&required_net_amount_out + &exclusive_gas_cost).max(public_amount_out.clone());

// What the hooks capture: everything the route produces above the commitment.
let surplus_amount = exclusive_route_amount_out - &committed_amount_out;
Expand Down Expand Up @@ -818,6 +834,17 @@ fn combine_with_surplus(
result
}

/// Returns `amount` raised by `EXCLUSIVE_ROUTE_IMPROVEMENT_BPS`.
///
/// The margin is rounded up and is at least one atomic unit, so it never rounds away on small
/// amounts: an exclusive route that clears it always leaves the user strictly better off.
fn with_improvement_margin(amount: &BigUint) -> BigUint {
let denominator = BigUint::from(10_000u64);
let margin = (amount * BigUint::from(EXCLUSIVE_ROUTE_IMPROVEMENT_BPS) + (&denominator - 1u64)) /
&denominator;
amount + margin.max(BigUint::from(1u64))
}

/// Returns `true` only for routes carrying exactly one exclusive leg, positioned as the terminal
/// leg of its path.
///
Expand Down Expand Up @@ -1835,19 +1862,31 @@ mod tests {
/// Head selection across the gate case matrix. Each route is given as `(gross, net)`;
/// expected is `(head amount_out, captured surplus)`. A surplus win prepends the pinned
/// quote to the public fallbacks; otherwise the public ranking is returned unchanged.
///
/// The improvement margin on these amounts is 1 (the one-atomic-unit floor) except in the
/// 1_000_000 cases, where 1 bps is 100.
#[rstest]
#[case::exclusive_beats_public((900, 900), (950, 950), (900, 900, Some(50)))]
#[case::exclusive_beats_public((900, 900), (950, 950), (901, 901, Some(49)))]
#[case::exclusive_below_public((950, 950), (900, 900), (950, 950, None))]
#[case::exclusive_ties_public_net((900, 900), (900, 900), (900, 900, None))]
#[case::exclusive_marginally_better((999, 999), (1000, 1000), (999, 999, Some(1)))]
#[case::exclusive_marginally_better((999, 999), (1000, 1000), (1000, 1000, Some(0)))]
#[case::exclusive_cannot_cover_public_gross((1000, 950), (990, 980), (1000, 950, None))]
// Gas-heavier exclusive route: the committed amount rises to max(1000, 950 + 140) = 1090 so
// the user still nets the public 950; the protocol captures 1100 - 1090 = 10.
#[case::exclusive_with_higher_gas((1000, 950), (1100, 960), (1090, 950, Some(10)))]
// Gas-heavier exclusive route: the committed amount rises to max(1000, 951 + 140) = 1091 so
// the user nets the public 950 plus the margin; the protocol captures 1100 - 1091 = 9.
#[case::exclusive_with_higher_gas((1000, 950), (1100, 960), (1091, 951, Some(9)))]
// Gas-cheaper exclusive route: the public amount is the larger bound (committed = 1000);
// the user keeps the
// gas saving (nets 960) and the protocol captures 100.
#[case::exclusive_with_lower_gas((1000, 950), (1100, 1060), (1000, 960, Some(100)))]
// 1 bps of 1_000_000 is 100: a candidate only 50 better than the public route is rejected.
#[case::exclusive_below_margin((1_000_000, 1_000_000), (1_000_050, 1_000_050),
(1_000_000, 1_000_000, None))]
// Exactly at the margin: the route is quoted and the whole 100 goes to the user, surplus 0.
#[case::exclusive_at_margin((1_000_000, 1_000_000), (1_000_100, 1_000_100),
(1_000_100, 1_000_100, Some(0)))]
// Above the margin: the user gets the 100, the protocol captures the remaining 400.
#[case::exclusive_above_margin((1_000_000, 1_000_000), (1_000_500, 1_000_500),
(1_000_100, 1_000_100, Some(400)))]
fn test_combine_head_selection(
#[case] public: (u64, u64),
#[case] exclusive: (u64, u64),
Expand Down Expand Up @@ -1956,14 +1995,14 @@ mod tests {
.expect("should have an exclusive swap");

// committed_leg = leg.amount_out * committed_route_out / realized_route_out
// = 1000 * 900 / 1000 = 900
assert_eq!(perm_swap.committed_amount_out(), Some(&BigUint::from(900u64)),);
// = 1000 * 901 / 1000 = 901
assert_eq!(perm_swap.committed_amount_out(), Some(&BigUint::from(901u64)),);
}

#[test]
fn test_combine_committed_leg_deduction() {
// leg = 995, committed = 900, realized = 1000: the route's excess (100) is deducted from
// the exclusive leg in full — committed_leg = 995 − 100 = 895, exactly, no rounding.
// leg = 995, committed = 901, realized = 1000: the route's excess (99) is deducted from
// the exclusive leg in full — committed_leg = 995 − 99 = 896, exactly, no rounding.
let responses = OrderResponses {
order_id: "test-order".to_string(),
quotes: vec![
Expand Down Expand Up @@ -2002,7 +2041,7 @@ mod tests {
.iter()
.find(|s| policy.is_exclusive(s.protocol_component()))
.expect("should have an exclusive swap");
assert_eq!(perm_swap.committed_amount_out(), Some(&BigUint::from(895u64)));
assert_eq!(perm_swap.committed_amount_out(), Some(&BigUint::from(896u64)));
}

#[test]
Expand All @@ -2025,9 +2064,9 @@ mod tests {

#[test]
fn test_combine_split_route_attribution() {
// Split route: public branch 600 + exclusive branch 500 = 1100 realized vs 1000
// committed (zero gas). Only the exclusive leg is stamped; the public branch flows to
// the user untouched.
// Split route: public branch 600 + exclusive branch 500 = 1100 realized vs 1001
// committed (zero gas, 1 unit improvement margin). Only the exclusive leg is stamped;
// the public branch flows to the user untouched.
let responses = OrderResponses {
order_id: "test-order".to_string(),
quotes: vec![
Expand Down Expand Up @@ -2058,7 +2097,7 @@ mod tests {
Some(&policy),
);

assert_eq!(*combined[0].amount_out(), BigUint::from(1000u64));
assert_eq!(*combined[0].amount_out(), BigUint::from(1001u64));
let route = combined[0]
.route()
.expect("surplus quote should have a route");
Expand All @@ -2070,15 +2109,15 @@ mod tests {
assert_eq!(public_leg.committed_amount_out(), None);

// The public branch (600) pays out in full, so the entire excess
// (1100 − 1000 = 100) is deducted from the exclusive leg: committed_leg = 500 − 100 =
// 400. The user receives 600 + 400 = 1000 (exactly the committed amount) and the hook
// captures all 100.
// (1100 − 1001 = 99) is deducted from the exclusive leg: committed_leg = 500 − 99 =
// 401. The user receives 600 + 401 = 1001 (exactly the committed amount) and the hook
// captures all 99.
let exclusive_leg = route
.swaps()
.iter()
.find(|s| policy.is_exclusive(s.protocol_component()))
.expect("should have an exclusive swap");
assert_eq!(exclusive_leg.committed_amount_out(), Some(&BigUint::from(400u64)));
assert_eq!(exclusive_leg.committed_amount_out(), Some(&BigUint::from(401u64)));
}

/// Builds an `OrderResponses` where both quotes carry explicit `amount_out_net_gas`.
Expand Down
58 changes: 57 additions & 1 deletion tools/hindsight/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,16 @@ and takes neither.
back-of-block (state N), and emits `RangeComparison` JSONL records. Exposes a Prometheus
metrics endpoint (`--metrics-port`). `--max-lag-blocks` (default 100, ~20 min on mainnet)
bounds how far it may fall behind chain head before rebuilding the solver.
`--propamm-pair <IN,OUT>` additionally injects a mock PropAMM pool (see below).

- **`report`** — Offline: read the `comparisons-YYYY-MM-DD.jsonl` files a `monitor` run wrote
(`--comparisons-dir`) and render a single self-contained HTML file (`-o`, defaults to
`<dir>/report.html`) with the dashboard's value views — the headline Fynd savings, win rate, and
median savings bps; the verdict split by trade count and by volume; per-solver/venue breakdowns;
top-saving trades; and the unsolved token tail. `--venue <name>` (repeatable, case-insensitive)
restricts the report to those venues. No chain, Tycho, or network access.
restricts the report to those venues. No chain, Tycho, or network access. Records from a
`--propamm-pair` run also get a "Mock PropAMM" section (winrate, captured flow, fee headroom, and
a per-order-pair breakdown).

## Environment

Expand All @@ -43,6 +46,9 @@ and takes neither.
| `ALLIUM_API_KEY` | Allium API key (`verify` only) |
| `ALLIUM_QUERY_ID` | Saved Allium query ID (`verify` only) |
| `HINDSIGHT_REGISTRY` | Override path for the decoder address-book TOML |
| `PROPAMM_PAIR` | Token pair the mock PropAMM mirrors, comma-separated (`monitor` only) |
| `PROPAMM_OFFSETS_BPS` | Price offsets in bps off the public best route, e.g. `-5,0,5` |
| `PROPAMM_PROBE_UNITS` | Trade size used to pick which real pool the mock mirrors |

## Architecture

Expand Down Expand Up @@ -101,6 +107,56 @@ self-contained HTML file.
| `aggregate.rs` | Pure aggregations over the records (verdicts, coverage, savings, per-group, movers) |
| `html.rs` | Renders the aggregates to a self-contained HTML file (inline CSS, `<div>` bars, no assets) |

### Mock PropAMM (`src/propamm/`)

Test scaffolding for ENG-6157 — sizes what a dynamic-underbidding PropAMM pool would win before the
pool exists. Off unless `monitor --propamm-pair` is set.

| File | Purpose |
|---|---|
| `mod.rs` | `Injector` — writes a synthetic exclusive component into the running solver's `MarketState` once per block and announces it on the market-event channel |
| `mirror.rs` | `MirrorPool` — a `ProtocolSim` that delegates to the best real pool for the pair and scales its price by `--propamm-price-pct`, charging no fee |
| `report.rs` | Per-order outcomes and run totals; `Record` is what lands in the comparisons JSONL |

Each order on the mirrored pair is solved **twice**: once with the mock neutralised (scaled to one
part per million, so it stays in the graph but loses every comparison), which yields the public best
route Fynd would otherwise have quoted; then again with the mock rescaled so its output lands
`--propamm-offsets-bps` off that number. So an offset means "this much better than the route Fynd
would have quoted", not "this much better than some single pool".

That makes every calibrated order an assertion. The report groups them by offset and judges each
group against the behaviour its price implies:

| offset | expectation |
|---|---|
| below market | never selected — the router requires a strict beat |
| at market | can only win on gas, and then there is no surplus, so the fee must be zero |
| above market | the fee taken cannot exceed the offset, since the offset is all the surplus there is |

A group with no selections above market is reported as *no data*, not a failure: winning there
depends on gas as well as price. Orders off the mirrored pair carry no offset and form no group —
calibration is only exact when the mock serves the whole order in one hop.

Fynd's existing exclusive-access routing does the rest: `FyndBuilder::exclusivity_policy` hides the
mock from every configured worker pool, and each pool is twinned with a `liquidity_scope = "all"`
copy that sees it. Because the router pins a surplus quote's `amount_out` to the public commitment,
the mock never changes hindsight's own win/loss verdicts — it only adds this second measurement.

Requires `fynd-core`'s `experimental` feature for `Solver::market_event_sender`. Not for production:
the mock prices a pool that does not exist on chain, so any calldata it produces is unexecutable.

Two things to get right when running it:

- **Set `EXCLUSIVE_SWAP_CONTROLLER_KEY`** (any throwaway key — nothing is executed). The encoder
fails fast on an exclusive leg with no signer, which turns every win into a failed quote and
silently reports a zero winrate.
- **Pick a pair that has flow.** Only orders whose own pair is the mirrored one get calibrated, and
those are a small slice of settled flow — 40 mainnet blocks yielded 9 calibrated orders on
ETH/USDT. On ethereum, ETH/USDC carries roughly twice ETH/USDT's volume; check a run's per-pair
table before committing to a long one. Filling the groups is a matter of blocks, so budget for it.
- **`--min-tvl 10`** is mandatory against `tycho-fynd-ethereum`, which rejects any other value with
`tvl_gt must be == 10`.

### Verdict model

Each re-solved trade produces a `top` (optimistic, state N-1) and `back` (pessimistic, state N)
Expand Down
5 changes: 5 additions & 0 deletions tools/hindsight/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,15 @@ anyhow = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }

# Timestamps on the mock PropAMM component
chrono = { workspace = true }

# Serialization
serde = { workspace = true }
serde_json = { workspace = true }
toml = { workspace = true }
# Registers the mock PropAMM pool state as a `Box<dyn ProtocolSim>` implementation
typetag = { workspace = true }

[lints.clippy]
pedantic = { level = "warn", priority = -1 }
Expand Down
1 change: 1 addition & 0 deletions tools/hindsight/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
mod decoder;
mod propamm;
mod report;
mod resolve;
mod telemetry;
Expand Down
Loading
Loading