Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,12 @@ before 1.0).

### Changed

- **Schema 21:** no `spent.idx`. Spent ranges are `8 × max(n_out,1)` from
txout meta (sparse `spent.off`). Open of `meta=20` unlinks leftover
`spent.idx` and rewrites `store/meta` to 21. A 20 binary refuses 21 `meta`.
- **Schema 22:** restore `spent.idx` (u32 stride-8). Spent slot is flags +
u40 spend fk + u16 vin. Occupied 15–21 Class A with creates refuses
(`wipe datadir and redo IBD`). Empty 15–21 rewrite `meta` to 22 and
unlink leftover `spent.off`. A 21 binary refuses 22 `meta`. Esplora
`/outspend(s)` emits `vin` from the slot (mempool overlay uses the hub
tx input index). First-wave Outs guess is `4+(max_vout+1)×38`.

### Added

Expand Down Expand Up @@ -181,10 +184,11 @@ before 1.0).
Core-functional `bitcoind` shim still ignores `-permitbaremultisig`.
`getnetworkhashps` is labeled dummy 2-work-per-block / elapsed (not Core
chainwork hashrate). [`docs/rpc.md`](docs/rpc.md) / [`OPERATOR.md`](OPERATOR.md).
- **Esplora outspend omits `vin`:** `/tx/:txid/outspend/:vout` and
`/outspends` no longer emit always-zero `vin`. `PointRecord` /
`put_spend` / `put_spend_batch` no longer take a dummy spending input
index (`spent.body` never stored it). [`COMPAT.md`](COMPAT.md).
- **Esplora outspend includes `vin`:** `/tx/:txid/outspend/:vout` and
`/outspends` emit the spending input index from the schema-22 spent
slot (`PointRecord.spending_vin`). Mempool overlay uses the hub tx’s
input index. Unspent remains `{spent:false}` with no `vin`.
[`COMPAT.md`](COMPAT.md).
- **Leftover index layouts refuse on open:** fuse8 **v1** sealed filters, flat
`tx.head.meta`, flat `*.idx.meta`, Shared (file) `scripthash.body`, and pack8
**Paged** (mode 10) fail closed with a one-line wipe/rebuild message (Class A
Expand Down
11 changes: 5 additions & 6 deletions COMPAT.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,10 @@ surfaces). Those need reverse indexes and explorer-only APIs we deliberately
omit. Block/tx **by full id** and address/**exact** scripthash history exist so
wallets and APIs can verify and sync—not so we become mempool.space.

**Product gap:** `/tx/:txid/outspend/:vout` and `/outspends` omit Blockstream
`vin` (spending input index). `spent.body` stores only `spending_tx_fk`; vin
is not on disk. Serving it would cold-read the spending tx’s `inwit`. Full
`/tx/:txid` JSON still has `vin[]`. Explorer flow-graphs that need outspend
`vin` are out of scope until that is a product goal.
**Product:** `/tx/:txid/outspend/:vout` and `/outspends` emit Blockstream
`vin` (spending input index) from the schema-22 spent slot. Mempool overlay
uses the hub tx’s input index. Unspent remains `{spent:false}` with no `vin`.
Full `/tx/:txid` JSON still has `vin[]`. Electrum has no outspend-vin surface.

## Intentional differences

Expand Down Expand Up @@ -176,7 +175,7 @@ via reverse proxy; app `ServeLimits` always on (same model as Electrum).
| Tip | done | `/blocks/tip/height`, `/blocks/tip/hash`. REST stamps `X-Bitcoin-Chain-Tip` / `X-Bitcoin-Chain-Tip-Height` (CORS-exposed): **live tip** for block/tx/header routes; **SH watermark** for `/address/` and `/scripthash/` so wallet JSON matches the SH join. Empty chain omits them (existing 503). If the pin dies mid-request: **503** `chain view moved`. |
| Blocks list | done | `/blocks`, `/blocks/:start_height` (10 summaries, newest-first) |
| Block | done | `/block/:hash` JSON, `/raw`, `/status`, `/header`, `/txids`, `/txid/:i`, `/txs[/:start]`. JSON `bits` is the compact-target **u32** (Esplora schema, not Core hex). `size` / `weight` are BIP144 total size and BIP141 weight (witness included). |
| Tx | done | `/tx/:txid` full JSON, `/hex`, `/raw`, `/status`, Electrum `/merkle-proof`, BIP37 `/merkleblock-proof`, `/outspend(s)` (**no** `vin` on outspend — explorer gap above). Mempool-only txs (not in Class A) use the wire body from the mempool hub (`vin`/`vout`/`size`/`weight`/`fee`, `status.confirmed` false) including `GET /tx/:txid/status`. Live `/outspend(s)` overlay mempool spends of confirmed coins; `?asof=` omits mempool. `?asof=<hash>` on `/status` and `/outspend(s)`: confirmed/spent as of that ancestor; 404 if not on chain. |
| Tx | done | `/tx/:txid` full JSON, `/hex`, `/raw`, `/status`, Electrum `/merkle-proof`, BIP37 `/merkleblock-proof`, `/outspend(s)` (`vin` from the spent slot; unspent omits it). Mempool-only txs (not in Class A) use the wire body from the mempool hub (`vin`/`vout`/`size`/`weight`/`fee`, `status.confirmed` false) including `GET /tx/:txid/status`. Live `/outspend(s)` overlay mempool spends of confirmed coins; `?asof=` omits mempool. `?asof=<hash>` on `/status` and `/outspend(s)`: confirmed/spent as of that ancestor; 404 if not on chain. |
| Address / scripthash | done | stats + `/utxo` + `/txs` + `/txs/mempool` + `/txs/chain[/:last_seen_txid]`; `/utxo` matches Electrum listunspent (mempool funding + drop mempool-spent confirmed); `/txs` and `/txs/mempool` use full Esplora tx JSON for mempool-only rows (wire from the hub). Last **one** SH join reused across sequential REST calls until SH-view **hash** changes; concurrent different SHs re-join. Needs SH finalize. Stamp is visible SH (durable + pending write-behind), matching live tip while jobs sit in RAM. `?asof=<hash>` on `/`, `/utxo`, `/txs`, `/txs/chain`: confirmed join at that ancestor **at or behind visible SH**, **no** mempool; headers are the asof hash; 404 if not on chain or ahead of visible SH. |
| Mempool / fees | done | `/mempool`, `/mempool/txids`, `/mempool/recent` (accept-order ring), `/fee-estimates` |
| `POST /tx` | done | broadcast via mempool hub; **503** if hub absent |
Expand Down
31 changes: 21 additions & 10 deletions OPERATOR.md
Original file line number Diff line number Diff line change
Expand Up @@ -581,7 +581,7 @@ counts, ingest OA, and refuse lines: [`SCHEMA.md`](./SCHEMA.md) and

## Schema upgrade

Live bytes: [`SCHEMA.md`](./SCHEMA.md) (`SCHEMA_VERSION = 21`). This section is
Live bytes: [`SCHEMA.md`](./SCHEMA.md) (`SCHEMA_VERSION = 22`). This section is
the operator copy-paste only — do not treat it as a second layout map.

Open **never silently wipes** a populated store (policy:
Expand All @@ -591,15 +591,26 @@ names the dirs. Corrupt files are **not** repaired in-process.

| Incoming `meta` | What this binary does |
|-----------------|------------------------|
| **21** | Open. |
| **20** | Unlink leftover `spent.idx`, rewrite `meta` to 21, then open. |
| **19** or **18**, empty `tx.head` and no `scripthash*` data | Rewrite `meta` to 21, then open. |
| **19** or **18**, occupied `tx.head` or any `scripthash*` | **Refuse.** Wipe `store/tx.head` and `store/scripthash*`, keep Class A, restart. |
| **17**, empty `tx.head` and no `scripthash*` data | Rewrite `meta` to 21, then open. |
| **17**, populated `tx.head` or any `scripthash*` | **Refuse.** Wipe those index dirs, keep Class A, restart. |
| **22** | Open. |
| **21**, empty Class A | Unlink leftover `spent.off`, rewrite `meta` to 22, then open. |
| **21**, occupied Class A | **Refuse.** Wipe datadir and redo IBD. |
| **20**, empty Class A | Unlink leftover `spent.off`, rewrite `meta` to 22, then open. |
| **20**, occupied Class A | **Refuse.** Wipe datadir and redo IBD. |
| **19** or **18**, empty Class A and empty `tx.head` / no `scripthash*` data | Rewrite `meta` to 22, then open. |
| **19** or **18**, occupied Class A | **Refuse.** Wipe datadir and redo IBD. |
| **19** or **18**, empty Class A, occupied `tx.head` or any `scripthash*` | **Refuse.** Wipe `store/tx.head` and `store/scripthash*`, keep Class A, restart. |
| **17**, empty Class A and empty `tx.head` / no `scripthash*` data | Rewrite `meta` to 22, then open. |
| **17**, occupied Class A | **Refuse.** Wipe datadir and redo IBD. |
| **17**, empty Class A, populated `tx.head` or any `scripthash*` | **Refuse.** Wipe those index dirs, keep Class A, restart. |
| Older than 17 with creates / leftover catalogs | **Refuse.** The error names files; often a full datadir wipe + IBD. Details: SCHEMA.md **13/14→17**, **15→17**, **16→17**. |

A **20 binary** refuses 21 `meta` (do not downgrade in place). A **19 binary** refuses 20+ `meta`.
A **21 binary** refuses 22 `meta` (do not downgrade in place). A **19 binary** refuses 20+ `meta`.

When the schema-22 Class A refuse fires, the log line is:

```text
schema 22 refuses schema-21 Class A with creates; wipe datadir and redo IBD
```

When the 20 index refuse fires, the log line is:

Expand Down Expand Up @@ -630,10 +641,10 @@ rm -rf "$DATADIR/store/tx.head" "$DATADIR/store/scripthash"*
# mkdir -p "$DATADIR/store/txout.idx"
# mv "$DATADIR/store/txout.idx.meta" "$DATADIR/store/txout.idx/meta"
# mv "$DATADIR/store/txout.idx."[0-9][0-9][0-9][0-9][0-9][0-9] "$DATADIR/store/txout.idx/"
# same for inwit.idx (spent.idx is unlinked on open)
# same for inwit.idx and spent.idx
```

Keep Class A (`txout` / `inwit` / `spent` + `txout.idx` / `inwit.idx`, `txid.body`, headers) and
Keep Class A (`txout` / `inwit` / `spent` + `txout.idx` / `inwit.idx` / `spent.idx`, `txid.body`, headers) and
Class C. Restart the same binary: `tx.head` rebuilds from Class A; with
`--shindex`, SH rematerializes. Do **not** `rm -rf store/`.

Expand Down
Loading
Loading