Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,9 @@ Required jobs: **`fmt`**, **`deny`**, **`clippy`**, **`ast-grep`**, **`test`**,
store + `--smoke` (not operator zips). Operator binaries are GitHub Releases
(`release.yml`). Releases (minor / patch / major): [`docs/releases.md`](docs/releases.md).
Label **`core-functional`** when the PR touches the Core functional harness
(and on every **ship** version-bump PR).
(and on every **ship** version-bump PR). Do **not** label ordinary net or
RPC PRs; the job is too slow for the default gate. Default `cargo test` is
the pin; owner rules: [`TESTING.md`](TESTING.md) (Default CI is the pin).

`origin` fetch/pull is HTTPS; `pushurl` is SSH (operator). This VM has **no**
GitHub App SSH key. The App token from `~/.config/rbitcoin-grok/gh-login.sh`
Expand Down Expand Up @@ -229,7 +231,9 @@ the agent VM. Perf A/B is operator-host only.
| **Refactor** | Remove the one-off | Still green: fold into the real shape; delete dual paths. |

Planning anatomy, INVEST, step template: [`docs/how-we-plan.md`](docs/how-we-plan.md).
Fixture size, one-entry-per-path, coverage bar: [`TESTING.md`](TESTING.md).
Fixture size, one-entry-per-path, coverage bar, default-CI vs nightly Core:
[`TESTING.md`](TESTING.md). Extend a catalog journey; do not add a twin or
treat Core functional as the PR pin.

The test must assert the **exact** contract, drive the **shipped** function,
fail with the **same class of error**, and use tiny `/tmp` fixtures.
Expand Down
8 changes: 6 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,9 @@ That is **not** the operator binary (`nix build .#rbitcoin-musl`). Details:
(return, store after reopen, peer/RPC/log line). Do not `include_str!`
production sources or markdown and `contains` identifiers, comments, or
call graphs. Fixture JSON/hex and tests that read **datadir** bytes are
not this rule.
not this rule. Prefer **one** [catalog journey](./TESTING.md) over a twin
unit for the same reject. Core functional is nightly, not a substitute
for that journey.
9. **RAM and CPU are design inputs.** This node indexes chain-scale
structures (tens of millions of keys, hundred-MiB arrays, GiB-class
heads). Iterating those structures is expensive. Every algorithm should
Expand Down Expand Up @@ -275,7 +277,9 @@ IO; they do not package zips. GitHub Releases:

## Code review checklist

- [ ] Behavior covered by a high-level scenario (or justified narrow test)
- [ ] Behavior covered by a catalog journey (or a justified unit next to a
pure helper). No twin for the same reject. Core functional is not the
PR pin ([`TESTING.md`](./TESTING.md)).
- [ ] No new silent dead branches
- [ ] No unused crate-root `pub` / `pub use` (principle 11). Tests drive
shipped functions, not `#[cfg(test)]` wrappers or `*_for_test`
Expand Down
44 changes: 39 additions & 5 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,38 @@

**Fewer scenario functions / store opens, not less coverage** — put more asserts on one carefully designed multi-stage journey.

### Default CI is the pin

Default `cargo test` owns operator- and peer-visible contracts (JSON-RPC,
Electrum TCP, Esplora HTTP, BIP324 P2P). Bitcoin Core’s Python functional
suite is a **nightly / ship / `core-functional` label** oracle. It is **not**
a required PR check and **not** required on PRs that merely touch net or RPC
(too slow). Nightly Core is not a license to delete in-tree tests.

When adding or folding a pin:

| Do | Do not |
|----|--------|
| Extend an existing [catalog](#scenario-catalog) journey (same `/tmp` pad, more asserts) | A new skinny scenario that remine-pads the same chain |
| Fold a twin unit once the journey hits the same shipped path | Twin unit + scenario for the same reject string |
| Keep guts the journey cannot hit | Delete handshake **format** needles, `decode_rpc_subset`, or BIP324 encode vectors waiting for Core |
| Live P2P/RPC on `cross_surface` / `integration_multinode` catalog tests | Grow `node_cli_and_surface_smoke` into a second live node |
| New P2P behavior on `p2p_timeout_*` / compact / feeler / inbound-full | Stuff more asserts onto `two_node` (`coverage.sh` skips it) |

Coverage (≥90% LCOV `LH`/`LF`) is a required PR job. If deleting a guts test
drops the bar, the journey did not cover the path — keep the guts or hit
those lines from the journey first.

Keep until a **default** journey hits the same lines: store packed / v17 /
fuse / SH machines, empty / truncated / v1 fuse refuse, unsorted pack/lag,
IBD wave fence / 8×8000, SH writebehind / uring CAS, leftover identity,
handshake format needles, `getaddr_cache_*`, eviction ranking, feeler silence
timeout, sole-preferred stall KEEP, `stamp_reject_names_*`,
`multi_hop_bad_prev_*`, structure s1–s18, rate-limiter, netgroup, subsidy
table. Optional leftovers (more HTTP methods on `cross_surface`, a tiny
legacy-head `Store::open` fixture, testnet 20-minute min-diff header walk)
are not a backlog.

### Parallel cargo test (same binary)

`cargo test` / `cargo llvm-cov test` run **one process per test binary**. Do not:
Expand Down Expand Up @@ -234,12 +266,12 @@ Prefer **one high-level scenario** per behavior cluster. Delete lower-level test
| `electrum_leftover_mempool_does_not_double_count` | Electrum | Relay-off leftover is confirmed, not a second mempool UTXO |
| `electrum_and_esplora_asof_hides_later_spend` | Electrum + Esplora | One pad: TCP `1.4.2-asof` plus HTTP `?asof=` hide a later spend; unknown asof errors; `GET /tx` `v0_p2wpkh` vout |
| `electrum_empty_chain_headers_subscribe_and_empty_scripthash` | Electrum | Empty store: `headers.subscribe` errors; scripthash history/balance/unspent/mempool empty |
| `electrum_tweaks_subscribe_streams_then_done` | Electrum | Cake `tweaks.subscribe`: one-height result, per-height notifies, then `done` |
| `electrum_tweaks_subscribe_streams_then_done` | Electrum | Cake `tweaks.subscribe`: one-height result, per-height notifies, `done`, and BIP352 hash-bind on a P2WPKH→P2TR spend. Keep zero-chunk / pre-taproot units |
| `electrum_max_connections_rejects_extra_client` | Electrum | TCP cap drops the extra client |
| `electrum_idle_timeout_disconnects_quiet_client` | Electrum | Idle timeout closes a quiet socket |
| `esplora_broadcast_visible_in_rpc_and_electrum` | Node + Electrum + Esplora + RPC | One `run_p2p` datadir: HTTP `sendrawtransaction` / `testmempoolaccept` (allowed, missing-or-spent, min-relay), Esplora `POST /tx` parent and mempool child appear in `getrawmempool` and Electrum mempool/history (`fee` on unconfirmed, including child `height = -1`); `generate` includes those txs; immature coinbase sendraw rejects. Keep `accept.rs` reject units and RPC dry-run orphan-count |
| `esplora_broadcast_visible_in_rpc_and_electrum` | Node + Electrum + Esplora + RPC | One `run_p2p` datadir: HTTP `sendrawtransaction` / `testmempoolaccept` (allowed, missing-or-spent, min-relay, RBF too-low reject + replacement), Esplora `POST /tx` parent and mempool child appear in `getrawmempool` and Electrum mempool/history (`fee` on unconfirmed, including child `height = -1`); `generate` includes those txs; immature coinbase sendraw rejects. Keep `accept.rs` reject units and RPC dry-run orphan-count |
| `two_node_header_and_block_sync` | P2P (**default + multinode CI**) | Seeder → peer 8-block IBD; peer `last_write` meter. **Not** re-run under `coverage.sh`. |
| `p2p_timeout_getaddr_and_keepalive_ping` | P2P (**default**) | One pad: v1-magic inbound drops at `peertimeout=1`, full-relay GetAddr cache 1000, headers-sync stall replace, self-connect refuses, AddrFetch `getaddr`/`addrv2` (no `getheaders`), one keepalive ping/pong. Sole-preferred stall KEEP stays a PeerHub unit. |
| `p2p_timeout_getaddr_and_keepalive_ping` | P2P (**default**) | One pad: v1-magic inbound drops at `peertimeout=1`, obsolete VERSION and pre-verack ping close the peer, full-relay GetAddr cache 1000, headers-sync stall replace, self-connect refuses, AddrFetch `getaddr`/`addrv2` (no `getheaders`), one keepalive ping/pong. Handshake **format** needles stay. Sole-preferred stall KEEP stays a PeerHub unit. |
| `p2p_compact_hb_getblocktxn_and_orphan` | P2P (**default**) | One mature pad: HB coinbase `cmpctblock`, 2-tx compact → `getblocktxn` + connect, orphan child GetData then parent accept (INV AlreadyHave). Does **not** pin depth-10 full-block serve, tokio-worker lock, or park-not-reject logs |
| `p2p_feeler_completes_and_closes` | P2P (**default**) | Outbound feeler: VERSION then close (`feeler connection completed`). No live follow; dummy has no completed inbound. Does **not** pin feeler silence timeout (`handshake_timeout_after_silence`) |
| `p2p_inbound_full_rejects_extra` | P2P (**default**) | `max_inbound=1`: second follow is refused; first inbound stays. Does **not** pin SelectNodeToEvict ranking |
Expand Down Expand Up @@ -426,8 +458,10 @@ The nightly job (`.github/workflows/core-functional.yml` →
`scripts/core-functional/nightly.sh`) warns — it does not fail — when a
newer Bitcoin Core release exists than the inventory pin. Label
**`core-functional`** on harness PRs and on **ship** version-bump PRs
([`docs/releases.md`](docs/releases.md)). Default `cargo test` does **not**
invoke Core’s Python suite.
([`docs/releases.md`](docs/releases.md)). It is **not** a required PR check,
including on PRs that touch net or RPC (too slow). Unlabeled PRs keep the
default cargo jobs. Default `cargo test` does **not** invoke Core’s Python
suite.

```bash
python3 scripts/core-functional/check_inventory.py
Expand Down
135 changes: 0 additions & 135 deletions crates/rbitcoin-electrum/src/server_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2987,138 +2987,3 @@ async fn tweaks_subscribe_pre_taproot_collapses_empty_heights() {
handle.shutdown().await;
let _ = std::fs::remove_dir_all(&dir);
}

#[test]
fn tweaks_subscribe_matches_engine_on_p2wpkh_spend() {
use bitcoin::hashes::{hash160, Hash};
use bitcoin::secp256k1::{PublicKey, Secp256k1, SecretKey};
use bitcoin::{Amount, OutPoint, ScriptBuf, Sequence, Transaction, TxIn, TxOut, Witness};
use rbitcoin_consensus::tweak_from_tx;
use rbitcoin_primitives::{Fk, Height};
use rbitcoin_query::TxApply;
use rbitcoin_store::{HeaderRecord, InputRecord, OutputRecord, TxRecord};

let (dir, q) = tmp_store();
let params = ChainParams::regtest();
let cfg = ElectrumConfig::for_params("127.0.0.1:0".parse().unwrap(), &params);

let secp = Secp256k1::new();
let sk = SecretKey::from_slice(&[2u8; 32]).unwrap();
let pk = PublicKey::from_secret_key(&secp, &sk);
let ser = pk.serialize();
let h160 = hash160::Hash::hash(&ser);
let mut p2wpkh = vec![0x00, 0x14];
p2wpkh.extend_from_slice(h160.as_ref());
let (xonly, _) = pk.x_only_public_key();
let mut p2tr = vec![0x51, 0x20];
p2tr.extend_from_slice(&xonly.serialize());

let mut genesis_txid = [0u8; 32];
genesis_txid[31] = 0xcb;
let mut merkle0 = [0u8; 32];
merkle0[5] = 0xec;
let h0 = HeaderRecord {
prev_fk: Fk::NULL,
version: 1,
timestamp: 1,
bits: 0x207fffff,
nonce: 0,
merkle_root: merkle0,
hash: merkle0,
};
let ta0 = TxApply {
tx: TxRecord {
txid: genesis_txid,
version: 1,
locktime: 0,
input_start_fk: Fk::NULL,
input_count: 1,
output_start_fk: Fk::NULL,
output_count: 1,
},
inputs: vec![InputRecord::coinbase(u32::MAX, vec![0x00], vec![])],
outputs: vec![OutputRecord::unspent(50_0000_0000, p2wpkh.clone())],
};
let fk0 = q.connect_block(Height(0), &h0, &[ta0]).unwrap();
let create_fk = q.block_tx_fks(Height(0)).unwrap()[0];

let mut spend_txid = [0u8; 32];
spend_txid[0] = 0x11;
spend_txid[31] = 0xcd;
let hash1 = rbitcoin_store::block_header_hash(1, &h0.hash, &[0x11; 32], 2, 0x207fffff, 1);
let h1 = HeaderRecord {
prev_fk: fk0,
version: 1,
timestamp: 2,
bits: 0x207fffff,
nonce: 1,
merkle_root: [0x11; 32],
hash: hash1,
};
let ta1 = TxApply {
tx: TxRecord {
txid: spend_txid,
version: 2,
locktime: 0,
input_start_fk: Fk::NULL,
input_count: 1,
output_start_fk: Fk::NULL,
output_count: 1,
},
inputs: vec![InputRecord {
prev_txid: genesis_txid,
create_fk,
prev_index: 0,
sequence: u32::MAX,
script_sig: vec![],
witness: vec![vec![0u8; 64], ser.to_vec()],
}],
outputs: vec![OutputRecord::unspent(49_0000_0000, p2tr.clone())],
};
q.connect_block(Height(1), &h1, &[ta1]).unwrap();

let mut header_sub = false;
let mut sh_subs = HashSet::new();
let v = dispatch(
"blockchain.tweaks.subscribe",
&json!([1, 1, false]),
&q,
&cfg,
&params,
None,
&mut header_sub,
&mut sh_subs,
)
.unwrap();

let engine_tx = Transaction {
version: bitcoin::transaction::Version::TWO,
lock_time: bitcoin::absolute::LockTime::ZERO,
input: vec![TxIn {
previous_output: OutPoint {
txid: bitcoin::Txid::from_byte_array(genesis_txid),
vout: 0,
},
script_sig: ScriptBuf::new(),
sequence: Sequence::MAX,
witness: Witness::from_slice(&[&[0u8; 64][..], &ser[..]]),
}],
output: vec![TxOut {
value: Amount::from_sat(49_0000_0000),
script_pubkey: ScriptBuf::from_bytes(p2tr),
}],
};
let engine_prev = vec![TxOut {
value: Amount::from_sat(50_0000_0000),
script_pubkey: ScriptBuf::from_bytes(p2wpkh),
}];
let expect = tweak_from_tx(&engine_tx, &engine_prev).unwrap();
let mut disp = spend_txid;
disp.reverse();
let key = rbitcoin_primitives::hex_encode(disp);
assert_eq!(
v["1"][key]["tweak"],
json!(rbitcoin_primitives::hex_encode(expect.tweak))
);
let _ = std::fs::remove_dir_all(&dir);
}
15 changes: 14 additions & 1 deletion crates/rbitcoin-net/src/peer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -377,14 +377,27 @@ impl HandshakePolicy<'static> {
}
}

/// Outbound BIP324 session after VERSION/VERACK with [`HandshakePolicy::plain`].
/// Outbound BIP324 session. [`Self::outbound_regtest`] completes VERSION/VERACK;
/// [`Self::outbound_bip324`] stops after transport so the caller can send a
/// custom first application message.
pub struct V2PlainSession {
reader: V2Reader,
writer: V2Writer,
tcp_shutdown: std::net::TcpStream,
}

impl V2PlainSession {
/// Dial-side BIP324 only (no VERSION). Caller sends the first application message.
pub async fn outbound_bip324(stream: TcpStream) -> Result<Self, NetError> {
let magic = Magic::REGTEST;
let (reader, writer, _wire, tcp_shutdown) = open_v2(stream, magic, false).await?;
Ok(Self {
reader,
writer,
tcp_shutdown,
})
}

/// Dial-side handshake on `stream`; `limit` bounds VERSION/VERACK.
pub async fn outbound_regtest(
stream: TcpStream,
Expand Down
Loading