diff --git a/AGENTS.md b/AGENTS.md index cf3098a8..9d5b6cdc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -130,7 +130,9 @@ Required jobs: **`fmt`**, **`deny`**, **`clippy`**, **`ast-grep`**, **`test`**, store + `--smoke` (not operator zips). Operator binaries are GitHub Releases (`release.yml`). Releases (minor / patch / major): [`docs/releases.md`](docs/releases.md). Label **`core-functional`** when the PR touches the Core functional harness -(and on every **ship** version-bump PR). +(and on every **ship** version-bump PR). Do **not** label ordinary net or +RPC PRs; the job is too slow for the default gate. Default `cargo test` is +the pin; owner rules: [`TESTING.md`](TESTING.md) (Default CI is the pin). `origin` fetch/pull is HTTPS; `pushurl` is SSH (operator). This VM has **no** GitHub App SSH key. The App token from `~/.config/rbitcoin-grok/gh-login.sh` @@ -229,7 +231,9 @@ the agent VM. Perf A/B is operator-host only. | **Refactor** | Remove the one-off | Still green: fold into the real shape; delete dual paths. | Planning anatomy, INVEST, step template: [`docs/how-we-plan.md`](docs/how-we-plan.md). -Fixture size, one-entry-per-path, coverage bar: [`TESTING.md`](TESTING.md). +Fixture size, one-entry-per-path, coverage bar, default-CI vs nightly Core: +[`TESTING.md`](TESTING.md). Extend a catalog journey; do not add a twin or +treat Core functional as the PR pin. The test must assert the **exact** contract, drive the **shipped** function, fail with the **same class of error**, and use tiny `/tmp` fixtures. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ca058154..b823a064 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -182,7 +182,9 @@ That is **not** the operator binary (`nix build .#rbitcoin-musl`). Details: (return, store after reopen, peer/RPC/log line). Do not `include_str!` production sources or markdown and `contains` identifiers, comments, or call graphs. Fixture JSON/hex and tests that read **datadir** bytes are - not this rule. + not this rule. Prefer **one** [catalog journey](./TESTING.md) over a twin + unit for the same reject. Core functional is nightly, not a substitute + for that journey. 9. **RAM and CPU are design inputs.** This node indexes chain-scale structures (tens of millions of keys, hundred-MiB arrays, GiB-class heads). Iterating those structures is expensive. Every algorithm should @@ -275,7 +277,9 @@ IO; they do not package zips. GitHub Releases: ## Code review checklist -- [ ] Behavior covered by a high-level scenario (or justified narrow test) +- [ ] Behavior covered by a catalog journey (or a justified unit next to a + pure helper). No twin for the same reject. Core functional is not the + PR pin ([`TESTING.md`](./TESTING.md)). - [ ] No new silent dead branches - [ ] No unused crate-root `pub` / `pub use` (principle 11). Tests drive shipped functions, not `#[cfg(test)]` wrappers or `*_for_test` diff --git a/TESTING.md b/TESTING.md index 79261081..b4762891 100644 --- a/TESTING.md +++ b/TESTING.md @@ -11,6 +11,38 @@ **Fewer scenario functions / store opens, not less coverage** — put more asserts on one carefully designed multi-stage journey. +### Default CI is the pin + +Default `cargo test` owns operator- and peer-visible contracts (JSON-RPC, +Electrum TCP, Esplora HTTP, BIP324 P2P). Bitcoin Core’s Python functional +suite is a **nightly / ship / `core-functional` label** oracle. It is **not** +a required PR check and **not** required on PRs that merely touch net or RPC +(too slow). Nightly Core is not a license to delete in-tree tests. + +When adding or folding a pin: + +| Do | Do not | +|----|--------| +| Extend an existing [catalog](#scenario-catalog) journey (same `/tmp` pad, more asserts) | A new skinny scenario that remine-pads the same chain | +| Fold a twin unit once the journey hits the same shipped path | Twin unit + scenario for the same reject string | +| Keep guts the journey cannot hit | Delete handshake **format** needles, `decode_rpc_subset`, or BIP324 encode vectors waiting for Core | +| Live P2P/RPC on `cross_surface` / `integration_multinode` catalog tests | Grow `node_cli_and_surface_smoke` into a second live node | +| New P2P behavior on `p2p_timeout_*` / compact / feeler / inbound-full | Stuff more asserts onto `two_node` (`coverage.sh` skips it) | + +Coverage (≥90% LCOV `LH`/`LF`) is a required PR job. If deleting a guts test +drops the bar, the journey did not cover the path — keep the guts or hit +those lines from the journey first. + +Keep until a **default** journey hits the same lines: store packed / v17 / +fuse / SH machines, empty / truncated / v1 fuse refuse, unsorted pack/lag, +IBD wave fence / 8×8000, SH writebehind / uring CAS, leftover identity, +handshake format needles, `getaddr_cache_*`, eviction ranking, feeler silence +timeout, sole-preferred stall KEEP, `stamp_reject_names_*`, +`multi_hop_bad_prev_*`, structure s1–s18, rate-limiter, netgroup, subsidy +table. Optional leftovers (more HTTP methods on `cross_surface`, a tiny +legacy-head `Store::open` fixture, testnet 20-minute min-diff header walk) +are not a backlog. + ### Parallel cargo test (same binary) `cargo test` / `cargo llvm-cov test` run **one process per test binary**. Do not: @@ -234,12 +266,12 @@ Prefer **one high-level scenario** per behavior cluster. Delete lower-level test | `electrum_leftover_mempool_does_not_double_count` | Electrum | Relay-off leftover is confirmed, not a second mempool UTXO | | `electrum_and_esplora_asof_hides_later_spend` | Electrum + Esplora | One pad: TCP `1.4.2-asof` plus HTTP `?asof=` hide a later spend; unknown asof errors; `GET /tx` `v0_p2wpkh` vout | | `electrum_empty_chain_headers_subscribe_and_empty_scripthash` | Electrum | Empty store: `headers.subscribe` errors; scripthash history/balance/unspent/mempool empty | -| `electrum_tweaks_subscribe_streams_then_done` | Electrum | Cake `tweaks.subscribe`: one-height result, per-height notifies, then `done` | +| `electrum_tweaks_subscribe_streams_then_done` | Electrum | Cake `tweaks.subscribe`: one-height result, per-height notifies, `done`, and BIP352 hash-bind on a P2WPKH→P2TR spend. Keep zero-chunk / pre-taproot units | | `electrum_max_connections_rejects_extra_client` | Electrum | TCP cap drops the extra client | | `electrum_idle_timeout_disconnects_quiet_client` | Electrum | Idle timeout closes a quiet socket | -| `esplora_broadcast_visible_in_rpc_and_electrum` | Node + Electrum + Esplora + RPC | One `run_p2p` datadir: HTTP `sendrawtransaction` / `testmempoolaccept` (allowed, missing-or-spent, min-relay), Esplora `POST /tx` parent and mempool child appear in `getrawmempool` and Electrum mempool/history (`fee` on unconfirmed, including child `height = -1`); `generate` includes those txs; immature coinbase sendraw rejects. Keep `accept.rs` reject units and RPC dry-run orphan-count | +| `esplora_broadcast_visible_in_rpc_and_electrum` | Node + Electrum + Esplora + RPC | One `run_p2p` datadir: HTTP `sendrawtransaction` / `testmempoolaccept` (allowed, missing-or-spent, min-relay, RBF too-low reject + replacement), Esplora `POST /tx` parent and mempool child appear in `getrawmempool` and Electrum mempool/history (`fee` on unconfirmed, including child `height = -1`); `generate` includes those txs; immature coinbase sendraw rejects. Keep `accept.rs` reject units and RPC dry-run orphan-count | | `two_node_header_and_block_sync` | P2P (**default + multinode CI**) | Seeder → peer 8-block IBD; peer `last_write` meter. **Not** re-run under `coverage.sh`. | -| `p2p_timeout_getaddr_and_keepalive_ping` | P2P (**default**) | One pad: v1-magic inbound drops at `peertimeout=1`, full-relay GetAddr cache 1000, headers-sync stall replace, self-connect refuses, AddrFetch `getaddr`/`addrv2` (no `getheaders`), one keepalive ping/pong. Sole-preferred stall KEEP stays a PeerHub unit. | +| `p2p_timeout_getaddr_and_keepalive_ping` | P2P (**default**) | One pad: v1-magic inbound drops at `peertimeout=1`, obsolete VERSION and pre-verack ping close the peer, full-relay GetAddr cache 1000, headers-sync stall replace, self-connect refuses, AddrFetch `getaddr`/`addrv2` (no `getheaders`), one keepalive ping/pong. Handshake **format** needles stay. Sole-preferred stall KEEP stays a PeerHub unit. | | `p2p_compact_hb_getblocktxn_and_orphan` | P2P (**default**) | One mature pad: HB coinbase `cmpctblock`, 2-tx compact → `getblocktxn` + connect, orphan child GetData then parent accept (INV AlreadyHave). Does **not** pin depth-10 full-block serve, tokio-worker lock, or park-not-reject logs | | `p2p_feeler_completes_and_closes` | P2P (**default**) | Outbound feeler: VERSION then close (`feeler connection completed`). No live follow; dummy has no completed inbound. Does **not** pin feeler silence timeout (`handshake_timeout_after_silence`) | | `p2p_inbound_full_rejects_extra` | P2P (**default**) | `max_inbound=1`: second follow is refused; first inbound stays. Does **not** pin SelectNodeToEvict ranking | @@ -426,8 +458,10 @@ The nightly job (`.github/workflows/core-functional.yml` → `scripts/core-functional/nightly.sh`) warns — it does not fail — when a newer Bitcoin Core release exists than the inventory pin. Label **`core-functional`** on harness PRs and on **ship** version-bump PRs -([`docs/releases.md`](docs/releases.md)). Default `cargo test` does **not** -invoke Core’s Python suite. +([`docs/releases.md`](docs/releases.md)). It is **not** a required PR check, +including on PRs that touch net or RPC (too slow). Unlabeled PRs keep the +default cargo jobs. Default `cargo test` does **not** invoke Core’s Python +suite. ```bash python3 scripts/core-functional/check_inventory.py diff --git a/crates/rbitcoin-electrum/src/server_tests.rs b/crates/rbitcoin-electrum/src/server_tests.rs index fe9861d2..3b5e5e3d 100644 --- a/crates/rbitcoin-electrum/src/server_tests.rs +++ b/crates/rbitcoin-electrum/src/server_tests.rs @@ -2987,138 +2987,3 @@ async fn tweaks_subscribe_pre_taproot_collapses_empty_heights() { handle.shutdown().await; let _ = std::fs::remove_dir_all(&dir); } - -#[test] -fn tweaks_subscribe_matches_engine_on_p2wpkh_spend() { - use bitcoin::hashes::{hash160, Hash}; - use bitcoin::secp256k1::{PublicKey, Secp256k1, SecretKey}; - use bitcoin::{Amount, OutPoint, ScriptBuf, Sequence, Transaction, TxIn, TxOut, Witness}; - use rbitcoin_consensus::tweak_from_tx; - use rbitcoin_primitives::{Fk, Height}; - use rbitcoin_query::TxApply; - use rbitcoin_store::{HeaderRecord, InputRecord, OutputRecord, TxRecord}; - - let (dir, q) = tmp_store(); - let params = ChainParams::regtest(); - let cfg = ElectrumConfig::for_params("127.0.0.1:0".parse().unwrap(), ¶ms); - - let secp = Secp256k1::new(); - let sk = SecretKey::from_slice(&[2u8; 32]).unwrap(); - let pk = PublicKey::from_secret_key(&secp, &sk); - let ser = pk.serialize(); - let h160 = hash160::Hash::hash(&ser); - let mut p2wpkh = vec![0x00, 0x14]; - p2wpkh.extend_from_slice(h160.as_ref()); - let (xonly, _) = pk.x_only_public_key(); - let mut p2tr = vec![0x51, 0x20]; - p2tr.extend_from_slice(&xonly.serialize()); - - let mut genesis_txid = [0u8; 32]; - genesis_txid[31] = 0xcb; - let mut merkle0 = [0u8; 32]; - merkle0[5] = 0xec; - let h0 = HeaderRecord { - prev_fk: Fk::NULL, - version: 1, - timestamp: 1, - bits: 0x207fffff, - nonce: 0, - merkle_root: merkle0, - hash: merkle0, - }; - let ta0 = TxApply { - tx: TxRecord { - txid: genesis_txid, - version: 1, - locktime: 0, - input_start_fk: Fk::NULL, - input_count: 1, - output_start_fk: Fk::NULL, - output_count: 1, - }, - inputs: vec![InputRecord::coinbase(u32::MAX, vec![0x00], vec![])], - outputs: vec![OutputRecord::unspent(50_0000_0000, p2wpkh.clone())], - }; - let fk0 = q.connect_block(Height(0), &h0, &[ta0]).unwrap(); - let create_fk = q.block_tx_fks(Height(0)).unwrap()[0]; - - let mut spend_txid = [0u8; 32]; - spend_txid[0] = 0x11; - spend_txid[31] = 0xcd; - let hash1 = rbitcoin_store::block_header_hash(1, &h0.hash, &[0x11; 32], 2, 0x207fffff, 1); - let h1 = HeaderRecord { - prev_fk: fk0, - version: 1, - timestamp: 2, - bits: 0x207fffff, - nonce: 1, - merkle_root: [0x11; 32], - hash: hash1, - }; - let ta1 = TxApply { - tx: TxRecord { - txid: spend_txid, - version: 2, - locktime: 0, - input_start_fk: Fk::NULL, - input_count: 1, - output_start_fk: Fk::NULL, - output_count: 1, - }, - inputs: vec![InputRecord { - prev_txid: genesis_txid, - create_fk, - prev_index: 0, - sequence: u32::MAX, - script_sig: vec![], - witness: vec![vec![0u8; 64], ser.to_vec()], - }], - outputs: vec![OutputRecord::unspent(49_0000_0000, p2tr.clone())], - }; - q.connect_block(Height(1), &h1, &[ta1]).unwrap(); - - let mut header_sub = false; - let mut sh_subs = HashSet::new(); - let v = dispatch( - "blockchain.tweaks.subscribe", - &json!([1, 1, false]), - &q, - &cfg, - ¶ms, - None, - &mut header_sub, - &mut sh_subs, - ) - .unwrap(); - - let engine_tx = Transaction { - version: bitcoin::transaction::Version::TWO, - lock_time: bitcoin::absolute::LockTime::ZERO, - input: vec![TxIn { - previous_output: OutPoint { - txid: bitcoin::Txid::from_byte_array(genesis_txid), - vout: 0, - }, - script_sig: ScriptBuf::new(), - sequence: Sequence::MAX, - witness: Witness::from_slice(&[&[0u8; 64][..], &ser[..]]), - }], - output: vec![TxOut { - value: Amount::from_sat(49_0000_0000), - script_pubkey: ScriptBuf::from_bytes(p2tr), - }], - }; - let engine_prev = vec![TxOut { - value: Amount::from_sat(50_0000_0000), - script_pubkey: ScriptBuf::from_bytes(p2wpkh), - }]; - let expect = tweak_from_tx(&engine_tx, &engine_prev).unwrap(); - let mut disp = spend_txid; - disp.reverse(); - let key = rbitcoin_primitives::hex_encode(disp); - assert_eq!( - v["1"][key]["tweak"], - json!(rbitcoin_primitives::hex_encode(expect.tweak)) - ); - let _ = std::fs::remove_dir_all(&dir); -} diff --git a/crates/rbitcoin-net/src/peer.rs b/crates/rbitcoin-net/src/peer.rs index cab95788..43c3db50 100644 --- a/crates/rbitcoin-net/src/peer.rs +++ b/crates/rbitcoin-net/src/peer.rs @@ -377,7 +377,9 @@ impl HandshakePolicy<'static> { } } -/// Outbound BIP324 session after VERSION/VERACK with [`HandshakePolicy::plain`]. +/// Outbound BIP324 session. [`Self::outbound_regtest`] completes VERSION/VERACK; +/// [`Self::outbound_bip324`] stops after transport so the caller can send a +/// custom first application message. pub struct V2PlainSession { reader: V2Reader, writer: V2Writer, @@ -385,6 +387,17 @@ pub struct V2PlainSession { } impl V2PlainSession { + /// Dial-side BIP324 only (no VERSION). Caller sends the first application message. + pub async fn outbound_bip324(stream: TcpStream) -> Result { + let magic = Magic::REGTEST; + let (reader, writer, _wire, tcp_shutdown) = open_v2(stream, magic, false).await?; + Ok(Self { + reader, + writer, + tcp_shutdown, + }) + } + /// Dial-side handshake on `stream`; `limit` bounds VERSION/VERACK. pub async fn outbound_regtest( stream: TcpStream, diff --git a/crates/rbitcoin-test/tests/cross_surface.rs b/crates/rbitcoin-test/tests/cross_surface.rs index 3dbf62a5..fc00ea7e 100644 --- a/crates/rbitcoin-test/tests/cross_surface.rs +++ b/crates/rbitcoin-test/tests/cross_surface.rs @@ -99,6 +99,40 @@ async fn jsonrpc(addr: SocketAddr, method: &str, params: Value) -> Value { serde_json::from_str(&text).unwrap_or_else(|e| panic!("rpc {method} json: {e} body={text}")) } +fn encode_tx(tx: &Transaction) -> String { + let mut raw = Vec::new(); + tx.consensus_encode(&mut raw).unwrap(); + rbitcoin_primitives::hex_encode(&raw) +} + +fn acs_spend(prev: Txid, input_sat: u64, fee: u64, spk: ScriptBuf) -> Transaction { + Transaction { + version: TxVersion::TWO, + lock_time: LockTime::ZERO, + input: vec![TxIn { + previous_output: OutPoint { + txid: prev, + vout: 0, + }, + script_sig: ScriptBuf::new(), + sequence: Sequence::ENABLE_RBF_NO_LOCKTIME, + witness: Witness::new(), + }], + output: vec![TxOut { + value: Amount::from_sat(input_sat - fee), + script_pubkey: spk, + }], + } +} + +fn mempool_has(mem: &Value, txid: &str) -> bool { + mem["result"] + .as_array() + .expect("getrawmempool array") + .iter() + .any(|v| v.as_str() == Some(txid)) +} + async fn electrum_rpc(stream: &mut TcpStream, id: u64, method: &str, params: Value) -> Value { let req = json!({"jsonrpc":"2.0","id": id, "method": method, "params": params}); let mut line = serde_json::to_string(&req).unwrap(); @@ -163,32 +197,12 @@ async fn esplora_broadcast_visible_in_rpc_and_electrum() { assert_eq!(count["result"], 102, "{count}"); let rpc_spk = ScriptBuf::from_bytes(vec![0x54]); - let rpc_spend = Transaction { - version: TxVersion::TWO, - lock_time: LockTime::ZERO, - input: vec![TxIn { - previous_output: OutPoint { - txid: rpc_cb, - vout: 0, - }, - script_sig: ScriptBuf::new(), - sequence: Sequence::ENABLE_RBF_NO_LOCKTIME, - witness: Witness::new(), - }], - output: vec![TxOut { - value: Amount::from_sat(50_0000_0000 - 1_000), - script_pubkey: rpc_spk, - }], - }; - let mut rpc_raw = Vec::new(); - rpc_spend.consensus_encode(&mut rpc_raw).unwrap(); - let rpc_hex = rbitcoin_primitives::hex_encode(&rpc_raw); + let rpc_spend = acs_spend(rpc_cb, 50_0000_0000, 1_000, rpc_spk); + let rpc_hex = encode_tx(&rpc_spend); let rpc_txid = rpc_spend.compute_txid().to_string(); let mut zero_fee = rpc_spend.clone(); zero_fee.output[0].value = Amount::from_sat(50_0000_0000); - let mut zero_raw = Vec::new(); - zero_fee.consensus_encode(&mut zero_raw).unwrap(); - let zero_hex = rbitcoin_primitives::hex_encode(&zero_raw); + let zero_hex = encode_tx(&zero_fee); let tma = jsonrpc(rpc_addr, "testmempoolaccept", json!([[zero_hex]])).await; assert_eq!(tma["result"][0]["allowed"], false, "{tma}"); assert_eq!( @@ -200,9 +214,8 @@ async fn esplora_broadcast_visible_in_rpc_and_electrum() { let sent = jsonrpc(rpc_addr, "sendrawtransaction", json!([rpc_hex])).await; assert_eq!(sent["result"], rpc_txid, "{sent}"); let mem = jsonrpc(rpc_addr, "getrawmempool", json!([])).await; - let ids = mem["result"].as_array().expect("getrawmempool array"); assert!( - ids.iter().any(|v| v.as_str() == Some(rpc_txid.as_str())), + mempool_has(&mem, &rpc_txid), "getrawmempool missing sendraw {rpc_txid}: {mem}" ); let miss = Transaction { @@ -233,26 +246,8 @@ async fn esplora_broadcast_visible_in_rpc_and_electrum() { ); let spk = ScriptBuf::from_bytes(vec![0x52]); - let spend = Transaction { - version: TxVersion::TWO, - lock_time: LockTime::ZERO, - input: vec![TxIn { - previous_output: OutPoint { - txid: coinbase_txid, - vout: 0, - }, - script_sig: ScriptBuf::new(), - sequence: Sequence::ENABLE_RBF_NO_LOCKTIME, - witness: Witness::new(), - }], - output: vec![TxOut { - value: Amount::from_sat(50_0000_0000 - 1_000), - script_pubkey: spk.clone(), - }], - }; - let mut raw = Vec::new(); - spend.consensus_encode(&mut raw).unwrap(); - let hex = rbitcoin_primitives::hex_encode(&raw); + let spend = acs_spend(coinbase_txid, 50_0000_0000, 1_000, spk.clone()); + let hex = encode_tx(&spend); let txid_hex = spend.compute_txid().to_string(); let (st, body) = http_post(esplora_addr, "/tx", &hex).await; @@ -267,13 +262,12 @@ async fn esplora_broadcast_visible_in_rpc_and_electrum() { assert_eq!(status_v["confirmed"], false, "{status_v}"); let mem = jsonrpc(rpc_addr, "getrawmempool", json!([])).await; - let ids = mem["result"].as_array().expect("getrawmempool array"); assert!( - ids.iter().any(|v| v.as_str() == Some(txid_hex.as_str())), + mempool_has(&mem, &txid_hex), "getrawmempool missing {txid_hex}: {mem}" ); assert!( - ids.iter().any(|v| v.as_str() == Some(rpc_txid.as_str())), + mempool_has(&mem, &rpc_txid), "getrawmempool dropped sendraw {rpc_txid}: {mem}" ); @@ -309,26 +303,13 @@ async fn esplora_broadcast_visible_in_rpc_and_electrum() { } let child_spk = ScriptBuf::from_bytes(vec![0x53]); - let child = Transaction { - version: TxVersion::TWO, - lock_time: LockTime::ZERO, - input: vec![TxIn { - previous_output: OutPoint { - txid: spend.compute_txid(), - vout: 0, - }, - script_sig: ScriptBuf::new(), - sequence: Sequence::ENABLE_RBF_NO_LOCKTIME, - witness: Witness::new(), - }], - output: vec![TxOut { - value: Amount::from_sat(50_0000_0000 - 2_000), - script_pubkey: child_spk.clone(), - }], - }; - let mut child_raw = Vec::new(); - child.consensus_encode(&mut child_raw).unwrap(); - let child_hex = rbitcoin_primitives::hex_encode(&child_raw); + let child = acs_spend( + spend.compute_txid(), + 50_0000_0000 - 1_000, + 1_000, + child_spk.clone(), + ); + let child_hex = encode_tx(&child); let child_txid = child.compute_txid().to_string(); let (st, body) = http_post(esplora_addr, "/tx", &child_hex).await; assert_eq!(st, 200, "POST /tx child: {body}"); @@ -371,6 +352,62 @@ async fn esplora_broadcast_visible_in_rpc_and_electrum() { "{child_mem_row}" ); + let low = acs_spend( + rpc_cb, + 50_0000_0000, + 1_000, + ScriptBuf::from_bytes(vec![0x55]), + ); + let low_hex = encode_tx(&low); + let tma = jsonrpc(rpc_addr, "testmempoolaccept", json!([[low_hex.clone()]])).await; + assert_eq!(tma["result"][0]["allowed"], false, "{tma}"); + assert_eq!( + tma["result"][0]["reject-reason"], "insufficient fee", + "{tma}" + ); + let rejected = jsonrpc(rpc_addr, "sendrawtransaction", json!([low_hex])).await; + assert_eq!(rejected["error"]["code"], -26, "{rejected}"); + assert_eq!( + rejected["error"]["message"], "insufficient fee", + "{rejected}" + ); + let mem = jsonrpc(rpc_addr, "getrawmempool", json!([])).await; + assert!( + mempool_has(&mem, &rpc_txid), + "too-low RBF must leave the original: {mem}" + ); + + let high = acs_spend( + rpc_cb, + 50_0000_0000, + 50_000, + ScriptBuf::from_bytes(vec![0x56]), + ); + let high_hex = encode_tx(&high); + let high_txid = high.compute_txid().to_string(); + let tma = jsonrpc(rpc_addr, "testmempoolaccept", json!([[high_hex.clone()]])).await; + assert_eq!(tma["result"][0]["allowed"], true, "{tma}"); + let mem = jsonrpc(rpc_addr, "getrawmempool", json!([])).await; + assert!( + mempool_has(&mem, &rpc_txid), + "testmempoolaccept must not RBF-evict: {mem}" + ); + assert!( + !mempool_has(&mem, &high_txid), + "trial replacement must not remain: {mem}" + ); + let replaced = jsonrpc(rpc_addr, "sendrawtransaction", json!([high_hex])).await; + assert_eq!(replaced["result"], high_txid, "{replaced}"); + let mem = jsonrpc(rpc_addr, "getrawmempool", json!([])).await; + assert!( + mempool_has(&mem, &high_txid), + "replacement missing from mempool: {mem}" + ); + assert!( + !mempool_has(&mem, &rpc_txid), + "replaced tx must leave mempool: {mem}" + ); + let mined = jsonrpc(rpc_addr, "generate", json!([1])).await; assert_eq!( mined["result"].as_array().map(|a| a.len()), @@ -386,11 +423,11 @@ async fn esplora_broadcast_visible_in_rpc_and_electrum() { let txs = blk["result"]["tx"].as_array().expect("mined tx array"); assert!( txs.len() >= 4, - "coinbase + sendraw + esplora parent + child: {blk}" + "coinbase + RBF replacement + esplora parent + child: {blk}" ); assert!( - txs.iter().any(|t| t["txid"] == rpc_txid), - "generate must include sendraw: {blk}" + txs.iter().any(|t| t["txid"] == high_txid), + "generate must include RBF replacement: {blk}" ); let cb_txid = txs[0]["txid"].as_str().expect("coinbase txid").to_string(); let cb_val = (txs[0]["vout"][0]["value"].as_f64().unwrap() * 100_000_000.0).round() as u64; diff --git a/crates/rbitcoin-test/tests/electrum_protocol.rs b/crates/rbitcoin-test/tests/electrum_protocol.rs index 6043403f..d0d650f5 100644 --- a/crates/rbitcoin-test/tests/electrum_protocol.rs +++ b/crates/rbitcoin-test/tests/electrum_protocol.rs @@ -970,27 +970,161 @@ async fn electrum_empty_chain_headers_subscribe_and_empty_scripthash() { } /// Cake isolate: JSON-RPC result is the first height, then one notification -/// per following height, then `{"message":"done"}`. A multi-height result is -/// treated as one event; no `done` leaves the isolate pinging forever. +/// per following height, then `{"message":"done"}`. Height 2 hash-binds a +/// P2WPKH→P2TR spend to `tweak_from_tx`. #[tokio::test] async fn electrum_tweaks_subscribe_streams_then_done() { - use rbitcoin_consensus::{accept_and_connect_block, Milestone}; - use rbitcoin_primitives::Height; + use bitcoin::hashes::{hash160, Hash}; + use bitcoin::secp256k1::{PublicKey, Secp256k1, SecretKey}; + use bitcoin::{Amount, OutPoint, ScriptBuf, Sequence, Transaction, TxIn, TxOut, Witness}; + use rbitcoin_consensus::{accept_and_connect_block, tweak_from_tx, Milestone}; + use rbitcoin_primitives::{Fk, Height}; + use rbitcoin_query::testutil::FixtureChain; + use rbitcoin_query::TxApply; + use rbitcoin_store::{HeaderRecord, InputRecord, OutputRecord, TxRecord}; let dir = TempDir::new().unwrap(); let q = Query::open_or_create_tiny(dir.path().join("store")).unwrap(); let params = ChainParams::regtest(); let genesis = bitcoin::blockdata::constants::genesis_block(bitcoin::Network::Regtest); accept_and_connect_block(&q, ¶ms, Height::GENESIS, &genesis, Milestone::NONE).unwrap(); - let _ = rbitcoin_consensus::pad_empty_from( - &q, - ¶ms, - genesis.block_hash(), - genesis.header.time, - 1, - 4, - 0, - ); + let (fk0, rec0) = q.header_at_height(Height::GENESIS).unwrap().unwrap(); + + let secp = Secp256k1::new(); + let sk = SecretKey::from_slice(&[2u8; 32]).unwrap(); + let pk = PublicKey::from_secret_key(&secp, &sk); + let ser = pk.serialize(); + let h160 = hash160::Hash::hash(&ser); + let mut p2wpkh = vec![0x00, 0x14]; + p2wpkh.extend_from_slice(h160.as_ref()); + let (xonly, _) = pk.x_only_public_key(); + let mut p2tr = vec![0x51, 0x20]; + p2tr.extend_from_slice(&xonly.serialize()); + + let mut create_txid = [0u8; 32]; + create_txid[31] = 0xcb; + let mut merkle1 = [0u8; 32]; + merkle1[0] = 1; + merkle1[5] = 0xec; + let hash1 = rbitcoin_store::block_header_hash(1, &rec0.hash, &merkle1, 2, 0x207fffff, 1); + let h1 = HeaderRecord { + prev_fk: fk0, + version: 1, + timestamp: 2, + bits: 0x207fffff, + nonce: 1, + merkle_root: merkle1, + hash: hash1, + }; + let ta1 = TxApply { + tx: TxRecord { + txid: create_txid, + version: 1, + locktime: 0, + input_start_fk: Fk::NULL, + input_count: 1, + output_start_fk: Fk::NULL, + output_count: 1, + }, + inputs: vec![InputRecord::coinbase(u32::MAX, vec![0x01], vec![])], + outputs: vec![OutputRecord::unspent(50_0000_0000, p2wpkh.clone())], + }; + let fk1 = q.connect_block(Height(1), &h1, &[ta1]).unwrap(); + let create_fk = q.block_tx_fks(Height(1)).unwrap()[0]; + + let mut spend_txid = [0u8; 32]; + spend_txid[0] = 0x11; + spend_txid[31] = 0xcd; + let merkle2 = [0x11; 32]; + let hash2 = rbitcoin_store::block_header_hash(1, &hash1, &merkle2, 3, 0x207fffff, 2); + let h2 = HeaderRecord { + prev_fk: fk1, + version: 1, + timestamp: 3, + bits: 0x207fffff, + nonce: 2, + merkle_root: merkle2, + hash: hash2, + }; + let ta2 = TxApply { + tx: TxRecord { + txid: spend_txid, + version: 2, + locktime: 0, + input_start_fk: Fk::NULL, + input_count: 1, + output_start_fk: Fk::NULL, + output_count: 1, + }, + inputs: vec![InputRecord { + prev_txid: create_txid, + create_fk, + prev_index: 0, + sequence: u32::MAX, + script_sig: vec![], + witness: vec![vec![0u8; 64], ser.to_vec()], + }], + outputs: vec![OutputRecord::unspent(49_0000_0000, p2tr.clone())], + }; + let fk2 = q.connect_block(Height(2), &h2, &[ta2]).unwrap(); + + let mut merkle3 = [0u8; 32]; + merkle3[0] = 3; + merkle3[5] = 0xec; + let hash3 = rbitcoin_store::block_header_hash(1, &hash2, &merkle3, 4, 0x207fffff, 3); + let mut dummy_txid = [0u8; 32]; + dummy_txid[0] = 3; + dummy_txid[31] = 0xcb; + let h3 = HeaderRecord { + prev_fk: fk2, + version: 1, + timestamp: 4, + bits: 0x207fffff, + nonce: 3, + merkle_root: merkle3, + hash: hash3, + }; + let ta3 = TxApply { + tx: TxRecord { + txid: dummy_txid, + version: 1, + locktime: 0, + input_start_fk: Fk::NULL, + input_count: 1, + output_start_fk: Fk::NULL, + output_count: 1, + }, + inputs: vec![InputRecord::coinbase(u32::MAX, vec![0x03], vec![])], + outputs: vec![OutputRecord::unspent(50_0000_0000, vec![0x51])], + }; + q.connect_block(Height(3), &h3, &[ta3]).unwrap(); + + let engine_tx = Transaction { + version: bitcoin::transaction::Version::TWO, + lock_time: bitcoin::absolute::LockTime::ZERO, + input: vec![TxIn { + previous_output: OutPoint { + txid: bitcoin::Txid::from_byte_array(create_txid), + vout: 0, + }, + script_sig: ScriptBuf::new(), + sequence: Sequence::MAX, + witness: Witness::from_slice(&[&[0u8; 64][..], &ser[..]]), + }], + output: vec![TxOut { + value: Amount::from_sat(49_0000_0000), + script_pubkey: ScriptBuf::from_bytes(p2tr.clone()), + }], + }; + let engine_prev = vec![TxOut { + value: Amount::from_sat(50_0000_0000), + script_pubkey: ScriptBuf::from_bytes(p2wpkh), + }]; + let expect = tweak_from_tx(&engine_tx, &engine_prev).unwrap(); + let mut disp = spend_txid; + disp.reverse(); + let spend_key = rbitcoin_primitives::hex_encode(disp); + let tweak_hex = rbitcoin_primitives::hex_encode(expect.tweak); let q = Arc::new(q); let (tip_tx, _) = broadcast::channel(4); @@ -1021,13 +1155,29 @@ async fn electrum_tweaks_subscribe_streams_then_done() { "JSON-RPC result must be one height, got {map:?}" ); assert!(map.contains_key("1"), "{map:?}"); + assert_eq!( + map["1"].as_object().map(|o| o.len()), + Some(0), + "height 1 is the P2WPKH create, not a tweak: {map:?}" + ); read_line_timeout(&mut reader, &mut resp, "tweaks 2").await; let n2: Value = serde_json::from_str(&resp).unwrap(); assert_eq!(n2["method"], "blockchain.tweaks.subscribe"); let p2 = n2["params"][0].as_object().expect("notify 2"); assert_eq!(p2.len(), 1); - assert!(p2.contains_key("2"), "{p2:?}"); + let h2_txs = p2["2"].as_object().expect("height 2 txs"); + assert_eq!(h2_txs.len(), 1, "{h2_txs:?}"); + assert_eq!(h2_txs[&spend_key]["tweak"], json!(tweak_hex), "{h2_txs:?}"); + assert_eq!( + h2_txs[&spend_key]["output_pubkeys"]["0"][0], + json!(rbitcoin_primitives::hex_encode(xonly.serialize())), + "{h2_txs:?}" + ); + assert_eq!( + h2_txs[&spend_key]["output_pubkeys"]["0"][1], + json!(49_0000_0000_u64) + ); read_line_timeout(&mut reader, &mut resp, "tweaks 3").await; let n3: Value = serde_json::from_str(&resp).unwrap(); @@ -1035,6 +1185,11 @@ async fn electrum_tweaks_subscribe_streams_then_done() { let p3 = n3["params"][0].as_object().expect("notify 3"); assert_eq!(p3.len(), 1); assert!(p3.contains_key("3"), "{p3:?}"); + assert_eq!( + p3["3"].as_object().map(|o| o.len()), + Some(0), + "height 3 has no P2TR spend: {p3:?}" + ); read_line_timeout(&mut reader, &mut resp, "tweaks done").await; let done: Value = serde_json::from_str(&resp).unwrap(); diff --git a/crates/rbitcoin-test/tests/integration_multinode.rs b/crates/rbitcoin-test/tests/integration_multinode.rs index 30db918a..433e8c66 100644 --- a/crates/rbitcoin-test/tests/integration_multinode.rs +++ b/crates/rbitcoin-test/tests/integration_multinode.rs @@ -95,6 +95,21 @@ async fn wait_ms_until( } } +async fn wait_v2_eof(sess: &mut rbitcoin_net::V2PlainSession, label: &str) { + let deadline = tokio::time::Instant::now() + Duration::from_secs(3); + loop { + match tokio::time::timeout(Duration::from_millis(100), sess.read_contents()).await { + Ok(Err(_)) => return, + Ok(Ok(_)) => {} + Err(_) => { + if tokio::time::Instant::now() >= deadline { + panic!("{label}"); + } + } + } + } +} + async fn seed_chain(node: &P2PNode, blocks: u32) { let genesis = regtest_genesis(); node.ingest_block(0, genesis.clone()).unwrap(); @@ -152,8 +167,9 @@ async fn two_node_header_and_block_sync() { } /// In-tree P2P client (no Core functional): peertimeout of a v1-magic inbound, -/// full-relay GetAddr cache (1000 / 23%), AddrFetch GetAddr (no getheaders), -/// one post-verack keepalive ping/pong, and headers-sync stall replace. +/// obsolete VERSION / pre-verack ping disconnect, full-relay GetAddr cache +/// (1000 / 23%), AddrFetch GetAddr (no getheaders), one post-verack keepalive +/// ping/pong, and headers-sync stall replace. #[tokio::test(flavor = "multi_thread", worker_threads = 4)] async fn p2p_timeout_getaddr_and_keepalive_ping() { use bitcoin::p2p::message::NetworkMessage; @@ -353,6 +369,81 @@ async fn p2p_timeout_getaddr_and_keepalive_ping() { } drop(raw); + let seed_addr = seed.local_addr; + let mut obsolete = rbitcoin_net::V2PlainSession::outbound_bip324( + tokio::net::TcpStream::connect(seed_addr) + .await + .expect("obsolete VERSION dial"), + ) + .await + .expect("obsolete VERSION BIP324"); + let obsolete_ver = { + use bitcoin::p2p::address::Address; + use bitcoin::p2p::message_network::VersionMessage; + use bitcoin::p2p::ServiceFlags; + VersionMessage { + version: 31799, + services: ServiceFlags::NONE, + timestamp: 0, + receiver: Address::new(&seed_addr, ServiceFlags::NONE), + sender: Address::new(&seed_addr, ServiceFlags::NONE), + nonce: 1, + user_agent: "/rbitcoin:test/".into(), + start_height: 0, + relay: true, + } + }; + obsolete + .write_contents( + &rbitcoin_net::encode_v2_contents(NetworkMessage::Version(obsolete_ver)) + .expect("encode obsolete VERSION"), + ) + .await + .expect("write obsolete VERSION"); + wait_v2_eof(&mut obsolete, "obsolete VERSION must close the peer").await; + + let mut pre_verack = rbitcoin_net::V2PlainSession::outbound_bip324( + tokio::net::TcpStream::connect(seed_addr) + .await + .expect("pre-verack ping dial"), + ) + .await + .expect("pre-verack BIP324"); + let ok_ver = { + use bitcoin::p2p::address::Address; + use bitcoin::p2p::message_network::VersionMessage; + use bitcoin::p2p::ServiceFlags; + VersionMessage { + version: 70016, + services: ServiceFlags::NONE, + timestamp: 0, + receiver: Address::new(&seed_addr, ServiceFlags::NONE), + sender: Address::new(&seed_addr, ServiceFlags::NONE), + nonce: 2, + user_agent: "/rbitcoin:test/".into(), + start_height: 0, + relay: true, + } + }; + pre_verack + .write_contents( + &rbitcoin_net::encode_v2_contents(NetworkMessage::Version(ok_ver)) + .expect("encode VERSION"), + ) + .await + .expect("write VERSION"); + pre_verack + .write_contents( + &rbitcoin_net::encode_v2_contents(NetworkMessage::Ping(1)).expect("encode ping"), + ) + .await + .expect("write ping prior to verack"); + wait_v2_eof( + &mut pre_verack, + "pre-verack ping must close at peertimeout=1", + ) + .await; + for id in peer .peers .snapshot() diff --git a/docs/README.md b/docs/README.md index 689b817f..2dc40678 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,7 +14,7 @@ update that file — do not paste a parallel spec. | Agent | [`AGENTS.md`](../AGENTS.md) | Short hard rules + pointers (not a second design book) | | On-disk | [`SCHEMA.md`](../SCHEMA.md) | Current bytes; soft migrate / bump / refuse; history in [`SCHEMA_HISTORY.md`](../SCHEMA_HISTORY.md) | | Confirm / store implementer | [`invariants.md`](./invariants.md) + [`concurrency.md`](./concurrency.md) | Stage IO, leftover union, roles, tip commit | -| Tests | [`TESTING.md`](../TESTING.md) | How to run, budgets, coverage | +| Tests | [`TESTING.md`](../TESTING.md) | How to run, budgets, coverage, default-CI pin vs nightly Core | | Peer full nodes | [`peer-clients.md`](./peer-clients.md) | Hornet / satd comparison; later-consideration tests and ideas | Planning a multi-step change: [`how-we-plan.md`](./how-we-plan.md). @@ -65,7 +65,7 @@ quality backlog). | [`CHANGELOG.md`](../CHANGELOG.md) | Release notes. | | [`SCHEMA.md`](../SCHEMA.md) | Current on-disk schema (`SCHEMA_VERSION` home). Soft migrate / bump / refuse. | | [`SCHEMA_HISTORY.md`](../SCHEMA_HISTORY.md) | Prior versions and migrations. | -| [`TESTING.md`](../TESTING.md) | Suite, budgets, coverage policy. | +| [`TESTING.md`](../TESTING.md) | Suite, budgets, coverage policy, default-CI pin vs nightly Core. | ## Confirm stage IO (one table) diff --git a/docs/core-functional.md b/docs/core-functional.md index 4d91e99b..e310e548 100644 --- a/docs/core-functional.md +++ b/docs/core-functional.md @@ -144,9 +144,11 @@ lists as done, but whose official script still fails on dialect, stay runs `scripts/core-functional/nightly.sh` on a nightly cron, on `workflow_dispatch`, and on PRs that are **ship versions**, or labeled **`core-functional`** / **`release`**. Unlabeled non-ship PRs keep the cargo -gates only. Label harness PRs and every version-bump ship PR (see -[`releases.md`](./releases.md)). The `release-extra` job on that workflow -fails when a ship PR did not get a green `core-functional`. +gates only. Do **not** make this job required on all PRs, or on PRs that +merely touch net or RPC (too slow). Default `cargo test` is the PR pin +([`TESTING.md`](../TESTING.md)). Label harness PRs and every version-bump +ship PR (see [`releases.md`](./releases.md)). The `release-extra` job on +that workflow fails when a ship PR did not get a green `core-functional`. The job sparse-inits the pin, checks the inventory, **warns** (does not fail) if a newer Bitcoin Core *release* exists than `inventory.toml` diff --git a/docs/how-we-plan.md b/docs/how-we-plan.md index eeb8e7b9..a0767d32 100644 --- a/docs/how-we-plan.md +++ b/docs/how-we-plan.md @@ -159,10 +159,14 @@ Aligned with AGENTS.md TDD + suite speed: | Prefer for Red | When | |----------------|------| +| Extend an existing default [catalog](../TESTING.md#scenario-catalog) journey | Operator/peer-visible RPC, Electrum, Esplora, BIP324 P2P | | Focused unit next to shipped fn | Pure helper, fast loop, expensive full path | | Slim scenario / integration | Stage boundaries, IBD/confirm wiring, store publish order | | One pin per contract | Not unit + twin integration for the same lines | +Core functional `run` scripts are a nightly oracle. They are **not** the Red +test for a default-CI story and **not** a reason to skip an in-tree journey. + | Plan-time rules | | |-----------------|--| | Each step declares Red tests **before** Green work | | @@ -239,6 +243,7 @@ Each step is independently green and shippable. | Spike disguised as implement | Named spike + follow-on story | | Green without refactor forever | Refactor phase required in the step template | | Plan ignores test runtime | Explicit unit vs scenario choice per step | +| Core functional as the default-CI Red | In-tree catalog journey; Core stays nightly | | Step done = “code compiles” | Step done = Red→Green→Refactor verify checklist | --- @@ -251,7 +256,7 @@ Before accepting a plan: - [ ] Every step has **Contract + Red + Green + Refactor + Verify** - [ ] No step larger than one Red→Green→Refactor without a spike - [ ] Vertical slices; horizontal deps called out as prior steps -- [ ] Test budget: suite stays fast; no unjustified full-store twins +- [ ] Test budget: suite stays fast; no unjustified full-store twins; Red is in-tree (catalog journey or unit), not Core functional - [ ] No production-scale default fixtures when tiny N still hits the branch (see TESTING.md) - [ ] Constraints cite project invariants (concurrency, IO split, no live heal, …) diff --git a/docs/quality.md b/docs/quality.md index 7f873b6e..d14111f8 100644 --- a/docs/quality.md +++ b/docs/quality.md @@ -262,7 +262,7 @@ included; tree at #318): | `TODO` / `FIXME` / `#[allow(` | **0** / **0** / **4** | | Coverage gate | **≥90%** LCOV `LH`/`LF` (required CI) | | Required CI | `fmt`, `deny`, `clippy`, `ast-grep`, `test`, `windows`, `macos`, `multinode`, `coverage` (+ CodeQL) | -| Extra CI | `release.yml` on `v*.*.*` / dispatch; `fuzz.yml` nightly; `miri.yml` nightly primitives; `core-functional.yml` nightly / labeled PR / **ship version-bump** (`release-extra` fails if a ship PR skipped it) | +| Extra CI | `release.yml` on `v*.*.*` / dispatch; `fuzz.yml` nightly; `miri.yml` nightly primitives; `core-functional.yml` nightly / labeled PR / **ship version-bump** (not required on ordinary net/RPC PRs; `release-extra` fails if a ship PR skipped it) | | rustc | **1.95** (`Cargo.toml` + `rust-toolchain.toml` + `dtolnay/rust-toolchain@1.95.0` + nixos-26.05 / shell) | | Nix | **nixos-26.05** + crane **0.24.0** | | Host cargo silos | `target/dev` (test) / `target/cov` (coverage) |