Skip to content

Auto-convert basecoin and CT to RingCT - #1055

Merged
Veil-Project merged 2 commits into
Veil-Project:masterfrom
ohcee:feature/auto-ct-to-ringct
Aug 24, 2026
Merged

Veil-Project merged 2 commits into
Veil-Project:masterfrom
ohcee:feature/auto-ct-to-ringct

Conversation

@ohcee

@ohcee ohcee commented Jun 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Implements opt-in automatic conversion of basecoin and CT balances to RingCT, addressing #1033 and #1054.

This is a substantially reworked version of the original PR based on a deeper review of the wallet's transaction building internals. The conversion no longer runs inside BlockConnected. It now runs as a periodic wallet scheduler task with confirmation depth, batching, and failure handling safeguards, and the two conversion hops are staged instead of chained as unconfirmed transactions. BlockConnected is untouched by this PR.

How it works

When -autoconvert=1 is set, a wallet scheduler task runs every 60 seconds. For each loaded wallet it:

  1. Selects up to 32 confirmed outputs (12+ confirmations), largest first. Matured CT outputs are lifted to RingCT first, otherwise a new basecoin batch is started
  2. If the selected amount reaches the threshold (default 10 VEIL, configurable via -autoconvertthreshold), builds a single batched transaction for that hop
  3. Tests acceptance against the mempool before committing to the wallet, so a rejection cannot strand inputs in a spent state
  4. Commits and broadcasts only after the mempool test passes

Why two hops

Consensus only permits RingCT outputs when the inputs are CT or RingCT (bad-txns-ringct-output-no-anonin, consensus/tx_verify.cpp), so basecoin cannot convert to RingCT in one transaction. Basecoin batches therefore convert to CT, and once those CT outputs reach 12 confirmations a later cycle lifts them to RingCT. Unlike the original design, the two hops are never chained while unconfirmed. Each stage only ever spends outputs with 12 or more confirmations.

Failure handling

  • AddStandardInputs/AddBlindedInputs persist a transaction record and mark inputs pending spend at build time. If anything fails after that point (build, mempool test, or commit), the new AnonWallet::UnwindPendingTransaction removes the record and clears the pending spend flags so the outputs remain spendable. Previously a failed broadcast could leave CT outputs invisible to the balance until a manual rescanringctwallet.
  • A commit stage failure additionally abandons the wallet transaction so its basecoin inputs become spendable again.
  • Consecutive failures back off exponentially (10 minutes doubling up to 6 hours) instead of retrying on every block.

Privacy considerations

  • Conversions are time based with random jitter (1 to 10 minutes between attempts), not block synchronous, so broadcasts do not deterministically trail the block that paid the wallet.
  • The 12 confirmation floor keeps conversions clear of short reorgs and further decorrelates conversion time from receipt time.
  • Known trade off: consolidation inherently links the converted inputs to each other on chain. The threshold and batch cap bound how often this happens, but users who need strict input isolation should not enable this option. Feedback on the default threshold is welcome.

Configuration

Disabled by default (opt-in). In veil.conf:

autoconvert=1
autoconvertthreshold=10   # optional, amount in VEIL

Both options appear in veild -help.

Safety gates

  • Skips during initial block download and until headers and blocks are fully synced
  • Skips when the wallet is locked or unlocked for staking only
  • Skips when wallet broadcast is disabled (-walletbroadcast=0)
  • Input count capped at 32 per transaction so sweeps of large wallets can never approach standard transaction size limits. Large balances convert over multiple cycles

Testing status

⚠️ Testing and review before merge.

Done so far (macOS arm64, daemon build):

  • Full end to end pipeline on regtest: 5500 VEIL across 110 coinbase outputs converted with -autoconvertthreshold=5. Four basecoin→CT batches (32/32/32/13 inputs, respecting the input cap) followed by the CT→RingCT lift after maturity. The entire balance arrived in RingCT with about 0.0073 in total fees and nothing stranded in any pool
  • Failure path exercised live: a consensus rejected transaction was caught by the mempool test before touching the wallet, the record and pending spend unwind ran, and exponential backoff engaged
  • Jittered scheduling observed between cycles (conversions at irregular multi minute intervals, not block locked)

Still to do:

  • Longer testnet soak (real network fees, mempool pressure, reorgs)
  • Encrypted wallet matrix: locked, staking only unlock, fully unlocked
  • Restart between cycles with a conversion pending
  • A repeatable regtest functional test for the conversion and unwind paths

Closes

#1033
#1054

@ohcee

ohcee commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator Author

✅ Additional Testing — June 10, 2026

Completed full end-to-end test on fresh wallet using latest binary (commit f456afcd5).

Transaction Flow

Step Action Result
1 Received 10 VEIL basecoin from zerocoin spend ✅ No crash
2 Auto-converted basecoin → CT on confirmation ✅ Broadcast
3 Received 1 VEIL CT (stealth to stealth) ✅ No crash
4 Auto-converted all CT → RingCT on confirmation ✅ Broadcast
5 Sent RingCT → RingCT manually ✅ Clean send

Debug Log

2026-06-10T18:35:22Z AutoConvert: converting 10.00 basecoin -> CT
2026-06-10T18:35:22Z [default wallet] Relaying wtx edcf03d4e843f961c69570a9b7bc564e93d8864d2797c2d66aaeb93e45a396a9
2026-06-10T18:39:08Z AutoConvert: converting 10.998902 CT -> RingCT
2026-06-10T18:39:08Z [default wallet] Relaying wtx 7f256fa6fa2108f6e26f8876cc453534c635f3541a086c27d6b7dd5455606082
2026-06-10T19:16:26Z [default wallet] Relaying wtx 8fe2010f92183ddd8a98f0f73f5c90e8d451bd5001d3d58e1039796d60f273b7

On-Chain Verification

  • Basecoin→CT: edcf03d4e843f961c69570a9b7bc564e93d8864d2797c2d66aaeb93e45a396a9
  • CT→RingCT: 7f256fa6fa2108f6e26f8876cc453534c635f3541a086c27d6b7dd5455606082
  • RingCT→RingCT send: 8fe2010f92183ddd8a98f0f73f5c90e8d451bd5001d3d58e1039796d60f273b7

@ohcee
ohcee force-pushed the feature/auto-ct-to-ringct branch from f456afc to e847010 Compare June 11, 2026 18:24
@seanPhill seanPhill added the Issue Type: Feature Request Additional feature that would make Veil better. label Jun 20, 2026
@ohcee

ohcee commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

This PR auto-converts basecoin and CT to RingCT on every block, closing #1033 and #1054. Tested on mainnet, conversions confirm correctly each block, and autoconvert=0 is available for anyone who needs funds to stay at the base layer instead.

@seanPhill — given your comment confirmed this solves the issues you raised, do we want more testing? Let me know if there's additional testing you'd want to see, or any case you think isn't covered yet.

@ohcee
ohcee force-pushed the feature/auto-ct-to-ringct branch from 46af28a to 5e0e48a Compare July 20, 2026 03:23
@ohcee

ohcee commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator Author

Force pushed a substantially reworked v2 and updated the description above.

After digging deeper into the wallet internals I found real problems with my original approach: a failed broadcast could strand CT balances (the wallet marks inputs pending spend at build time and nothing undid it), the BlockConnected hook did heavy transaction building on the validation path, and failures retried on every block. The rework moves conversion to a wallet scheduler task, batches at most 32 outputs with a 12 confirmation floor, tests mempool acceptance before committing, and unwinds wallet state on any failure.

Behavior changes if you tried v1: the default minimum is now 10 VEIL (-autoconvertthreshold), conversions happen minutes after maturity rather than instantly per block, and large balances convert over multiple cycles.

Testing and review before merge.

@ohcee ohcee self-assigned this Jul 28, 2026
@ohcee
ohcee requested a review from seanPhill July 28, 2026 15:07
@ohcee

ohcee commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed a failure-path test suite for the unwind machinery: src/wallet/test/autoconvert_tests.cpp (305 lines, two cases).

  • autoconvert_mempool_reject_unwinds — drives the public AutoConvertToRingCT() end-to-end with mock time, forces a guaranteed mempool rejection via an externally-signed conflicting spend of a wallet coinbase, and asserts the wallet is left bit-identical (records, wallet txs, balance, spendable set). A timing probe then distinguishes a real backed-off failure from a silently skipped attempt, and a real conversion is performed past the backoff.
  • unwind_restores_pending_spend — direct do/undo symmetry of MarkInputsAsPendingSpend + SaveRecord vs UnwindPendingTransaction.

Local run:

wallet/test/autoconvert_tests.cpp: Entering test suite "autoconvert_tests"
Entering test case "autoconvert_mempool_reject_unwinds" ... Leaving; testing time: 475571us
Entering test case "unwind_restores_pending_spend" ... Leaving; testing time: 68088us
*** No errors detected

@ohcee ohcee changed the title Auto-convert basecoin and CT to RingCT on block confirmation Auto-convert basecoin and CT to RingCT Aug 13, 2026
@ohcee
ohcee requested a review from Veil-Project August 20, 2026 16:19
ohcee added 2 commits August 23, 2026 20:11
When -autoconvert is enabled, a wallet scheduler task periodically
converts batches of confirmed outputs toward RingCT. Consensus only
permits RingCT outputs from CT or RingCT inputs, so basecoin batches
convert to CT and later cycles lift the matured CT outputs to RingCT.

* runs off the validation path on the wallet scheduler with random
  jitter between attempts
* converts at most 32 outputs with 12 or more confirmations per
  transaction, once the batch reaches -autoconvertthreshold (default 10)
* tests mempool acceptance before committing to the wallet
* unwinds the transaction record and pending spend flags if a built
  transaction cannot be broadcast, and backs off exponentially after
  repeated failures
* skips during IBD, before full sync, while locked or staking only
  unlocked, and when wallet broadcast is disabled

Testing and review before merge.
The one part of the feature mainnet happy-path testing cannot reach: a
conversion rejected by the mempool must unwind completely.

autoconvert_mempool_reject_unwinds drives the public
AutoConvertToRingCT() through its gates with mock time. A conflicting
spend of one of the wallet's own coinbases, signed with the raw key and
placed in the mempool from outside the wallet, guarantees the next
conversion attempt builds successfully (record saved, inputs marked
pending-spend) and then fails mempool test-acceptance. Asserts the
unwind leaves no transaction record, no wallet transaction, and the
balance and spendable-coin set bit-identical.

The backoff probe doubles as proof the failure really executed: a
failed attempt backs off exactly 600s while a NOTHING_TO_CONVERT pass
reschedules within at most 540s, so probing at +570s with the conflict
cleared distinguishes a genuine failure (still blocked) from an attempt
that silently never ran (would convert and fail the mempool-empty
check). Past the backoff the conversion must then go through for real:
a batch spending all mature coinbases, CT outputs, record and wallet
transaction keyed by the same txid.

unwind_restores_pending_spend locks the do/undo symmetry directly:
MarkInputsAsPendingSpend + SaveRecord (what AddStandardInputs and
AddBlindedInputs do at build time) followed by UnwindPendingTransaction
restores ORF_PENDING_SPEND and erases the record.

The fixture wires a wallet the way CreateWalletFromFile does (HD seed,
zerocoin wallet, AnonWallet on a mock db, stealth/blinding contexts)
and creates an in-memory pzerocoinDB, which ConnectBlock needs once the
regtest chain passes height 20 (accumulator checksums every 10 blocks).
X16RT blocks after the PowUpdateTimestamp always retarget to the pow
limit, so extending the fixture chain is safe from the regtest DGW
overflow.
@ohcee
ohcee force-pushed the feature/auto-ct-to-ringct branch from 09eb5db to 9f4d975 Compare August 24, 2026 03:18

@seanPhill seanPhill left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK

As this is opt-in this can be merged, and further testnet testing can be done.
If anything in the NITS or suggestions (help text, for example) can be done, it can be done later.


PR Review: #1055 – Auto-convert basecoin and CT to RingCT
Overall recommendation: Approve with minor comments / additional testing recommended before merge.
This is a well-thought-out and carefully reworked implementation of a feature that addresses real user pain points (#1033 and #1054). Moving the logic out of BlockConnected into a wallet scheduler task, staging the two conversion hops, adding a mempool pre-check, and implementing proper unwind on failure are all significant improvements over any earlier approach. The design shows a clear understanding of the wallet’s internal state machine and the consensus constraints around RingCT inputs.
Strengths

Correct default and safety posture: The feature is fully opt-in (autoconvert=0 by default). This is the right call. Exchanges, services, and any operator who needs balances to remain in basecoin for accounting or operational reasons are not surprised. Combined with the existing -exchangesandservicesmode, the risk surface for accidental conversion is appropriately low.
Robust failure handling: The combination of “test mempool acceptance before committing wallet state” + AnonWallet::UnwindPendingTransaction + exponential backoff is excellent. The two unit tests (autoconvert_mempool_reject_unwinds and unwind_restores_pending_spend) give good confidence that stranded inputs are avoided.
Privacy-conscious scheduling: Time-based execution with jitter (instead of block-synchronous) and the 12-confirmation maturity requirement are sensible choices that reduce correlation risk.
Practical batching limits: Cap of 32 inputs and a configurable threshold (default 10 VEIL) keep transaction size and frequency under control.
Clear documentation in the PR description: The two-hop explanation, consensus rule reference, and explicit privacy trade-off note are helpful for future maintainers and users.

Specific feedback / questions

Interaction with -exchangesandservicesmode
The PR description does not explicitly state whether the autoconvert scheduler respects (or is affected by) -exchangesandservicesmode. Even though autoconvert is opt-in, it would be cleaner to either:
Document that the two flags are independent, or
Make autoconvert a no-op when exchangesandservicesmode is active (defensive).
A one-line comment or short note in the help text would remove any ambiguity for exchange operators.

Wallet locking / staking-only unlock
Skipping when the wallet is locked or unlocked only for staking is correct. Please confirm (or add a brief comment) that the scheduler does not attempt to unlock the wallet itself under any circumstances.
Restart / pending conversion behaviour
The pending testing list already calls this out, which is good. A quick note in the PR (or a small functional test) confirming that a wallet restart while a conversion is pending leaves no permanent pending-spend flags would be reassuring.
Threshold and privacy trade-off
Default of 10 VEIL feels reasonable. The PR already solicits feedback on this value — I would keep it. The consolidation risk is correctly called out; users who care about input isolation can simply leave the feature disabled.
Logging
When a conversion batch is built and broadcast (or fails and is unwound), is there a clear, non-spammy log line at a reasonable level (info/debug)? Operators monitoring exchange wallets will want to see these events without having to dig through debug logs.

Testing status (as of the latest description)
Completed work looks solid (regtest multi-batch conversion, failure/unwind paths, jitter observation, and a fresh-wallet flow).
Still outstanding items that I would like to see before final merge:

Longer testnet soak under real mempool pressure and fee conditions.
Explicit matrix for encrypted wallets (locked / staking-only / fully unlocked).
Restart-with-pending-conversion case.
Optional: a higher-level functional/regtest test that exercises the full basecoin → CT → RingCT path end-to-end and asserts final balances and no residual pending spends.

Minor nits

The help text for -autoconvert and -autoconvertthreshold should explicitly state that the feature is disabled by default and that conversion is two-hop.
Consider adding a short note in the release notes / user-facing docs once this lands, explaining the opt-in nature and the privacy vs convenience trade-off.

Summary
This is high-quality work. The rework to a scheduler-based, staged, failure-resilient design is the right architecture. The opt-in default correctly protects exchanges and operators who need basecoin balances.
Once the remaining test matrix items are exercised (or at least the restart and encrypted-wallet cases are manually verified), this is ready to merge.
Nice job on the careful state management and the clear documentation of trade-offs.

@Veil-Project Veil-Project left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

The description itself recommends more "soak testing"on testnet.

This update is fine, and more testnet soaking will begin tonight.

Upon this commit it will contain numerous important and useful upgrades.

@Veil-Project
Veil-Project merged commit d26cd6b into Veil-Project:master Aug 24, 2026
9 checks passed
@ohcee
ohcee deleted the feature/auto-ct-to-ringct branch August 24, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Issue Type: Feature Request Additional feature that would make Veil better.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants