diff --git a/src/content/docs/developer/integrations/tx-lifecycle.md b/src/content/docs/developer/integrations/tx-lifecycle.md index f8c9633..1aa41db 100644 --- a/src/content/docs/developer/integrations/tx-lifecycle.md +++ b/src/content/docs/developer/integrations/tx-lifecycle.md @@ -38,7 +38,7 @@ Do not use **included**, **removed**, **accepted**, or **confirmed** as a paymen | How are transactions prioritized? | Candidate selection iterates by descending `gasPrice`. Equal-price ordering is not a client contract. | | How does replacement work? | A transaction that conflicts on a spending ID replaces the existing transaction only when its gas price is strictly higher. Raising only the gas limit is insufficient. Moonlight uses account and nonce as its spending ID; Phoenix conflicts on spent nullifiers. | | What happens when the mempool is full? | A higher-priced transaction can evict the lowest-priced entry. Node operators configure the capacity; the default is 10,000 transactions. | -| When do transactions expire? | Expiry is local node policy, not a field in the transaction. The default residence is three days, checked hourly, and both values are configurable. Clients must not treat three days as a network guarantee. | +| When do transactions expire? | Expiry is local node policy, not a field in the transaction. Rusk's built-in defaults are three days with hourly checks, while node-installer `v0.5.22` configures mainnet and testnet nodes for 30 minutes with checks every five minutes. Clients must use the policy of their node instead of treating either value as a network guarantee. | | What happens to future Moonlight nonces? | A transaction with a nonce gap is briefly staged outside the real mempool while the node waits for intermediate nonces. It emits `deferred`, is invisible to `mempoolTxs`, and is admitted only if the gap is filled before retries end. | | Can every node publish transactions? | A full Rusk node can receive and rebroadcast transactions. Its operator can disable or restrict public HTTP access, apply ACLs, or rate-limit propagation. | diff --git a/src/content/docs/operator/archive-node.md b/src/content/docs/operator/archive-node.md index 83e0bb7..b0983b0 100644 --- a/src/content/docs/operator/archive-node.md +++ b/src/content/docs/operator/archive-node.md @@ -1,111 +1,103 @@ --- title: Run an archive node -description: Learn about Dusk archive nodes that store and give access to Dusk’s historical data. +description: Install and operate a Rusk node with finalized historical indexes. --- -Archive nodes extend the functionality of [Provisioners](/operator/provisioner) by also preserving a complete historical record of the Dusk blockchain. While provisioners focus on the current state and consensus duties, archive mode adds long-term historical data access for applications, users, researchers, and auditors. +An archive node runs the archive build of Rusk and retains finalized historical indexes in addition to normal chain state. Applications use it for queries such as `moonlightHistory`, `fullMoonlightHistory`, and `finalizedEvents`. -Archive nodes are commonly run as data-serving infrastructure and do not need to stake DUSK. However, archive mode is built on top of the regular node stack, so an archive node can also be configured to participate in consensus if you choose to stake and run it that way. +Archive mode does not require staking. An archive node can also participate in consensus, but production API infrastructure is usually kept separate from provisioner duties so query load and maintenance do not compete with consensus. -In short, archive nodes: -- Provide additional historical data (such as events emitted by contracts) that is not stored by a Provisioner node -- Expose archive-only GraphQL queries such as `moonlightHistory`, `fullMoonlightHistory`, and `finalizedEvents` -- Can also participate in consensus by staking DUSK, though that is optional +## Requirements -:::tip[Run an Archive node] -If you want to quickly launch & run an archive node, you can use the node installer by following [the archive guide](/operator/archive-node). -::: - -## Archive Node Specifications +Use Ubuntu 24.04 LTS and begin with at least: -Archive nodes store and serve historical data and require large storage capacity, efficient processing for concurrent requests, and fast Internet connectivity. The following recommended specifications serve as a baseline for Archive Nodes. Over time, storage requirements will increase. +| CPU | RAM | Storage | Network | +|---|---|---|---| +| 4 cores at 2 GHz | 8 GB | 500 GB | 100 Mbps | -| CPU | RAM | Storage | Network Connection | -| :------------- | :--- | :------ | :----------------- | -| 4 cores; 2 GHz | 8 GB | 500 GB | 100 Mbps | +This is an initial planning baseline, not a capacity guarantee. Archive storage grows over time; monitor actual utilization and expansion lead time. -## Run an archive node -> A step-by-step guide to setting up a Dusk archive node. +## Install -The following guide will explain you how to install and setup an archive node on Ubuntu 24.04 through the [node installer](https://github.com/dusk-network/node-installer). This installer will set up Rusk as a service on your server, preconfigure parts of the node, and provide a couple of helper scripts. +Run the released node installer with the archive feature: -Install Rusk with the archive feature enabled by pasting the following command in your server terminal: ```sh -curl --proto '=https' --tlsv1.2 -sSfL https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh | sudo bash -s -- --feature archive +curl --proto '=https' --tlsv1.2 -sSfL \ + https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh \ + | sudo bash -s -- --feature archive ``` -This will install an archive node with the network being set to mainnet. - -## Configure Rusk +Add `--network testnet` before `--feature archive` for a testnet archive node. -You now should have successfully installed Rusk. - -A quick check with: - -```sh -ruskquery version -``` - -Should tell you, that you are running the latest installer version. - -## Start your node - -If you've configured everything correctly, you can now start rusk: -```sh -service rusk start -``` +Start Rusk and monitor synchronization: -Your node will now start syncing. You can check if it indeed is by running: ```sh +sudo systemctl start rusk +systemctl is-active rusk +ruskquery peers ruskquery block-height ``` -It is best to wait until your node is synced up. You can find the latest block height on [the block explorer](https://explorer.dusk.network/). Alternatively, consider [fast-syncing](/operator/guides/fast-sync) for a quicker method. +Compare the height with the matching [network](/operator/networks/). + +An archive expected to answer complete history must synchronize from genesis or restore a trusted backup that includes the archive databases. The installer's `download_state` snapshot restores execution state, but it does not backfill archive indexes for blocks before that snapshot. -## Enable http +## Enable the HTTP API -If you want to serve archive data to the outside world, your node needs to enable the http capabilities. This can be done by adding +The installed API is not intended to be public by default. For local access or a reverse proxy on the same host, add this to `/opt/dusk/conf/rusk.toml`: ```toml -# rusk.toml [http] listen = true -listen_address = '0.0.0.0:8080' +listen_address = '127.0.0.1:8080' ``` -To your rusk.toml file in your system's `/opt/dusk/conf` folder. That's it. +Then restart Rusk: -Now you can query the archive for data with an external client application. +```sh +sudo systemctl restart rusk +``` -## Test archive endpoint +Use `0.0.0.0:8080` only when the API must accept traffic on an external interface. Restrict it with a firewall, access policy, rate limits, or a controlled reverse proxy. Public `8080/tcp` access is not required for synchronization or consensus. -You can check which GraphQL queries are available by retrieving the schema (SDL). On RUES, an empty body returns the schema: -```bash -curl -s -X POST "http://:8080/on/graphql/query" -``` +:::caution[Updates regenerate rusk.toml] +Rerun the installer with `--feature archive` during every update. The installer regenerates `/opt/dusk/conf/rusk.toml`, so review the new file and reapply the intended `[http]` settings before returning the API to service. +::: -This should now return a different schema than a normal node returns. Archive-enabled nodes expose additional historical queries such as `moonlightHistory`, `fullMoonlightHistory`, and `finalizedEvents`. +## Verify archive queries -A regular node also exposes `checkBlock`, but the `onlyFinalized: true` behavior below is archive-only. +List the archive fields exposed by canonical GraphQL: -In order to test this endpoint, you can run the following command. +```sh +curl -fsS -X POST "http://127.0.0.1:8080/graphql" \ + -H "Content-Type: application/json" \ + --data-raw '{"query":"{ __schema { queryType { fields { name } } } }"}' \ + | jq -r '.data.__schema.queryType.fields[].name' \ + | grep -E '^(moonlightHistory|fullMoonlightHistory|finalizedEvents|checkBlock)$' +``` + +Query a known block: -```bash -curl -s -X POST "http://:8080/on/graphql/query" \ - --data-raw '{ block(height: 1) { header { hash } } }' | jq . +```sh +curl -fsS -X POST "http://127.0.0.1:8080/graphql" \ + -H "Content-Type: application/json" \ + --data-raw '{"query":"{ block(height: 1) { header { height hash } } }"}' \ + | jq . ``` -Then use the returned block hash with `checkBlock`: +To verify finalized-block lookup, replace `` with the returned hash: -```bash -curl -s -X POST "http://:8080/on/graphql/query" \ - --data-raw 'query { checkBlock(height: 1, hash: "", onlyFinalized: true) }' | jq . +```sh +curl -fsS -X POST "http://127.0.0.1:8080/graphql" \ + -H "Content-Type: application/json" \ + --data-raw '{"query":"{ checkBlock(height: 1, hash: \"\", onlyFinalized: true) }"}' \ + | jq . ``` -On a non-archive node, `onlyFinalized: true` returns an error. +Archive history is populated as blocks finalize. Verify the earliest range your application requires before declaring the service ready. -## Stake with archive node +See [Scan Moonlight deposits](/developer/integrations/historical_events/) for a bounded history consumer and [HTTP API](/developer/integrations/http-api/) for query and access-policy details. -It is possible to stake and participate in consensus while the archive node is running. This is usually not recommended, but is possible since the archive is built on top of a normal provisioner node and therefore has all the capabilities to do so. +## Stake from an archive node -You can read the [node wallet guide](/operator/guides/node-wallet-setup) for a step-by-step instruction on setting up the wallet, depending on if you want the archive node to participate in consensus too or not. +Staking is optional. If this host will also be a provisioner, follow [Set up the node wallet](/operator/guides/node-wallet-setup/) and apply the same monitoring, key-isolation, and update procedures as any other provisioner. diff --git a/src/content/docs/operator/faq.md b/src/content/docs/operator/faq.md index e0cab13..9ae1619 100644 --- a/src/content/docs/operator/faq.md +++ b/src/content/docs/operator/faq.md @@ -63,7 +63,9 @@ Yes. The node only needs the consensus key file (`consensus.keys`). Export it fr Example (adjust user/IP): ```bash -scp consensus.keys duskadmin@:/opt/dusk/conf/ +scp consensus.keys duskadmin@:~/consensus.keys +ssh duskadmin@ \ + 'sudo install -o root -g dusk -m 640 ~/consensus.keys /opt/dusk/conf/consensus.keys && rm -f ~/consensus.keys' ``` #### What if I lose access to my server or keys? @@ -80,13 +82,15 @@ rusk-wallet stake --amt 3000 #### How can I recover my node if the state is corrupted? -Reload from a snapshot and restart: +For a default mainnet or testnet node, reload from a snapshot and restart: ```bash -download_state -service rusk start +sudo download_state +sudo systemctl start rusk ``` +For an archive node that requires complete history, follow the archive-specific [re-sync guidance](/operator/guides/manual-resync/#restore-a-published-state). + #### How can I run a Dusk node on Docker? We don't support a production-ready Docker image. For ephemeral (non-persistent) usage: @@ -111,9 +115,10 @@ Without the node installer, configure `rusk.toml`: [kadcast] public_address = ":" listen_address = ":" # Optional -bootstrapping_nodes = ["165.22.193.63:9000", "167.172.175.19:9000"] ``` +Retain the bootstrapping nodes supplied for the selected network. + #### How do I change the HTTP API port? In `rusk.toml`: @@ -121,27 +126,20 @@ In `rusk.toml`: ```toml [http] listen = true -listen_address = "0.0.0.0:8080" +listen_address = "127.0.0.1:8081" ``` -Or via CLI: - -```bash ---http-listen-addr 0.0.0.0:8081 -``` +The installer regenerates `rusk.toml` during upgrades, so review and reapply this setting afterward. Bind an external interface only when required, and restrict public API traffic. #### How can I get data from testnet or mainnet nodes? See [/developer/integrations/http-api](/developer/integrations/http-api) and [/developer/integrations/historical_events](/developer/integrations/historical_events). -#### How can I relay my internal port 8080 when using RUES? - -```bash -socat tcp-listen:8081,reuseaddr,fork tcp:localhost:8080 -``` - #### How can I perform a liveness check on my node? - `ruskquery block-height` (should increase over time) - `ruskquery peers` -- `systemctl status rusk` +- `systemctl is-active rusk` +- `ruskquery info` + +See [Maintain and monitor a node](/operator/maintenance-monitoring/) for network-tip comparison, resources, logs, and alerts. diff --git a/src/content/docs/operator/guides/fast-sync.md b/src/content/docs/operator/guides/fast-sync.md index 0635454..d9a7a8b 100644 --- a/src/content/docs/operator/guides/fast-sync.md +++ b/src/content/docs/operator/guides/fast-sync.md @@ -1,34 +1,55 @@ --- -title: Fast-sync your node -description: Speed up your Dusk node synchronization by leveraging pre-available snapshot. +title: Fast-sync a node +description: Replace local chain state with a published mainnet or testnet snapshot. --- -The node installer comes with an easy to use fast syncing tool. To significantly reduce the time required to sync your node to the latest published state, you can use the `download_state` command. This command stops your node and replaces its current state with the latest published state from one of Dusk's archive nodes. +The node installer provides `download_state` for mainnet and testnet. It downloads a published state, stops Rusk, replaces the local state and chain database, and leaves the service stopped for operator verification. -## Available states +:::caution +Fast sync replaces current chain state under `/opt/dusk/rusk`. It does not replace consensus keys or node configuration, but you should still verify you selected the intended network before confirming. +::: + +Fast sync does not backfill archive indexes for blocks before the snapshot. Do not use a state snapshot alone to bootstrap an archive node that promises complete historical data. + +## List snapshots + +The command detects the network from `/opt/dusk/conf/rusk.toml`: -To see the available published states, run: ```sh download_state --list ``` -## Download state +Override detection only when necessary: + +```sh +download_state --network testnet --list +``` + +## Replace state + +Download the latest snapshot: -To install the latest state, simply run: ```sh -download_state +sudo download_state ``` -Once it tells you the operation is complete, run the following command to start your node again: +Or select one of the listed block heights: + ```sh -service rusk start +sudo download_state ``` -This process bootstraps your node with the latest available state snapshot, allowing it to sync the remaining blocks much faster than starting from genesis. +The tool downloads to a unique temporary file and removes it afterward. A successful command still leaves Rusk stopped. + +## Restart and verify -:::note -If you are experiencing errors in downloading the state, it might be due to some remnants of previous state syncing. Try to clean up with: ```sh -sudo rm /tmp/state.tar.gz +sudo systemctl start rusk +systemctl is-active rusk +ruskquery peers +ruskquery block-height ``` -::: \ No newline at end of file + +Check the height again after roughly 30 seconds. The node should continue syncing from the snapshot toward the current [network tip](/operator/networks/). + +If the service fails, inspect `systemctl status rusk`, `/var/log/rusk.log`, and `/var/log/rusk_recovery.log` before retrying. See [Re-sync a node](/operator/guides/manual-resync/) for the decision process. diff --git a/src/content/docs/operator/guides/manual-resync.md b/src/content/docs/operator/guides/manual-resync.md index 217cf83..c5902d5 100644 --- a/src/content/docs/operator/guides/manual-resync.md +++ b/src/content/docs/operator/guides/manual-resync.md @@ -1,74 +1,42 @@ --- -title: Manually re-sync your node -description: Learn how to force a Dusk node resync when it becomes stuck on a fork. +title: Re-sync a node +description: Diagnose a stalled node and restore it from a supported state snapshot. --- -Resyncing a Dusk node is sometimes a necessary procedure if your node experiences issues such as getting stuck on a block or if there's a need to recover from a network split. Below is a detailed guide on how to manually resync your Dusk node. Before proceeding, ensure you have access to your node's console and necessary permissions to execute the commands. +Re-sync only after confirming the node is stalled or on unusable local state. A node that is behind but continuing to advance usually needs time, not state replacement. -## Preliminary Check: Determine the Need for Resync - -Before initiating a resync, confirm if your node is indeed stuck or lagging behind the network. Execute the following command to check your current block height: +## Confirm the problem ```sh +systemctl is-active rusk +ruskquery info | jq '{version, chain_id, kadcast_address}' +ruskquery peers ruskquery block-height +sleep 30 +ruskquery block-height +sudo tail -n 100 /var/log/rusk.log ``` -Compare the output with the latest block height available on the [Dusk Network Explorer](https://explorer.dusk.network/). - -## Steps to Resync Your Node - -If your node is confirmed to be stuck (e.g. at block 50636) or significantly behind the current block height, follow these steps to resync: - -### 1. Unstake (if applicable) - -If you are staked, the first step is to unstake to prevent any potential loss of stake due to node downtime: - -```sh -rusk-wallet unstake -``` - -### 2. Stop the Rusk Service - -Stop the Rusk service to prevent any new data from being written to the database: - -```sh -service rusk stop -``` - -### 3. Use the fast syncing tool - -Retrieve the latest published state through the fast syncing tool: -```sh -download_state -``` - -For more details see the [fast syncing page](/operator/guides/fast-sync). +Compare the local height and chain ID with the selected [network](/operator/networks/). Fix service, version, peer, firewall, or wrong-network problems before replacing valid state. -### 4. Restart Rusk +## Restore a published state -With the old data removed, restart the Rusk service to begin syncing from scratch: +For a default mainnet or testnet node, use the installer-managed fast-sync tool: ```sh -service rusk start +download_state --list +sudo download_state +sudo systemctl start rusk ``` -### 5. Monitor Sync Progress +Then verify service state, peers, and increasing block height. See [Fast-sync a node](/operator/guides/fast-sync/) for snapshot selection and network overrides. -Monitor the progress of your node's sync by checking the last block accepted by your node: +For an archive node, a state snapshot does not reconstruct pre-snapshot historical indexes. Restore a known-good backup that includes the archive databases or synchronize the archive from genesis when complete history is required. -```sh -ruskquery block-height -``` - -### 6. Restake (if applicable) +Routine re-sync does not require unstaking. Keep the recovery window short and monitor provisioner faults. If the node will remain unavailable, its keys may be compromised, or penalties are increasing, follow [Slashing recovery](/operator/guides/slashing-recovery/). -Once your node is close to the current block height, you can restake your DUSK tokens: - -```sh -rusk-wallet stake --amt -``` -Replace `` with the number of DUSK tokens you wish to stake. +## Full reset -## Conclusion +`ruskreset` deletes local state, archive data, wallet cache, and Rusk logs. It is not the first response to ordinary lag or a recoverable snapshot failure. -Following these steps should successfully resync your Dusk node with the network. It's crucial to periodically check your node's health and sync status to ensure it remains in good standing within the network. For more information and updates on the Dusk Network, refer to the [official GitHub page](https://github.com/dusk-network/rusk). +Use it only when a network reset announcement or Dusk support procedure explicitly requires a full reset. Preserve diagnostic logs before running it, confirm the selected network, and be prepared to synchronize state again afterward. diff --git a/src/content/docs/operator/guides/node-wallet-setup.md b/src/content/docs/operator/guides/node-wallet-setup.md index cdbb896..e6ebbbd 100644 --- a/src/content/docs/operator/guides/node-wallet-setup.md +++ b/src/content/docs/operator/guides/node-wallet-setup.md @@ -44,10 +44,12 @@ You will be asked to provide your recovery phrase/mnemonic, **in lowercase**, an #### Export consensus key -Once you've done so, run the following command to export a **consensus key** for the given wallet: +Once you've done so, export a **consensus key** for the given wallet to your home directory, then install it with the ownership and permissions expected by Rusk: ```sh -rusk-wallet export -d /opt/dusk/conf -n consensus.keys +rusk-wallet export -d "$HOME" -n consensus.keys +sudo install -o root -g dusk -m 640 \ + "$HOME/consensus.keys" /opt/dusk/conf/consensus.keys ``` To participate in consensus, Rusk needs your consensus keys. These keys are used to sign and vote for blocks. You will be asked to set an encryption password for the consensus key, make sure to remember it. @@ -55,7 +57,7 @@ To participate in consensus, Rusk needs your consensus keys. These keys are used Now, run the following script and provide the **same password** (for consensus key) from before. This will set the password as an environment variable for Rusk to use. ```sh -sh /opt/dusk/bin/setup_consensus_pwd.sh +sudo sh /opt/dusk/bin/setup_consensus_pwd.sh ``` #### Start your node @@ -66,7 +68,7 @@ If you prefer to sync from genesis (e.g., for auditing or archival purposes), do Start your node with: ```sh -service rusk start +sudo systemctl start rusk ``` Once the node is running, check its sync progress by running: diff --git a/src/content/docs/operator/guides/rollback-node-update.md b/src/content/docs/operator/guides/rollback-node-update.md index 7765470..7ad7ad3 100644 --- a/src/content/docs/operator/guides/rollback-node-update.md +++ b/src/content/docs/operator/guides/rollback-node-update.md @@ -1,93 +1,87 @@ --- title: Roll back a node update -description: Learn how to reinstall a previous Dusk node installer release when an update needs to be rolled back. +description: Reinstall a pinned node-installer release when rollback is supported by the network. --- -Rolling back means reinstalling a previous `node-installer` release. This can be useful if a newly installed node release has a problem and the network has not yet activated a protocol change that requires it. +Rollback installs the Rusk and Rusk Wallet versions pinned by an older node-installer release. Use it only when that software still supports the active network. :::caution -Do not roll back across a network upgrade or activation unless the Dusk team explicitly instructs operators to do so. If the chain already requires the newer node version, an older node may stop following the network. +Do not roll back across a protocol activation unless Dusk explicitly instructs operators to do so. An incompatible node can stop following the network, and an archive database migration may not support downgrade. ::: -## Before you roll back +## Before rollback -Check the version currently installed: +Record the current state and preserve intentional configuration: ```sh ruskquery version -``` - -Check whether the node is syncing: - -```sh +ruskquery info | jq '{version, chain_id, kadcast_address}' +ruskquery peers ruskquery block-height -tail -n 50 /var/log/rusk.log +systemctl is-active rusk ``` -If the node is only stuck or behind, try [fast-sync](/operator/guides/fast-sync) or [manual resync](/operator/guides/manual-resync) before rolling back. +Confirm that the failure is caused by the installed release. A stalled or corrupt state normally requires [re-sync](/operator/guides/manual-resync/), not older software. -## Roll back mainnet +Rollback regenerates the same managed files as an upgrade, including `rusk.toml` and the systemd unit. It preserves consensus keys, protected service environment files, and chain state, but it does not restore an older database snapshot. -Replace `vX.Y.Z` with the installer release you want to roll back to. +## Install a pinned release -For example, to roll back to installer release `v1.2.3`, set: +Set the node-installer release approved for rollback: -```sh -INSTALLER_VERSION="v1.2.3" ```sh INSTALLER_VERSION="vX.Y.Z" +``` -sudo service rusk stop +Use the command matching the node's existing network and role. The installer stages and verifies downloads before stopping Rusk, so do not stop the service first. +### Mainnet provisioner or full node + +```sh curl --proto '=https' --tlsv1.2 -sSfL \ "https://github.com/dusk-network/node-installer/releases/download/${INSTALLER_VERSION}/node-installer.sh" \ | sudo bash - -sudo service rusk start ``` -## Roll back testnet - -Use the same pinned installer release, but pass the testnet flag. +### Mainnet archive node ```sh -INSTALLER_VERSION="vX.Y.Z" +curl --proto '=https' --tlsv1.2 -sSfL \ + "https://github.com/dusk-network/node-installer/releases/download/${INSTALLER_VERSION}/node-installer.sh" \ + | sudo bash -s -- --feature archive +``` -sudo service rusk stop +### Testnet provisioner or full node +```sh curl --proto '=https' --tlsv1.2 -sSfL \ "https://github.com/dusk-network/node-installer/releases/download/${INSTALLER_VERSION}/node-installer.sh" \ | sudo bash -s -- --network testnet - -sudo service rusk start ``` -## Verify the rollback - -Confirm the installed version: +### Testnet archive node ```sh -ruskquery version +curl --proto '=https' --tlsv1.2 -sSfL \ + "https://github.com/dusk-network/node-installer/releases/download/${INSTALLER_VERSION}/node-installer.sh" \ + | sudo bash -s -- --network testnet --feature archive ``` -Check the service: +For direct root automation, append `--user `. -```sh -service rusk status -``` +## Restart and verify -Check whether the node is progressing: +Review regenerated configuration and reapply only intentional settings that remain valid for the older release. Then start Rusk: ```sh +sudo systemctl start rusk +systemctl is-active rusk +ruskquery info | jq '{version, chain_id, kadcast_address}' +ruskquery peers ruskquery block-height -tail -F /var/log/rusk.log +sudo tail -n 50 /var/log/rusk.log ``` -If block height does not progress, compare your height with the explorer and consider [fast-syncing the node](/operator/guides/fast-sync). - -## Notes +Confirm that height continues to increase and remains close to the selected network. For an archive, verify required historical queries before returning API traffic. -- Use a pinned installer release URL. Do not use `latest` for rollback. -- Rollback changes the installed node software and service configuration. It does not automatically restore an older chain state. -- Archive nodes may have database migrations that are not safe to downgrade. If an archive node fails after rollback, update back to the supported version or resync the archive state. -- If you operate a provisioner, monitor the node after rollback. Downtime or running an incompatible version can affect consensus participation. See [Slashing prevention and recovery](/operator/guides/slashing-recovery). +If the older release cannot open current state or archive data, reinstall the supported current release. Do not repeatedly alternate versions against the same database. diff --git a/src/content/docs/operator/guides/slashing-recovery.md b/src/content/docs/operator/guides/slashing-recovery.md index c2c661b..6f43ff8 100644 --- a/src/content/docs/operator/guides/slashing-recovery.md +++ b/src/content/docs/operator/guides/slashing-recovery.md @@ -25,22 +25,13 @@ Start by restoring healthy node operation. Slashing is a symptom; the first prio ### 1. Check the installed version ```sh +ruskquery info | jq -r '.version' ruskquery version ``` -If the network has a required release, upgrade to it: +The first command reports the running Rusk version; the second checks the installed node-installer release. -```sh -curl --proto '=https' --tlsv1.2 -sSfL https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh | sudo bash -sudo service rusk start -``` - -For testnet: - -```sh -curl --proto '=https' --tlsv1.2 -sSfL https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh | sudo bash -s -- --network testnet -sudo service rusk start -``` +If the network has a required release, follow [Upgrade a node](/operator/guides/upgrade-node/) with the existing network and feature flags. This is especially important for archive nodes. If a newly installed release is known to be problematic and the network can still run the previous version, follow [Rollback a node update](/operator/guides/rollback-node-update). @@ -50,17 +41,19 @@ If a newly installed release is known to be problematic and the network can stil ruskquery block-height ``` -Compare the height with the explorer. If the node is stuck or far behind, use [fast-sync](/operator/guides/fast-sync): +Compare the height with the explorer. If a default node is stuck or far behind, use [fast-sync](/operator/guides/fast-sync): ```sh -download_state -sudo service rusk start +sudo download_state +sudo systemctl start rusk ``` +For an archive node that requires complete historical indexes, follow the archive-specific [re-sync guidance](/operator/guides/manual-resync/#restore-a-published-state) instead. + ### 3. Check service status and logs ```sh -service rusk status +systemctl status rusk tail -n 100 /var/log/rusk.log ``` @@ -113,7 +106,7 @@ Replace `` with the amount you want to stake. The new stake must mature - Keep consensus keys backed up and readable by the Rusk service. - Never run the same consensus key on multiple active nodes. - Avoid running experimental or mismatched binaries on a staked provisioner. -- Use [fast-sync](/operator/guides/fast-sync) when a node falls behind instead of waiting for a long resync from genesis. +- Use [fast-sync](/operator/guides/fast-sync) for a default node when state recovery is required; preserve complete-history requirements for archive nodes. ## Related guides diff --git a/src/content/docs/operator/guides/upgrade-node.mdx b/src/content/docs/operator/guides/upgrade-node.mdx index ecdc901..98f921a 100644 --- a/src/content/docs/operator/guides/upgrade-node.mdx +++ b/src/content/docs/operator/guides/upgrade-node.mdx @@ -1,111 +1,108 @@ --- -title: Upgrade your node -description: Learn how to update your Dusk node to the latest version of the Nocturne testnet. +title: Upgrade a node +description: Apply a supported node-installer release and verify the node afterward. --- -A network may occasionally be upgraded with new features and performance optimizations. +The node installer pins the supported Rusk and Rusk Wallet versions for each network. Rerunning it downloads and verifies replacement binaries before stopping the current Rusk service. -To make the upgrade process as flawless as possible, the [node installer](https://github.com/dusk-network/node-installer) script may be updated from time to time. This script can be run in a non-destructive way, meaning it changes only what is needed. It will gracefully shut down Rusk for you. +:::caution[Keep the same role] +Repeat the network and feature flags used for the existing node. Omitting `--feature archive` replaces an archive installation with the default Rusk binary. +::: -## How to Upgrade +## Before upgrading -import { Tabs, TabItem } from '@astrojs/starlight/components'; +Record the current state: - - +```sh +ruskquery version +ruskquery info | jq '{version, chain_id, kadcast_address}' +ruskquery peers +ruskquery block-height +systemctl is-active rusk +``` - To upgrade to the latest mainnet version, run: +Retain a secure copy of consensus keys and intentional configuration. The installer preserves: - ```bash - # mainnet upgrade - curl --proto '=https' --tlsv1.2 -sSfL https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh | sudo bash - ``` +- `/opt/dusk/conf/consensus.keys`; +- `/opt/dusk/services/dusk.conf`; +- `/opt/dusk/services/rusk.conf.user`; and +- chain state under `/opt/dusk/rusk`. - - +It regenerates `/opt/dusk/conf/rusk.toml`, `genesis.toml`, the Rusk systemd unit, managed binaries and scripts, and the selected user's wallet configuration. Save the old `rusk.toml` for comparison if it contains intentional changes, but do not copy it over the new network configuration wholesale. - To upgrade to the latest Nocturne testnet version, run: +## Run the installer - ```bash - # testnet upgrade - curl --proto '=https' --tlsv1.2 -sSfL https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh | sudo bash -s -- --network testnet - ``` +Choose the command matching the node. - - +### Mainnet provisioner or full node -Once it's done and gives no errors, start Rusk again: ```sh -sudo service rusk start +curl --proto '=https' --tlsv1.2 -sSfL \ + https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh \ + | sudo bash ``` -If everything is up and running again, query the node a couple of times to see if the block height is progressing: -```sh -ruskquery block-height -``` +### Mainnet archive node -Or keep an eye on the logs to see if it's making and accepting new blocks: ```sh -tail -F /var/log/rusk.log +curl --proto '=https' --tlsv1.2 -sSfL \ + https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh \ + | sudo bash -s -- --feature archive ``` -If you see new blocks being accepted, you're up and running again. +### Testnet provisioner or full node -You can see the status of the Rusk service here: ```sh -service rusk status +curl --proto '=https' --tlsv1.2 -sSfL \ + https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh \ + | sudo bash -s -- --network testnet ``` -If it is not running after starting it, and reports an error, you can check your logs to see if something is wrong, like a wrong password or no peers being found: +### Testnet archive node + ```sh -tail -n 30 /var/log/rusk.log +curl --proto '=https' --tlsv1.2 -sSfL \ + https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh \ + | sudo bash -s -- --network testnet --feature archive ``` -Unable to figure it out yourself? Visit our [Node Runner Troubleshooting](https://discord.com/channels/847466263064346624/1118582421055606805) on Discord. - -If everything else fails, check out the [manual resync](/operator/guides/manual-resync) instructions. +When an automation process runs the installer directly as root, append `--user ` so the wallet configuration is written to the intended home directory. A normal interactive `sudo` invocation uses `SUDO_USER` automatically. -## Roll back an update +The installer stops Rusk only after downloads and binary version checks pass. It enables and reloads the systemd unit, but leaves Rusk stopped. -If a newly installed release causes problems and the network has not activated changes that require it, you can reinstall a previous `node-installer` release. +## Review and restart -Use a pinned installer release instead of `latest`, and only roll back across a network upgrade if the Dusk team instructs operators to do so. +If the previous `rusk.toml` contained intentional operator settings, compare it with the generated file and reapply only the required values. Archive API operators commonly need to restore their `[http]` configuration. -See: [Roll back a node update](/operator/guides/rollback-node-update). +Start the service: -## Nocturne Reset - -:::note[Info] -This only applies for testnet. -::: - -Sometimes the testnet gets reset. In that case you need to reset the state of Nocturne. - -1. To reset the state of Nocturne, download the latest version of our installer: ```sh -curl --proto '=https' --tlsv1.2 -sSfL https://github.com/dusk-network/node-installer/releases/latest/download/node-installer.sh | sudo bash -s -- --network testnet +sudo systemctl start rusk ``` -2. Reset all the data on your node: -```sh -ruskreset -``` -Type in **Y** to accept the deletion of all the state. +Then verify all layers: -3. Start Rusk: ```sh -service rusk start +systemctl is-active rusk +systemctl show rusk -p ActiveState -p SubState -p NRestarts +ruskquery info | jq '{version, chain_id, kadcast_address}' +ruskquery peers +ruskquery block-height +sudo tail -n 50 /var/log/rusk.log ``` -4. Check if you're already staking by running: -```sh -rusk-wallet stake-info -``` -If you already have DUSK staked, wait until the chain starts producing blocks. You can check [our explorer](https://testnet.apps.dusk.network/explorer/) to see if the chain is progressing or when it will produce the genesis block. +Check the height again after roughly 30 seconds and compare it with the matching [network explorer or public endpoint](/operator/networks/). For an archive node, also run an archive GraphQL query before returning API traffic to it. -5. If you do not have testnet DUSK, request it from the [faucet](/operator/networks#how-to-get-testnet-tokens). +If Rusk fails to start, inspect: -6. Stake your testnet DUSK: ```sh -rusk-wallet stake --amt 1000 # Or however much you want to stake +sudo systemctl status rusk --no-pager +sudo journalctl -u rusk --since "15 minutes ago" --no-pager +sudo tail -n 100 /var/log/rusk_recovery.log ``` + +Do not repeatedly rerun the installer until the first failure is understood. See [Troubleshooting](/operator/troubleshooting/) and [Roll back a node update](/operator/guides/rollback-node-update/) when appropriate. + +## Testnet resets + +Testnet can be reset independently of mainnet. Follow an announced reset by reinstalling with `--network testnet`, then use the supported [fast-sync](/operator/guides/fast-sync/) or reset procedure specified in the announcement. Do not reset state merely because a normal upgrade temporarily lags the network. diff --git a/src/content/docs/operator/maintenance-monitoring.md b/src/content/docs/operator/maintenance-monitoring.md index fb444a2..b3c783a 100644 --- a/src/content/docs/operator/maintenance-monitoring.md +++ b/src/content/docs/operator/maintenance-monitoring.md @@ -1,42 +1,138 @@ --- -title: Maintenance & Monitoring -description: Maintenance & Monitoring information for Dusk node runners. +title: Maintain and monitor a node +description: Check node health, plan maintenance, and respond to common operational failures. --- -# Tips +Monitor a Dusk node from both the host and an external system. A running process alone is not enough: the node must have peers, advance with the network, and retain enough disk and memory headroom. -Here below you can find some recommendations on how to run your node. Especially when running a Provisioner, it is important to make sure that the node is managed in a secure way, as well as having risk-mitigation strategies. +## Check node health -It is recommended to use a dedicated server with only necessary services, as this minimizes the attack surface. +Run these checks after installation, upgrades, reboots, and alerts: +| Signal | Command | Healthy result | +|---|---|---| +| Service | `systemctl is-active rusk` | `active` | +| Restart count | `systemctl show rusk -p ActiveState -p SubState -p NRestarts` | Active, running, and not repeatedly restarting | +| Rusk version and network | `ruskquery info` | Expected `version`, `chain_id`, and Kadcast address | +| Peers | `ruskquery peers` | Non-zero and reasonably stable | +| Chain progress | `ruskquery block-height` | Height increases and remains close to the network tip | +| Installer version | `ruskquery version` | Current installer release | -:::tip[Recommended Setup] -The recommended setup for network participants looking to stake and use the network is to run a Provisioner node on a VPS or server, and a Prover locally on their machine/laptop. This ensures the most efficient configuration where the full resources of your provisioner node is at disposal to the consensus, while maximizing privacy by proving privacy-preserving transactions locally. -::: +`ruskquery version` checks the installer, not the running Rusk binary. Read the Rusk version from `ruskquery info` or run `rusk --version`. -## Monitoring +Sample the local height more than once: -Effective monitoring and alerting systems are crucial to avoid slashing events. There are several tools available for real-time monitoring and alerting, which are particularly important for provisioners participating in consensus. Implementing these systems helps ensure continuous performance and timely responses to potential issues. +```sh +ruskquery block-height +sleep 30 +ruskquery block-height +``` -At minimum, monitor whether the Rusk service is running, whether `ruskquery block-height` is progressing, and whether your node is close to the network tip. If your provisioner was slashed or is at risk of being slashed, follow [Slashing prevention and recovery](/operator/guides/slashing-recovery). +An unchanged sample is not conclusive by itself. Alert when the local height remains unchanged while the public network advances, or when its lag keeps increasing. -## Keys Management +To read the mainnet tip independently: -Proper management of your cryptographic keys is essential to ensure the security of your node. +```sh +curl -fsS -X POST "https://nodes.dusk.network/graphql" \ + -H "Content-Type: application/json" \ + --data-raw '{"query":"{ block(height: -1) { header { height } } }"}' \ + | jq -r '.data.block.header.height' +``` -For this reason, *consensus keys* can be strictly limited to signing consensus messages, such as block proposal, validation, and voting. +Use `https://testnet.nodes.dusk.network/graphql` for testnet. -Any other critical operations, such as un-staking, withdrawing funds or sending out Dusk can be separated into multiple different keys. +## Inspect resources and logs -## Sentry Nodes -Denial-of-service (DoS) attacks occur when an attacker floods a server with excessive traffic, preventing it from maintaining its internet connection. Attackers who scan the network may attempt to identify the IP addresses of provisioner nodes and disrupt their operations by overwhelming them with traffic. +```sh +df -h /opt/dusk /var/log +free -h +uptime +sudo tail -n 200 /var/log/rusk.log +sudo tail -n 100 /var/log/rusk_recovery.log +sudo journalctl -u rusk --since "30 minutes ago" --no-pager +``` -A recommended approach to mitigate this risk is to use a sentry node architecture. In this setup, validator nodes connect only to trusted full nodes. Validators may use private data centers with direct connections to major cloud providers, which, in turn, connect to sentry nodes. These sentry nodes act as an intermediary, absorbing the burden of any potential DoS attacks, allowing provisioners to remain secure. +The installer rotates `/var/log/rusk*.log` daily, retaining ten compressed rotations. Alert before either the state volume or log volume is exhausted; 80% utilization is a practical warning point when growth is not yet understood. -It’s recommended for Provisioners to implement load balancing and distribute incoming traffic across multiple sentry nodes, further reducing the risk of a single node being overwhelmed and enhancing the defense against DoS attacks. +Alert immediately when: -## Firewalls -A well-configured firewall is another critical layer of defense. Firewalls use predefined rules to filter incoming and outgoing traffic, blocking any suspicious or unauthorized requests. For instance, a virtual private server (VPS) firewall should block all ports that are not essential to your services, only allowing legitimate traffic. You can configure your firewall to permit traffic only from trusted sentry nodes, preventing unauthorized access and reducing the risk of attacks. Make sure to check what is the required firewalls configuration. +- the service is inactive or its restart count rises repeatedly; +- the node has no peers beyond a short startup interval; +- block height stops while the selected network continues; +- disk space is close to exhaustion; or +- logs repeatedly report startup, storage, consensus-key, or `chain.stalled` errors. -## SSH keys -When setting up a provisioner on a cloud instance, it's recommended to use SSH keys instead of passwords for secure access. SSH keys are more secure, with key lengths up to 4096 bits, and offer greater protection against server-side compromises. Even if a server is breached, the SSH key remains safe, as it is never exposed during the authentication process. To further secure your SSH key, protect it with a strong passphrase, and make sure to back it up securely in case the device storing it is compromised. +For a provisioner, also check `rusk-wallet stake-info` from the wallet host and investigate a rising fault count. Do not use reward changes alone as a short-term health check. + +## Update Rusk + +Use the [node upgrade procedure](/operator/guides/upgrade-node/). The installer stages and verifies new binaries before stopping Rusk, but it does not start the service after installation. + +Always rerun the installer with the same network and feature flags used for the node. In particular, an archive node must retain `--feature archive`; omitting it installs the default Rusk binary. + +The installer preserves chain state, `consensus.keys`, `/opt/dusk/services/dusk.conf`, and `/opt/dusk/services/rusk.conf.user`. It regenerates managed binaries, the systemd unit, `/opt/dusk/conf/rusk.toml`, `genesis.toml`, helper scripts, and the selected user's wallet configuration. Review the regenerated `rusk.toml` and reapply only intentional local settings instead of restoring an old file wholesale. + +## Update the operating system + +Use Ubuntu 24.04 LTS, the version supported by the node installer. Apply OS updates during a controlled maintenance window: + +```sh +sudo apt update +apt list --upgradable +sudo apt upgrade +``` + +The node installer installs its own prerequisites but does not perform a full OS upgrade. After package updates, verify Rusk again even when a reboot is not requested. + +Ubuntu commonly indicates a required reboot through `/var/run/reboot-required`: + +```sh +if [ -f /var/run/reboot-required ]; then + cat /var/run/reboot-required +fi +``` + +Before rebooting, confirm the node was healthy and that no second node is using the same consensus key. Then reboot and verify the enabled service after reconnecting: + +```sh +sudo systemctl stop rusk +sudo reboot +``` + +```sh +systemctl is-active rusk +ruskquery peers +ruskquery block-height +``` + +Keep provisioner downtime brief. If the node does not recover promptly, diagnose it instead of repeatedly rebooting. + +## Protect keys and configuration + +Keep the owner wallet and recovery material off the node. Back up the consensus key and its password separately, encrypted and access-controlled. Possession of both is sufficient to operate that provisioner. + +Never run the same consensus key on two nodes at once. Conflicting consensus messages can cause a hard penalty. When migrating a provisioner, stop and verify the old service before starting the replacement. + +Record or securely back up: + +- `/opt/dusk/conf/consensus.keys`; +- the consensus-key password stored through `/opt/dusk/services/dusk.conf`; +- intentional overrides in `/opt/dusk/services/rusk.conf.user`; and +- any intentional changes to `/opt/dusk/conf/rusk.toml` that must be reviewed after an installer update. + +Chain state under `/opt/dusk/rusk` is replaceable through a published state snapshot. Do not copy a live database as a substitute for key backups. + +## Respond to failures + +| Symptom | First actions | +|---|---| +| Service will not start | Check `systemctl status rusk`, `journalctl -u rusk`, and `rusk_recovery.log`. Startup checks report missing consensus keys or passwords explicitly. | +| Service runs but height does not advance | Check peers, the selected chain ID, `/var/log/rusk.log`, and `9000/udp` reachability. Compare against the matching public network. | +| Node is behind but advancing | Continue monitoring. Use [fast sync](/operator/guides/fast-sync/) when the gap is operationally significant. | +| State is corrupt or the node remains stalled | Use the supported [fast-sync or re-sync procedure](/operator/guides/manual-resync/). Do not delete individual database files while Rusk is running. | +| Disk is nearly full | Stop Rusk before moving state, expand or replace the volume, then verify ownership and restart. Do not delete current state or archive files blindly. | +| Provisioner faults or penalties increase | Restore node health first, then follow [slashing recovery](/operator/guides/slashing-recovery/). | + +The default node requires inbound `9000/udp` for Kadcast. Rusk's HTTP port `8080/tcp` does not need to be public for consensus or local `ruskquery` checks. Keep it private unless the node intentionally serves API traffic, and restrict public access with firewall and application-level controls. + +See [Troubleshooting](/operator/troubleshooting/) for symptom-specific commands. diff --git a/src/content/docs/operator/networks.md b/src/content/docs/operator/networks.md index 2a57b26..240ac5a 100644 --- a/src/content/docs/operator/networks.md +++ b/src/content/docs/operator/networks.md @@ -1,6 +1,6 @@ --- -title: Choose a Network -description: Explore the scope of different Dusk’s environments, including testnet and mainnet. +title: Choose a network +description: Compare Dusk mainnet, testnet, and development environments. --- Dusk offers multiple network environments to support different stages of development, testing, and live usage. Each network serves a unique purpose, allowing developers, testers, and users to interact with the Dusk network under various conditions. @@ -12,8 +12,8 @@ The available networks include: | Network | Chain ID | Genesis File(s) | Public Endpoints | Availability | Blockexplorer | |---------------------|----------|-------------------------------|----------------------------------------|----------------|----------------| -| **[Mainnet](/operator/networks#mainnet)** | 1 | [Genesis](https://github.com/dusk-network/node-installer/blob/af115a7e4e05099c306bd16221fd4cf2de474e89/conf/mainnet.genesis) | [https://nodes.dusk.network](https://nodes.dusk.network) | Public | [Explorer](https://apps.dusk.network/explorer/) | -| **[Nocturne Testnet](/operator/networks#nocturne-testnet)**| 2 | [Genesis](https://github.com/dusk-network/node-installer/blob/af115a7e4e05099c306bd16221fd4cf2de474e89/conf/testnet.genesis) | [https://testnet.nodes.dusk.network](https://testnet.nodes.dusk.network) | Public | [Explorer](https://apps.testnet.dusk.network/explorer/) | +| **[Mainnet](/operator/networks#mainnet)** | 1 | [Genesis](https://github.com/dusk-network/node-installer/blob/main/conf/mainnet.genesis) | [https://nodes.dusk.network](https://nodes.dusk.network) | Public | [Explorer](https://explorer.dusk.network/) | +| **[Nocturne Testnet](/operator/networks#nocturne-testnet)**| 2 | [Genesis](https://github.com/dusk-network/node-installer/blob/main/conf/testnet.genesis) | [https://testnet.nodes.dusk.network](https://testnet.nodes.dusk.network) | Public | [Explorer](https://apps.testnet.dusk.network/explorer/) | | **[Lunare Devnet](/operator/networks#lunare-devnet)** | 3 | Internal Only | Internal Only | Internal only | N/A | --- diff --git a/src/content/docs/operator/overview.mdx b/src/content/docs/operator/overview.mdx index 8e388cf..94b829e 100644 --- a/src/content/docs/operator/overview.mdx +++ b/src/content/docs/operator/overview.mdx @@ -27,7 +27,7 @@ Run Dusk infrastructure by first choosing the network and node role that match y - + diff --git a/src/content/docs/operator/troubleshooting.md b/src/content/docs/operator/troubleshooting.md index 8ab9f9b..5b0bca0 100644 --- a/src/content/docs/operator/troubleshooting.md +++ b/src/content/docs/operator/troubleshooting.md @@ -1,113 +1,99 @@ --- -title: Troubleshooting -description: Learn how to solve issues when setting up a node on Dusk. +title: Troubleshoot a node +description: Diagnose Rusk startup, connectivity, synchronization, state, and wallet problems. --- -Here you can find answers explaining how to solve errors you may encounter. +Start with current evidence instead of resetting state immediately: -#### kadcast::handling: NETWORK MISMATCH -This error occurs when one of the peers of your node is running on a different chain. You can safely ignore this error unless your node hasn't been upgraded correctly. - -#### Consensus msg discarded: reason="too far in the future" -This means your node is receiving consensus messages for rounds much higher than your current tip (more than ~10 blocks ahead). Common causes: - -- Your node is still syncing or stuck behind the network tip -- You're connected to the wrong network/chain ID - -Check your block height (`ruskquery block-height`) against the explorer. If you are stuck, consider fast-syncing (`download_state`) or a manual resync. - -#### My provisioner was slashed -First, restore healthy node operation: check the installed version, sync height, service status, logs, and staking status. Then monitor the node to make sure it keeps progressing. +```sh +systemctl is-active rusk +sudo systemctl status rusk --no-pager +ruskquery info +ruskquery peers +ruskquery block-height +sudo tail -n 100 /var/log/rusk.log +``` -Follow: [Slashing prevention and recovery](/operator/guides/slashing-recovery). +## Rusk will not start -#### Unable to resolve domain: invalid socket address -Such errors usually indicate DNS problems. Check your DNS settings. +Inspect systemd and recovery output: -#### What should I do if the ruskreset command aborts? -When prompted: +```sh +sudo journalctl -u rusk --since "30 minutes ago" --no-pager +sudo tail -n 100 /var/log/rusk_recovery.log +``` -`WARNING: This operation will DELETE your Rusk state, wallet cache, and logs. -Are you sure you want to proceed? (Y/n):` +Common pre-start failures include: -Answer with a capital **Y**, otherwise the operation will abort. +| Message or symptom | Action | +|---|---| +| Missing `consensus.keys` | Install the intended key at `/opt/dusk/conf/consensus.keys` with `root:dusk` ownership and mode `640`. | +| `DUSK_CONSENSUS_KEYS_PASS` not set | Run `sudo sh /opt/dusk/bin/setup_consensus_pwd.sh`; do not place the password in shell history or logs. | +| Rusk restarts every ten seconds | Read the first failure in `journalctl` and `rusk_recovery.log` instead of waiting through repeated restarts. | +| Port already in use | Identify the listener with `sudo ss -ltnup`; stop or reconfigure the conflicting service. | -#### PersistenceError(Os { code: 2, kind: NotFound, message: "No such file or directory" }) -Run the `ruskreset` command to fix this. +The installer-managed unit writes Rusk output to `/var/log/rusk.log`; systemd logs service transitions and pre-start failures. -#### "stake" command not recognized -Use the current CLI help output as the source of truth for your installed wallet version. The normal staking flow uses `stake`, for example: +## No peers or chain progress -```bash -rusk-wallet --help -rusk-wallet stake --help -rusk-wallet stake --amt 3000 -``` +Check the local network identity and sample height twice: -#### Serialization error -First, wait for your node to be fully synced. If the node is running fine, use the following command for further details: -```bash -rusk-wallet --log-level debug +```sh +ruskquery info | jq '{version, chain_id, kadcast_address}' +ruskquery peers +ruskquery block-height +sleep 30 +ruskquery block-height ``` -#### Connection to Rusk Failed, some operations won't be available -This error is expected on the first installation because the node isn't up yet. +Then verify: -Anyways, ensure you properly followed the instructions and that you are using the latest version and check the logs for further details. +- the node uses the intended mainnet or testnet configuration; +- inbound `9000/udp` reaches the Kadcast public address; +- NAT overrides in `/opt/dusk/services/rusk.conf.user` are correct; and +- the host has working DNS, outbound HTTPS, and UDP connectivity. -```bash -ruskquery version -``` +Public `8080/tcp` access is not required for consensus. It is only needed when the node intentionally serves HTTP API clients. -```bash -tail -F /var/log/rusk.log -n 50 -``` +`NETWORK MISMATCH` from an individual peer can be ignored when your own `chain_id` is correct and the node otherwise progresses. Repeated `too far in the future` messages usually mean the node is far behind or on the wrong network. A persistent `chain.stalled` condition requires checking peers, height, version, and Kadcast reachability before replacing state. -#### Port conflicts when running multiple services -You can use the following to identify what's running on the conflicting port (e.g., 8080) and resolve the issue: +## State or storage errors -```bash -netstat -tuplen -``` +If the node is behind but advancing, keep monitoring. If it remains stalled on unusable state, follow [Re-sync a node](/operator/guides/manual-resync/): -You can also verify if something else is running on port 8080 by running: - -```bash -curl -si --request POST "http://127.0.0.1:8080/on/node/info" | head +```sh +download_state --list +sudo download_state +sudo systemctl start rusk ``` -If the response is not from Rusk (for example, HTML/404, or missing the `Rusk-Version` header), another service is likely using that port. -#### "chain.stalled" error -This indicates your node cannot process blocks due to: +Do not delete `chain.db`, individual state files, or archive databases while Rusk is running. `ruskreset` also deletes wallet cache, archive data, and diagnostic logs; use it only when an announced reset or support procedure explicitly requires it. -Port not being forwarded or incorrect Kadcast address in `/opt/dusk/services/rusk.conf.default`. -To fix the issues, ensure the required ports (9000/udp, 8080/tcp) are accessible and verify your Kadcast addresses (for example `KADCAST_PUBLIC_ADDRESS` and, if needed behind NAT, `KADCAST_LISTEN_ADDRESS`). -To check the state of your node, you can run: +When disk space is exhausted, stop Rusk before moving data. Expand or replace the volume, preserve file ownership, then start and verify the service. Blindly deleting state is not a disk-cleanup strategy. -```bash -rusk-wallet stake-info -``` +## Wallet and staking errors -#### "Invalid Data" or "500 Internal Server Error" during staking -Ensure you're using the latest version of `rusk-wallet`. Update by cloning the repository and reinstalling: +If Rusk Wallet reports that its Rusk connection failed, verify the local service and API first: -```bash -cargo install --path rusk/rusk-wallet +```sh +systemctl is-active rusk +curl -si -X POST "http://127.0.0.1:8080/on/node/info" | head ``` +A Rusk response includes a `Rusk-Version` header. HTML, an unrelated `404`, or no listener indicates a port conflict or disabled local API. -#### "PersistenceError" caused by "InvalidData" or "pointer out of bounds" -These errors indicate that your node's state is corrupted. To fix this, reload from a snapshot by running: +Use the installed CLI help as the command reference for that wallet version: -```bash -download_state +```sh +rusk-wallet --version +rusk-wallet --help +rusk-wallet stake --help ``` -(and confirm the warning by typing 'Y'). Then, restart the node with: -```bash -service rusk start -``` +Do not update Rusk Wallet by cloning and compiling the Rusk repository on an installer-managed node. Rerun the [node upgrade procedure](/operator/guides/upgrade-node/) with the node's existing network and feature flags. + +For active stake, owner separation, fault diagnosis, or penalties, see [Slashing recovery](/operator/guides/slashing-recovery/). +## Still unresolved -#### Compilation errors on libraries when building rusk-wallet -Check if you are launching `make` from the root of the **rusk** directory (instead of another directory, such as **rusk-wallet**). The binary will be found in `./rusk/target/release/rusk-wallet`. +Preserve the relevant timestamps, Rusk and installer versions, chain ID, local and network heights, peer count, and log excerpt. Remove keys, passwords, mnemonics, IPs you consider private, and other secrets before sharing diagnostics.