diff --git a/documentation/operators/src/nodes/nym-node.md b/documentation/operators/src/nodes/nym-node.md index 307082505c..c33cce3ef3 100644 --- a/documentation/operators/src/nodes/nym-node.md +++ b/documentation/operators/src/nodes/nym-node.md @@ -39,20 +39,22 @@ If you are not well familiar with `nym-node` setup, automation, and `nymtun0` co curl -o network_tunnel_manager.sh -L https://gist.githubusercontent.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77/raw/9d785d6ee3aa2970553633eccbd89a827f49fab5/network_tunnel_manager.sh && chmod +x network_tunnel_manager.sh ``` -3. If you have a running `nym-node` or `nym-gateway` (alone or service), stop the process - - In case your node was a `nym-gateway`, [**migrate to `nym-node`**](setup.md#migrate) now! +3. In case you had a`nym-gateway` running, stop the process (directly or systemd service), and [**migrate to `nym-node`**](setup.md#migrate) now! -4. Check Nymtun IP tables: +4. Apply the rules: +```sh +sudo ./network_tunnel_manager.sh apply_iptables_rules +``` +5. Check Nymtun IP tables: ```sh sudo ./network_tunnel_manager.sh check_nymtun_iptables ``` - - if there's no process running it shouldn't get anything -5. Display IPv6: +6. Display IPv6: ```sh sudo ./network_tunnel_manager.sh fetch_and_display_ipv6 ``` - - If you have a `global ipv6` address this is good, if not the next step should fix it + - You should see a `global ipv6` address: ~~~admonish example collapsible=true title="Correct `./network_tunnel_manager.sh fetch_and_display_ipv6` output:" ```sh iptables-persistent is already installed. @@ -61,21 +63,18 @@ operation fetch_ipv6_address_nym_tun completed successfully. ``` ~~~ -6. Apply the rules: -```sh -sudo ./network_tunnel_manager.sh apply_iptables_rules -``` - - and check them again like in point 4. - 7. (If you didn't have a `nym-node` service yet) Create `systemd` [automation and configuration file](configuration.md#systemd), reload and enable the service 8. Start `nym-node` service: ```sh sudo service nym-node start && journalctl -u nym-node -f -n 100 ``` +9. Upgrade your node version in the smart contract, [using Nym wallet](manual-upgrade.md#step-2-updating-your-node-information-in-the-smart-contract) - If you don't run this as an upgrade but started a fresh new node, you need to [bond](bonding.md) the gateway now. After that finish the verification steps below. -9. After a minute of running properly, check `nymtun0`: +10. After a minute of running properly, validate your `nymtun0` routing: + + - Display the address ```sh ip addr show nymtun0 ``` @@ -94,9 +93,8 @@ ip addr show nymtun0 ``` ~~~ -10. Validate your IPv6 and IPv4 networking by running a joke via Mixnet: + - Run a joke through the Mixnet - you should get two jokes (IPv4 and IPv6 routing): + ```sh sudo ./network_tunnel_manager.sh joke_through_the_mixnet ``` - -Make sure that you get the validation of IPv4 and IPv6 connectivity, in case of problems, check [troubleshooting page](../troubleshooting/vps-isp.md#incorrect-gateway-network-check). After proceed to [bonding](bonding.md). diff --git a/documentation/operators/src/nodes/setup.md b/documentation/operators/src/nodes/setup.md index 11c8cc5f29..23863e36d3 100644 --- a/documentation/operators/src/nodes/setup.md +++ b/documentation/operators/src/nodes/setup.md @@ -28,7 +28,7 @@ Currently `nym-node` binary enables to run only one `--mode` at a time. In the f Every `exit-gateway` mode is basically an `entry-gateway` with NR (Network Requester) and IPR (IP Packet Router) enabled. This means that every `exit-gateway` is automatically seen as an `entry-gateway` but not the opposite. -Gateway operators can check out the performance, connectivity and much more in our new tool [harbourmaster.nymtech.net](https://harbourmaster.nymtech.net/). +Gateway operators can check out the node performance, connectivity and much more in our new tool [harbourmaster.nymtech.net](https://harbourmaster.nymtech.net/). To determine which mode your node is running, you can check the `:8080/api/v1/roles` endpoint. For example: ``` diff --git a/documentation/operators/src/testing/gateway-probe.md b/documentation/operators/src/testing/gateway-probe.md index f24aa8284e..3fe82a19ea 100644 --- a/documentation/operators/src/testing/gateway-probe.md +++ b/documentation/operators/src/testing/gateway-probe.md @@ -24,7 +24,7 @@ git clone https://github.com/nymtech/nym-vpn-client.git 2. Build `nym-gateway-probe`: ```sh -cd nym-vpn-client +cd nym-vpn-client/nym-vpn-core cargo build --release -p nym-gateway-probe ``` @@ -48,7 +48,7 @@ Options: ``` ~~~ -To run the client, simply add a flag `--gateway` with a targeted gateway identity key. +To run the client, simply add a flag `--gateway` with a targeted gateway identity key. ```sh ./target/release/nym-gateway-probe --gateway @@ -74,6 +74,6 @@ For any `nym-node --mode exit-gateway` the aim is to have this outcome: } ``` +If your Gateway is blacklisted, the probe will not work. + If you don't provide a `--gateway` flag it will pick a random one to test. - - diff --git a/documentation/operators/src/troubleshooting/nodes.md b/documentation/operators/src/troubleshooting/nodes.md index 6cb0d5c192..9731617769 100644 --- a/documentation/operators/src/troubleshooting/nodes.md +++ b/documentation/operators/src/troubleshooting/nodes.md @@ -124,13 +124,13 @@ nym-node 103349 root 57u IPv6 1333229976 0t0 TCP [2a03:b0c0:3:d0::ff3:f ##### Query `systemd` journal with `journalctl` -``` +```sh sudo journalctl -u nym-node -o cat | grep "Since startup mixed" ``` If you have created `nym-node.service` file (i.e. you are running your Ny, Node via `systemd`) then this command shows you how many packets have you mixed so far, and should return a list of messages like this: -``` +```sh 2021-05-18T12:35:24.057Z INFO nym_node::node::metrics > Since startup mixed 233639 packets! 2021-05-18T12:38:02.178Z INFO nym_node::node::metrics > Since startup mixed 233739 packets! 2021-05-18T12:40:32.344Z INFO nym_node::node::metrics > Since startup mixed 233837 packets! @@ -146,7 +146,7 @@ A `build-info` command prints the build information like commit hash, rust versi For example `./target/debug/nym-node --no-banner build-info --output json` will return: -``` +```sh {"binary_name":"nym-network-requester","build_timestamp":"2023-07-24T15:38:37.00657Z","build_version":"1.1.23","commit_sha":"c70149400206dce24cf20babb1e64f22202672dd","commit_timestamp":"2023-07-24T14:45:45Z","commit_branch":"feature/simplify-cli-parsing","rustc_version":"1.71.0","rustc_channel":"stable","cargo_profile":"debug"} ``` @@ -154,13 +154,13 @@ For example `./target/debug/nym-node --no-banner build-info --output json` will ##### Scan ports with `nmap`: -``` +```sh nmap -p 1789 -Pn ``` If your Nym Node is configured properly it should output something like this: -``` +```sh bob@desktop:~$ nmap -p 1789 95.296.134.220 -Pn Host is up (0.053s latency). @@ -169,9 +169,16 @@ PORT STATE SERVICE 1789/tcp open hello ``` +##### Check with `telnet` + +Your node should connect to telnet when running: +```sh +telnet +``` + ##### Query online nodes: -``` +```sh curl --location --request GET 'https://validator.nymtech.net/api/v1/mixnodes/' ``` @@ -184,6 +191,15 @@ You can query Gateways by replacing `nym-mixnodes` with `nym-gateways` in the ab We currently have an API set up returning our metrics tests of the network. There are two endpoints to ping for information about your Mix Node, `report` and `history`. Find more information about this in the [Mixnodes metrics documentation](../nodes/maintenance.md#metrics--api-endpoints). +For more information about available endpoints and their status, you can refer to: +```sh +# for http +http://:8080/api/v1/swagger/#/ + +# for https reversed proxy +https:///api/v1/swagger/#/ +``` + ### Why is my node not mixing any packets? If you are still unable to see your node on the dashboard, or your node is declaring it has not mixed any packets, there are several potential issues: @@ -229,12 +245,6 @@ Check that you have provided the correct information when bonding your Nym Node > All delegated stake will be lost when un-bonding! However the Nym Node must be operational in the first place for the delegation to have any effect. -#### Missing `announce-host` flag - -On certain cloud providers such as AWS and Google Cloud, you need to do some additional configuration of your firewall and use `--host` with your **local ip** and `--announce-host` with the **public ip** of your Mix Node host. - -If the difference between the two is unclear, contact the help desk of your VPS provider. - ### Running on a local machine behind NAT with no fixed IP address Your ISP has to be IPv6 ready if you want to run a Nym Node on your local machine. Sadly, in 2020, most of them are not and you won't get an IPv6 address by default from your ISP. Usually it is an extra paid service or they simply don't offer it. @@ -262,43 +272,7 @@ Most of the `ERROR` and `WARN` messages in your node logs are benign - as long a More specific errors and warnings are covered below. -#### `tokio runtime worker` error -If you are running into issues with an error including the following: - -``` -thread 'tokio-runtime-worker' panicked at 'Failed to create TCP listener: Os { code: 99, kind: AddrNotAvailable, message: "Cannot assign requested address" }' -``` - -Then you need to `--announce-host ` and `--host ` on startup. This issue is addressed [above](#missing-`announce-host`-flag) - ### What is `verloc` and do I have to configure my Nym Node to implement it? `verloc` is short for _verifiable location_. Mix Nodes and Gateways now measure speed-of-light distances to each other, in an attempt to verify how far apart they are. In later releases, this will allow us to algorithmically verify node locations in a non-fake-able and trustworthy manner. @@ -322,17 +296,69 @@ Nym API measures performance by routing traffic through the Mixnet. If the avera In case your Gateway appeared on the [blacklist](https://validator.nymtech.net/api/v1/gateways/blacklisted), it's because there is some flaw in the configuration. The most common sources of problems are: - Bonding before starting the node/service -- Bonding before opening [the needed ports](../nodes/vps-setup.md#configure-your-firewall) +- Bonding before opening [needed ports](../nodes/vps-setup.md#configure-your-firewall) - VPS restarted without operator having a [systemd automation](../nodes/configuration.md#systemd) or some alert notification flow setup What to do: -- Make sure your node is running and do not stop it if there is no need -- Open all needed ports -- Wait until your node gets above 50% of performance (average of last 24h) - this will likely take several hours, up to a day. During this time your node is tested by `nym-api` and every positive response picks up your Gateway's routing score. +1. Make sure your node is running and do not stop it if there is no need +2. Open all needed [needed ports](../nodes/vps-setup.md#configure-your-firewall) +3. Check your `config.toml` - often people have filled `hostname` without such hostname being configured or a wrong IP address after moving their node. +4. [Check Gateway Connectivity](#check-gateway-connectivity) +5. See logs of your Gateway and search [for errors](#nym-node-errors) - if you find any unusual one, you can ask in the [Element Node Operators](https://matrix.to/#/#operators:nymtech.chat) channel + - If your logs show that your Node has `cover down: 0.00` that means that the embedded IPR and NR is not sending any cover traffic. +6. +7. When all problems addressed:Wait until your node gets above 50% of performance (average of last 24h) - this will likely take several hours, up to a day. During this time your node is tested by `nym-api` and every positive response picks up your Gateway's routing score. **Do not restart your Nym Node without reason, your routing score will only get worse!** +### Check Gateway connectivity + +**1. Check out the API endpoints** + +Start with checking if your Gateway IPR and NR is active. To determine which mode your node is running, you can check the `:8080/api/v1/roles` endpoint. For example: +``` +# for http +http://:8080/api/v1/roles + +# for https reversed proxy +https:///api/v1/roles +``` + +Everything necessary will exist on your node by default. For instance, if you're running a mixnode, you'll find that a NR (Network Requester) and IPR (IP Packet Router) address exist, but they will be ignored in `mixnode` mode. + +For more information about available endpoints and their status, you can refer to: +```sh +# for http +http://:8080/api/v1/swagger/#/ + +# for https reversed proxy +https:///api/v1/swagger/#/ +``` + +**2. Configure IPv4 and IPv6 tables and rules** + +- In case you haven't, follow the steps in the node [configuration](../nodes/configuration.md) chapter [connectivity test and configurastion](../nodes/configuration.md#connectivity-test-and-configuration). + + +**3. Test connectivity** + +- Telnet - from your local machine try to connect to your VPS bu running: + +```sh +telnet +``` + +- [Websocket wcat](https://github.com/websockets/wscat): + - Install on your local machine: +```sh +sudo apt install node-ws +``` + - Run `wscat` pointing to the IP of your VPS with port `9000`: +``` +wscat -c ws://: +``` + ### My exit Gateway "is still not online..." The Nyx chain epoch takes up to 60 min. To prevent the Gateway getting blacklisted, it's essential to start it before the bonding process and let it running. In case it already got [blacklisted](#my-gateway-is-backlisted) check the steps above.