From ce067db401f13420b823ec3064148e410a4df305 Mon Sep 17 00:00:00 2001 From: import this <97586125+serinko@users.noreply.github.com> Date: Tue, 23 Apr 2024 09:14:27 +0000 Subject: [PATCH] [DOCs]: Guides for NYM-NODE (#4541) * initialise nym node pages * new TOC flow and archive section * vps setup page draft * syntax fix * syntax fix * syntax fix * add ulimit setup * initialise nym-node page * add ports * syntax edit * syntax edit * add flags * add setup, run and bonding pages * add info * syntax edit * add configuration page * syntax fix * syntax fix * add vps troubleshooting * add vps troubleshooting * create nyx configuration page * add quickflow * add quickflow * add quickflow * add quickflow * add quickflow * add quickflow * syntax fix * syntax fix * move vps setup to configuration * syntax fix * syntax fix * syntax fix * edit points * finish nym-node setup.md page * new connnectivity configuration - page finished * finish proxy and landing page guide * finish nymvisor update * finish performance testing upgdate * finish faq pages update * fix troubleshooting * fix troubleshooting * fix manual-upgrade * finish introduction page * update introduction * update summary * correct all links * update graphs * update cmdruns * syntax edit and spellcheck * updated mdbook plugins * mdbook update * update modules * remove redundant * fix version cmdrun * removed smoosh-faw to archive * syntax fix * attempt to update mdbook admonish * try dirty workaround hack * try dirty workaround hack * try dirty workaround hack * try dirty workaround hack * try dirty workaround hack * try dirty workaround hack * try dirty workaround hack * try dirty workaround hack * try dirty workaround hack * address review comments * address review comments * address review comments * PR ready to merge --- .github/workflows/cd-docs.yml | 20 +- .github/workflows/ci-docs.yml | 20 +- documentation/dev-portal/book.toml | 2 +- documentation/docs/book.toml | 2 +- documentation/operators/book.toml | 2 +- .../operators/nym_themes/mdbook-admonish.css | 39 +- .../nym_themes/mdbook-admonish.css.bak | 341 +++++++++ documentation/operators/src/SUMMARY.md | 46 +- .../operators/src/archive/archive.md | 7 + .../operators/src/archive/faq/mixnodes-faq.md | 97 +++ .../operators/src/archive/faq/smoosh-faq.md | 108 +++ .../src/{ => archive}/nodes/gateway-setup.md | 34 +- .../src/{ => archive}/nodes/mix-node-setup.md | 42 +- .../nodes/network-requester-setup.md | 21 +- .../{ => archive/nodes}/preliminary-steps.md | 6 +- .../src/archive/nodes/setup-guides.md | 15 + .../operators/src/binaries/building-nym.md | 8 +- .../src/binaries/pre-built-binaries.md | 4 +- .../operators/src/faq/general-faq.md | 42 ++ .../operators/src/faq/nym-nodes-faq.md | 32 + documentation/operators/src/faq/nyx-faq.md | 29 + documentation/operators/src/faq/smoosh-faq.md | 10 +- .../operators/src/images/ip_table_prompt.png | Bin 0 -> 21173 bytes .../operators/src/images/ipv6_64.png | Bin 0 -> 20426 bytes documentation/operators/src/introduction.md | 36 +- .../operators/src/legal/exit-gateway.md | 2 +- .../operators/src/legal/landing-pages.md | 358 +-------- documentation/operators/src/nodes/bonding.md | 68 ++ .../operators/src/nodes/configuration.md | 305 ++++++++ .../operators/src/nodes/maintenance.md | 700 ++---------------- .../operators/src/nodes/manual-upgrade.md | 68 +- documentation/operators/src/nodes/nym-node.md | 101 +++ .../operators/src/nodes/nymvisor-upgrade.md | 36 +- .../operators/src/nodes/nyx-configuration.md | 268 +++++++ .../operators/src/nodes/preliminary-steps.md | 10 + .../src/nodes/proxy-configuration.md | 482 ++++++++++++ .../operators/src/nodes/setup-guides.md | 11 - documentation/operators/src/nodes/setup.md | 198 +++++ .../operators/src/nodes/vps-setup.md | 235 ++++++ .../operators/src/nodes/wallet-preparation.md | 17 + .../operators/src/testing/node-setup.md | 19 +- .../operators/src/testing/performance.md | 6 +- .../operators/src/testing/templates.md | 2 +- .../src/tokenomics/mixnet-rewards.md | 46 ++ .../nym-node.md} | 155 ++-- .../src/troubleshooting/validators.md | 16 + .../src/troubleshooting/vps-setup.md | 34 + 47 files changed, 2821 insertions(+), 1279 deletions(-) create mode 100644 documentation/operators/nym_themes/mdbook-admonish.css.bak create mode 100644 documentation/operators/src/archive/archive.md create mode 100644 documentation/operators/src/archive/faq/mixnodes-faq.md create mode 100644 documentation/operators/src/archive/faq/smoosh-faq.md rename documentation/operators/src/{ => archive}/nodes/gateway-setup.md (79%) rename documentation/operators/src/{ => archive}/nodes/mix-node-setup.md (77%) rename documentation/operators/src/{ => archive}/nodes/network-requester-setup.md (86%) rename documentation/operators/src/{ => archive/nodes}/preliminary-steps.md (90%) create mode 100644 documentation/operators/src/archive/nodes/setup-guides.md create mode 100644 documentation/operators/src/faq/general-faq.md create mode 100644 documentation/operators/src/faq/nym-nodes-faq.md create mode 100644 documentation/operators/src/faq/nyx-faq.md create mode 100644 documentation/operators/src/images/ip_table_prompt.png create mode 100644 documentation/operators/src/images/ipv6_64.png create mode 100644 documentation/operators/src/nodes/bonding.md create mode 100644 documentation/operators/src/nodes/configuration.md create mode 100644 documentation/operators/src/nodes/nym-node.md create mode 100644 documentation/operators/src/nodes/nyx-configuration.md create mode 100644 documentation/operators/src/nodes/preliminary-steps.md create mode 100644 documentation/operators/src/nodes/proxy-configuration.md delete mode 100644 documentation/operators/src/nodes/setup-guides.md create mode 100644 documentation/operators/src/nodes/setup.md create mode 100644 documentation/operators/src/nodes/vps-setup.md create mode 100644 documentation/operators/src/nodes/wallet-preparation.md create mode 100644 documentation/operators/src/tokenomics/mixnet-rewards.md rename documentation/operators/src/{nodes/troubleshooting.md => troubleshooting/nym-node.md} (71%) create mode 100644 documentation/operators/src/troubleshooting/validators.md create mode 100644 documentation/operators/src/troubleshooting/vps-setup.md diff --git a/.github/workflows/cd-docs.yml b/.github/workflows/cd-docs.yml index 9b6e6facc5..7460210e00 100644 --- a/.github/workflows/cd-docs.yml +++ b/.github/workflows/cd-docs.yml @@ -9,7 +9,7 @@ jobs: steps: - uses: actions/checkout@v3 - name: Install Dependencies (Linux) - run: sudo apt-get update && sudo apt-get install -y build-essential curl wget libssl-dev libudev-dev squashfs-tools protobuf-compiler + run: sudo apt-get update && sudo apt-get install -y build-essential curl wget libssl-dev libudev-dev squashfs-tools protobuf-compiler git - name: Install rsync run: sudo apt-get install rsync - uses: rlespinasse/github-slug-action@v3.x @@ -30,9 +30,25 @@ jobs: - name: Remove existing Nym config directory (`~/.nym/`) run: cd documentation && ./remove_existing_config.sh continue-on-error: false - - name: Build all projects in documentation/ & move to ~/dist/docs/ +# This is the original flow +# - name: Build all projects in documentation/ & move to ~/dist/docs/ +# run: cd documentation && ./build_all_to_dist.sh +# continue-on-error: false +# This is a dirty hack replacement which builds on develop and then on current branch + + - name: Save current branch to ~/current_branch + run: git rev-parse --abbrev-ref HEAD > ~/current_branch + - name: Git pull & switch to develop + run: git pull && git checkout develop + - name: Build all projects in documentation/ & move to ~/dist/docs/ on develop run: cd documentation && ./build_all_to_dist.sh continue-on-error: false + - name: Switch to current branch + run: git checkout $echo "$(cat ~/current_branch)" + - name: Build all projects in documentation/ & move to ~/dist/docs/ on current branch + run: cd documentation && ./build_all_to_dist.sh && rm ~/current_branch + continue-on-error: false +# End of replacemet - name: Post process run: cd documentation && ./post_process.sh diff --git a/.github/workflows/ci-docs.yml b/.github/workflows/ci-docs.yml index 7e67a9f432..858a8ce69d 100644 --- a/.github/workflows/ci-docs.yml +++ b/.github/workflows/ci-docs.yml @@ -13,7 +13,7 @@ jobs: steps: - uses: actions/checkout@v3 - name: Install Dependencies (Linux) - run: sudo apt-get update && sudo apt-get install -y build-essential curl wget libssl-dev libudev-dev squashfs-tools protobuf-compiler + run: sudo apt-get update && sudo apt-get install -y build-essential curl wget libssl-dev libudev-dev squashfs-tools protobuf-compiler git - name: Install rsync run: sudo apt-get install rsync - uses: rlespinasse/github-slug-action@v3.x @@ -34,9 +34,25 @@ jobs: - name: Remove existing Nym config directory (`~/.nym/`) run: cd documentation && ./remove_existing_config.sh continue-on-error: false - - name: Build all projects in documentation/ & move to ~/dist/docs/ +# This is the original flow +# - name: Build all projects in documentation/ & move to ~/dist/docs/ +# run: cd documentation && ./build_all_to_dist.sh +# continue-on-error: false +# This is a dirty hack replacement which builds on develop and then on current branch + + - name: Save current branch to ~/current_branch + run: git rev-parse --abbrev-ref HEAD > ~/current_branch + - name: Git pull & switch to develop + run: git pull && git checkout develop + - name: Build all projects in documentation/ & move to ~/dist/docs/ on develop run: cd documentation && ./build_all_to_dist.sh continue-on-error: false + - name: Switch to current branch + run: git checkout $echo "$(cat ~/current_branch)" + - name: Build all projects in documentation/ & move to ~/dist/docs/ on current branch + run: cd documentation && ./build_all_to_dist.sh && rm ~/current_branch + continue-on-error: false +# End of replacemet - name: Deploy branch to CI www continue-on-error: true diff --git a/documentation/dev-portal/book.toml b/documentation/dev-portal/book.toml index 8533a08344..582e47460e 100644 --- a/documentation/dev-portal/book.toml +++ b/documentation/dev-portal/book.toml @@ -24,7 +24,7 @@ turn-off = false [preprocessor.admonish] command = "mdbook-admonish" -assets_version = "3.0.0" # do not edit: managed by `mdbook-admonish install` +assets_version = "3.0.2" # do not edit: managed by `mdbook-admonish install` # https://gitlab.com/tglman/mdbook-variables/ [preprocessor.variables.variables] diff --git a/documentation/docs/book.toml b/documentation/docs/book.toml index 33505cb707..21da30b6c1 100644 --- a/documentation/docs/book.toml +++ b/documentation/docs/book.toml @@ -25,7 +25,7 @@ turn-off = true [preprocessor.admonish] command = "mdbook-admonish" -assets_version = "3.0.0" # do not edit: managed by `mdbook-admonish install` +assets_version = "3.0.2" # do not edit: managed by `mdbook-admonish install` # https://gitlab.com/tglman/mdbook-variables/ [preprocessor.variables.variables] diff --git a/documentation/operators/book.toml b/documentation/operators/book.toml index d987773f01..baf906dc17 100644 --- a/documentation/operators/book.toml +++ b/documentation/operators/book.toml @@ -24,7 +24,7 @@ turn-off = true [preprocessor.admonish] command = "mdbook-admonish" -assets_version = "3.0.0" # do not edit: managed by `mdbook-admonish install` +assets_version = "3.0.2" # do not edit: managed by `mdbook-admonish install` # https://gitlab.com/tglman/mdbook-variables/ [preprocessor.variables.variables] diff --git a/documentation/operators/nym_themes/mdbook-admonish.css b/documentation/operators/nym_themes/mdbook-admonish.css index e0a3365532..45aeff0511 100644 --- a/documentation/operators/nym_themes/mdbook-admonish.css +++ b/documentation/operators/nym_themes/mdbook-admonish.css @@ -1,20 +1,4 @@ @charset "UTF-8"; -:root { - --md-admonition-icon--admonish-note: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-abstract: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-info: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-tip: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-success: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-question: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-warning: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-failure: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-danger: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-bug: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-example: url("data:image/svg+xml;charset=utf-8,"); - --md-admonition-icon--admonish-quote: url("data:image/svg+xml;charset=utf-8,"); - --md-details-icon: url("data:image/svg+xml;charset=utf-8,"); -} - :is(.admonition) { display: flow-root; margin: 1.5625em 0; @@ -71,6 +55,8 @@ a.admonition-anchor-link::before { padding-inline: 4.4rem 1.2rem; font-weight: 700; background-color: rgba(68, 138, 255, 0.1); + print-color-adjust: exact; + -webkit-print-color-adjust: exact; display: flex; } :is(.admonition-title, summary.admonition-title) p { @@ -86,6 +72,8 @@ html :is(.admonition-title, summary.admonition-title):last-child { width: 2rem; height: 2rem; background-color: #448aff; + print-color-adjust: exact; + -webkit-print-color-adjust: exact; mask-image: url('data:image/svg+xml;charset=utf-8,'); -webkit-mask-image: url('data:image/svg+xml;charset=utf-8,'); mask-repeat: no-repeat; @@ -119,6 +107,25 @@ details[open].admonition > summary.admonition-title::after { transform: rotate(90deg); } +:root { + --md-details-icon: url("data:image/svg+xml;charset=utf-8,"); +} + +:root { + --md-admonition-icon--admonish-note: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-abstract: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-info: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-tip: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-success: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-question: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-warning: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-failure: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-danger: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-bug: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-example: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-quote: url("data:image/svg+xml;charset=utf-8,"); +} + :is(.admonition):is(.admonish-note) { border-color: #448aff; } diff --git a/documentation/operators/nym_themes/mdbook-admonish.css.bak b/documentation/operators/nym_themes/mdbook-admonish.css.bak new file mode 100644 index 0000000000..e0a3365532 --- /dev/null +++ b/documentation/operators/nym_themes/mdbook-admonish.css.bak @@ -0,0 +1,341 @@ +@charset "UTF-8"; +:root { + --md-admonition-icon--admonish-note: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-abstract: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-info: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-tip: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-success: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-question: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-warning: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-failure: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-danger: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-bug: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-example: url("data:image/svg+xml;charset=utf-8,"); + --md-admonition-icon--admonish-quote: url("data:image/svg+xml;charset=utf-8,"); + --md-details-icon: url("data:image/svg+xml;charset=utf-8,"); +} + +:is(.admonition) { + display: flow-root; + margin: 1.5625em 0; + padding: 0 1.2rem; + color: var(--fg); + page-break-inside: avoid; + background-color: var(--bg); + border: 0 solid black; + border-inline-start-width: 0.4rem; + border-radius: 0.2rem; + box-shadow: 0 0.2rem 1rem rgba(0, 0, 0, 0.05), 0 0 0.1rem rgba(0, 0, 0, 0.1); +} +@media print { + :is(.admonition) { + box-shadow: none; + } +} +:is(.admonition) > * { + box-sizing: border-box; +} +:is(.admonition) :is(.admonition) { + margin-top: 1em; + margin-bottom: 1em; +} +:is(.admonition) > .tabbed-set:only-child { + margin-top: 0; +} +html :is(.admonition) > :last-child { + margin-bottom: 1.2rem; +} + +a.admonition-anchor-link { + display: none; + position: absolute; + left: -1.2rem; + padding-right: 1rem; +} +a.admonition-anchor-link:link, a.admonition-anchor-link:visited { + color: var(--fg); +} +a.admonition-anchor-link:link:hover, a.admonition-anchor-link:visited:hover { + text-decoration: none; +} +a.admonition-anchor-link::before { + content: "§"; +} + +:is(.admonition-title, summary.admonition-title) { + position: relative; + min-height: 4rem; + margin-block: 0; + margin-inline: -1.6rem -1.2rem; + padding-block: 0.8rem; + padding-inline: 4.4rem 1.2rem; + font-weight: 700; + background-color: rgba(68, 138, 255, 0.1); + display: flex; +} +:is(.admonition-title, summary.admonition-title) p { + margin: 0; +} +html :is(.admonition-title, summary.admonition-title):last-child { + margin-bottom: 0; +} +:is(.admonition-title, summary.admonition-title)::before { + position: absolute; + top: 0.625em; + inset-inline-start: 1.6rem; + width: 2rem; + height: 2rem; + background-color: #448aff; + mask-image: url('data:image/svg+xml;charset=utf-8,'); + -webkit-mask-image: url('data:image/svg+xml;charset=utf-8,'); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-size: contain; + content: ""; +} +:is(.admonition-title, summary.admonition-title):hover a.admonition-anchor-link { + display: initial; +} + +details.admonition > summary.admonition-title::after { + position: absolute; + top: 0.625em; + inset-inline-end: 1.6rem; + height: 2rem; + width: 2rem; + background-color: currentcolor; + mask-image: var(--md-details-icon); + -webkit-mask-image: var(--md-details-icon); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-size: contain; + content: ""; + transform: rotate(0deg); + transition: transform 0.25s; +} +details[open].admonition > summary.admonition-title::after { + transform: rotate(90deg); +} + +:is(.admonition):is(.admonish-note) { + border-color: #448aff; +} + +:is(.admonish-note) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(68, 138, 255, 0.1); +} +:is(.admonish-note) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #448aff; + mask-image: var(--md-admonition-icon--admonish-note); + -webkit-mask-image: var(--md-admonition-icon--admonish-note); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-abstract, .admonish-summary, .admonish-tldr) { + border-color: #00b0ff; +} + +:is(.admonish-abstract, .admonish-summary, .admonish-tldr) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(0, 176, 255, 0.1); +} +:is(.admonish-abstract, .admonish-summary, .admonish-tldr) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #00b0ff; + mask-image: var(--md-admonition-icon--admonish-abstract); + -webkit-mask-image: var(--md-admonition-icon--admonish-abstract); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-info, .admonish-todo) { + border-color: #00b8d4; +} + +:is(.admonish-info, .admonish-todo) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(0, 184, 212, 0.1); +} +:is(.admonish-info, .admonish-todo) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #00b8d4; + mask-image: var(--md-admonition-icon--admonish-info); + -webkit-mask-image: var(--md-admonition-icon--admonish-info); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-tip, .admonish-hint, .admonish-important) { + border-color: #00bfa5; +} + +:is(.admonish-tip, .admonish-hint, .admonish-important) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(0, 191, 165, 0.1); +} +:is(.admonish-tip, .admonish-hint, .admonish-important) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #00bfa5; + mask-image: var(--md-admonition-icon--admonish-tip); + -webkit-mask-image: var(--md-admonition-icon--admonish-tip); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-success, .admonish-check, .admonish-done) { + border-color: #00c853; +} + +:is(.admonish-success, .admonish-check, .admonish-done) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(0, 200, 83, 0.1); +} +:is(.admonish-success, .admonish-check, .admonish-done) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #00c853; + mask-image: var(--md-admonition-icon--admonish-success); + -webkit-mask-image: var(--md-admonition-icon--admonish-success); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-question, .admonish-help, .admonish-faq) { + border-color: #64dd17; +} + +:is(.admonish-question, .admonish-help, .admonish-faq) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(100, 221, 23, 0.1); +} +:is(.admonish-question, .admonish-help, .admonish-faq) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #64dd17; + mask-image: var(--md-admonition-icon--admonish-question); + -webkit-mask-image: var(--md-admonition-icon--admonish-question); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-warning, .admonish-caution, .admonish-attention) { + border-color: #ff9100; +} + +:is(.admonish-warning, .admonish-caution, .admonish-attention) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(255, 145, 0, 0.1); +} +:is(.admonish-warning, .admonish-caution, .admonish-attention) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #ff9100; + mask-image: var(--md-admonition-icon--admonish-warning); + -webkit-mask-image: var(--md-admonition-icon--admonish-warning); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-failure, .admonish-fail, .admonish-missing) { + border-color: #ff5252; +} + +:is(.admonish-failure, .admonish-fail, .admonish-missing) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(255, 82, 82, 0.1); +} +:is(.admonish-failure, .admonish-fail, .admonish-missing) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #ff5252; + mask-image: var(--md-admonition-icon--admonish-failure); + -webkit-mask-image: var(--md-admonition-icon--admonish-failure); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-danger, .admonish-error) { + border-color: #ff1744; +} + +:is(.admonish-danger, .admonish-error) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(255, 23, 68, 0.1); +} +:is(.admonish-danger, .admonish-error) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #ff1744; + mask-image: var(--md-admonition-icon--admonish-danger); + -webkit-mask-image: var(--md-admonition-icon--admonish-danger); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-bug) { + border-color: #f50057; +} + +:is(.admonish-bug) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(245, 0, 87, 0.1); +} +:is(.admonish-bug) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #f50057; + mask-image: var(--md-admonition-icon--admonish-bug); + -webkit-mask-image: var(--md-admonition-icon--admonish-bug); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-example) { + border-color: #7c4dff; +} + +:is(.admonish-example) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(124, 77, 255, 0.1); +} +:is(.admonish-example) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #7c4dff; + mask-image: var(--md-admonition-icon--admonish-example); + -webkit-mask-image: var(--md-admonition-icon--admonish-example); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +:is(.admonition):is(.admonish-quote, .admonish-cite) { + border-color: #9e9e9e; +} + +:is(.admonish-quote, .admonish-cite) > :is(.admonition-title, summary.admonition-title) { + background-color: rgba(158, 158, 158, 0.1); +} +:is(.admonish-quote, .admonish-cite) > :is(.admonition-title, summary.admonition-title)::before { + background-color: #9e9e9e; + mask-image: var(--md-admonition-icon--admonish-quote); + -webkit-mask-image: var(--md-admonition-icon--admonish-quote); + mask-repeat: no-repeat; + -webkit-mask-repeat: no-repeat; + mask-size: contain; + -webkit-mask-repeat: no-repeat; +} + +.navy :is(.admonition) { + background-color: var(--sidebar-bg); +} + +.ayu :is(.admonition), +.coal :is(.admonition) { + background-color: var(--theme-hover); +} + +.rust :is(.admonition) { + background-color: var(--sidebar-bg); + color: var(--sidebar-fg); +} +.rust .admonition-anchor-link:link, .rust .admonition-anchor-link:visited { + color: var(--sidebar-fg); +} diff --git a/documentation/operators/src/SUMMARY.md b/documentation/operators/src/SUMMARY.md index c7167fe898..9ae97e8296 100644 --- a/documentation/operators/src/SUMMARY.md +++ b/documentation/operators/src/SUMMARY.md @@ -12,13 +12,17 @@ # Operators Guides -- [Mixnet Nodes Setup](nodes/setup-guides.md) - - [Preliminary Steps](preliminary-steps.md) - - [Mix Node](nodes/mix-node-setup.md) - - [Gateway](nodes/gateway-setup.md) - - [Network Requester](nodes/network-requester-setup.md) +- [Preliminary Steps](nodes/preliminary-steps.md) + - [Nym Wallet Preparation](nodes/wallet-preparation.md) + - [VPS Setup](nodes/vps-setup.md) +- [Nym Node](nodes/nym-node.md) + - [Setup & Run](nodes/setup.md) + - [Configuration](nodes/configuration.md) + - [WSS & Reversed Proxy](nodes/proxy-configuration.md) + - [Bonding](nodes/bonding.md) - [Nyx Validator Setup](nodes/validator-setup.md) - [Nym API Setup](nodes/nym-api.md) + - [Validator & API Configuration](nodes/nyx-configuration.md) - [Maintenance](nodes/maintenance.md) - [Manual Node Upgrade](nodes/manual-upgrade.md) - [Automatic Node Upgrade: Nymvisor Setup and Usage](nodes/nymvisor-upgrade.md) @@ -28,12 +32,12 @@ - [Prometheus & Grafana](testing/prometheus-grafana.md) - [ExploreNYM scripts](testing/explorenym-scripts.md) -- [Troubleshooting](nodes/troubleshooting.md) - + +# Troubleshooting + +- [VPS Setup](troubleshooting/vps-setup.md) +- [Nym Node](troubleshooting/nym-node.md) +- [Validators](troubleshooting/validators.md) # Token Economics @@ -43,10 +47,11 @@ # FAQ -- [Mix Nodes](faq/mixnodes-faq.md) -- [Project Smoosh](faq/smoosh-faq.md) +- [General Operators FAQ](faq/general-faq.md) +- [Nym Nodes](faq/nym-nodes-faq.md) +- [Nyx & Validators](faq/nyx-faq.md) -# Legal Forum +# Community & Legal Forum - [Exit Gateway](legal/exit-gateway.md) - [Community Counsel](legal/community-counsel.md) @@ -56,6 +61,19 @@ - [Landing Pages](legal/landing-pages.md) - [How to Add Info](legal/add-content.md) +--- +# Archive + +- [Why archive?](archive/archive.md) +- [Mixnet Nodes Setup](archive/nodes/setup-guides.md) + - [Preliminary Steps](archive/nodes/preliminary-steps.md) + - [Mix Node](archive/nodes/mix-node-setup.md) + - [Gateway](archive/nodes/gateway-setup.md) + - [Network Requester](archive/nodes/network-requester-setup.md) +- [FAQ: Mix Nodes](archive/faq/mixnodes-faq.md) +- [FAQ: Project Smoosh](archive/faq/smoosh-faq.md) + + --- # Misc. - [Code of Conduct](coc.md) diff --git a/documentation/operators/src/archive/archive.md b/documentation/operators/src/archive/archive.md new file mode 100644 index 0000000000..58bfde316e --- /dev/null +++ b/documentation/operators/src/archive/archive.md @@ -0,0 +1,7 @@ +# Archived Pages + +This section contains old but still relevant pages/guides, archived for backwards compatibility. The content of the pages is not updated. See the top of every page informing you about the last time of update. + +Pages listed in archive section will eventually be terminated as they will become completely irrelevant with time. + + diff --git a/documentation/operators/src/archive/faq/mixnodes-faq.md b/documentation/operators/src/archive/faq/mixnodes-faq.md new file mode 100644 index 0000000000..b893ae5d8a --- /dev/null +++ b/documentation/operators/src/archive/faq/mixnodes-faq.md @@ -0,0 +1,97 @@ +# Frequently Asked Questions + +```admonish warning +**This is an archived page for backwards compatibility. The content of this page is not updated since April 19th 2024. Eventually this page will be terminated!** +``` + +## Nym Mixnet + +To see different stats about Nym Mixnet live, we recommend you to visit [status.notrustverify.ch](https://status.notrustverify.ch/d/CW3L7dVVk/nym-mixnet?orgId=1) built by [No Trust Verify](https://notrustverify.ch/) crew, one of the squads within Nym core community. + + + + +### Is there an explorer for Nym Mixnet? + +Yes, there are several places, some are built by Nym core community: + +* [Nym Explorer](https://explorer.nymtech.net/) +* [Guru Explorer](https://mixnet.explorers.guru/) +* [ExploreNYM](https://explorenym.net/) + +### What determines the rewards when running a Mix Node? + +The stake required for a Mix Node to achieve maximum rewards is called Mix Node saturation point. This is calculated from the staking supply (all circulating supply + part of unlocked tokens). The target level of staking is to have 40% of the staking supply locked in Mix Nodes. + +The node stake saturation point, which we denote by Nsat, is given by the stake supply, target level of staking divided between the rewarded nodes. + +This design ensures the nodes aim to have a same size of stake (reputation) which can be done by delegation staking, as well as it secures a whale prevention and decentralization of staking, as any higher level of delegated $NYM than Nsat per node results in worsening reward ratio. On the contrary, the more Mix Nodes are active, the lower is Nsat. The equilibrium is reached when the staked tokens are delegated equally across the active Mix nodes and that's our basis for this incentive system. + + + + +The rewarded nodes are the nodes which will receive some rewards by the end of the given epoch. These can be separated further separated into: + +1. Active: Top *N* nodes of the rewarded set (currently all of them but this can change), these are nodes which are used by the clients and mix packets. + +2. Standby: Bottom *N* nodes of the rewarded set, they don't mix data from the clients but are used for testing. Their reward is smaller. + +For more detailed calculation, read our blog post [Nym Token Economics update](https://blog.nymtech.net/nym-token-economics-update-fedff0ed5267). More info on staking can be found [here](https://blog.nymtech.net/staking-in-nym-introducing-mainnet-mixmining-f9bb1cbc7c36). And [here](https://blog.nymtech.net/want-to-stake-in-nym-here-is-how-to-choose-a-mix-node-to-delegate-nym-to-c3b862add165) is more info on how to choose a Mix Node for delegation. And finally an [update](https://blog.nymtech.net/quarterly-token-economic-parameter-update-b2862948710f) on token economics from July 2023. + + + +*More graphs and stats at [stats.notrustverify.ch](https://status.notrustverify.ch/d/CW3L7dVVk/nym-mixnet?orgId=1&from=1703074861988&to=1705666862004).* + +### Which VPS providers would you recommend? + +Consider in which jurisdiction you reside and where do you want to run a Mix Node. Do you want to pay by crypto or not and what are the other important particularities for your case? We always recommend operators to try to choose smaller and decentralised VPS providers over the most known ones controlling a majority of the internet. We receive some good feedback on these: Linode, Ghandi, Flokinet and Exoscale. Do your own research and share with the community. + +### Why is a mix node setup on a self-hosted machine so tricky? + +We don't recommend this setup because it's really difficult to get a static IP and route IPv6 traffic. + +### What's the Sphinx packet size? + +The sizes are shown in the configs [here](https://github.com/nymtech/nym/blob/1ba6444e722e7757f1175a296bed6e31e25b8db8/common/nymsphinx/params/src/packet_sizes.rs#L12) (default is the one clients use, the others are for research purposes, not to be used in production as this would fragment the anonymity set). More info can be found [here](https://github.com/nymtech/nym/blob/4844ac953a12b29fa27688609ec193f1d560c996/common/nymsphinx/anonymous-replies/src/reply_surb.rs#L80). + +### Why a Mix Node and a Gateway cannot be bonded with the same wallet? + +Because of the way the smart contract works we keep it one-node one-address at the moment. + +### Which nodes are the most needed to be setup to strengthen Nym infrastructure and which ones bring rewards? + +Ath this point the most crutial component needed are [Exit Gateways](../../legal/exit-gateway.md). + +### Are Mix Nodes whitelisted? + +Nope, anyone can run a Mix Node. Purely reliant on the node's reputation (self stake + delegations) & routing score. + +## Validators and tokens + +### What's the difference between NYM and uNYM? + +1 NYM = 1 000 000 uNYM + + + +### Why some Nyx blockchain operations take one hour and others are instant? + +This is based on the definition in [Nym's CosmWasm](https://github.com/nymtech/nym/tree/develop/common/cosmwasm-smart-contracts) smart contracts code. + +Whatever is defined as [a pending epoch event](https://github.com/nymtech/nym/blob/b07627d57e075b6de35b4b1a84927578c3172811/common/cosmwasm-smart-contracts/mixnet-contract/src/pending_events.rs#L35-L103) will get resolved at the end of the current epoch. + +And whatever is defined as [a pending interval event](https://github.com/nymtech/nym/blob/b07627d57e075b6de35b4b1a84927578c3172811/common/cosmwasm-smart-contracts/mixnet-contract/src/pending_events.rs#L145-L172) will get resolved at the end of the current interval. + +### Can I run a validator? + +We are currently working towards building up a closed set of reputable validators. You can ask us for coins to get in, but please don't be offended if we say no - validators are part of our system's core security and we are starting out with people we already know or who have a solid reputation. + +### Why is validator set entry whitelisted? + +We understand that the early days of the Nyx blockchain will face possible vulnerabilities in terms of size - easy to disrupt or halt the chain if a malicious party entered with a large portion of stake. Besides that, there are some legal issues we need to address before we can distribute the validator set in a fully permissions fashion. + +### Why does Nym do airdrops? + +It is part of ensuring decentralisation - we need to avoid a handful of people having too much control over the token and market. Of course ideally people will stake the tokens and contribute to the project at this stage. We run surveys to better understand what people are doing with their tokens and what usability issues there are for staking. Any feedback is appreciated as it helps us improve all aspects of using the token and participating in the ecosystem. diff --git a/documentation/operators/src/archive/faq/smoosh-faq.md b/documentation/operators/src/archive/faq/smoosh-faq.md new file mode 100644 index 0000000000..8ca144b8e7 --- /dev/null +++ b/documentation/operators/src/archive/faq/smoosh-faq.md @@ -0,0 +1,108 @@ +# Project Smoosh - FAQ + +```admonish warning +**This is an archived page for backwards compatibility. We have switched to [`nym-node` binary](../../nodes/nym-node.md), please [migrate](../../nodes/setup.md#migrate) your nodes. The content of this page is not updated since April 19th 2024. Eventually this page will be terminated!** +``` + +> We aim on purpose to make minimal changes to reward scheme and software. We're just 'smooshing' together stuff we already debugged and know works. +> -- Harry Halpin, Nym CEO + +
+ +This page refer to the changes which are planned to take place over Q3 and Q4 2023. As this is a transition period in the beginning (Q3 2023) the [Mix Nodes FAQ page](mixnodes-faq.md) holds more answers to the current setup as project Smoosh refers to the eventual setup. As project Smoosh gets progressively implemented the answers on this page will become to be more relevant to the current state and eventually this FAQ page will be merged with the still relevant parts of the main Mix Nodes FAQ page. + +If any questions are not answered or it's not clear for you in which stage project Smoosh is right now, please reach out in Node Operators [Matrix room](https://matrix.to/#/#operators:nymtech.chat). + + +### What are the changes? + +Project Smoosh will have four steps, please follow the table below to track the dynamic progress: + +| **Step** | **Status** | +| :--- | :--- | +| **1.** Combine the `nym-gateway` and `nym-network-requester` into one binary | ✅ done | +| **2.** Create [Exit Gateway](../../legal/exit-gateway.md): Take the `nym-gateway` binary including `nym-network-requester` combined in \#1 and switch from [`allowed.list`](https://nymtech.net/.wellknown/network-requester/standard-allowed-list.txt) to a new [exit policy](https://nymtech.net/.wellknown/network-requester/exit-policy.txt) | ✅ done | +| **3.** Combine all the nodes in the Nym Mixnet into one binary, that is `nym-mixnode`, `nym-gateway` (entry and exit) and `nym-network-requester`. | ✅ done | +| **4.** Adjust reward scheme to incentivise and reward Exit Gateways as a part of `nym-node` binary, implementing [zkNym credentials](https://youtu.be/nLmdsZ1BsQg?t=1717). | ✅ done | + +These steps will be staggered over time - period of several months, and will be implemented one by one with enough time to take in feedback and fix bugs in between. +Generally, the software will be the same, just instead of multiple binaries, there will be one Nym Node (`nym-node`) binary. Delegations will remain on as they are now, per our token economics (staking, saturation etc) + +### What does it mean for Nym nodes operators? + +We are exploring two potential methods for implementing binary functionality in practice and will provide information in advance. The options are: + +1. Make a selection button (command/argument/flag) for operators to choose whether they want their node to provide all or just some of the functions nodes have in the Nym Mixnet. Nodes functioning as Exit Gateways (in that epoch) will then have bigger rewards due to their larger risk exposure and overhead work with the setup. + +2. All nodes will be required to have the Exit Gateway functionality. All nodes are rewarded the same as now, and the difference is that a node sometimes (some epochs) may be performing as Exit Gateway sometimes as Mix node or Entry Gateway adjusted according the network demand by an algorithm. + +### Where can I read more about the Exit Gateway setup? + +We created an [entire page](../../legal/exit-gateway.md) about the technical and legal questions around Exit Gateway. + +### What is the change from allow list to deny list? + +The operators running Gateways would have to “open” their nodes to a wider range of online services, in a similar fashion to Tor exit relays. The main change will be to expand the original short [`allowed.list`](https://nymtech.net/.wellknown/network-requester/standard-allowed-list.txt) to a more permissive setup. An [exit policy](https://nymtech.net/.wellknown/network-requester/exit-policy.txt) will constrain the hosts that the users of the Nym VPN and Mixnet can connect to. This will be done in an effort to protect the operators, as Gateways will act both as SOCKS5 Network Requesters, and exit nodes for IP traffic from Nym VPN and Mixnet clients. + +### How will the Exit policy be implemented? + +Follow the dynamic progress of exit policy implementation on Gateways below: + +| **Step** | **Status** | +| :--- | :--- | +| **1.** By default the [exit policy](https://nymtech.net/.wellknown/network-requester/exit-policy.txt) filtering is disabled and the [`allowed.list`](https://nymtech.net/.wellknown/network-requester/standard-allowed-list.txt) filtering is going to continue be used. This is to prevent operators getting surprised by upgrading their Gateways (or Network Requesters) and suddenly be widely open to the internet. To enable the new exit policy, operators must use `--with-exit-policy` flag or modify the `config.toml` file. | ✅ done | +| **2.** The exit policy is part of the Gateway setup by default. To disable this exit policy, operators must use `--disable-exit-policy` flag. | ✅ done | +| **3.** The exit policy is the only option. The `allowed.list` is completely removed. | ✅ done | + +Keep in mind the table above only relates to changes happening on Gateways. For the Project Smoosh progress refer to the [table above](./smoosh-faq.md#what-are-the-changes). Whether Exit Gateway functionality will be optional or mandatory part of every active Nym Node depends on the chosen [design](./smoosh-faq.md#what-does-it-mean-for-nym-nodes-operators). + +### Can I run a Mix Node only? + +It depends which [design](./smoosh-faq.md#what-does-it-mean-for-nym-nodes-operators) will ultimately be used. In case of the first - yes. In case of the second option, all the nodes will be setup with Exit Gateway functionality turned on. + +## Token Economics & Rewards + +```admonish info +For any specifics on Nym token economics and Nym Mixnet reward system, please read the [Nym token economics paper](https://nymtech.net/nym-cryptoecon-paper.pdf). +``` + +### What are the incentives for the node operator? + +In the original setup there were no incentives to run a `nym-network-requester` binary. After the transition all the users will buy multiple tickets of zkNyms credentials and use those as [anonymous e-cash](https://arxiv.org/abs/2303.08221) to pay for their data traffic ([`Nym API`](https://github.com/nymtech/nym/tree/master/nym-api) will do the do cryptographical checks to prevent double-spending). All collected fees get distributed to all active nodes proportionally to their work by the end of each epoch. + +### How does this change the token economics? + +The token economics will stay the same as they are, same goes for the reward algorithm. + +### How are the rewards distributed? + +This depends on [design](./smoosh-faq.md#what-does-it-mean-for-nym-nodes-operators) chosen. In case of \#1, it will look like this: + +As each operator can choose what roles their nodes provide, the nodes which work as open Gateways will have higher rewards because they are the most important to keep up and stable. Besides that the operators of Gateways may be exposed to more complication and possible legal risks. + +The nodes which are initialized to run as Mix Nodes and Gateways will be chosen to be on top of the active set before the ones working only as a Mix Node. + +I case we go with \#2, all nodes active in the epoch will be rewarded proportionally according their work. + +In either way, Nym will share all the specifics beforehand. + +### How will be the staking and inflation after project Smoosh? + +Nym will run tests to count how much payment comes from the users of the Mixnet and if that covers the reward payments. If not, we may need to keep inflation on to secure incentives for high quality Gateways in the early stage of the transition. + +### When project smooth will be launched, it would be the mixmining pool that will pay for the Gateway rewards based on amount of traffic routed ? + +Yes, the same pool. Nym's aim is to do minimal modifications. The only real modification on the smart contract side will be to get into top X of 'active set' operators will need to have open Gateway function enabled. + +### What does this mean for the current delegators? + +From an operator standpoint, it shall just be a standard Nym upgrade, a new option to run the Gateway software on your node. Delegators should not have to re-delegate. + +## Legal Questions + +### Are there any legal concerns for the operators? + +So far the general line is that running a Gateway is not illegal (unless you are in Iran, China, and a few other places) and due to encryption/mixing less risky than running a normal VPN node. For Mix Nodes, it's very safe as they have "no idea" what packets they are mixing. + +There are several legal questions and analysis to be made for different jurisdictions. To be able to share resources and findings between the operators themselves we created a [Community Legal Forum](../../legal/exit-gateway.md). + diff --git a/documentation/operators/src/nodes/gateway-setup.md b/documentation/operators/src/archive/nodes/gateway-setup.md similarity index 79% rename from documentation/operators/src/nodes/gateway-setup.md rename to documentation/operators/src/archive/nodes/gateway-setup.md index 321a26498b..6a363e1818 100644 --- a/documentation/operators/src/nodes/gateway-setup.md +++ b/documentation/operators/src/archive/nodes/gateway-setup.md @@ -1,22 +1,26 @@ # Gateways -> The Nym gateway was built in the [building nym](../binaries/building-nym.md) section. If you haven't yet built Nym and want to run the code, go there first. +```admonish warning +**This is an archived page for backwards compatibility for existing node operators. To start a new node or migrate, follow the [`nym-node` guides](../../nodes/nym-node.md).** The content of this page is not updated since April 19th 2024. Eventually this page will be terminated! +``` + +> The Nym gateway was built in the [building nym](../../binaries/building-nym.md) section. If you haven't yet built Nym and want to run the code, go there first. ```admonish info -As a result of [Project Smoosh](../faq/smoosh-faq.md), the current version of `nym-gateway` binary also contains `nym-network-requester` functionality which can be enabled [by the operator](./gateway-setup.md#initialising-gateway-with-network-requester). This combination is a basis of ***Nym Exit Gateway*** node - an essential piece in our new setup. Please read more in our [Project Smoosh FAQ](../faq/smoosh-faq.md) and [Exit Gateway](../legal/exit-gateway.md) pages. We recommend operators begin to shift their setups to this new combined node, instead of operating two separate binaries. +As a result of [Project Smoosh](../faq/smoosh-faq.md), the current version of `nym-gateway` binary also contains `nym-network-requester` functionality which can be enabled [by the operator](./gateway-setup.md#initialising-gateway-with-network-requester). This combination is a basis of ***Nym Exit Gateway*** node - an essential piece in our new setup. Please read more in our [Project Smoosh FAQ](../faq/smoosh-faq.md) and [Exit Gateway](../../legal/exit-gateway.md) pages. We recommend operators begin to shift their setups to this new combined node, instead of operating two separate binaries. ``` > Any syntax in `<>` brackets is a user's unique variable. Exchange with a corresponding name without the `<>` brackets. ## Current version ``` - + ``` ## Preliminary steps -Make sure you do the preparation listed in the [preliminary steps page](../preliminary-steps.md) before setting up your Gateway. +Make sure you do the preparation listed in the [preliminary steps page](preliminary-steps.md) before setting up your Gateway. ## Gateway setup @@ -37,7 +41,7 @@ You can check that your binaries are properly compiled with: ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -50,7 +54,7 @@ You can also check the various arguments required for individual commands with: ## Initialising your Gateway -As Nym developers build towards [Exit Gateway](../legal/exit-gateway.md) functionality, operators can now run their `nym-gateway` binary with inbuilt Network Requester and include the our new [exit policy](https://nymtech.net/.wellknown/network-requester/exit-policy.txt). Considering the plan to [*smoosh*](../faq/smoosh-faq.md) all the nodes into one binary and have wide opened Exit Gateways, we recommend this setup, instead of operating two separate binaries. +As Nym developers build towards [Exit Gateway](../../legal/exit-gateway.md) functionality, operators can now run their `nym-gateway` binary with inbuilt Network Requester and include the our new [exit policy](https://nymtech.net/.wellknown/network-requester/exit-policy.txt). Considering the plan to [*smoosh*](../faq/smoosh-faq.md) all the nodes into one binary and have wide opened Exit Gateways, we recommend this setup, instead of operating two separate binaries. ```admonish warning Before you start an Exit Gateway, read our [Operators Legal Forum](../legal/exit-gateway.md) page and [*Project Smoosh FAQ*](../faq/smoosh-faq.md). @@ -74,7 +78,7 @@ If we follow the previous example with `` chosen `superexitgateway`, adding ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -84,7 +88,7 @@ Additionally #### Add Network Requester to an existing Gateway -If you already [upgraded](./manual-upgrade.md) your Gateway to the [latest version](./gateway-setup.md#current-version) and initialised without a Network Requester, you can easily change its functionality to Exit Gateway with a command `setup-network-requester`. +If you already [upgraded](../../nodes/manual-upgrade.md) your Gateway to the [latest version](./gateway-setup.md#current-version) and initialised without a Network Requester, you can easily change its functionality to Exit Gateway with a command `setup-network-requester`. See the options: @@ -94,7 +98,7 @@ See the options: ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -114,7 +118,7 @@ Say we have a Gateway with `` as `new-gateway`, originally initialised and r ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -159,7 +163,7 @@ To check available configuration options use: ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -171,7 +175,7 @@ The following command returns a Gateway on your current IP with the `` of `s ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -189,7 +193,7 @@ The `run` command starts the Gateway: ## Bonding your Gateway ```admonish info -Before you bond your Gateway, please make sure the [firewall configuration](./maintenance.md#configure-your-firewall) is setup so your Gateway can be reached from the outside. You can also setup [WSS on your Gateway](./maintenance.md#run-web-secure-socket-wss-on-gateway) and [automate](./maintenance.md#vps-setup-and-automation) your Gateway to simplify the operation overhead. We highly recommend to run any of these steps before bonding to prevent disruption of your Gateway's routing score later on. +Before you bond your Gateway, please make sure the [firewall configuration](./maintenance.md#configure-your-firewall) is setup so your Gateway can be reached from the outside. You can also setup WSS and automate your Gateway to simplify the operation overhead. We highly recommend to run any of these steps before bonding to prevent disruption of your Gateway's routing score later on. ``` ### Via the Desktop wallet (recommended) @@ -239,7 +243,7 @@ It will look something like this (as `` we used `supergateway`): * And paste it into the wallet nodal, press `Next` and confirm the transaction. -![Paste Signature](../images/wallet-screenshots/wallet-gateway-sign.png) +![Paste Signature](../../images/wallet-screenshots/wallet-gateway-sign.png) *This image is just an example, copy-paste your own base58-encoded signature.* * Your Gateway is now bonded. @@ -252,5 +256,5 @@ If you want to bond your Gateway via the CLI, then check out the [relevant secti ## Maintenance -For Gateway upgrade, firewall setup, port configuration, API endpoints, VPS suggestions, automation, WSS setup and more, see the [maintenance page](./maintenance.md) +For Gateway upgrade, firewall setup, port configuration, API endpoints, VPS suggestions, automation, WSS setup and more, see the [maintenance page](../../nodes/maintenance.md) diff --git a/documentation/operators/src/nodes/mix-node-setup.md b/documentation/operators/src/archive/nodes/mix-node-setup.md similarity index 77% rename from documentation/operators/src/nodes/mix-node-setup.md rename to documentation/operators/src/archive/nodes/mix-node-setup.md index 55ace09e71..43424fb9b1 100644 --- a/documentation/operators/src/nodes/mix-node-setup.md +++ b/documentation/operators/src/archive/nodes/mix-node-setup.md @@ -1,23 +1,27 @@ # Mix Nodes -> The Nym Mix Node binary was built in the [building nym](../binaries/building-nym.md) section. If you haven't yet built Nym and want to run the code, go there first. +```admonish warning +**This is an archived page for backwards compatibility for existing node operators. To start a new node or migrate, follow the [`nym-node` guides](../../nodes/nym-node.md).** The content of this page is not updated since April 19th 2024. Eventually this page will be terminated! +``` + +> The Nym Mix Node binary was built in the [building nym](../../binaries/building-nym.md) section. If you haven't yet built Nym and want to run the code, go there first. > Any syntax in `<>` brackets is a user's unique variable. Exchange with a corresponding name without the `<>` brackets. ## Current version ``` - + ``` The `nym-mix node` binary is currently one point version ahead of the rest of the platform binaries due to a patch applied between releases. ## Preliminary steps -Make sure you do the preparation listed in the [preliminary steps page](../preliminary-steps.md) before setting up your Mix Node. +Make sure you do the preparation listed in the [preliminary steps page](preliminary-steps.md) before setting up your Mix Node. ## Mix node setup -Now that you have built the [codebase](../binaries/building-nym.md), set up your [wallet](https://nymtech.net/docs/wallet/desktop-wallet.html), and have a VPS with the `nym-mix node` binary, you can set up your Mix Node with the instructions below. +Now that you have built the [codebase](../../binaries/building-nym.md), set up your [wallet](https://nymtech.net/docs/wallet/desktop-wallet.html), and have a VPS with the `nym-mix node` binary, you can set up your Mix Node with the instructions below. To begin, move to `/target/release` directory from which you run the node commands: @@ -37,7 +41,7 @@ Which should return a list of all available commands. ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -59,7 +63,7 @@ To check available configuration options for initializing your node use: ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -72,7 +76,7 @@ If `` was `my-node`, the output will look like this: ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -119,9 +123,9 @@ From `v1.1.3`, if you unbond your Mix Node that means you are leaving the mi To initialise, run and bond your Mix Node are the minimum steps to do in order for your Mix Node to work. However we recommend to do a few more steps before bonding. These steps will make it easier for you as a node operator on a long run as well as for others to possibly delegate Nym tokens to your Mix Node. These steps are: - [Describe your Mix Node](./mix-node-setup.md#node-description-optional) -- [Configure your firewall](./maintenance.md#configure-your-firewall) -- [Automate your Mix Node](./maintenance.md#vps-setup-and-automation) -- Set the [ulimit](./maintenance.md#set-the-ulimit-via-systemd-service-file), in case you haven't automated with [systemd](./maintenance.md#set-the-ulimit-on-non-systemd-based-distributions) +- [Configure your firewall](../../nodes/maintenance.md#configure-your-firewall) +- [Automate your Mix Node](../../nodes/maintenance.md#vps-setup-and-automation) +- Set the [ulimit](../../nodes/maintenance.md#set-the-ulimit-via-systemd-service-file), in case you haven't automated with [systemd](../../nodes/maintenance.md#set-the-ulimit-on-non-systemd-based-distributions) ### Bond via the Desktop wallet (recommended) @@ -141,8 +145,8 @@ It will look something like this: ~~~admonish example collapsible=true title="Console output" ``` - - + + ``` ~~~ @@ -155,7 +159,7 @@ It will look something like this: * And paste it into the wallet nodal, press `Next` and confirm the transaction. -![Paste Signature](../images/wallet-screenshots/wallet-sign.png) +![Paste Signature](../../images/wallet-screenshots/wallet-sign.png) *This image is just an example, copy-paste your own base58-encoded signature* * Your node will now be bonded and ready to mix at the beginning of the next epoch (at most 1 hour). @@ -189,8 +193,8 @@ Change directory by `cd ///` and run the following on th ~~~admonish example collapsible=true title="Console output" ``` - - + + ``` ~~~ @@ -224,8 +228,8 @@ Change directory by `cd ///` and run the following on th ~~~admonish example collapsible=true title="Console output" ``` - - + + ``` ~~~ @@ -276,9 +280,7 @@ There are also 2 community explorers which have been created by [Nodes Guru](htt - [Mainnet](https://mixnet.explorers.guru/) - [Sandbox testnet](https://sandbox.mixnet.explorers.guru/) -For more details see [Troubleshooting FAQ](../nodes/troubleshooting.md) - ## Maintenance -For Mix Node upgrade, firewall setup, port configuration, API endpoints, VPS suggestions, automation and more, see the [maintenance page](./maintenance.md) +For Mix Node upgrade, firewall setup, port configuration, API endpoints, VPS suggestions, automation and more, see the [maintenance page](../../nodes/maintenance.md) diff --git a/documentation/operators/src/nodes/network-requester-setup.md b/documentation/operators/src/archive/nodes/network-requester-setup.md similarity index 86% rename from documentation/operators/src/nodes/network-requester-setup.md rename to documentation/operators/src/archive/nodes/network-requester-setup.md index 90bd37d3fe..035c49a862 100644 --- a/documentation/operators/src/nodes/network-requester-setup.md +++ b/documentation/operators/src/archive/nodes/network-requester-setup.md @@ -1,21 +1,21 @@ -# Network Requesters +# Network Requester -> Nym Network Requester was built in the [building nym](../binaries/building-nym.md) section. If you haven't yet built Nym and want to run the code, go there first. - -```admonish info -As a result of [Project Smoosh](../faq/smoosh-faq.md), the current version of `nym-gateway` binary also contains `nym-network-requester` functionality which can be enabled [by the operator](./gateway-setup.md#initialising-gateway-with-network-requester). This combination is a basis of Nym Exit Gateway node - an essential piece in our new setup. Please read more in our [Project Smoosh FAQ](../faq/smoosh-faq.md) and [Exit Gateways Page](../legal/exit-gateway.md). We recommend operators begin to shift their setups to this new combined node, instead of operating two separate binaries. +```admonish warning +**This is an archived page for backwards compatibility for existing node operators. To start a new node or migrate, follow the [`nym-node` guides](../../nodes/nym-node.md).** The content of this page is not updated since April 19th 2024. Eventually this page will be terminated! ``` +> Nym Network Requester was built in the [building nym](../../binaries/building-nym.md) section. If you haven't yet built Nym and want to run the code, go there first. + > Any syntax in `<>` brackets is a user's unique variable. Exchange with a corresponding name without the `<>` brackets. ## Current version ``` - + ``` ## Preliminary steps -Make sure you do the preparation listed in the [preliminary steps page](../preliminary-steps.md) before setting up your Network Requester. +Make sure you do the preparation listed in the [preliminary steps page](preliminary-steps.md) before setting up your Network Requester. ## Network Requester Whitelist @@ -129,7 +129,7 @@ The `./nym-network-requester --help ` command can be used to show a list of avai ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -153,7 +153,7 @@ In the following we used `example`. ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -218,7 +218,4 @@ This command should return the following: { "status": "ok" } ``` -## Maintenance - -For Network Requester upgrade (including an upgrade from `= v1.1.10`), firewall setup, port configuration, API endpoints, VPS suggestions, automation and more, see the [maintenance page](./maintenance.md). diff --git a/documentation/operators/src/preliminary-steps.md b/documentation/operators/src/archive/nodes/preliminary-steps.md similarity index 90% rename from documentation/operators/src/preliminary-steps.md rename to documentation/operators/src/archive/nodes/preliminary-steps.md index 2f7cb2008b..88cf7c7512 100644 --- a/documentation/operators/src/preliminary-steps.md +++ b/documentation/operators/src/archive/nodes/preliminary-steps.md @@ -1,6 +1,10 @@ # Preliminary Steps -> The Nym `mixnode`, `gateway` and `network-requester` binaries were built in the [building nym](./binaries/building-nym.md) section. If you haven't yet built Nym and want to run the code, go there first. +```admonish warning +**This is an archived page for backwards compatibility. The content of this page is not updated since April 19th 2024. Eventually this page will be terminated!** +``` + +> The Nym `mixnode`, `gateway` and `network-requester` binaries were built in the [building nym](../../binaries/building-nym.md) section. If you haven't yet built Nym and want to run the code, go there first. There are a couple of steps that need completing before starting to set up your mix node, gateway or a network requester: diff --git a/documentation/operators/src/archive/nodes/setup-guides.md b/documentation/operators/src/archive/nodes/setup-guides.md new file mode 100644 index 0000000000..957c94ab51 --- /dev/null +++ b/documentation/operators/src/archive/nodes/setup-guides.md @@ -0,0 +1,15 @@ +# Node Setup Guides + +```admonish warning +**This is an archived page for backwards compatibility. The content of this page is not updated since April 19th 2024. Eventually this page will be terminated!** +``` + +To setup any type of Nym's node, start with building [Nym's platform](../../binaries/building-nym.md) on the machine (VPS) where you want to run the node. Nodes will need to be bond to Nym's wallet, setup one [here](https://nymtech.net/docs/wallet/desktop-wallet.html). + +This section contains setup guides for the following node types: +* [Mix Node](mix-node-setup.md) +* [Gateway](gateway-setup.md) +* [Network Requester](network-requester-setup.md) +* [Validator](../../nodes/validator-setup.md) + + diff --git a/documentation/operators/src/binaries/building-nym.md b/documentation/operators/src/binaries/building-nym.md index 7d7b1a506a..eb6234bb60 100644 --- a/documentation/operators/src/binaries/building-nym.md +++ b/documentation/operators/src/binaries/building-nym.md @@ -54,16 +54,14 @@ cargo build --release # build your binaries with **mainnet** configuration Quite a bit of stuff gets built. The key working parts are: -* [mix node](../nodes/mix-node-setup.md): `nym-mixnode` -* [gateway node](../nodes/gateway-setup.md): `nym-gateway` +* [Nym Node](../nodes/nym-node.md): `nym-node` +* [Validator](../nodes/validator-setup.md) * [websocket client](https://nymtech.net/docs/clients/websocket-client.html): `nym-client` * [socks5 client](https://nymtech.net/docs/clients/socks5-client.html): `nym-socks5-client` * [webassembly client](https://nymtech.net/docs/clients/webassembly-client.html): `webassembly-client` -* [network requester](../nodes/network-requester-setup.md): `nym-network-requester` * [nym-cli tool](https://nymtech.net/docs/tools/nym-cli.html): `nym-cli` * [nym-api](../nodes/nym-api.md): `nym-api` - -[//]: # (* [nymvisor](../nodes/nymvisor-upgrade.md): `nymvisor`) +* [nymvisor](../nodes/nymvisor-upgrade.md): `nymvisor` The repository also contains Typescript applications which aren't built in this process. These can be built by following the instructions on their respective docs pages. * [Nym Wallet](https://nymtech.net/docs/wallet/desktop-wallet.html) diff --git a/documentation/operators/src/binaries/pre-built-binaries.md b/documentation/operators/src/binaries/pre-built-binaries.md index 1256df3d19..b42e5bb9c6 100644 --- a/documentation/operators/src/binaries/pre-built-binaries.md +++ b/documentation/operators/src/binaries/pre-built-binaries.md @@ -28,7 +28,5 @@ Now you can use your binary, initialise and run your Nym Node. Follow the guide **Node setup and usage guides:** -* [Mix nodes](../nodes/mix-node-setup.md) -* [Gateways](../nodes/gateway-setup.md) -* [Network requesters](../nodes/network-requester-setup.md) +* [Nym Nodes](../nodes/nym-node.md) * [Validators](../nodes/validator-setup.md) diff --git a/documentation/operators/src/faq/general-faq.md b/documentation/operators/src/faq/general-faq.md new file mode 100644 index 0000000000..df8587872b --- /dev/null +++ b/documentation/operators/src/faq/general-faq.md @@ -0,0 +1,42 @@ +# General Operators FAQ + +## Nym Mixnet + +To see different stats about Nym Mixnet live, we recommend you to visit [status.notrustverify.ch](https://status.notrustverify.ch/d/CW3L7dVVk/nym-mixnet?orgId=1) built by [No Trust Verify](https://notrustverify.ch/) crew, one of the squads within Nym core community. + + + + +### Is there an explorer for Nym Mixnet? + +Yes, there are several places, some are built by Nym core community: + +* [Nym Explorer](https://explorer.nymtech.net/) +* [Guru Explorer](https://mixnet.explorers.guru/) +* [ExploreNYM](https://explorenym.net/) + +### Which VPS providers would you recommend? + +Consider in which jurisdiction you reside and where do you want to run a Mix Node. Do you want to pay by crypto or not and what are the other important particularities for your case? We always recommend operators to try to choose smaller and decentralised VPS providers over the most known ones controlling a majority of the internet. We receive some good feedback on these: Linode, Gandi, Flokinet and Exoscale. Do your own research and share with the community. + +### Why is a node setup on a self-hosted machine so tricky? + +We don't recommend this setup because it's really difficult to get a static IP and route IPv6 traffic. + +### What's the Sphinx packet size? + +The sizes are shown in the configs [here](https://github.com/nymtech/nym/blob/1ba6444e722e7757f1175a296bed6e31e25b8db8/common/nymsphinx/params/src/packet_sizes.rs#L12) (default is the one clients use, the others are for research purposes, not to be used in production as this would fragment the anonymity set). More info can be found [here](https://github.com/nymtech/nym/blob/4844ac953a12b29fa27688609ec193f1d560c996/common/nymsphinx/anonymous-replies/src/reply_surb.rs#L80). + +### Why a Mix Node and a Gateway cannot be bonded with the same wallet? + +Because of the way the smart contract works we keep it one-node one-address at the moment. + +### Which nodes are the most needed to be setup to strengthen Nym infrastructure and which ones bring rewards? + +Ath this point the most crutial component needed are [Exit Gateways](../legal/exit-gateway.md). + +### Are Nym Nodes whitelisted? + +Nope, anyone can run a Nym Node. whether your node is chosen to mix is purely reliant on the node's performance and reputation (self stake + delegations). + + diff --git a/documentation/operators/src/faq/nym-nodes-faq.md b/documentation/operators/src/faq/nym-nodes-faq.md new file mode 100644 index 0000000000..d0a65b2ec0 --- /dev/null +++ b/documentation/operators/src/faq/nym-nodes-faq.md @@ -0,0 +1,32 @@ +# Nym Nodes related Frequently Asked Questions + +### What determines the rewards when running a `nym-node --mode mixnode`? + +The stake required for a Mix Node to achieve maximum rewards is called Mix Node saturation point. This is calculated from the staking supply (all circulating supply + part of unlocked tokens). The target level of staking is to have 40% of the staking supply locked in Mix Nodes. + +The node stake saturation point, which we denote by Nsat, is given by the stake supply, target level of staking divided between the rewarded nodes. + +This design ensures the nodes aim to have a same size of stake (reputation) which can be done by delegation staking, as well as it secures a whale prevention and decentralization of staking, as any higher level of delegated $NYM than Nsat per node results in worsening reward ratio. On the contrary, the more Mix Nodes are active, the lower is Nsat. The equilibrium is reached when the staked tokens are delegated equally across the active Mix nodes and that's our basis for this incentive system. + + + +The rewarded nodes are the nodes which will receive some rewards by the end of the given epoch. These can be separated further separated into: + +1. Active: Top *N* nodes of the rewarded set (currently all of them but this can change), these are nodes which are used by the clients and mix packets. + +2. Standby: Bottom *N* nodes of the rewarded set, they don't mix data from the clients but are used for testing. Their reward is smaller. + + +For more detailed calculation, read our blog post [Nym Token Economics update](https://blog.nymtech.net/nym-token-economics-update-fedff0ed5267). More info on staking can be found [here](https://blog.nymtech.net/staking-in-nym-introducing-mainnet-mixmining-f9bb1cbc7c36). And [here](https://blog.nymtech.net/want-to-stake-in-nym-here-is-how-to-choose-a-mix-node-to-delegate-nym-to-c3b862add165) is more info on how to choose a Mix Node for delegation. And finally an [update](https://blog.nymtech.net/quarterly-token-economic-parameter-update-b2862948710f) on token economics from July 2023. + + + + + +*More graphs and stats at [stats.notrustverify.ch](https://status.notrustverify.ch/d/CW3L7dVVk/nym-mixnet?orgId=1&from=1703074861988&to=1705666862004).* + + diff --git a/documentation/operators/src/faq/nyx-faq.md b/documentation/operators/src/faq/nyx-faq.md new file mode 100644 index 0000000000..bbef752dfe --- /dev/null +++ b/documentation/operators/src/faq/nyx-faq.md @@ -0,0 +1,29 @@ +## Validators and tokens + +### What's the difference between NYM and uNYM? + +1 NYM = 1 000 000 uNYM + + + +### Why some Nyx blockchain operations take one hour and others are instant? + +This is based on the definition in [Nym's CosmWasm](https://github.com/nymtech/nym/tree/develop/common/cosmwasm-smart-contracts) smart contracts code. + +Whatever is defined as [a pending epoch event](https://github.com/nymtech/nym/blob/b07627d57e075b6de35b4b1a84927578c3172811/common/cosmwasm-smart-contracts/mixnet-contract/src/pending_events.rs#L35-L103) will get resolved at the end of the current epoch. + +And whatever is defined as [a pending interval event](https://github.com/nymtech/nym/blob/b07627d57e075b6de35b4b1a84927578c3172811/common/cosmwasm-smart-contracts/mixnet-contract/src/pending_events.rs#L145-L172) will get resolved at the end of the current interval. + +### Can I run a validator? + +We are currently working towards building up a closed set of reputable validators. You can ask us for coins to get in, but please don't be offended if we say no - validators are part of our system's core security and we are starting out with people we already know or who have a solid reputation. + +### Why is validator set entry whitelisted? + +We understand that the early days of the Nyx blockchain will face possible vulnerabilities in terms of size - easy to disrupt or halt the chain if a malicious party entered with a large portion of stake. Besides that, there are some legal issues we need to address before we can distribute the validator set in a fully permissions fashion. + +### Why does Nym do airdrops? + +It is part of ensuring decentralisation - we need to avoid a handful of people having too much control over the token and market. Of course ideally people will stake the tokens and contribute to the project at this stage. We run surveys to better understand what people are doing with their tokens and what usability issues there are for staking. Any feedback is appreciated as it helps us improve all aspects of using the token and participating in the ecosystem. diff --git a/documentation/operators/src/faq/smoosh-faq.md b/documentation/operators/src/faq/smoosh-faq.md index 32774a65e6..08ca34169c 100644 --- a/documentation/operators/src/faq/smoosh-faq.md +++ b/documentation/operators/src/faq/smoosh-faq.md @@ -5,7 +5,7 @@
-This page refer to the changes which are planned to take place over Q3 and Q4 2023. As this is a transition period in the beginning (Q3 2023) the [Mix Nodes FAQ page](mixnodes-faq.md) holds more answers to the current setup as project Smoosh refers to the eventual setup. As project Smoosh gets progressively implemented the answers on this page will become to be more relevant to the current state and eventually this FAQ page will be merged with the still relevant parts of the main Mix Nodes FAQ page. +This page refer to the changes which are planned to take place over Q3 and Q4 2023. As this is a transition period in the beginning (Q3 2023) the [Mix Nodes FAQ page](nym-nodes-faq.md) holds more answers to the current setup as project Smoosh refers to the eventual setup. As project Smoosh gets progressively implemented the answers on this page will become to be more relevant to the current state and eventually this FAQ page will be merged with the still relevant parts of the main Mix Nodes FAQ page. If any questions are not answered or it's not clear for you in which stage project Smoosh is right now, please reach out in Node Operators [Matrix room](https://matrix.to/#/#operators:nymtech.chat). @@ -28,8 +28,8 @@ Project Smoosh will have four steps, please follow the table below to track the | :--- | :--- | | **1.** Combine the `nym-gateway` and `nym-network-requester` into one binary | ✅ done | | **2.** Create [Exit Gateway](../legal/exit-gateway.md): Take the `nym-gateway` binary including `nym-network-requester` combined in \#1 and switch from [`allowed.list`](https://nymtech.net/.wellknown/network-requester/standard-allowed-list.txt) to a new [exit policy](https://nymtech.net/.wellknown/network-requester/exit-policy.txt) | ✅ done | -| **3.** Combine all the nodes in the Nym Mixnet into one binary, that is `nym-mixnode`, `nym-gateway` (entry and exit) and `nym-network-requester`. | 🛠️ in progress | -| **4.** Adjust reward scheme to incentivise and reward Exit Gateways as a part of `nym-node` binary, implementing [zkNym credentials](https://youtu.be/nLmdsZ1BsQg?t=1717). | 🛠️ in progress | +| **3.** Combine all the nodes in the Nym Mixnet into one binary, that is `nym-mixnode`, `nym-gateway` (entry and exit) and `nym-network-requester`. | ✅ done | +| **4.** Adjust reward scheme to incentivise and reward Exit Gateways as a part of `nym-node` binary, implementing [zkNym credentials](https://youtu.be/nLmdsZ1BsQg?t=1717). | ✅ done | These steps will be staggered over time - period of several months, and will be implemented one by one with enough time to take in feedback and fix bugs in between. Generally, the software will be the same, just instead of multiple binaries, there will be one Nym Node (`nym-node`) binary. Delegations will remain on as they are now, per our token economics (staking, saturation etc) @@ -57,8 +57,8 @@ Follow the dynamic progress of exit policy implementation on Gateways below: | **Step** | **Status** | | :--- | :--- | | **1.** By default the [exit policy](https://nymtech.net/.wellknown/network-requester/exit-policy.txt) filtering is disabled and the [`allowed.list`](https://nymtech.net/.wellknown/network-requester/standard-allowed-list.txt) filtering is going to continue be used. This is to prevent operators getting surprised by upgrading their Gateways (or Network Requesters) and suddenly be widely open to the internet. To enable the new exit policy, operators must use `--with-exit-policy` flag or modify the `config.toml` file. | ✅ done | -| **2.** The exit policy is part of the Gateway setup by default. To disable this exit policy, operators must use `--disable-exit-policy` flag. | 🛠️ in progress | -| **3.** The exit policy is the only option. The `allowed.list` is completely removed. | 🛠️ in progress | +| **2.** The exit policy is part of the Gateway setup by default. To disable this exit policy, operators must use `--disable-exit-policy` flag. | ✅ done | +| **3.** The exit policy is the only option. The `allowed.list` is completely removed. | ✅ done | Keep in mind the table above only relates to changes happening on Gateways. For the Project Smoosh progress refer to the [table above](./smoosh-faq.md#what-are-the-changes). Whether Exit Gateway functionality will be optional or mandatory part of every active Nym Node depends on the chosen [design](./smoosh-faq.md#what-does-it-mean-for-nym-nodes-operators). diff --git a/documentation/operators/src/images/ip_table_prompt.png b/documentation/operators/src/images/ip_table_prompt.png new file mode 100644 index 0000000000000000000000000000000000000000..dadd006b5871b1d1bbdf4671139bdce9a7a0d71c GIT binary patch literal 21173 zcmeFZXH-<%x-E*jRDzNbB`y#ogOW1}l5&P=wI)B%uo$A`Mda3_&7K?cNG<$ zYvJJB#Ngmu6UMy`{v}p^IU4+R!&O>Q2NxH2YF>Q~2j@>5#plm--llENc=~FOoc`F0 z8;l{${J7oBP5lRn#r_-xxKK)(8cXXYGyRI$(92tt~$ZLOpd@2$q z_V*{c&kKZqeR_90yY|LkpXQobxBvd1Zy*2PuYr~LUsw-TV2MD~@?Pai0I**Lkk}PI3V{@e!8FrBrp`oBl0J)3Ze=7b2|| zS&W!ZA}jf{P$s6g|88&{C1f}gzkd+q!_|;{X}NhOsov@&p)bgcgX5Ikcv8iQgL7f0 zau}zj?`tFM+xsOxi;#UzcT3R(K_{Kl-6zRMNT-A>9}egxRxMb_{br%+7GxRMjZ`Ol zkh_1_<34v#cUcyDikP2*rH*MQxX?fSaSBO+Ohb?yhd!=C#~HKYlMFaGm(Jv#;Dpz; zy91Zzck+FGea*JALUH+K;dUyGNj=?>#4VUr2lshtQ-MN!hUZSrMXCHbVYgMhS&~jY zqZ{aZEO=)d)6p`^)$Zak1jDEn+Uq#GPn5uY(1}aevi*z%2I(ang}wCdN0<)ipidDQ(lCl!GDANG zbH9;mci(u$J7YJY3V!DvweX@0!}Ec{k;K{PdTZJC*}xWS zOreQ3_W4(~y$odZSP=1H1JUl*QI>kR+{$gqx0htP@1S1V`*StD^hZNYqpfeb7|c6N zGqsKpVoJt>YA&3q&C0FO(eynHhBsOV5L9X%1Ux*@H+Ox-sz}%q_D=fZYb5y}&5{N8 z-diInVEL+bVjcY8yqV(FcJ{~}$p^#Jh&sSS7c=LYSb$UMKg!G`S$ zk3gcdi}$~%wPw>};U0kp1xo6i>6S`LdX(HuQi*+Y3Be3fdXupu7njuQW+iSbZAk5Q z|8Im`n=@Y{MJt`=I(AYpC*MqE&g2Mp3k}q8Umf?xu>UYKF8l%s87;&KXbl}Evf>;f zk0|>+Gc@Wjgk8%=jf^<0yx90GyAdrtdv~MrVedGc#eE*%VFV9(qsAnj$^V(!BX>R=OFY2ygrGC8dK+$VN7Z-J|gwzPX0gcODJ+mMg< z)zXgbLbPmPh=Z-6(S})9aGeSaSJz*?&6~uhQeY#UGxR(N)++OnI3>1?M%)~=$}Evc zEI8Gpc3*JF4sAGu>3&!_EA&mwjhH|g=N-#rwYfvaHd1zHx(hXogY#qE<$!9)g4C9S ze&fk3++0a{IG)X>PTn}zVxw6m@v^YOU%tFFijB=su_%BRRl>%ls-2a;OsBfXxDqul zte{=LWCn$~d>)x|r4PoMe+;>(Xzkqn?9i|!*@l^WHMxqyO40MpD3eNm85*iGgf|P> z&C6(W>LGUI#e9_2o?Li|IgIFo`@2AKH$9`6gTM>8%qDAOQ1OF1HZc7?zp~_9x6q;i zRXR~Y^cgeSVMosW{1!G5nsjDxDSi=i5F7F;{Daj*#II0VF>1X89Gs}d4+xdrGEjkC zRCBZ_xh+&`=hqf1$VEXKsS&hV^Y83K(|x>>d3lRMqBBbHJ1?B?!{|BUzSH@twK%;P zmH2Y!mVCy((`a69I>)hvV9SZ5`NUJ*YVzcB|*o zffz_l-skkMe&ru7*PbO7UqJ5gTdrMtznnwZ@{bNE`xhY$ZFp?%3$ndUY{E^SC+XOz3ZL$EwyBduUa;@Wa-hs4=X>(y$Qxhh3p4S?bgnC{+mZr59bu8#^ zfgXp1(x~Xxl?*kq)GA|jr;MeWQ(zA+IWNH4x){BW;O#+6W53vth_p~L8Q*2*n z1sQRvJX)*@&_(tl740YLM0sD(y(@fq&E4tQVijuRS9HJy1$R95$%gc^dOF%0j~a_F z=U01OFD;(ZlH{_HI$Y!t5|;azM$#y^=-tPMT5>hjh5eHA0U)RC?J9sZYIDP7?(Oeo zy0T}nHX}N;bgz{fd34xItqSpqNAqMCE0^tbo1djHJ_&{fbdCG`2zYxrO*de{`-qu< zL4}RzBLlj0$7s~L(abvfpv~L;ZN8|eR=`~J{F33$cVaXjKXbU{=JYCPo(TxElU&;N zrTBm?9)Z62OiI%SxtKc%Q8nl0iVCe7E~X%H@35#@*(0-yjo8q=BIn!Z^_Qq*?5B;t zLXyAj^~ai|$M=6{GF+wm{O7gdVx2%z$-l(eME42t6pqboeW$BUILW{Y-7=O%NWVt0#9+ zNeAoQOU)3x?7=)gJ0xLo5V8LaABnVc!+pDlIyVW9DjhP4t#eRF?aS7fN_BlasdV*I zy+p`$wFl^DU`bchHFNrpIo&UKRV@tW(iFjoD5>(@;Oj_xRg*@goRNI6FW~^ z_ep&uRp|`zqij+3mdl9u6SJA-;l=23cTTUXx}P&LyTziK zb@6bHh`I#>MgoQ}cBGfck##q4ewcHhhb*fCp&>4pk6q4vj2$|x2<|4DEjhD z!TtG{^E`>%R{-yx$zS#UxxM+}!ccYCqhs2S=vLe`iNR1GSNbTF7f3J4t6AZlyDx-e z&asyL8cx);Qt0aV8-@yiqE@#PB=(e%7K-gfThmJ`HvC)&06Jg5`wbcs}M)|aHAGCEyNo4>s3hCd_boov{sWc6J zaoaqMJha+gOHf!iovEqh^W(hG?qO&1Rv}{_gy_$`j@8_7BZ<*P3FjsLCcV0;pL?y% zt*TU2W{(359X-me4zKkP6c$~Qk~#`=^q(#0Z6ypdFbg&NV9qQPj0JKy>dM(n_;^+| zp`Oe_hVpESj{D@;x>#6@7f*ST?aXQy=p56IA-<^O zrV2sW-oAdZ9t&0RLEh|esrmfujD=cuFJYmawtgHMFpH1JhUw)S2_kN8er71dcAq9q zEyGk_ZcQ(AvhlcKg^Nn$XzqoFCalA|ktLEQXYAp0Q>w6QdrNICY2Ea|@CdBeOjsA! z`NZw4vH+c{diiuUk4!o!_@0M!{84I8Fu71N;v!I)B+u1SyMCZ?^FmyR+L2fIyLEtg zt%>t0S{F4mJiN+vWRQoZkp7u)VWpi%*woed^d1F zPJrht;=G{p=FOX8BY|6l8^t5GWTcLtsg4m1o@94}du!T#U;6K+ng{iQ)wrj5-*yu7@+4j*+jeq%FvkV(uadC?Nv28Bw}LU@xwd_CYTGAnysCY@qx z?AN+o#uULQ!h>g=XU}oe6Yyed*k^6}I4+!$$9A?g_^z=&r*Xwb_(!8+Ia0Ze*54u( zl$X1$0}mdY1V?P-)2=D+&)V@*NtsnSq7u_XLl3jf8O7Xf%@6U*eb$pzavuy+Zcj9s z!A3@5W~eHEiQl&Fjze}Zh{A`O5urP`TJ8LTnVEB#Sn@!CffOfT2&3c?^}MiPpryfh zpEuSiA8blS+whP2T}=F7xlTs5lUzz}xtLiDXHYL?lD;_+z4XF4Ybo9xr(!s7q0~gx zdVI~RL9B90)&a}fRYrFjx;UuqfevlmX0_!_&Q(b{m#zq=>^Cc?{*s13K#$dbg?f^G zBz{t+dF|STA^lhTUvX*WA2{0jrtHqd;a5ziI%AGodgX8KRaeIhBvhA> zcabXhZ#=aSqSk|2dOCfdTwRt>dbS&;PgqUG%$onzA|zI?x~jU&$Mm~X!$SZ28&U!e zlRsWeI8Uc}#bLeAYs<(GN4_Rid9b3B*Zx{sE(!_^$GKd_9{tdUC2X@FyB??K-oj3t zR7%e9`i4-x!|W@P;#>AZ5&Um3J|K}IsK*lMIUYa$ zle^^kDmM?9jOeBOZ8thx!zsceI825b-M3QiHhFt)O_g@6Ywtl-(y2*}YMmGB_i>pg zgafu0dLIxVY52amCC9R{5nUXehdp#BQ(Q@K_6`~=F`*YZOWzoQUFr=HI7 zLU_DUF~bxF0({peB+o0%XkFa)hB%sZT$1))pV*+@$0#xCMBi0h3L|6AEV^M} zKUtaWk#2SU#*NKwjSLyZs8||Q8B~}~!cqt-yp)|!ynw={(u=DQNx~wZh|R5T>B; z=62UpCPRp@X96Qp>%BI+RFGR8k<=G%SoR7l{}bEN67V$7trzZ~m7Vhjdcr7cL?~!! zHOpR4I2e02O~PQu8Rw?~g@8WtvXp0a;OVk{h)8rWVe3vbsJ_MMC1@|9Jlh>B$+NDB z|3Sae<@*S$ZI#gk%JP-D!(u1ds}<8W+PE&vYTuORAPIg3YI!+l#5Us5@YiO)N+X9+ zDW{3aO50`q8MEzQU!T!3Jh!%&9+0T1sz}L`xpKA z@y=t*I2keq(Kwxom{QLWw|npvrc*GU>)`6T+Ytrj-_fAUV4^Otv0+bO=VcdXWK_Y` zEYxwnt!_BPiZ3kD!j+2P(3lGyG`|H3Btwv88TV%Hm=W@Ok6w1?DM)hd#6OVVU!`MF zg4m52(Dx1<7edMMP7i!0&J>Ja-M^DORkP()Y(|7;J6xag8!PeiP}dvs0ZH9{cj}__ zl4C`xxYB-v>0=T+D-~DL?-cs5!*a&z}c5OzExv?y6X6D=WuLZcuv!f_mF~ z4do_UC3kMUt9Sc$Tvw%Y>!_rkng2#XxAZ4E39)m`B8k`dy=ra=&=wrMi{NwIONPfMt~t0h`YuM{$}-a2FtTH8u$ZyhQp^>t|B?aei}{thV+l}dMA zwTuY|)wC}~$Os^y0ew-}$j0yAl>k4gfBE94Fd*mbgST$|ur9IKlr1dD0F>+4yT${t zHRVf%@fg;d&8u&gU&1uW5vJAk)y`+@PNfTgRkh$Xoc&z*bfVn3dNv`&L0d~}A1Val zGOqVt4&&)NyniRSsP(yu%D3g}($bytQ^m0C@*dxaWLa5R`-#TkZEZU7x3#a2CuO8) z6;&E}f)tdMOAjz{iN!hPzLSXOp|6oU=g;6mUV`j8tW$N$QzWhJBl&y3mA8x&*|fXo z`N*UvStEPGae2((detw|y!LzKGE6P~T5Yq(w?e8F)C@^|LD9O$ZQy^R<2&KvdUEcGq*CHn#$MQT9ULAV(GVRJUfx_|fU?#GO zgk*kYV0XnOBzCZUv}|)96Ds51?I`9vP9+Nh^CwZ>;l>OXFRysgD=~%UuU4pdSFapg zPlwVk%@moc0TL=tLlt1LTT;qK99x9nHFii5Meku zUavGN9_>-aB_Y@OQ-n=@jtpab%=I=(?MF@PB*Za4RE=RQw_$z*Jos2y!5lxMa<&hMIBEoc{p3~pCA%8!v zYD;v#JrbMhDX}qVKCuik^(=nIW8u3%Scb6k${~_xAf?WFv{zOdH?Y|Wv$l5^X>Zcx zp-Fe{9sc3jz$n}5z7omyq-j~%pm)>7co~wH7gKC%FKWBgfZZ4i`-JPd6gXA#&HlnA zBuAnDOyVkMnNSgp206l#bfQB(#iyt775-XmQ+6-^ZV~uzy}Sgh4XjX!CRfkQX)ezj zYd3N39ZyrGuV#iaw8ScP4U%$jfY#F${9#_%UsuUHzI=|EW?it(%O<)wfuhJEI$`RZ z`~*x2o8$0~GszNrUq3(4@2uetyvf{`&dfr$@$uP-X=vQAO40@fpO&G4^yFjKzokF9 z-44dNZBp-c44tC9>^$+5GVVCRj*&S`3Py2}7VE)OAH^MOFW)mnbnh*+Nik+``VRHQ z*pqY9Ov60K*tFZ}g`HR>FQf_8_sq!oElr=_e?Z~CJ$)1A?(OBBP&Ub0w;s~c&ejtM z`Srbjw&`}M8NzEr$2Smxbmqc)$j|>AHeyhp-(_*K=n0Yv)+EGV`5g+~{$234N5V?2 zgOZIj`)vqm*f0RbOOKAyz#z4>wdH;>Ej6>UPp{8alMoTI504qf)bmnzFRh%2dhPcD z#81s%6qijq*_~-oS?12!Anv`O;neNO>x@gCZC&T$9DYwdRV<+V*z9;ehg#6vfBUS< ze@LiHEWKR2e_6#ML98+f3gs3Swnom! ztjC)#?f#4{654;A!1#b@xoNDO>P;Utt)v@fd*-K1NTkUAh{*nd%eI{a z-Z5zpxABb_aq8*}%c=yl^&g!!_ZcZ?+D8eOi#ZLaevoWq`$6YUD%m>NIkg#&u9<55 zEiNw357@8-qw{Ikk&nbV3rgTA=YeK^ap6x5pGb+1kHv--5NTr{oP5T7#$ep&>FMPK z-BGiS+iA?tp526v<=gV6^rts`6}sOGwT~aR=GW9DNfp{+HEsMwAWL%zF$Ih7}^2f8LEjkWPPPa=qeNSS9vXYW% z)jrG~(jnooc~nDPGgfr`dwqBnK2_$q8x1$C$<50nV8A?h@F=cJODEA{aV=)CtBb8I zjB*b>Oe^LA@JCRPFh;wmnwJNnlq5S|X>;C}3O{|@opz6l`gGk*zvR(#d0BEcDs!Qx z?r~dk%X1Bv?)^EaU-b#bhTzS>Nu|s3{>KUx((0Doy}c5B#KFF$9V4T3Ym2myPP&W6dd<_)_#?lG#o@wNhs_QE~{%Mf^qMmP_wcDEe$`F zD*9f8eu)7!2?-S$Srr5s#iEu*7=Vj zdAjDqy(CtdAOjGcNa9(%g!e*pTKTC84RS|97_uEp#1hG6gfT1qRTP;eDJl8n$)(O+ zkfJwk+$f)YIFaefCT7{DA9h-qq+3>|i8yn~GC9>_Pbhrd<`P0g4+D6yNAIRg*o$cX<%|ZK4tJ@%we4FMmx?vXL@rS0XB@Ss~JmJ$^jue>V@$vkj9OZ1X7m@$; z0&p39XB6jTno@si#0=qSE5$C2?Po73bK)I8wcw!Mx0wM|2by#xV7Z^$WIv^Y>-71|}`sE>#Daabo25Nc6ha)uvD?G#AHsL?z zJkL%IJR)cWoSM%p&rc)xsUp(rE4QX6`VZrX=tU2f09+G2`4-B@62Q;F$hcXN!`Ejl z=I-c<>D;TUphUUV6Ve(Bog5Bq0}5y^v$XFEd~PJ8l=2SyI9!GU8h}`Jn)@nwn}`Nr zp#mthM0-^*vGgak$m_G@;k^|hIq-d_gQ7M~&3OW;^hX21M%ktnE2(z+WS`}<-Q0eD zsR<@$Cufo}^ICBLRMA6n>3B|E-=X-up2QM=R3FiuTQY8LOVO_a{QBVH|20nid>ie? z3d;ad8Uwl0>y2RgJa!vvSm|_pdnX_^F>yA35ba=Q>wmZ{aeH9Wc4up_B2FzYRls#H zT`V+4bG}s$Q;I&TsGG!e82j=b0d!KD zYZ7Cu{eR3tehG|tGDAL>qRy+u4r!7+9LCctxdTFWvrhZp3|C5%J%heA#TPp)x#n=i zGA9PjyzjIX+?4FGvExGsc^BdGSdy>9VhRYhf)A-`{o&`^DH;PV4gWAV zglCJ>0n$^i)Re11%Ix->l;c9sIvfs{35F*k-~V~}#2E)?x)~dkT8i=Yn+Oxh%p`0o zMM{X+g<{`YJVb~e*rnR*#sC^+%KIDKD8MqrsbOdO7}7n+Glz2oseii{wd-`c^PoN8 z53<`pKHZwEDg}il!?ed~loYksDJS{bAx_!x5U6YKCO#Z1_atzblbqkZ=Euy`oJwZYCul^Q2iJbTx4{t+k#D{ zs;XCr`9dT~Lq_r-5%~;AW0&;+p#!1-a8ym>xrSIOU9?yljq5gP&v3>2j$2m*{nZhz zf7JgEcDwV6OTRk)_ocr7C&?eB=xl|d-Vp?9_Zj5$K;oEIP0}Z=d~ddaUOib{43X{h z5sW#W_Q=!m!Nodc!nYes?noy+ACV{2B^W@POZeRB-C@0m-4Su~lj=Rx_8|KYihSNT zYWVj7YaoN`L9cp)CklD?vEQTt)8I7kDW9>NB8jXS_|7xt_`QC8&yieTYxG*Y*Nc$u zj-AFAi2Tk&n)GWFg zq9QG$HO-XG>o<{!D+yb*=RmUn3<38RXzu6`dn8yd9~GIvo;1+VTIX`q?U}*r%U1C> zkM1{p6qBQzR2v{A0!E@3D>f8bZBsDHGaHv7^3r?%{7};LXdQF?=Jmcqi#Kn$&FWVg zb`Bmrnk9(BwL`+)hbcFV=>jgM+cY7KVxAZAzkaodh(?zaepP`F5PG?BdVaYsp3dRA z){jm?$qRPjzO2#|x7>7()ANXX{GxR$OicO{K1B~9{ov)r`8Nk+k0w6Hg`x?13Jy~y zL}1Hemj&-kqgM}YB!zzc63g(crz+ESl>JhMtLs`fhphAc<5>r_3~|2sRvTKeCgHaV z$V$v-*F8#tTNJ(ERu!{q7By6hU_pEQ;`W@{C=oWys$8?;V4 z*_(Fc@%*mi5kzcOG6M*-Y7)pDPA#UBU+VICe!M)qyuDhjTz~>|t+MS+sau;llv`ah z=oWp)n;|AvdrZE2-^q|L8Bred87E_^Z01|S4AY$?=Z=NQ(kOKci?`rUGH=eo*6Chn zj9Fh>x*Dyw=lW!cE-H5=BqG2{>*>48PceX=asm+x&hI2KFK2C_Fjd``T4QZLf!S)` zLXAP{y%rNee|k|@ThmU6-rtYi;bNF?j~$Sday@zRdZFq2_g@FJMgle`WAqLuhtxLy z^fT|?H-Yb0JD?mBlzfHvw)U5MI$OWDS@ZkDZD&Tg!~ioa-|^gI<9DuO(`s)jsG_5} zsufV8OP7V8$-TY3Mpd?VXnq^R5(L&pU}Hc_IT{|wm6w%0OGtKTpyW2VINrFTxamTR zpHFnRFH4mitV_57#M~B6DX{aZHMp1~vDgsXX4X-j5COOc*Ec*D^C3_^Ud&zU5lyk` zukrk0#g1`+zW?2MIn$JC{n!F7uyO-ED%5f3iPgB?N))XG+--dWuu5hDA`|D!gjW4& zV*6+wTCuA2>6~JwJI7`J&)eR`Fib|uEr_ZH2HL(v*~C6D|gMwe|?cF%D6-Y({}T?!t&Bm?L>ea z)v(BNd4?|a_lI*C3v3NxndQ4B`c3>4GnO$qpC*WI6Cn}M-?h$7MV_X0E?et)urFqT zpruUcoB8=HD@l(EtyS61xV^m81Z;a$WaM`rxqZSvAHbIg)Qn$#bUx5qAbA1g0q(JKi%Tf{}->{H@)Gge-$b4{7-L^!8N7YCuG zvb<%eO8}4r%QOmfs3O{}`B&bsfI%Wn%Cid_zLZ7sU9y;BPL(R=UTUvqY57WMREd;H__Xir;PD#N(Qz#b@V(w{`ok5JeX zXLA<+z2kaYb$t+nXWzapMMq2TKW36NZnW))Y?28%;Lze<+08VMW9h>j9C#npZLAc7 zXz)U9y@Eth`=eKi$8LX)42|$FFd^ug1+oU<{Tm!mCHg+@0?3MhOZ6(0xOq+aL&f-F z`=9^GkaHC79G+>^m~x@Zm?jLqznsXF>6pEb^&ZT6ViwS3IW=m6-I(f%WxCU)-O?ic zrZ?^D7z(r#DevAn77l%4>7(06sQ}9boamg5JGVkhOf5mbG20rc0<=E8YT7IRbaM_6 z_$TLP1Mt!&-&M8OHQeXrH@&tdu`#X40@J3A%UNn#;yKTg-*qh?11X^8K@0HAq9!K& zFed6252#tSTOWvmC@&vt3yY%zhX(zCy{eh;9{YbIKu`=}l&IQTUUr@#FLRHMkHM4p zwB-J+jEU_nKsH8S9#64{QO-pPtt)55wm)ggF6Z-TF*FoDj0&s z!8im)w}A|?^|Lx7!T!uS@%h1kgL4l zG47ub@|z^=Z(vj7Sx2&oK}!@Nv>iHGCJ4_Pe zx7LuyC808^wS+xN72JAU^TO6BMJXrjGk(-3Tr%l+cHQY1+FQ47RoWrDeH;7(Cy|cR z{`*UYJ=Z&XdX!-!RI0hp0#B{0Cf^~%qM7a(1JUoF^tO^ikDltnK!#a^M_|t3zaaK! z=28V9l`k*KGfLg`U?T+uHXV_4(8tb2GggYE%w2M=!S17_Q@>%J`V2p38bZNS?k;WpvRnLtcL zC$8f6r4T(lfiL62$ya4$^7RK<)U7&8wC7EH@h9HBFC*k027TvLPe?CM`vA|ZZ3^1r zkR9(2{yw;>`e?ZQuJI|x@3*qAmr=+Uq4HpP($fs&ayCfS+EKk4a(Jj=mAFi$=W)v649PPy=(kKc0VVBEu zK3Ue}sdpg8tG%ip+y3_Lw&o|?l-gL|(Ctr?lyaZEQvW9CwBgdp+AIU2E?O-fx4(<| zD&}h9CG`=4vMSJ8{}x~4mn_=4zN{)=Lp*PsipXJEgS4@HBq@BACljMo*0%mI)fYdS zKkNf8MVuQoj2zKfE_)FsGcojphjf{6x~$f9sk4+T#eEQkjft5jdjzK?d64^7Us16u zV79T!etjc>1I~{G)HtWUJCJ+CJZ;2YkJnNOWOx_&_3&CMiIflACZa1bfFA&uL`t?p zz%Vp4M1Ss|TW+~n4xiAX3+AeEoKaL&-8>0HtA1ynx3t0 zJ`N&I;xXee=I(F&+10+@q4i6Mye3Junxl;y<~KWSnxjNfVb88v{^i)D$RnBMWd)Fh zxKce+Aj(4=Hsr24V4k0?L!TB5&-QpnfKd9Y`k7R>M>J9sx&BNTE7#x;>vrFsu2zim zrWUgIxtw7e9MlAjF(L0UfI~JdrbYwH_RGzpZ^;KL=2awkz{?rzv1_L z^o;+fBv8Fit3NUOlY{7?Rr#Hi+1XhUzv`qU{lH-NYR3uPiZ@j^Zu=MNBN89drzSBe z#YTWf4DP(rYT$kO@&X*`^ZiwJ{mLox#4i+9`cqMyceTpof9p8kG_VjQ(eApU8`^N9f?91=d ziQn{xSfxJ&*^Lp=k`74f`^}#%(+BLAFpE@su;R@oFUsBfXCSPLRWf>J=bHdGkA%0! z#;;85*JtLE#%IU7BBE&vZbgo&8JZDbGHQl%Q2uyA)ckjJe&o@ z(Z^%NJ%4|H!1cPO3CN2NIk<>MC<%LdIEl9uJ$&aI>`r()_uY#Om&izFD|gM9R1w+c zxtAPA;8aX$VgZIr`Sg>ql%*W$J%({Q)%FVY5yd28R^>0)^gLMu+A?2WbZ3twi8*z+ z?qabKp`o_OOx?k(GCnKLUhm7m+eCdepinPMZC>g!vF1N@a{4vGf%obaLt?;$q>W*% zlM`svc`6SA=6+-z9T9jO9-~TGCMzB2jG2sz^-@)GVftigr0v<#G6bi`>uTv?HPfyu zZoUAzdl0dV6?tX1hD6RbrM_MJ^+X;{*Vba2wKQS<-&KRvM_qE()}QoXb#CkB$1#uM z)`>bU4w&&|W*zvcs)!lSk=ii}y*(ALum&z*cwX%{`F>(WeDnNQN!r0(FCVtrmkD_P z)DW0^B$6&`H|AWqnh6-VP0k+j@=p62$C@gFnYgjBl3tZc$oKiBnCCXP$^OS>e`B%u zKT7SVJDH7v%VrjZ@n4$`XN{86lwUh@lMQd5Rc_?@rkb3p*KZ@ZpC!+R!ifdXU9LLK z^a|^kAc-8(%Qd{Y%!Du11-VVZ$-4XXZ?3+{joeziq}|N!7KWdKCZuGzNav)pO7?`otc@r6o91aL)~0S zl@Y5oz9BwfQ1kP>C}2}+`7R3^srKLd$gC{Mqr;kSP&N((6hO#9ey=?v%o&gq3?c>; zIiNC(DqqUJHR#Ni+A~jJ(%iQfKeflM58vy}z}hT#FB4eRy|lM%?dWJQJ)i7JDh~aG zdzXM9PY?pp3WXDQE`YI8xYp&b zvGSP|ziVITBNj+}nj9;`K=8LI1>rr}M9`DUbFyciFT0VfJ({kmeHip9EbMKae1}NI zJ^cc$vX{-*{Z2ffg6^|m(5o_X0F7j_%FgSw-vs?oKJtFRid*9`LEIY^Ghv_61u(aK zgsPEA^Fsy=@`j5~H>Pf)lWdwQWnX8Dd7PjQ34T*yQv{GN8d6^Jr+|5|y-{Fk5J4hg zyZdVl{KT&9z!+ei?2(xX1IQhn#sH}TpRtRT9u?d&M4LLmb3OXKPcHRIgP%ubq`QgE z6eWjmPb+@@j889^6_CCPtvcmY6zokAfCzgHaiR|iM5SX{jVB(Xfp}#CYd_fXk%K>Z zj5?3CZ=a+buERN1HXKgx$L~6O#s&ah^yU>%b(S$C+=>j*^C=xZ%^c=o zv>Yo>?qYJmxJJlj+br+%02V{KHX+v+#P0$1u>x1>9(}d?t4IQkbfP>* zEnU>EF04=PGtKUL7KJ%7<4b%z4VcILLPlvttNZV5bdgyZA;x>RA52)nraJTfLfybq z_we1!gHeI2iEsncBM!6wdn9abZiDg6#~v#?OSvs~K4P#TwG0q1SU#Md;S<&jHZ@EeTN!odtCY@ zZe3b!z}%A9Cq`B>^LEy2=VQMe9j*AT!9pR}Xa*a(Lp>O<5HbucJuM1ETqg@SIBO0J zS>-trE;YX~eNBdW&XINdEaY2mt~+XIpr;4bCfHXvmY;&0{}J)l8;Of|*AVE>K`0>s zTXz5?=zWDBG4gUE0$t+eZxe|p*0R6S%Ta0@v7;2ZII10DMi?QO23oF76I}i9KSh*x zjV=G&D*H6}|G@pq!$6^Umo10b+b%OlBtwYkMZN+S>X-^_iIhECKg2Oyc}oU=*^+=Mtm7l9!u{$DqTWfIuJtmk)d- zTtP<`8`H(c9(WAE<_y@G46kYz-sH`p<&fi%G5#>9|pP1e{yZw6amz5^L4{iS3)dNtJedK+j_xj9y8 z;Ru7r`7ix%FXsY!0I=&Wn0Rlj?FdXCwF(XJ0&*?f2=Nn%Tq!6296~MP0%&4e*N1RJ9jyBhZF2{TrY9r+=+7 z1Q7!fy%C633_t&iJV5v>4eZM&)xCNzz(1t|1wlo|O~8Yv3ikO+p7n&1S@H7m*^Lx} z+6n=n{eRH*fmcprASBR;2t*ih1!=%40v2Qc53ZZAJ4Xc$G0N#oJ!Qq-GuxB8R2wmq%29$P1!^E*1Frr3a; z2twRlF~5x$z=P=#2<3y6o0ifGc`Q&uOJN6Y3A$#z>Pkw4cx+5arYI*6FOVjQ%`N9v z$ZR9_XP^;Bq3oU8Ci)pFB{F zq?e6ly~!k^qN)tG*?>U~AQ(X2)I84^DTuQQA@pic9`gW`1Y6`^^7lL6d5@iG5^1-8 zL(K%e-JP2cTec%QcVl|82PbCa;+ol)ZE;797}dT!2Bvz;p0=ⅈgc}6wrqVCT8YK zFZ5`64ott~Xy{BKu$g%7hu915zg?$t*_s~s;M6u%qpqnAaWlB)uNk>&{QUfOkm_3Z zVjo7*99Mz;PqzY}!A2y_4kyMgE^IA~*GEuTGO4s#n5Pgrq>enAsoQyA zVQ>UYbFm<4vW63ErKrw>0vBR>85zj5pL;twx2`J84?jT106Lk5=6AML9wDs}5f%SR zMH@5~G3q(_+pA`0+Ry5Ydhkh%ZIyTL+96jI5k9*Hwm`|}fQHqWd;7HMHnm`(9bD?a z@-{^Nn1(lWmcOWVL=zrn=7KGhMqGfhE5jbyr*6uvZEbBS+*i|t`>q3o=;h;c`{p^y z?`idr50DL!0SIxsl`KO(N<0RzRb?={L&&$n9XlyI0WX}mr0>B{BB2`cn` z;K`SOnkfrjlcp4n%mpc)72xXQ3z!|EoZ=UAXTRusBGc=6>m~&dd+WWKGoJ;Diio;d z=p2=0cL2|#QbNKPK67Y1!_Kt{Z=|d6D7s9prJ0$~Bm%Z;UvKJP4 zEc`C}L2)IQC%Ae*PvNd$m&u8HDn12amHN(Jhv~hYE5(vi1!=2MI`-xhh zQ0a0Yx(}}9SV4;={U$$5R7e)TsPmoYH0zrx_GzedP5k2~MUiQCVVAY2=k^D{Oc^WM zOjcg3s3QHuE!DvOsB^b^4t=t$Y{oG12omY|Fpe_w(+rS%C#R+xH`K6M7J5=mF#8SwK@fIMjQTP-18 z0M>(TlhagXBI!@(T>kiX2@K({BYoo*aQoyUscL~-0=B;axl+w!(qnsC@8W2t;ZPk~ z{U`QSA@M?vpu~;qXL_2;BeL+NURwyKFkFtajpZaf8;~&(i-29Gsdgd`D3co`VKEX^EaK zU{Qihl?n1h{{32=x&Crp*%W?T2rA~p_B?^MWy>G`AJ(79|8w4`bA?cGa5%QBHWQHy zxD3ymBRKZ2-Xem7d<;IchZgNRcgMxsHCqj`Pm0 z(t0#!D_la0J~!pE!FM91xiqQ^30|*s90}ma+v^G4r^i-J)W7dZ-|tTwzI6LpV{X^; z2A4 z@Z!9i`t{eniB&EuqrBY(9`I_fatb2js@+uQittiimnA20`-~K365dg`^bTL%1Mdw1 zd(CqRiTGyf(=Yl)F8dHCX27c!?vP?vl~P0`NEHDsHL7v%>Hqoj z6!epH(W(OD9T9Y{R!&#|ue)gs<6zUNTU2DI+qwu2&ORO3X6T#$SKVwbo#66Xp5AxrKIa zT*p~51rS@EvZNMZ2Ut`6??MwkR4%WjvLE7b4lFvAKuIe0V&GosWSRNSugr5$Ek%6S z#+MZNnwpxx=B9e$tjS?@?lZT^Tu4n6%@erZ7S&9>C#%LgBJ}kii-A#=*WRV>+syDg zZ8QWU)tK+zPpB$4PpsZocEGYr@>FT8yR9308gCzN9lf3`bbru5bRY8nbaQ4wOec@?7?| z=$aiW$Qpxe3t}u|h3g(Vt%CN({j>pT7B5>HUG4@4gEgj#bv+=$PgjHatrxsC zTKOW(R=069^P63yE@^e~F`&E=%Bt?9-itlai9u=0=e(#l&+#s!-daNH9vl$3^=!(weHO z;$(%p`X3#Sj3h**oV5#Nzo)HWowPx|zH=j1sQHDIXKtB#=oN0u))Ou>c&}-=3@h55 zB`xkX*UWcSw-P#E%X0+Oj{}{T(gUKTp>f^PYZw=<@FvFW$DVEsn+THkI1IEVM!cBe z&Y2MGImt=aXASrFXdGkTkVpfiqY3=C`=d4c1-Q7Cylw47#9$GAy_gHC&b!JOEmSJZ zi{{t6JC9EfRmv@2+nH2c34x5r<#HoqV=JGSw`+BSM`>XK&G}i!*wE1ODKw|woNr^Z z`qPnPN z@+8a~p$%Mt__xZLxe^2-m$#@~F@goyS1L@-K1{aS|L4Jy$vIBL-fuVhW7n0IEebuH+iF-~^KOV+F-<@IS;MLJu zEdp5jXp!BA>CZbFNMD=is1>4` z8XW&r07E)Cf*Q*U_aVu&IdLj(5k&K-90EPR8hCQ5%l!6X%H`g~1gt$h8+u$x4jI&l zU^9v_-%pUU?UYpkRj5Y_mz0TPS%dSnyNeNuu(JTA3jYYb}#4k&$N?l97}@lNaLq++MCw2tIcn&Ym57Kwj0hypxI z1vnLci6Wtli?-53yK<#6R=I=l4^)MK7mbM_|j&P4&mW zhX|b+h%~<5{K?Mjo&D9^A|=lk|Ve?e7ax Y>;J(l$}G=t+^mJWioH(iJv*DXC3zm$HCgJGWFgZQ#WlR z8CT&ygM(B4e4YOs4(@XWp{WcU+&|BCFyY`%M3P>>!3lh$20wrEfA?*j6iH`1CK*K; zJsx4xi6BQL3f5yd(wLZs8BvMu;NCgesE@ul03UZ~;czF40 zd0pV(u3sk%#yMS}o=ncnm{-jY&|7(5J)NFy?yojD*gKZTO;qEQ%y4pX9W6fa=Sjo; zGh1<}bJk}SPjYaGF3Q{FxlW_lLWZTMtZZp!7WN(`8EVk%qhVmMTt7_Y<>6uj%XtR( zyT&SXl-ka&bk~I>XzXsyxT9V8`jVi(y!h8+NkRQg$LjBAfzN|QDMT%y^(6BSD=dcZ z8bYP6VjwJVa6Sl!->4xSBc?Sa=%O<-Gl7Vxr6naDu6Hb#kOqgk@I$9H4Lk(rs9EA4^qAxxWz6oY4~d(Ki_ z`FC)o+-&(Op|Bf==+53nnmDCn_er*okUw}HBnn~3F`WOZyTq|#CU3*hEHeTEg2D3g zf}I^4+>f?8LAl-@TFRiQDfPocyHHdDY;2y_6jPmp6K!(av`SeW!voZ@(iQL7+q|zh z*x7x~&Uw>@-5lL+?e-;Y-8fw@mc8GI>qmcKdIfi#3Z5)e7{+~h7?lwFAwVfpsKad? zH&#RhDGC``9BF!X_HEF?p&j+GZD|?0C}&-r9U3TJp}=|z8HwBLyQ94Q?p_-5pvb65 zFRu#`r1=G1tg!CBn5;7oA7qJ_6fReWuSI|6MmV=1f`=Xce5lC_4&ne|0M@JLIktPQlD*ta2ANG~wM zuz761ny!8Er08IRd_fNnZ`V9Ode~9B?d7ri)3oqf| zeg}Z-h4Ubn1BZ~bKMtG)#A>Sl|KFA>qUBT9&X0mIy#26lbVeE^VsI)#g(PPgex2w)!})^ zQdlrE@yq96Cw7LU6kSvvci+d-LsBy-wKz2*V)DDn+Zl%?O$~!6^i;05c{MhD!7x#( z3}Qhyc3M^kk=4&FS7b9&Gj4|yo)bY;*vOkZRq;2$!M+c^tdOP!DAalByGk`QwdJFg zuh{dMvb-)0#-x**`wVY1m*YJl?=2;yY+uAT799igwSJs|*>_cGSa2u96n1l!&pLub zLL?>3%r4U!PG89k6j|xB%nlC5iBsZWW7{uQ^-Xnq?B%_}MyBRCh^wnh%nsi?_qx*4BddsKrLidUv|70JpChIr3<+>}e!auW{t1XKgAjBe`>KU$U%$pkM?Ci?b zGgb|+iv1HO0b6QmwE9^hm-Ki&P;1uLH z=F+V`%SG%t++%LS?d8^XR>9@Y3Nt73!+ZqZ9O74KGE*xa79(buUpjl3;;dt3gMy52 zLflz@U2cSFIsX+IKK@S3D>P9gf}MR<-AgD8k5z_BRYT(<@0E#fn>R&}K(l)fp^2C} ziu3!XXG5qrI&x~0(8=}FcnhUWK{8%rd{Jp8+)0)WG}Z5@y*kG->MQhx`6*_v0ekq@=}YmIWfD(@1pnHprvjv6q#^_ z!uB+JcU0asH@X(=?}$r43RUUU892Fw*yuj}bVX!L4e|2ylmG?RmSG2Dqo-lG^(u@W zkr}WOARQeoY_R?2lx#*7uS5+x=ji5H!4nTJUz!MWsfI3Fp{3w-Z= zJq7F+a%mfmmkReS-w@6@T#t1kQK+fK=SOTwi5v3zopHoUJGnVk)>c{yy-gZ9T8WX7 zSIoDvbUEHCFOe%$rjx@P6~^;AwcU|mPdS)riqFn|CTeLqcH*3k#~LK{dS}+6|3`o- z+tl7D@`Pk-H(cjio)d7B)|17k}DQ_%_Q7# z^Jj2A4FHo|%k_QU1|0w#R>aGD#oY0B7bdvR#~4d}sGv!=Usi$E?4$GU|1&`OH*lu5 zO2pmT(8Siwz>vD*ThIOvsR4^KCISX71a$}^Y7Cn7A<-IlL2Yeqaj`5pI#RGG4i1h` zmJ&^Tm^fvIA~ze`&BXXKxV*Y%nz6RXW)v%8PMj|lfgvSLgkpi+KVr3AMco2VbX33# z_`X%*SczZhlNm7~ZFdIXM?*tHoguOF%(q{A1~XI@<`mFLBU?S~T8N(HULlg1=xWRw*Ym|IyD<>Y*s zEUm1pEGhyW>BHF==;z7P;Y>GOnP1A@Mm1~yWSG2mG@7i?8qD3+^AopS{o32xJ2WKE z&(E)*pkTHzJ2kbjxyj4J^RuYa0uRo|bI^Q%>=K(e)zZ4`$%t5ek!ry);wG;$R)-cn z0wdt_xuvPQ+_c&A3r-DTpSR})ku5x>=L_&9O4zsl@G*Vf7ki1hxw*~F&1cS>92~8l zu-^H30?7n8A4gfUfeV$*fTM*tm5r3VJ+U|1P8M%5k(}XOXuWnnI!vMvRtQ%-WU{tF zT0M-+%<}W|rxyrptgVqzQ21T<&gpaD;pi$hA@Ix4pIBeNQAs@BRPO!wf*ppA8Pd`X3zP?F3iL~%UXz>C9jjS*>Svw_`kRBQ7#SI{%UqtAU}t26F6n!fIdZJJ zfy$!rvaP4HwrUP>ae2~yLv~v#JDT7az(p0#|jXm9lcYK!J=8WFs59BoU zF&z~fhXe1oHRuv#u=->;q5VV6cjPptTS`=GT3&bPX~X94@X6+11$O={E>^8}gehog z8fNF`%jBz?r3psLd0gIaF3k&fX^FwnC*_084!6t&{ZUovEusS3peZh>@l6D ze=jE{wjQBXK^sXF)#iJDt|PnmRt^Q0STa*mRPyrr=A+J!xB)(|`wj494To<#M#bo` zCoF4Y4#d^eYD!DxQ<7BE%4=&&tE#M)EA%`B+MAkoPqG7txRUZ3^CTkgV!{RM_Qy%cXKo;Fq|>OBG@H-7s>Ew1%yYMGXRo26qVaawXCPQ# zYIU7mS~}R?os5inT~ob!!0#b0X0Qy9`g8w#UehrVJ)(Wh9$3n&U{Us_DDKa)-v&&x zm4C|16)9{VY`kOLuz2q4E0jfsMbzQjm&x2wK0PzjOok;#g-IJtw!X2WtTG_ra&2j; zeLXdRCB&`>*KWkDU#`2Wf?xYar#i8L{c+L|5^tJ6duA!YgxlBMdvkXqU4Y%@EzJJu zdsr1lfXF8&mz`xOx{Yd#Bq)y@Lo1T7m6%{0WMIlaJ%c6pbL}sWygN*W>*&pkH?`?eNM|^@u*t#sA zffUuF1so$b68f|G`32$!lPptXeV?^>i&4B06e|hStY8uH->@696uKK9p{&)jclvIV zoV1u>9s)vKmHCsql;gr_>g4E;HCb?QXSMH=YR5E+tgk=y6qc2rtk^Z3Tal^0D>1Yz z>efDc?C5Ct@E2DQ5V+RmC*C9vuF{OnM_$+SiSp&6i zUUWaEEi7>4;l#m7?0XbaDBN#!JqkiUJK9YA_Wgd(l_q9%6w2C2j3anr^AHOM2l;z$ zxe%r-B%<69u!FvZCN5;m%KE+vbZof&M08e@BdHgNTTK57F{r&kYe9{xQ09bx$gt*7?BcTz?Jk@h=zkae2)gFDXG zgqA%TOeFUtZKErfQ6V+<<8(WDwStxs(6CqrxzvX6U$t zySy*-3tr3dyDB}L4lx(LHZTB(Yw&P!Cw9->A`#)n5ST3ngcJ#}an{zKR*iW&)x7;6 zAvqNiVK09D@{4?xN#^%8Gimi_k!;*j8?23uo7({Z#iAmsu2y2S!lI%g4Q|Kha6&K5 z_T|tPA&n-rRe|H7tVWLzNOgb?ZNgOYfpt;6%a6>wc2mh$?iZWY{Y|T`6oz0f$Hp32 zSQKWDhYRDly1Mf6@+w0xKUaKUv`&08uT^Q(XvEE!-(IUQ?6Bn5w#Kg%`yEW<(o!ZO zA|mSJ!i`TnH#axRCB&aA#P>73VKy!jZbx*61LyDy`v<}Vx= z8jp2+P!RIJ_S8(}&Jim6kZ|bUkB_>S-@qEH?wmmvOpz~`++}{;L4-TGsbd3-`XBc- zMrI=vQ0cG!`>6GQ3qSw2PokSloh9JwcW|sWTnLC-&=5pl`&_Y@I;Kyr!Et}(xqrV~ zwb|XsI_+pJ*H~G{VYAnRxy?5GYMxxImY+_|2P#Y%1!1p)4S>2652#p#9D9l{4oL_t zTPoaQYFXq~T773{3mY~LTdsNvM?<1X1P3R6=@)OV7LJ@X3N&T#@GdO6R9ZXg=(D*0 znMf={K7Kz}dvUe47RO4`b2aj7XsBmCO!}TmOY~{nfS44+^{KN9{G=hQ)aP$&)+Zeu znL+S|ZaO{joFz+Ll?%0b%ht8J-Wj@bT8KOejY3)z4r%E=eP@qgWM;-f4jv4>^2AO~ z@Y_X)JKI^A&h?&En!53K0ZKu_&V*~b3-)S(rs)78TNR)T)DB9SycMF^@~J$E#X6Ga zGs}o<`&tzqm->p~;z)*`z)-;nuU@QF$3Sxf}6v5Y7pWfvFk)npqvtZWsRl-%5TydfTI zxTzMW93SFEp;f^5KK6SZs`lRa8r{|P!iL)(OuD8XB{5fUEkIZ;v^>8%sQ;pdy3u07XmGHf3{Q{2|O%|*FU>C)x>1^135ZA;f@Iu zC~N$9Ua`aF6EovzXvhas;{+7lrd5wZo;2JQ|BQ*O4h^aA^_P~K4jvgd^+`HwOi$ze zqO4wNrN%@W8HqLtl(M#-02q3US-0JFlF#Jr+n})r#5X2baa`qk)Gae{e%C&J;-)Uf zPRIcwl{Iw)!ZbV`KTd{jDt%k(O;FKD9wQWI352<{FS~DiXIC0{J$ajMxQ~~J1S&f^ zvXmC11tn;wHh=Ei-Xa7je~*jfavP$fl|q!TU=gKzzpt>7Sn@Jhf{$+@SMqh=W2W7M zotDLH^k`-W6CIQB&YtWDOrc~Zgn1gMEQeZT-8`#&%)XvQaqn~ zPV0=vlGwUj?Yn)ZvF`*s1DzE7HZZ&j^e}NBp{Hcd4xpUSU?cDE?Ex7qH6eYW6)Rhr z$MyDLtWr)iyRx;___DR9<%(ntc7uwFDwk9d9ifCB(PI@uI%?2(KD1w0Kg!$D<;6ZV50nO2fAR-#c zjT{p$Elqx@PRA>Lg!0wa8nZjaY~{H%7LSZrUhq4k;JpPShJhiaqx9zDKKPUE?YRz& z%VF_!=QMt84YSER0J;ef=KaR5b7@ILGRralC;UZCA}%|uh@M3GE%2X71tNja3%U%W zR>jlZ20@j!zNVp&$>sf2tqb@XsJ*Qc&_O6;km#S{x{k+-t0LeoEG#qwNZkakM72fL z8jq2WE7f7Nj)?2Ych6Q#%mh$`*pA1QYTrU>s$aMC^vpe}%PnLzm$S^(Hp{IgY8GPL zwhNjn1K45WqmXvvniA4{;gk-K;JnUG;pj}1HoB{R@OG8V+O@h1BX}=wnJA5~bVrxu z_9+Q@Kjt*22KsMF8a7?t_sD^E0#reQnr&xm>u%%`h}mWTtjN5Q8zyGFV6$t!GTq!_ zTEkIlTH1qv??U-%gFG=ep8zET^#w*uAOA^TmKXwRZC#aSff8oe`J=C+j_1p$g7MD5 z>YmFsZ!cZ>Vq+0)?=LTo%!Ei&@y#uIUhF*nHqtNo<((FiOCxMa!6 z$r&Cl8*{WQO{FTT8s3yr1Vm}}l+ua{ZkLlcZ%BJh4iyw46(Xb1MMt91UWk8ka}q~# z-P;*5*P5Q5rpJ1(xXm@AX(%WF@P9-NRz|rR6|9F)l+LnS8R(sdwsv&bgWablL&L-U zrqvcPM-s;?C@|XXwdI^EUd!AB3(G%!CX^Rf&%X296S%38H(%3sNJmkoco zrL3X?0Oq*x&?zR;AoMI&Wd?IY#^_m@zHX#Qm{4U)kgNMhxHg|E7M(+ zA*I}BW5w7TsHu|Lt<(pK(-096DyDFdQcNuv{3y#EF53E7@#@8w=xDA>-El*t6PQk$ zA6A+{VtjI8aWO+3-gl;Yz%36{0ajpSOj&e4=v?9A;M!wzNnuI~2?=R#CRFrL#g>?6 zg2Ppam(2IW7qkTW*w3HuQkY2Wm?eC?mOzJ;S7#?kzaiMd|+!A z{aB^M&c-5iI#9EXlD4Ty4~CJjLYjct#PvS+G4Uhpy3-fPE^Y1YLcy{+RW8}4PjvyA zq=7xaUY~BW)<40;$yiaw7P5^08L&MHF{mIby&gJn; zqqhpH&_Xe5Yv-euu5P_~mw9?G@Oe6hhwtp56~k}#XO7^_spXUm4b^J#A|qb|XLJoS zZe?rZ`p!#^SwbydQ)eLIPyqs2?M>z9-q_y4g0zj;T3cg=i4XeomeUpc6TZWDt#get zfT-y~w_yaY4VW;#22FHms0Nkiic!V_C=<{r92O@}6Cv)@{*Ucxzb))f_Qp#0s~-jR zZif|my)O_^N$#H~%BD0}5-19%atzNL{k?y`kDl(liGsc<9cRX>XvxXSHt8Gc-md2* z)IRz?*KIlIQ_2F3B7VSBP%z{GPFy2#VQ#M5xW=&@a$s9V(A@k7LuHPK?mZ(vrt z2>cyun|-f~1C{e5UnlESX)4gr%aGwf#t@`(A`w zFLh+I(=!bU7~ZH(rD@-hFB7g_iUB!p#AKEL7^c+J`R-)h*PM53DM2}HpV5&|u7Cj! z9fn>V9a3{9r`Z&8JFFP5x3u1WnN1o0bL|LQxmtnDc9HUYCgID`MN-|I7jq8MI;#ti!;q0|*mT)O$57K>h>~*1-ru zqChL8xfnOOw*xXzn}8rCWo`*Cah3Kx-Ghw1VS7=SBNzpyhfYqg%3dcWVjy9yO;m%C z4E}JDk;-O*#$hiZ0S3{*?(T@>@fhMh+H>3ueiQiIC{XE!fS%!a|L?eEcm0(VHKbCmJ@~V5l$&31GB*`Gy*klf(W4lGUM< z2@K~nVT*!|+ppr3%&B-zTpYIJ6N%W!`DxyG8X5qC*v+rqfCAYT?c$u8t~w^rBTA$6 zdM(-FveHsG3w#HFGqUrGc8`t<3PQM-8eMK=ua1sPBwtbl5fY`L-Jfm;hB|h>eXh6k z@&)`KJH?|W7~T#1hYle>-{<$H{93hm=%PpV8Ka>g?)!V?+1V7F$z@R&JHLMf`V&*f z%aoUA04cVlxENjZwZSqq%64!tM9A@$I86LtV{L8cqf)-AnE42g_v5dqMfXxB=DRD@ zuU{jR_LQyh*2RMRZLb>O}|*h8RBWYsPAuTG+fdI>U)x%!?-CopLf#&#JNh^c*hNKJ!~Xt07!vD+FaJW3&!D1D_5NRBr_iP>S&?eX`V8@sgeO9 zWcUb6D>G+sVxTC-(8Anqwe_*ED1Kqg?(X*WX2oJXDo}t%gAEf}U0o`Vv3}F)Mf`c| zD?fjeT)_?ic8Tx0iueG-ZI;olwgLYEXMo7g>MW$be3O>bY_0Jv;+3s_N3-56OK9|c z@UnH-^X;matDiO1|7i5yUG*%tG?_Hqf;llgF?GK8XM9axondo(S8cIgT*BQmIUzyY zz4BLu5^UZl;BJ|NYE)C9 zfR=T&TC8T@fB6MkHPB7p1~uy211vs~XoD_FnfJxf+7Us_zOsV*RJ zHdyz4?nSmprOPIv1#KVrL4sbhj>?~%QO!U4^~LY&cL;i|!qUAk362Zy|1+f^L^YU45K(CaudBvw1$ndkk$ zZC)bzKoh^1f6!_>eSY%wXQ<;Te849le?(upP5P%(?rIi$xgBR_R)?bA`cpfjg__uy zl=Xc?3jSrKMgxeVrk)n}-F8-GIUv$pc4y*OEE&`q=Vosr8-wxylRvwG2X-wuVEBah zHbaI3romF;fUe2V#J751ye1|1s?gzl+>S3+hOcOQJ zbC{KuOeH7malKQs&HA3bhBBwz_^D?bCK-c#2HPFphUMl|6bgDR%}tqF8FA5gZQTd0($Ej5qb~L$8dcg<$*a1ZrnN(w{XuncN)X-q7FE=~x znHFDCUOuu7^*@-+dFjk$_t5686*#>Le97$C}%1@huZIBnjmMe)#)!!$sqxg1t8lD}d9 z-iAS|C4eOdY5tU1R3rmalzdJ0YyA{>`IYPhOt^Pf26r@(s7Jfgxy8j^AVVdZjea+2 zU}zZoJ)O_>%mXXeAIQ)m%Brf(Mu)8OG&WsZhi>&;`xo6Yv(%m%rx#&(q*o&kIk-!U(oqnF9qDFK4y0K za5~@CeoL7I#y^kO{nSdvB2WMv9S07w{5iO|!oNm>HaHYg*YP6e^@dXuvEp$0{CiWa z_iVg%B`tdWrMsF*NltHwXd>z?p0dN%`_7Mcbu-?H&?3Q+DP zn?(Rhg%z)QNy1ZY)di3vl~sCL^6}2cr}c!ebf6+Hy3Nfm0P7svEou(Rp!z$~C-(pcSj#1|p=INO+KWlZ{6QbZ=;!y!P)Zo1_HkaLSnqP}mW zG?gp>VEsd>!WMuL+&b`IMH9W3Rf`Wn2M*c(@xB;2I;pU4QrvAnh%IdvUs&&1Kl2k( zz36Ix#_Z1`LmBHKU}$d6l{MN>Qx}U(Y)5pTIwT+@%tmhpqIiURdmxi{{p|z2)$W{N z0{D9B>mPN7;Lb5uHzy`0CRM*|p8T{;+~{7PJ>+d^M~&@&jKp?aNH{sC|A-;r;I^4O zJvD2OyU??}S%2_pBRQ}GWCxJxC|AUEvpTnkfn^Lbh#P64bu(q$-aPo#oZh5uBya$n z2RSwbL>0sY3&cWkpT}Qokz9m20`@wGF;P-jC_?w`Hu~3OuHNUYnBwB%?0|rbCvnxU$`-=Y=jcOTWfdg;au_^TahQ?Ck7E!h*UUC?=G-miaTnVn($pK$Qwak^ewZpa(E~vfKH&H2oka40sSIcQRFc#gS5X`k znNX$4x5<10Zgmsw>kDP&T;-}KJc4~6s>O?kFos}Pn9c^aimo{?4qklLaj4Dt5~b~Y>Kd!Hg7^sPPf zyoe>mP(_EHA1|=;WA&r!S#9wb4(kWpJ8+d{#i=Y3;QESW7+EX6*?|O#p6fXw{*J5T z{!&FuM-Br274f_G?;ou8#F?4P#6m$HqaweE{%&1OfiOyhy!KG4Xs)TtxF#y{brlF1 zw#2WoY6zLf7u)=V3VHOEg!c8wB|}<$65~?qeNMl;Q5o>I?c^@lWVFXki^_QCLPCO^ zSS*uUSC0nkNTRKvafUrzT zw6{M3VS1LuevbA7&t(4j(IyTSQ5Hn`D>?%mldPhGS}hZDusuG}yoYH52rzn@&XzrPJZ z2vP=f#Y8MOCzI{jV|&BVQdfDkjkOI~Z!dR;3!rE~90v7QS6j=Io_a-1=6AL^)1+qX zWUXMw{P%dg@&-d27}G~95g5q9pblm-yq%JzvwyzRu@%aSc}cIMKz3}XGCm6gf1vBY z-j%0z?fw}rt4pgL{;4jj#5xMYV<8|Q)O~nzPIU9M?bIsYtfGqPYw^A61g^*n_^Of? zqIxY|z0fA!`P8A|CwJRV%IfDApifTC%n%UnyxCdRQW2JRJpDv}s4?Ht^gnK63Us~9 zupvyX%hB02gWXC$0^Gy%b1-Q5@3*U(Z@o<3<^n(xp0v5K^P@yA3uux7?JLckPxpCM zb$SQu&=JPs)RoDSJ|IOVmFrJ(|HBFrWjSygq+GxQHj#=E@Z1uJ>4U*`57*BAIK#0j z#T+#70wHrMZyZFQ6?4H!s4zkmD#NEsVD`_%Z1 z)8V<%r`Zk%5l@NDJy)W-x>^tb=Huo2I9aGj>VMz88!V?qxU(OFOqj`ZtR^6wwX)3; zwCa8G$@bH?4SwRFF(8})*a5Vl+^|Jcd-HnZT}oyJ1&Q@QO`IGp)z4X@z)A2=<GN4sh-u3~B7&R(t&O&~*9)OhMpWmvovI$EHP#(=yA#vXhC@K4AF)RQi+ zE|QoZht^LOvrvcohiFQ^8Spnp61@)>r<6&QDV%f#U3)Ynq|Gx8K&P$a^(c*dU%)^- z#MKRa+kgq>cHZyS5%mV{WLdR%A=tLZBv!f>AZRp_Qrw}8;cXPf%$t+X+@*Zy*HxXs z0@T!GzakN^y?#Qr8fKJ(GC%rU8Olpg!y3vuFp+|Q!vS2&dU-HP=c>FrB5Ky#4Hg0S za6ee7C@b3>Z`&vHC(U|*DP~pjj=8BPONI)wrI>-Wao&$>=T#v; zg{1;e++TxoKrCW^l~tr(-piFURGR!?#T_Xvc658vl7lj~(Pg}~pK%tK6>|8|f!p(! zwsZpA=gSGFqy)WF?f8QEsVOSVu)NH}{302(t(2?qaA8C=B1|6J{*qY&;LA8UIlbe4 zpU4nz4?20aswR^GolGM>NG!T*@ z((;5(p{}NemGwPR!QJ`rd}~bSk_cEpaY-!bNK@0(05n-~pav=L<(^lbK*?G>LE^OyFl#49 zQnjL~Zz=CbGCMx4Uy^{g9Nbr+0AYxN)1Dg`i1z51AWzfO>~?l_QC=JeOBV)j4G;hT z+XNE6RRJ3^T;pGeeUH(S1mg1`Z+Oxj)7rhY`ngRZVbG#S^^4y%vC4X*RZro zbyq{so*2!CGM~;g5JW9!R|0{&1YwBmfof70=J^)o`(q^M=794;%kK-l%m9!>g=2PR z)y<&I52n_8q2p^(0%Pm9`~G*pCdjON2>1bl1T6ol*;&Mx*WFg5n@(>?Z%Wjp@cbdi z^}Vka`k-0_j<3R;&4;fi0`6&fc^CF4b|HM>4y&_n3H6)dY0{LcMd}I)x`5q2d$`TZ z6a3}ie>UWa8O!53am^)xw4mnb_zKg1m;ABj3w-EJC*)S?*oI$rkv+*aS_H(W*VeFd zD^~A)_a!M~zoHlaEY)xEg8@_C#dY`1I%cMzf0N$@GIB6wL+DT`cdS-9E;jD&j%5rX z+fuH#RuDQcq1-71IGGLNC4rj6U^Vo=`@`62-(yo;g$`69{;P7a<0J;@;6m7FxsUaC zB)D2+^`!WD=qlBuEP}?O^xTa}dn39jQJiZAuzJUL`2b5oj`><#QZbu9*%wV}Wnt6x zpV53>O+&|9TcGBrq^9aV0d%n)wshU_SPw0cx=N@$2U@~~jb?F4Q3fiiH;hZmz+iZ_ z&-?f1c2yz@8m`BqHhh{0GkbGB*9i}2=l2ps0u=O{VA9)i=|PknKydl5Kr*?G8CP!D zqO+>(U~Ee)z&_TwqR0iP-@j+_AT`g-UCDn%s^8%$#e@suf4E35tBv{*{`3C60ErGS zuW$e@-|Cvqq(m?DQK3mzJ1VD0UYv(!j}_ngV-PY#|5;*K8lb07-tRcAE^D?v<28 z%I@+fb+EM6ul@vflb~8_${dc~Ct%vWw$(x9XDb)4! z+)qx%f$iW@>q?`5zrJ>GJ{m12j~4TFp~@G_jj~evKx3cFUbiDvaW*WxJ@azM(FCN& zgXh2hjs?OFQZ_hS7q3pbSUSQZy9gA^-JM%$nHJ^BfSNB061 zIZM!AM_IdjY&sEHUPUWhGjwztirQG*d@HIgK9klU4&OOH?`V4>t)M^{=e|PExA)E| zTJ@!%KbPHT9B}7BZsX|TduFDffVaz=7MGiw2rX^Nh$y*Fj&2jv`M@%PGX z0Uoe%G~>AeXxd)K#NTZY;G_a~(&zN-0$9IpF1520Q?F55)U~y1&DM3nE=J20=|^S> z%JEF!P(GJR-FNO>S*$#sTj_c6XJ_Hv8&gH`eQEdviCXIS?KZ3DR=~b>_q}TO0M>ww zCQC_CS)1GXB_O!Kwgk=uXzIbDfBv^|YEn44>x?%zIM`sXTUb!+wFjW?T>f7P2nDsK zmIKmm`oX}!mpY!lj%1@3>_c_;F*`2tt?jJ@**Qa}-v}N=->@12Qrw2H+gW++SRM zH4BkC2Qt`Seo;cQ4DZ|?W&|*0Xq~uuM0akWTJH;$ zP*QjWZx^BffH+)kLM4(U4+53cLF~#&K~Me9C?bW+ac7qzNe1fc^U?H{ZxH21pyArt zm4!OK>Ns9%v4bI)Rl@;Qyvk;^kuBHj>de~GvVWNIA6`PxLd+Gj%+0JQh`#5PW0#Et zM_^Pxv$6bXvb}ZrHv?5T8ccepK<} zGyP>=z>LeH#cwIu{@)o!bZlzRD-|wMHjs?F%A=71IBs1!%C&g z_&t!v$0|P+t-yz(x*bl%%G0#C@5k8ygd6eX}*Q3V6WbJavC~mA8!+Lsn>tDck zE)7lfHa}PVhzY_F8wj{r-*%Q4QS@Y`@Mu#*LZ%{}p3}rigZN78y(nhb#>NggQ7b^% zz6%>8o7*OKVA-+N%L62P@GNNg3<`hlkcdhL%Ws_m zuRK764l{y-V+unQ=sLg)Qjm|t5Cwa3$U$E~MvuAo^X^O?6o{@Eq98?5TT|!g3#01c@17qPcUibez)!{^Fy*$=4{@I1^($m$P#e1-`jXK>}^st z$5wUfc>}gtKKE^&-uNgg02$)l*le;cOMK77*x0w`oqs^pkdW9NfN-p<8MX6WwBX!g zNUx+|h&>`BG^;?d+4t%Zv`FwKM3Hf)=HyiWdk`;9{=X!ot?8Kw;9#fnS&52oBu44g z1NaPJ1ifn=C6O?fL2^0l$%986-sq}bY)%iX@Z`$GU;`5rB1iaAn^!g)!PP4|u_;&7!$!EN{Y+126tY_}cQ90-!8 zyEPK&czl4*GtZu^qxOGl(zmzOU)vul1yO|RGh`6VnwR({W6HF(E37zQ^ECGGU7I}& zo5SbtY8TTrXwvPTX@Q)e5`k3pw@Ox~1qCRJ%gb3n?*3@GnX-z9`cOsdiUrhgKAK{Y zG}VUviT|)D&D)^7Eg;$ZUfd2_j6$mCZPl=_fFg&R#*B`1j{jPQNKnv#CO&ww=0$u@ zlAx6C(yr6<_{GGfrsCp~O@z{d0?Eld85S&19_s4q-{q6^wiQ*kGuR8eI!sIk4(Era zURac0PA4(RWGnAHDLuoKlnepZhvO?5FwsDHSHS`l3nKt?B?=Jv`Y5PaUQ}jqx1R(; zIAVxTBKfMozX(B}$Qws`A?oYfHbu!{l?v93C}WZC&ebq{eSL{+jzGFGzrfqm%fr(v zAyMl9D8XDT*$^`Se^0Hx=}Zooo^6D!4?R7F(Ml>P92Q!|7s?a*KWt?Hh$GzSK)CP9aHcBur3SG^jenjDSENt4PsjkpGRTq1r$g4YP~7Zqyz zCh(IJQwhu0%m|=EJ`Y6&yW1Pn`;p6cXdtmIZP&ZycEtP9-28TODoj^bHx%tDxP~cc z>>{n2F2L8E&lmWS%?o^_F`>SDy8kn-LFK&i z*VWBJW3Qq=xT&!kg(MrjH} zfCUZ^UsO_h!;l!4mb^OEErAp}V2Wo^2BZ%70%#F@lN{9aNAMzksS!ktAz`xCaOsxtwoyt|!)PZ>wA|-CrRSsrb=>hjJ5jHMP zN-91tKQCy-_kq)+v;15_**_}5aw^zySCCPFUBM)gF8+hwi2WVhcQ2|YBRCWF`b?7l;`L7{F!QHrSH@+ z3;Gt=W^i6dRI|mB%IQ8{ooM5*+6G|S$yJ-{wxQ8DKe#b=hh1|X0V3o?0GG93dj>Bb zD4`U1&$7(gFS*q=4#H=UGp8C`VCyr6=eU4k+eydh8T6zdtmk}V7c85lA^+2;&(-oI=E zFcmjOvdOdsGh)-fuaxd1zn+bWNp3_krKgWjG86@+!`8ya#N3+Fd!^2Uk31z;-F{TI zT({+V?zl0{*?r$fh@(|c%RTDJKhDg?M9_C3K`uFWa1grd;zj1a`sn-Qieb9kS!{#X z0O59+ot>?D$KB9S(^H_7A@Fz|Hi?p4&M`JihQlGidx0VPRW?6lDBTfk?h`c}1H^WD zOoWa>z)XZ7rsY2H%hTvjvQoN`6eR{fNOkkgadUC5wBP+cpI3$tn44ZM3zcdBV{mAw z6!--_l(G52J)jk!i<$%m=Hyg>Y;$gX{m7VzKgeGrW&aJr#g6+30Zi;Y6KOTry-FM_ zt1HX@!^dX=tm3wD-a(}l5FRkHlY+}mA-Ca#hO`|p##4tAIn)mF#~rckXr^B z8MG=Ef0pG>_pNGOJ{M|w2JTH3N}DzW@;5`D#{SGgS&|$bBr2YSJbM9HlG20L#Yf~M zA!WmB)HtBwyd{Uk~In&VV?Z@avuA?+KLCf7jr@G}W*gMDw}-$w?}h`RwHpDlx;(#2<=CxE<7a34@0^ z0~$7XFC_dq;I0X=v9XU9+$|4Tp|4+`176j3_ai%@?V0ak{2L+;NwoB zqLdOAuSdTlKiQ>ZvGJW4Sl(%?YH2m(*2L2&G=F>@0^MZMZ&n>|#Wgnv>kiiqCXCfq z`^P;!%uAD(yTrnd&F)GS4=w(EO)@&jj|JDN?FZ(zX;7{5;-cci#fP-1C=NIuRl6q* z;1Uj$X8Eh1cYp{q&uG`_&I(S6?F>RzlclP&fKW%=z^{&6McjXRir?n8H6U4GC@8q8 z8TI1#BoM;}%g_x$MEW*LFNSzYEMGN0+|QYZbGz#o_@Dt`5af!|I4rh)D0aKrj~v=a zsJ*Zeimc2O@^<`VEN*8|W>XlgMiZ^2rY7huR9atde>o~_g!->B1z^O0;05s6K)&vq zbe<>0dk~>8m5bnxY1IH;XnBD+2*ZpB-eHLHyB*WU%7A**C7xW(~B>)Wq?CWM`EP_B9t6{hgL|=aL$Hn;>KD#A( zn7FL}{rSJ{5jd0Z-Q#7HPXbeH%DlGrU2@qSv(714{w*$X(B^UAy5zc0@e(&Z2P6!P zGuz=kurcQukpz+kXeBTFz*fy%P+ru&=DuTtPI<;LSgP*=~MZPTI{=oGB zUPUy)^$=L~=fU~KR{j43T?eB0MZfKAZ4g4I&Ys`7{huOWlG@#d2KuivGMo3Go0q@o z*EO}ZwR`p*iinDyIBvAuNc8pfep$9?-lFA87tS*^H9-g+KY2#Bw0QGblrP;Z7K`y$ zlTO!Jw|d2_xr-Leon<9ci&u(dWwGZ;No@+bC7|V&Nk{)K@h!Dq!on{P88pDi$dJWi z|GsPAgt5Ury?RttRqsD^L?{%>77NRl&x8fQ-z>;J^7s6yw;YmUb1lBuba2`=-y5Cc^EhfnA|CU=KeOuVo+zD?9!$w zM{~pG+FCNGkMinf1sHhQYQ|;PEIL=%!-mG7k;Mq@x>-8iSASN3p_Lw0jL@Yt_O9_3 zE~aXS?v*bKElODHLj3m%W|cP1?Pqv*g=5s?s_Dn`Cic|3z0C1-apR~1Z~dG!Eg979 zS4&>JZ#X;4#z0HuLJDj0pSib}JL+ouYgwjrO3?l{z3eoT);iZUiKhOQtEWjG?hcP3 z`&zX8$skUJaExrxZ@K+QYIjT3mn9O3qk@K0R8$zcx_$h8pFVr8Wax@Yi#KnheCg&E zSF=?Vt*oM@)Wq2Jf9>6UOjTtZ!13QX=e`RU?pwSY@@h$Bg}eqL2!<@3&goW}>uszTowZu) zbnYKV2SEk{Kzb2mn}@qsdNEQ6#Z0M>}To zj9B?mTZ$wqv!VbXKP|%Q^j2j@1Hh65rA9?vFkSaD?7`FK?W*F5;Ksyvz+R@78)IhkjrnMu-EQYTU5udJ zZ~qOdt191pf7gcfYa4(3sjReEE|<+BvhClK4@}3;G+`~$u+`l+?t0`CBloV==}pkc z0AOIk^;L`cuRe#%!#Z6I?U{OVjg1Q4q2MM>+QS6>C;(6-;oA%VD2kxn%&^t%_OgkZ z=^H&IYGnNrF5bRItZs@Vp4vBcTMOF5M5{1E^B)raiy@{E&WnBrcEUv(86CULpoghY zbI9*Vj!vrynilt&QMy@V2;HBUgA^(3ScK_yI<9rf88e#~s8kAtLL!kcO!x~&p}OBb z>1nCSw=ewW$kF!B&dm>Rg3!%sZr&GjT5d97Ez&}r+)gul9!>Ut&}{Rp`QXya)p5_1 z$0%iF?pqzVaF70C)KUpWm?m9|407&~feCj>CjT|TLakgOC2DI@=eC-N#0oznge`;z zM!x|-p^!VB*UjVOVXvjKEc?fwZ7j>0OoJzynoCRW2Y{r+1Y=K+-QfU$Uw?0LI;NI3 zGP#@}h<=l4aA*hsf~LiN#Dv7f4>M4xgX9r&Dnez_MJg)H>$}>2s`WI(FyrGB-!~lR z?i893ALnp5xqIVW?&<9|Ugoy3S_+bVLb&xzFPlfPUgz0s}At5+RB7~3-oFx%LNC?i72q7c{XGw$*5`wcNLI?@LSrQ?Hgy1ZR5JEz5 kmP804AvjASgpiQG0YXu}@U^z^RsaA107*qoM6N<$f<{1<-v9sr literal 0 HcmV?d00001 diff --git a/documentation/operators/src/introduction.md b/documentation/operators/src/introduction.md index 5d1df7d880..c4829009bb 100644 --- a/documentation/operators/src/introduction.md +++ b/documentation/operators/src/introduction.md @@ -1,25 +1,45 @@ # Introduction -This is Nym's Operators guide, containing information and setup guides for the various pieces of Nym Mixnet infrastructure (Mix Node, Gateway and Network Requester) and Nyx blockchain validators. +This is Nym's Operators guide, containing information and setup guides for the various pieces of Nym Mixnet infrastructure and Nyx blockchain validators. -If you are new to Nym and want to learn about the mixnet, explore kickstart options and demos, learn how to integrate with the network, and follow developer tutorials check out the [Developer Portal](https://nymtech.net/developers/). +``` + ┌─►mix──┐ mix mix + │ │ + Entry │ │ Exit +client ───► Gateway ──┘ mix │ mix ┌─►mix ───► Gateway ───► internet + │ │ + │ │ + mix └─►mix──┘ mix +``` + +If you are new to Nym and want to learn about the Mixnet, explore kickstart options and demos, learn how to integrate with the network, and follow developer tutorials check out the [Developer Portal](https://nymtech.net/developers/). If you want to dive deeper into Nym's architecture, clients, nodes, and SDK examples visit the [technical docs](https://nymtech.net/docs/). ## Popular pages + **Binary Information** + * [Building Nym](binaries/building-nym.md) * [Pre-built Binaries](binaries/pre-built-binaries.md) **Node setup and usage guides:** -* [Mix nodes](nodes/mix-node-setup.md) -* [Gateways](nodes/gateway-setup.md) -* [Network requesters](nodes/network-requester-setup.md) + +* [Nym Node](nodes/nym-node.md) +* [Nymvisor](nodes/nymvisor-upgrade.md) * [Validators](nodes/validator-setup.md) +* [Nym API Setup](nodes/nym-api.md) **Maintenance, troubleshooting and FAQ** -* [Maintenance](nodes/maintenance.md) -* [Troubleshooting](nodes/troubleshooting.md) -* [FAQ](faq/mixnodes-faq.md) + +* [FAQ](faq/nym-nodes-faq.md) +* [Maintenance](nodes/maintenance.md) +* [Troubleshooting](troubleshooting/nym-node.md) + +**Community Legal Forum** + +* [Exit Gateway](legal/exit-gateway.md) +* [Community Counsel](legal/community-counsel.md) +* [How to Add Info](legal/add-content.md) diff --git a/documentation/operators/src/legal/exit-gateway.md b/documentation/operators/src/legal/exit-gateway.md index fa36eb3105..b34aa75743 100644 --- a/documentation/operators/src/legal/exit-gateway.md +++ b/documentation/operators/src/legal/exit-gateway.md @@ -8,7 +8,7 @@ This page is a part of Nym Community Legal Forum and its content is composed by This document presents an initiative to further support Nym’s mission of allowing privacy for everyone everywhere. This would be achieved with the support of Nym node operators operating Gateways and opening these to any online service. Such setup needs a **clear policy**, one which will remain the **same for all operators** running Nym nodes. The [proposed **Exit policy**](https://nymtech.net/.wellknown/network-requester/exit-policy.txt) is a combination of two existing safeguards: [Tor Null ‘deny’ list](https://tornull.org/) and [Tor reduced policy](https://tornull.org/tor-reduced-reduced-exit-policy.php). -All the technical changes on the side of Nym nodes - ***Project Smoosh*** - are described in the [FAQ section](../faq/smoosh-faq.md). +All the technical changes on the side of Nym nodes - ***Project Smoosh*** - are described in the [FAQ section](../archive/faq/smoosh-faq.md). ```admonish warning Nym core team cannot provide comprehensive legal advice across all jurisdictions. Knowledge and experience with the legalities are being built up with the help of our counsel and with you, the community of Nym node operators. We encourage Nym node operators to join the operator channels ([Element](https://matrix.to/#/#operators:nymtech.chat), [Discord](https://discord.com/invite/nym), [Telegram](https://t.me/nymchan_help_chat)) to share best practices and experiences. diff --git a/documentation/operators/src/legal/landing-pages.md b/documentation/operators/src/legal/landing-pages.md index adcda45548..0e39e56cef 100644 --- a/documentation/operators/src/legal/landing-pages.md +++ b/documentation/operators/src/legal/landing-pages.md @@ -10,360 +10,4 @@ Exit Gateway landing page is a great and transparent way to prevent possible tro ## Avril 14th Exit Gateways -The following snippet needs be modified as described below according to the public identity that you may want to show on this public notice, i.e. your graphics and your email. -It would allow you to serve it as a landing page resembling the one proposed by [Tor](https://gitlab.torproject.org/tpo/core/tor/-/raw/HEAD/contrib/operator-tools/tor-exit-notice.html) but with all the changes needed to adhere to the Nym's operators case. - -[Avril 14th](https://avril14th.org/) is a team of professionals and optimistic realists eager to contribute to a future of privacy, decentralization and connectivity. - -When done with the customization needs as described below, you'll need to upload the file and reference it as the `landing_page_assets_path` on the `[http]` section of the config.toml file: -``` -landing_page_assets_path = '' -``` - -### HTML file customization - -- add your favicon logo on the line: -``` - -``` - -and your header logo on the line: -``` - -``` - -by either setting the URl to the image (if you're hosting it publicly, i.e. on your web server) -``` -href="" -``` -and -``` -src="" - -``` -**or** by adding the image inline as base64 encoded image -``` -href="href="data:image/x-icon;base64,AAABAAMA...."" -``` -and -``` -src="href="data:image/x-icon;base64,AAABAAMA...."" -``` - - -- add the email address you're willing to use for being contacted. -``` -maintainer -``` - -- If you're running the node within the US check the sections marked as FIXME, add your DNS name and uncomment those. -*** - -```html - - - - -This is a NYM Exit Gateway - - - - - - -
-

This is a NYM Exit Gateway

-

- -

- -

-You are most likely accessing this website because you've had some issue with -the traffic coming from this IP. This router is part of the NYM project, which is -dedicated to create outstanding -privacy software that is legally compliant without sacrificing integrity or -having any backdoors. -This router IP should be generating no other traffic, unless it has been -compromised.

- -

-The Nym mixnet is operated by a decentralised community of node operators -and stakers. The Nym mixnet is trustless, meaning that no parts of the system -nor its operators have access to information that might compromise the privacy -of users. Nym software enacts a strict principle of data minimisation and has -no back doors. The Nym mixnet works by encrypting packets in several layers -and relaying those through a multi-layered network called a mixnet, eventually -letting the traffic exit the Nym mixnet through an exit gateway like this one. -This design makes it very hard for a service to know which user is connecting to it, -since it can only see the IP-address of the Nym exit gateway:

- -

- -Illustration showing how a user might connect to a service through the Nym network. The user first sends their data through three daisy-chained encrypted Nym nodes that exist on three different continents. Then the last Nym node in the chain connects to the target service over the normal internet. - - - - - - - - - - - - - - - - - - - - - -The user -This server -Your service -Nym network link -Unencrypted link - - - - - - -

- -

-Read more about how Nym works.

- -

-Nym relies on a growing ecosystem of users, developers and researcher partners -aligned with the mission to make sure Nym software is running, remains usable -and solves real problems. While Nym is not designed for malicious computer -users, it is true that they can use the network for malicious ends. This -is largely because criminals and hackers have significantly better access to -privacy and anonymity than do the regular users whom they prey upon. Criminals -can and do build, sell, and trade far larger and more powerful networks than -Nym on a daily basis. Thus, in the mind of this operator, the social need for -easily accessible censorship-resistant private, anonymous communication trumps -the risk of unskilled bad actors, who are almost always more easily uncovered -by traditional police work than by extensive monitoring and surveillance anyway.

- -

-In terms of applicable law, the best way to understand Nym is to consider it a -network of routers operating as common carriers, much like the Internet -backbone. However, unlike the Internet backbone routers, Nym mixnodes do not -contain identifiable routing information about the source of a packet and do -mix the user internet traffic with that of other users, making communications -private and protecting not just the user content but the metadata -(user's IP address, who the user talks to, when, where, from what device and -more) and no single Nym node can determine both the origin and destination -of a given transmission.

- -

-As such, there is little the operator of this Exit Gateway can do to help you -track the connection further. This Exit Gateway maintains no logs of any of the -Nym mixnet traffic, so there is little that can be done to trace either legitimate or -illegitimate traffic (or to filter one from the other). Attempts to -seize this router will accomplish nothing.

- - - - - - -

To decentralise and enable privacy for a broad range of services, this -Exit Gateway adopts an Exit Policy -in accordance with the Tor Null ‘deny’ list -and the Tor reduced policy, -which are two established safeguards. -

- -

-That being said, if you still have a complaint about the router, you may email the - maintainer. If complaints are related - to a particular service that is being abused, the maintainer will submit that to the - NYM Operators Community in order to add it to the Exit Policy cited above. -If approved, that would prevent this router from allowing that traffic to exit through it. -That can be done only on an IP+destination port basis, however. Common P2P ports are already blocked.

- -

-You also have the option of blocking this IP address and others on the Nym network if you so desire. - The Nym project provides a - web service to fetch a list of all IP addresses of Nym Gateway Exit nodes that allow exiting to a -specified IP:port combination. Please be considerate when using these options.

- -
- - -``` -*** -### Reverse proxy -You may set up a [reverse proxy](https://www.nginx.com/resources/glossary/reverse-proxy-server/) in order to serve this landing page with proper SSL and DNS management, i.e. to resolve it to https://nym-exit. or https://nym-readme.. - -The following assumes that you're owning a domain and that you've already set the Let's Encrypt certificates on your hosting, and you've copied those on your Gateway, i.e. copy the two Let's Encript pem files on your Gateway's home folder. -Else you may obtain a Let's Encrypt certificate using a -[-certonly procedure](https://eff-certbot.readthedocs.io/en/latest/using.html#getting-certificates-and-choosing-plugins). - -``` -sudo apt install nginx -``` -If you're running your Gateway (you should...) exposing only the needed ports as listed [here](https://nymtech.net/operators/nodes/maintenance.html?highlight=port#ports), you need to add the Nginx service to your ufw configuration: -``` -sudo ufw app list -sudo ufw allow 'Nginx Full' -sudo ufw reload -``` -Disable the default Nginx landing page -``` -sudo systemctl status nginx -sudo unlink /etc/nginx/sites-enabled/default -sudo systemctl restart nginx -``` -Add your endpoint configuration to Nginx changing occurrencies below with your domain name, including the top-level domain, i.e. *avril4th.org* - -``` -sudo nano /etc/nginx/sites-available/nym-exit. -``` - -``` -server { - listen 443 ssl http2; - listen [::]:443 ssl http2; - - server_name nym-exit.; - - ssl_certificate /fullchain.pem; - ssl_certificate_key /privkey.pem; - - access_log /var/log/nginx/access.log; - error_log /var/log/nginx/error.log; - - location / { - proxy_pass http://127.0.0.1:8080; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - } -} - -server { - listen 80; - listen [::]:80; - - if ($host = nym-exit.) { - return 301 https://$host$request_uri; - } - - server_name www.; - - return 301 https://$request_uri; -} -``` - -``` -sudo ln -s /etc/nginx/sites-available/nym-exit. /etc/nginx/sites-enabled/ -sudo nginx -t -sudo systemctl restart nginx -``` - -When done with the customization needs as described below, you'll need to upload the file and reference it as the `landing_page_assets_path` on the `[http]` section of the config.toml file that you'll find in the `~/.nym/gateways//config/config.toml` file -``` -nano ${HOME}/.nym/gateways//config/config.toml -``` -change -``` -landing_page_assets_path = '' -``` - -If you're running the Gateway as a service as described [here](https://nymtech.net/operators/nodes/maintenance.html#systemd) and [here](https://nymtech.net/operators/nodes/maintenance.html#following-steps-for-nym-nodes-running-as-systemd-service) you may just restart your service -``` -service nym-gateway stop -service nym-gateway start -``` -and check for the page being served either checking the service logs -``` -sudo journalctl -u nym-gateway.service | grep 8080 -``` -where you should read out -``` -... Started NymNodeHTTPServer on 0.0.0.0:8080 -``` -or just point your browser to the URI which you set above, such as https://nym-exit. +Visit [Reversed Proxy: Avril 14th Exit Gateways Guide](../nodes/proxy-configuration.md#reversed-proxy-avril-14th-exit-gateways-guide) and follow the guide to setup your landing page. diff --git a/documentation/operators/src/nodes/bonding.md b/documentation/operators/src/nodes/bonding.md new file mode 100644 index 0000000000..cb20f7b3f0 --- /dev/null +++ b/documentation/operators/src/nodes/bonding.md @@ -0,0 +1,68 @@ +# Bonding Nym Node + +```admonish caution +If you unbond your Nym Node that means you are leaving the mixnet and you will lose all your delegations (permanently). You can join again with the same identity key, however, you will start with **no delegations**. +``` + +Nym Mixnet operators are rewarded for their work every epoch (60 minutes). To prevent centralisation, [Nym API](nym-api.md) is ran by distributed validators on Nyx blockchain. + +You are asked to `sign` a transaction and bpnd your node to Nyx blockchain so that the Mixnet smart contract is able to map your nym address to your node. This allows us to create a nonce for each account and defend against replay attacks. + +**Before you bond your `nym-node` make sure you went through all the previous steps** + +1. [Build](../binaries/building-nym.md) or [download](../binaries/pre-built-binaries.md) `nym-node` binary +2. [Configure VPS](vps-setup.md) correctly +3. [Prepare Nym wallet](wallet-preparation.md) +4. [Setup & Run](setup.md) the node +5. [Configure](configuration.md) the node (Optionally setup automation, WSS, reversed proxy) + +```admonish warning +Do not bond your node to the API if the previous steps weren't finished. Bad connectivity, closed ports, or other poor setup will result in your node getting blacklisted. +``` + +## Bond via the Desktop wallet (recommended) + +You can bond your `nym-node` via the Desktop wallet. + +1. Open your wallet, and head to the `Bond` page, then select the node type `Mixnode` and input your node details. Press `Next`. + - To find out your `nym-node` details, run `./nym-node bonding-information` + - To get a correct host address, run `echo "$(curl -4 https://ifconfig.me)"` + + +2. Enter the `Amount`, `Operating cost` and `Profit margin` and press `Next`. + +3. You will be asked to run a `sign` command with your `nym-node` - copy and paste the long signature as the value of `--contract-msg` and run it. + +``` +./nym-node sign --contract-msg +``` + + +``` +~~~ +--> +4. Copy the resulting signature string and paste it into the wallet nodal, press `Next` and confirm the transaction: + +```sh +# This is just an example, copy the one from your process +>>> The base58-encoded signature is: +2bbDJSmSo9r9qdamTNygY297nQTVRyQaxXURuomVcRd7EvG9oEC8uW8fvZZYnDeeC9iWyG9mAbX2K8rWEAxZBro1 +``` + +![Paste Signature](../images/wallet-screenshots/wallet-sign.png) +*This image is just an example, copy-paste your own base58-encoded signature* + +5. Your node will now be bonded and ready to recieve traffic, latest at the beginning of the next epoch (at most 1 hour) + + +If everything worked, you'll see your node running on the either the [Sandbox testnet network explorer](https://sandbox-explorer.nymtech.net) or the [mainnet network explorer](https://explorer.nymtech.net), depending on which network you're running. + + +## Bond via the CLI (power users) + +If you want to bond your Mix Node via the CLI, then check out the [relevant section in the Nym CLI](https://nymtech.net/docs/tools/nym-cli.html#bond-a-mix-node) docs. diff --git a/documentation/operators/src/nodes/configuration.md b/documentation/operators/src/nodes/configuration.md new file mode 100644 index 0000000000..814028dd46 --- /dev/null +++ b/documentation/operators/src/nodes/configuration.md @@ -0,0 +1,305 @@ +# Nym Node Configuration + +## Basic Changes + +Nym Node can be configured directly by editing the config file (`config.toml`) located at `~/.nym/nym-nodes//config/config.toml` (by default `~/.nym/nym-nodes/default-nym-node/config/config.toml`) or through commands on the binary. + +### Commands & Examples + +Disable sharing of system hardware info with the network: + +```sh +./nym-node run --id --deny-init --mode entry-gateway -w --expose-system-hardware false --expose-system-info false +``` + +Note: `--expose-system-info false` supersedes `--expose-system-hardware false`. If both are present with conflicting values, the system hardware will not be shown. + + +## VPS Setup and Automation + +> Replace `` variable with type of node you run, preferably `nym-node` (depreciated `nym-mixnode`, `nym-gateway` or `nym-network-requester`). + +### Automating your node with nohup, tmux and systemd + +Although it’s not totally necessary, it's useful to have the Mix Node automatically start at system boot time. We recommend to run your remote operation via [`tmux`](maintenance.md#tmux) for easier management and a handy return to your previous session. For full automation, including a failed node auto-restart and `ulimit` setup, [`systemd`](maintenance.md#systemd) is a good choice. + +> Do any of these steps and run your automated node before you start bonding process! + +#### nohup + +`nohup` is a command with which your terminal is told to ignore the `HUP` or 'hangup' signal. This will stop the node process ending if you kill your session. + +```sh +nohup ./ run # use all the flags you use to run your node +``` + +#### tmux + +One way is to use `tmux` shell on top of your current VPS terminal. Tmux is a terminal multiplexer, it allows you to create several terminal windows and panes from a single terminal. Processes started in `tmux` keep running after closing the terminal as long as the given `tmux` window was not terminated. + +Use the following command to get `tmux`. + +| Platform | Install Command | +| :--- | :--- | +| Arch Linux|`pacman -S tmux` | +| Debian or Ubuntu|`apt install tmux` | +| Fedora|`dnf install tmux` | +| RHEL or CentOS|`yum install tmux` | +| macOS (using Homebrew | `brew install tmux` | +| macOS (using MacPorts) | `port install tmux` | +| openSUSE | `zypper install tmux` | + +In case it didn't work for your distribution, see how to build `tmux` from [version control](https://github.com/tmux/tmux#from-version-control). + +**Running tmux** + +Now you have installed tmux on your VPS, let's run a Mix Node on tmux, which allows you to detach your terminal and let your `` run on its own on the VPS. + +* Pause your `` +* Start tmux with the command +```sh +tmux +``` +* The tmux terminal should open in the same working directory, just the layout changed into tmux default layout. +* Start the `` again with a command: +```sh +./ run # use all the flags you use to run your node +``` +* Now, without closing the tmux window, you can close the whole terminal and the `` (and any other process running in tmux) will stay active. +* Next time just start your teminal, ssh into the VPS and run the following command to attach back to your previous session: +```sh +tmux attach-session +``` +* To see keybinding options of tmux press `ctrl`+`b` and after 1 second `?` + +#### systemd + +To automate with `systemd` use this init service file by saving it as `/etc/systemd/system/nym-node.service` and follow the [next steps](#following-steps-for-nym-nodes-running-as-systemd-service). + +1. Open text editor +```sh +nano /etc/systemd/system/nym-node.service +``` + +2. Paste this file +```ini +[Unit] +Description=Nym Node +StartLimitInterval=350 +StartLimitBurst=10 + +[Service] +User= +LimitNOFILE=65536 +ExecStart=/home///nym-node run # add all the flags you use to run your node +KillSignal=SIGINT +Restart=on-failure +RestartSec=30 + +[Install] +WantedBy=multi-user.target +``` + +3. Save the file + +```admonish note +Make sure your `ExecStart ` and `run` command are correct! + +Example: If you have built nym in the `$HOME` directory on your server, your username is `jetpanther`, and node `` is `puma`, then the `ExecStart` line (command) in the script located in `/etc/systemd/system/nym-mixnode.service` for Nym Mixnode might look like this: +`ExecStart=/home/jetpanther/nym/target/release/nym-node run --id puma`. + +Basically, you want the full `///nym-mixnode run --id `. If you are unsure about your `///`, then `cd` to your directory where you run your `` from and run `pwd` command which returns the full path for you. +``` + + +#### Following steps for Nym nodes running as `systemd` service + +Once your init file is save follow these steps: + +1. Reload systemctl to pickup the new unit file +```sh +systemctl daemon-reload +``` + +2. Enable the newly created service: + +```sh +systemctl enable nym-node.service +``` + +3. Start your `` as a `systemd` service: + +```sh +service nym-node start +``` + +This will cause your `` to start at system boot time. If you restart your machine, your `` will come back up automatically. + +**Useful systemd commands** + +- You can monitor system logs of your node by running: +```sh +journalctl -u nym-node -f +``` + +- Or check a status by running: +```sh +systemctl status .service +# for example systemctl status nym-node.service +``` + +- You can also do `service stop` or `service restart`. + +**Note:** if you make any changes to your `systemd` script after you've enabled it, you will need to run: + +```sh +systemctl daemon-reload +``` + +This lets your operating system know it's ok to reload the service configuration. Then restart your ``. + + +## Connectivity Test and Configuration + +```admonish info +**This chapter is relevant only for operators running `entry-gateway` and `exit-gateway` mode.** +``` + +During our ongoing testing events [Fast and Furious](https://nymtech.net/events/fast-and-furious) we found out, that after introducing IP Packet Router (IPR) and [Nym exit policy](https://nymtech.net/.wellknown/network-requester/exit-policy.txt) on embedded Network Requester (NR) by default, only a fragment of Gateways routes correctly through IPv4 and IPv6. We built a useful monitor to check out your Gateway (`nym-node --mode exit-gateway`) at [harbourmaster.nymtech.net](https://harbourmaster.nymtech.net/). + + +While we're working on Rust implementation to have these settings as a part of the binary build, we wrote a script to solve these connectivity requirements in the meantime we wrote a script [`network_tunnel_manager.sh`](https://gist.github.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77) to support the operators to configure their servers and address all the connectivity requirements. + +The `nymtun0` interface is dynamically managed by the `exit-gateway` service. When the service is stopped, `nymtun0` disappears, and when started, `nymtun0` is recreated. + +The script should be used in a context where `nym-node --mode exit-gateway` is running to fully utilise its capabilities, particularly for fetching IPv6 addresses or applying network rules that depend on the `nymtun0` interface. + +1. Download `network_tunnel_manager.sh`, make executable and run: + +```sh +curl -o network_tunnel_manager.sh -L https://gist.githubusercontent.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77/raw/9d785d6ee3aa2970553633eccbd89a827f49fab5/network_tunnel_manager.sh && chmod +x network_tunnel_manager.sh && ./network_tunnel_manager.sh +``` + +Here is a quick command explanation, for more details on the `network_tunnel_manager.sh` script, refer to the [overview](https://gist.github.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77) under the code block. + +~~~admonish example collapsible=true title="A summarized usage of `network_tunnel_manager.sh`" +```sh +summary: +This is a comprehensive script for configuring network packet forwarding and iptables rules, +aimed at ensuring smooth operation of a tunnel interface. +It includes functionality for both setup and tear-down of nymtun network configurations, +alongside diagnostics for verifying system settings and network connectivity. + +* fetch_ipv6_address_nym_tun - Fetches the IPv6 address assigned to the 'nymtun0'. +* fetch_and_display_ipv6 - Displays the IPv6 address on the default network device. +* apply_iptables_rules - Applies necessary IPv4 and IPv6 iptables rules. +* remove_iptables_rules - Removes applied IPv4 and IPv6 iptables rules. +* check_ipv6_ipv4_forwarding - Checks if IPv4 and IPv6 forwarding are enabled. +* check_nymtun_iptables - Check nymtun0 device +* perform_ipv4_ipv6_pings - Perform ipv4 and ipv6 pings to google +* check_ip6_ipv4_routing - Check ipv6 and ipv4 routing +* joke_through_the_mixnet - Run a joke through the mixnet via ipv4 and ipv6 + +``` +~~~ + + - To run the script next time, just enter `./network_tunnel_manager.` + +2. Make sure your `nym-node --mode exit-gateway` service is up running + +3. Check Nymtun IP tables: +```sh +sudo ./network_tunnel_manager.sh check_nymtun_iptables +``` + +~~~admonish example collapsible=true title="Correct `./network_tunnel_manager.sh check_nymtun_iptables` output:" +```sh + +iptables-persistent is already installed. +network Device: eth0 +--------------------------------------- + +inspecting IPv4 firewall rules... +Chain FORWARD (policy DROP 0 packets, 0 bytes) + 0 0 ufw-reject-forward all -- * * 0.0.0.0/0 0.0.0.0/0 + 0 0 ACCEPT all -- nymtun0 eth0 0.0.0.0/0 0.0.0.0/0 + 0 0 ACCEPT all -- eth0 nymtun0 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED + 0 0 ACCEPT all -- nymtun0 eth0 0.0.0.0/0 0.0.0.0/0 + 0 0 ACCEPT all -- eth0 nymtun0 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED + 0 0 ACCEPT all -- nymtun0 eth0 0.0.0.0/0 0.0.0.0/0 + 0 0 ACCEPT all -- eth0 nymtun0 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED +--------------------------------------- + +inspecting IPv6 firewall rules... +Chain FORWARD (policy DROP 0 packets, 0 bytes) + 0 0 ufw6-reject-forward all * * ::/0 ::/0 + 0 0 ACCEPT all eth0 nymtun0 ::/0 ::/0 state RELATED,ESTABLISHED + 0 0 ACCEPT all nymtun0 eth0 ::/0 ::/0 + 0 0 ACCEPT all eth0 nymtun0 ::/0 ::/0 state RELATED,ESTABLISHED + 0 0 ACCEPT all nymtun0 eth0 ::/0 ::/0 + 0 0 ACCEPT all eth0 nymtun0 ::/0 ::/0 state RELATED,ESTABLISHED + 0 0 ACCEPT all nymtun0 eth0 ::/0 ::/0 +operation check_nymtun_iptables completed successfully. +``` +~~~ + + - if there's no process running it wouldn't return anything + +4. Display IPv6: +```sh +sudo ./network_tunnel_manager.sh fetch_and_display_ipv6 +``` + - if you have a `global ipv6` address this is good + +~~~admonish example collapsible=true title="Correct `./network_tunnel_manager.sh fetch_and_display_ipv6` output:" +```sh +iptables-persistent is already installed. +Using IPv6 address: 2001:db8:a160::1/112 #the address will be different for you +operation fetch_ipv6_address_nym_tun completed successfully. +``` +~~~ + +5. Apply the rules: +```sh +sudo ./network_tunnel_manager.sh apply_iptables_rules +``` + + - The process may prompt you if you want to save current IPv4 rules, choose yes. + +![](../images/ip_table_prompt.png) + + - and check them again like in point 3 + +6. Check `nymtun0` interface: +```sh +ip addr show nymtun0 +``` +~~~admonish example collapsible=true title="Correct `ip addr show nymtun0` output:" +```sh +# your addresses will be different +8: nymtun0: mtu 1420 qdisc fq_codel state UNKNOWN group default qlen 500 + link/none + inet 10.0.0.1/16 scope global nymtun0 + valid_lft forever preferred_lft forever + inet6 2001:db8:a160::1/112 scope global + valid_lft forever preferred_lft forever + inet6 fe80::ad08:d167:5700:8c7c/64 scope link stable-privacy + valid_lft forever preferred_lft forever` +``` +~~~ + +10. Validate your IPv6 and IPv4 networking by running a joke via Mixnet: +```sh +sudo ./network_tunnel_manager.sh joke_through_the_mixnet +``` + +Make sure that you get the validation of IPv4 and IPv6 connectivity. If there are still any problems, please refer to [troubleshooting section](../troubleshooting/vps-setup.md#incorrect-gateway-network-check) + +## Next Steps + +There are a few more good suggestions for `nym-node` VPS configuration, especially to be considered for `exit-gateway` functionality, like Web Secure Socket or Reversed Proxy setup. Visit [Proxy configuration](proxy-configuration.md) page to see the guides. + +After proceed to [bonding](bonding.md). + + + diff --git a/documentation/operators/src/nodes/maintenance.md b/documentation/operators/src/nodes/maintenance.md index af3c0c7b97..54627d38b5 100644 --- a/documentation/operators/src/nodes/maintenance.md +++ b/documentation/operators/src/nodes/maintenance.md @@ -14,122 +14,6 @@ For example `./target/debug/nym-network-requester --no-banner build-info --outpu {"binary_name":"nym-network-requester","build_timestamp":"2023-07-24T15:38:37.00657Z","build_version":"1.1.23","commit_sha":"c70149400206dce24cf20babb1e64f22202672dd","commit_timestamp":"2023-07-24T14:45:45Z","commit_branch":"feature/simplify-cli-parsing","rustc_version":"1.71.0","rustc_channel":"stable","cargo_profile":"debug"} ``` - -## Run Web Secure Socket (WSS) on Gateway - -Now you can run WSS on your Gateway. - -### WSS on a new Gateway - -These steps are for an operator who is setting up a [Gateway](gateway-setup.md) for the first time and wants to run it with WSS. - -1. New flags will need to be added to the `init` and `run` command. The `--host` option should be replaced with these flags: - -- `--listening-address`: The IP address which is used for receiving sphinx packets and listening to client data. -- `--public-ips`: A comma separated list of IP’s that are announced to the `nym-api`. In the most cases `--public-ips` **is the address used for bonding.** - -```sh ---listening-address 0.0.0.0 --public-ips "$(curl -4 https://ifconfig.me)" -``` - -- `--hostname` (optional): This flag is required if the operator wishes to run WSS. It can be something like `mainnet-gateway2.nymtech.net`. - -2. Make sure to enable all necessary [ports](maintenance.md#configure-your-firewall) on the Gateway: - -```sh -sudo ufw allow 1789,1790,8000,9000,9001,22/tcp, 9001/tcp -``` - -The Gateway will then be accessible on something like: *http://85.159.211.99:8080/api/v1/swagger/index.html* - -Are you seeing something like: *this node attempted to announce an invalid public address: 0.0.0.0.*? - -Please modify `[host.public_ips]` section of your config file stored as `~/.nym/gateways//config/config.toml`. - -### WSS on an existing Gateway - -In case you already run a working Gateway and want to add WSS on it, here are the pre-requisites to running WSS on Gateways: - -* You need to use the latest `nym-gateway` binary [version](./gateway-setup.md#current-version) and restart it. -* That will add the relevant fields to update your config. -* These two values will be added and need to be amended in your config.toml: - -```sh -clients_wss_port = 0 -hostname = "" -``` - -Then you can run this: - -```sh -port=$1 // in the example below we will use 9001 -host=$2 = // this would be a domain name registered for your Gateway for example: mainnet-gateway2.nymtech.net - - -sed -i "s/clients_wss_port = 0/clients_wss_port = ${port}/" ${HOME}/.nym/gateways/*/config/config.toml -sed -i "s|hostname = ''|hostname = '${host}'|" ${HOME}/.nym/gateways/*/config/config.toml -``` -The following shell script can be run: - -```sh -#!/bin/bash - -if [ "$#" -ne 2 ]; then - echo "Usage: sudo ./install_run_caddy.sh " - exit 1 -fi - -host=$1 -port_value=$2 - -apt install -y debian-keyring debian-archive-keyring apt-transport-https -apt --fix-broken install - -curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg - -curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list - -apt update -apt install caddy - -systemctl enable caddy.service - -cd /etc/caddy - -# check if Caddyfile exists, if it does, remove and insert a new one -if [ -f Caddyfile ]; then - echo "removing caddyfile inserting a new one" - rm -f Caddyfile -fi - -cat <> Caddyfile -${host}:${port_value} { - @websockets { - header Connection *Upgrade* - header Upgrade websocket - } - reverse_proxy @websockets localhost:9000 -} -EOF - -cat Caddyfile - -echo "script completed successfully!" - -systemctl restart caddy.service -echo "have a nice day!" -exit 0 - -``` - -Although your Gateway is Now ready to use its `wss_port`, your server may not be ready - the following commands will allow you to set up a properly configured firewall using `ufw`: - -```sh -ufw allow 9001/tcp -``` - -Lastly don't forget to restart your Gateway, now the API will render the WSS details for this Gateway: - ## Configure your firewall Although your `` is now ready to receive traffic, your server may not be. The following commands will allow you to set up a firewall using `ufw`. @@ -151,7 +35,7 @@ sudo ufw status Finally open your `` p2p port, as well as ports for ssh and ports for verloc and measurement pings: ```sh -# for Mix Node, Gateway and Network Requester +# for nym-node sudo ufw allow 1789,1790,8000,9000,9001,22/tcp # in case of setting up WSS on Gateway add: @@ -173,11 +57,11 @@ For more information about your node's port configuration, check the [port refer ## VPS Setup and Automation -> Replace `` variable with `nym-mixnode`, `nym-gateway` or `nym-network-requester` according the node you running on your machine. +> Replace `` variable with type of node you run, preferably `nym-node` (depreciated `nym-mixnode`, `nym-gateway` or `nym-network-requester`). ### Automating your node with nohup, tmux and systemd -Although it’s not totally necessary, it's useful to have the Mix Node automatically start at system boot time. We recommend to run your remote operation via [`tmux`](maintenance.md#tmux) for easier management and a handy return to your previous session. For full automation, including a failed node auto-restart and `ulimit` setup, [`systemd`](maintenance.md#systemd) is a good choice. +Although it’s not totally necessary, it's useful to have the Mix Node automatically start at system boot time. We recommend to run your remote operation via [`tmux`](maintenance.md#tmux) for easier management and a handy return to your previous session. For full automation, including a failed node auto-restart and `ulimit` setup, [`systemd`](maintenance.md#systemd) is a good choice. > Do any of these steps and run your automated node before you start bonding process! @@ -186,7 +70,7 @@ Although it’s not totally necessary, it's useful to have the Mix Node automati `nohup` is a command with which your terminal is told to ignore the `HUP` or 'hangup' signal. This will stop the node process ending if you kill your session. ```sh -nohup ./ run --id # where `` is the id you set during the `init` command and depends on which node you starting +nohup ./ run # use all the flags you use to run your node ``` #### tmux @@ -195,15 +79,15 @@ One way is to use `tmux` shell on top of your current VPS terminal. Tmux is a te Use the following command to get `tmux`. -Platform|Install Command ----|--- -Arch Linux|`pacman -S tmux` -Debian or Ubuntu|`apt install tmux` -Fedora|`dnf install tmux` -RHEL or CentOS|`yum install tmux` -macOS (using Homebrew|`brew install tmux` -macOS (using MacPorts)|`port install tmux` -openSUSE|`zypper install tmux` +| Platform | Install Command | +| :--- | :--- | +| Arch Linux|`pacman -S tmux` | +| Debian or Ubuntu|`apt install tmux` | +| Fedora|`dnf install tmux` | +| RHEL or CentOS|`yum install tmux` | +| macOS (using Homebrew | `brew install tmux` | +| macOS (using MacPorts) | `port install tmux` | +| openSUSE | `zypper install tmux` | In case it didn't work for your distribution, see how to build `tmux` from [version control](https://github.com/tmux/tmux#from-version-control). @@ -219,7 +103,7 @@ tmux * The tmux terminal should open in the same working directory, just the layout changed into tmux default layout. * Start the `` again with a command: ```sh -./ run --id +./ run # use all the flags you use to run your node ``` * Now, without closing the tmux window, you can close the whole terminal and the `` (and any other process running in tmux) will stay active. * Next time just start your teminal, ssh into the VPS and run the following command to attach back to your previous session: @@ -230,77 +114,18 @@ tmux attach-session #### systemd -To automate with `systemd` use this init service file and follow the steps below. - -##### For Mix Node - -```ini -[Unit] -Description=Nym Mixnode -StartLimitInterval=350 -StartLimitBurst=10 - -[Service] -User= -LimitNOFILE=65536 -ExecStart=/home///nym-mixnode run --id -KillSignal=SIGINT -Restart=on-failure -RestartSec=30 - -[Install] -WantedBy=multi-user.target -``` - -* Put the above file onto your system at `/etc/systemd/system/nym-mixnode.service` and follow the [next steps](maintenance.md#following-steps-for-nym-nodes-running-as-systemd-service). - -##### For Gateway - -```ini -[Unit] -Description=Nym Gateway -StartLimitInterval=350 -StartLimitBurst=10 - -[Service] -User=nym -LimitNOFILE=65536 -ExecStart=/home///nym-gateway run --id -KillSignal=SIGINT -Restart=on-failure -RestartSec=30 - -[Install] -WantedBy=multi-user.target -``` - -* Put the above file onto your system at `/etc/systemd/system/nym-gateway.service` and follow the [next steps](maintenance.md#following-steps-for-nym-nodes-running-as-systemd-service). - -##### For Network Requester - -```ini -[Unit] -Description=Nym Network Requester -StartLimitInterval=350 -StartLimitBurst=10 - -[Service] -User=nym # replace this with whatever user you wish -LimitNOFILE=65536 -# remember to add the `--enable-statistics` flag if running as part of a service grant and check the path to your nym-network-requester binary -ExecStart=/home/nym/nym-network-requester run --id -KillSignal=SIGINT -Restart=on-failure -RestartSec=30 - -[Install] -WantedBy=multi-user.target -``` -* Put the above file onto your system at `/etc/systemd/system/nym-network-requester.service` and follow the [next steps](maintenance.md#following-steps-for-nym-nodes-running-as-systemd-service). - ##### For Nymvisor > Since you're running your node via a Nymvisor instance, as well as creating a Nymvisor `.service` file, you will also want to **stop any previous node automation process you already have running**. +To automate with `systemd` use this init service file by saving it as `/etc/systemd/system/nymvisor.service` and follow the [next steps](#following-steps-for-nym-nodes-running-as-systemd-service). + +1. Open text editor +```sh +nano /etc/systemd/system/nymvisor.service +``` + +2. Paste this file + ``` [Unit] Description=Nymvisor @@ -308,9 +133,9 @@ StartLimitInterval=350 StartLimitBurst=10 [Service] -User=nym # replace this with whatever user you wish +User= # replace this with whatever user you wish LimitNOFILE=65536 -ExecStart=/home///nymvisor run run --id +ExecStart=/home///nymvisor run --id KillSignal=SIGINT Restart=on-failure RestartSec=30 @@ -319,74 +144,57 @@ RestartSec=30 WantedBy=multi-user.target ``` -* Put the above file onto your system at `/etc/systemd/system/nymvisor.service` and follow the [next steps](maintenance.md#following-steps-for-nym-nodes-running-as-systemd-service). +3. Save the file + +```admonish note +Make sure your `ExecStart ` and `run` command are correct! + +Example: If you have built nym in the `$HOME` directory on your server, your username is `jetpanther`, and node `` is `puma`, then the `ExecStart` line (command) in the script located in `/etc/systemd/system/nym-mixnode.service` for Nym Mixnode might look like this: +`ExecStart=/home/jetpanther/nym/target/release/nym-node run --id puma`. + +Basically, you want the full `///nym-mixnode run --id `. If you are unsure about your `///`, then `cd` to your directory where you run your `` from and run `pwd` command which returns the full path for you. +``` + #### Following steps for Nym nodes running as `systemd` service -Change the `` in `ExecStart` to point at your `` binary (`nym-mixnode`, `nym-gateway` or `nym-network-requester`), and the `` so it is the user you are running as. - -Example: If you have built nym in the `$HOME` directory on your server, your username is `jetpanther`, and node `` is `puma`, then the `ExecStart` line (command) in the script located in `/etc/systemd/system/nym-mixnode.service` for Nym Mixnode might look like this: - -`ExecStart=/home/jetpanther/nym/target/release/nym-mixnode run --id puma`. - -Basically, you want the full `///nym-mixnode run --id `. If you are unsure about your `///`, then `cd` to your directory where you run your `` from and run `pwd` command which returns the full path for you. - -Once done, save the script and follow these steps: +Once your init file is save follow these steps: +1. Reload systemctl to pickup the new unit file ```sh systemctl daemon-reload -# to pickup the new unit file ``` -Enable the newly created service: +2. Enable the newly created service: ```sh -# for Mix Node -systemctl enable nym-mixnode.service - -# for Gateway -systemctl enable nym-gateway.service - -# for Network Requester -systemctl enable nym-network-requester.service - -# for Nymvisor -systemctl enable nymvisor.service +systemctl enable nym-node.service ``` -Start your `` as a `systemd` service: +3. Start your `` as a `systemd` service: ```sh -# for Mix Node -service nym-mixnode start - -# for Gateway -service nym-gateway start - -# for Network Requester -service nym-network-requester.service - -# for Nymvisor -service nymvisor.service start +service nym-node start ``` This will cause your `` to start at system boot time. If you restart your machine, your `` will come back up automatically. -You can monitor system logs of your node by running: +**Useful systemd commands** + +- You can monitor system logs of your node by running: ```sh -journalctl -f -u .service -# for example journalctl -f -u nym-mixnode.service +journalctl -u -f ``` -Or check a status by running: +- Or check a status by running: ```sh systemctl status .service -# for example systemctl status nym-mixnode.service +# for example systemctl status nym-node.service ``` -You can also do `service stop` or `service restart`. +- You can also do `service stop` or `service restart`. -Note: if you make any changes to your `systemd` script after you've enabled it, you will need to run: +**Note:** if you make any changes to your `systemd` script after you've enabled it, you will need to run: ```sh systemctl daemon-reload @@ -395,167 +203,6 @@ systemctl daemon-reload This lets your operating system know it's ok to reload the service configuration. Then restart your ``. -##### For Validator - -Below is a `systemd` unit file to place at `/etc/systemd/system/nymd.service` to automate your validator: - -```ini -[Unit] -Description=Nyxd -StartLimitInterval=350 -StartLimitBurst=10 - -[Service] -User= # change to your user -Type=simple -Environment="LD_LIBRARY_PATH=/home//" # change to correct path -ExecStart=/home///nymd start # change to correct path -Restart=on-failure -RestartSec=30 -LimitNOFILE=infinity - -[Install] -WantedBy=multi-user.target -``` - -Proceed to start it with: - -```sh -systemctl daemon-reload # to pickup the new unit file -systemctl enable nymd # to enable the service -systemctl start nymd # to actually start the service -journalctl -f -u nymd # to monitor system logs showing the service start -``` - -##### For Nym API - -Below is a `systemd` unit file to place at `/etc/systemd/system/nym-api.service` to automate your API instance: - -```ini -[Unit] -Description=NymAPI -StartLimitInterval=350 -StartLimitBurst=10 - -[Service] -User= # change to your user -Type=simple -ExecStart=/home///nym-api start # change to correct path -Restart=on-failure -RestartSec=30 -LimitNOFILE=infinity - -[Install] -WantedBy=multi-user.target -``` - -Proceed to start it with: - -```sh -systemctl daemon-reload # to pickup the new unit file -systemctl enable nym-api # to enable the service -systemctl start nym-api # to actually start the service -journalctl -f -u nym-api # to monitor system logs showing the service start -``` - - -### Setting the ulimit - -Linux machines limit how many open files a user is allowed to have. This is called a `ulimit`. - -`ulimit` is 1024 by default on most systems. It needs to be set higher, because Mix Nodes make and receive a lot of connections to other nodes. - -If you see errors such as: - -```sh -Failed to accept incoming connection - Os { code: 24, kind: Other, message: "Too many open files" } -``` - -This means that the operating system is preventing network connections from being made. - -#### Set the `ulimit` via `systemd` service file - -> Replace `` variable with `nym-mixnode`, `nym-gateway` or `nym-network-requester` according the node you running on your machine. - -The ulimit setup is relevant for maintenance of Nym Mix Node only. - -Query the `ulimit` of your `` with: - -```sh -# for nym-mixnode, nym-gateway and nym-network-requester: -grep -i "open files" /proc/$(ps -A -o pid,cmd|grep | grep -v grep |head -n 1 | awk '{print $1}')/limits - -# for nyx validator: -grep -i "open files" /proc/$(ps -A -o pid,cmd|grep nymd | grep -v grep |head -n 1 | awk '{print $1}')/limits -``` - - - -You'll get back the hard and soft limits, which looks something like this: - -```sh -Max open files 65536 65536 files -``` - -If your output is **the same as above**, your node will not encounter any `ulimit` related issues. - -However if either value is `1024`, you must raise the limit via the systemd service file. Add the line: - -```sh -LimitNOFILE=65536 -``` - -Reload the daemon: - -```sh -systemctl daemon-reload -``` - -or execute this as root for system-wide setting of `ulimit`: - -```sh -echo "DefaultLimitNOFILE=65535" >> /etc/systemd/system.conf -``` - -Reboot your machine and restart your node. When it comes back, use: -```sh -# for nym-mixnode, nym-gateway and nym-network-requester: -cat /proc/$(pidof )/limits | grep "Max open files" - -# for validator -cat /proc/$(pidof nym-validator)/limits | grep "Max open files" -``` -Make sure the limit has changed to 65535. - -#### Set the ulimit on `non-systemd` based distributions - -In case you chose tmux option for Mix Node automation, see your `ulimit` list by running: - -```sh -ulimit -a - -# watch for the output line -n --n: file descriptors 1024 -``` - -You can change it either by running a command: - -```sh -ulimit -u -n 4096 -``` - -or editing `etc/security/conf` and add the following lines: - -```sh -# Example hard limit for max opened files -username hard nofile 4096 - -# Example soft limit for max opened files -username soft nofile 4096 -``` - -Then reboot your server and restart your Mix Node. - ## Moving a node In case of a need to move a node from one machine to another and avoiding to lose the delegation, here are few steps how to do it. @@ -567,256 +214,29 @@ The following examples transfers a Mix Node (in case of other nodes, change the Assuming both machines are remote VPS. * Make sure your `~/.ssh/.pub` is in both of the machines `~/.ssh/authorized_keys` file -* Create a `mixnodes` folder in the target VPS. Ssh in from your terminal and run: +* Create a `nym-nodes` folder in the target VPS. SSH in from your terminal and run: ```sh # in case none of the nym configs was created previously mkdir ~/.nym #in case no nym Mix Node was initialized previously -mkdir ~/.nym/mixnodes +mkdir ~/.nym/nym-nodes ``` -* Move the node data (keys) and config file to the new machine by opening a local terminal (as that one's ssh key is authorized in both of the machines) and running: +* Move the node data (keys) and config file to the new machine by opening your **local terminal** (as that one's ssh key is authorized in both of the machines) and running: ```sh -scp -r -3 @:~/.nym/mixnodes/ @:~/.nym/mixnodes/ +scp -r -3 @:~/.nym/nym-nodes @:~/.nym/nym-nodes/ ``` -* Re-run init (remember that init doesn't overwrite existing keys) to generate a config with the new listening address etc. +* Re-initialise (`run` command) the node to generate a config with the new listening address. * Change the node smart contract info via the wallet interface. Otherwise the keys will point to the old IP address in the smart contract, and the node will not be able to be connected, and it will fail up-time checks. * Re-run the node from the new location. -## Virtual IPs and hosting via Google & AWS - -For true internet decentralization we encourage operators to use diverse VPS providers instead of the largest companies offering such services. If for some reasons you have already running AWS or Google and want to setup a `` there, please read the following. - -On some services (AWS, Google, etc) the machine's available bind address is not the same as the public IP address. In this case, bind `--host` to the local machine address returned by `$(curl -4 https://ifconfig.me)`, but that may not the public IP address to bond your `` in the wallet. - -You can run `ifconfig` command. For example, on a Google machine, you may see the following output: - -```sh -ens4: flags=4163 mtu 1460 - inet 10.126.5.7 netmask 255.255.255.255 broadcast 0.0.0.0 - ... -``` - -The `ens4` interface has the IP `10.126.5.7`. But this isn't the public IP of the machine, it's the IP of the machine on Google's internal network. Google uses virtual routing, so the public IP of this machine is something else, maybe `36.68.243.18`. - -To find the right IP configuration, contact your VPS provider for support to find the right public IP and use it to bond your `` with the `nym-api` via Nym wallet. - -On self-hosted machine it's a bit more tricky. In that case as an operator you must be sure that your ISP allows for public IPv4 and IPv6 and then it may be a bit of playing around to find the right configuration. One way may be to bind your binary with the `--host` flag to local address `127.0.0.1` and run `echo "$(curl -4 https://ifconfig.me)"` to get a public address which you use to bond your Mix Node to `nym-api` via Nym wallet. - -It's up to you as a node operator to ensure that your public and private IPs match up properly. - -## Nym API (previously 'Validator API') endpoints - -Numerous API endpoints are documented on the Nym API (previously 'Validator API')'s [Swagger Documentation](https://validator.nymtech.net/api/swagger/index.html). There you can also try out various requests from your browser, and download the response from the API. Swagger will also show you what commands it is running, so that you can run these from an app or from your CLI if you prefer. - -### Mix Node Reward Estimation API endpoint - -The Reward Estimation API endpoint allows Mix Node operators to estimate the rewards they could earn for running a Nym Mix Node with a specific `MIX_ID`. - -> The `` can be found in the "Mix ID" column of the [Network Explorer](https://explorer.nymtech.net/network-components/mixnodes/active). - -The endpoint is a particularly common for Mix Node operators as it can provide an estimate of potential earnings based on factors such as the amount of traffic routed through the Mix Node, the quality of the Mix Node's performance, and the overall demand for Mix Nodes in the network. This information can be useful for Mix Node operators in deciding whether or not to run a Mix Node and in optimizing its operations for maximum profitability. - -Using this API endpoint returns information about the Reward Estimation: - -```sh -/status/mixnode//reward-estimation -``` - -Query Response: - -```sh - "estimation": { - "total_node_reward": "942035.916721770541325331", - "operator": "161666.263307386408152071", - "delegates": "780369.65341438413317326", - "operating_cost": "54444.444444444444444443" - }, -``` - -> The unit of value is measured in `uNYM`. - -- `estimated_total_node_reward` - An estimate of the total amount of rewards that a particular Mix Node can expect to receive during the current epoch. This value is calculated by the Nym Validator based on a number of factors, including the current state of the network, the number of Mix Nodes currently active in the network, and the amount of network traffic being processed by the Mix Node. - -- `estimated_operator_reward` - An estimate of the amount of rewards that a particular Mix Node operator can expect to receive. This value is calculated by the Nym Validator based on a number of factors, including the amount of traffic being processed by the Mix Node, the quality of service provided by the Mix Node, and the operator's stake in the network. - -- `estimated_delegators_reward` - An estimate of the amount of rewards that Mix Node delegators can expect to receive individually. This value is calculated by the Nym Validator based on a number of factors, including the amount of traffic being processed by the Mix Node, the quality of service provided by the Mix Node, and the delegator's stake in the network. - -- `estimated_node_profit` - An estimate of the profit that a particular Mix node operator can expect to earn. This value is calculated by subtracting the Mix Node operator's `operating_costs` from their `estimated_operator_reward` for the current epoch. - -- `estimated_operator_cost` - An estimate of the total cost that a particular Mix Node operator can expect to incur for their participation. This value is calculated by the Nym Validator based on a number of factors, including the cost of running a Mix Node, such as server hosting fees, and other expenses associated with operating the Mix Node. - -### Validator: Installing and configuring nginx for HTTPS -#### Setup -[Nginx](https://www.nginx.com/resources/glossary/nginx) is an open source software used for operating high-performance web servers. It allows us to set up reverse proxying on our validator server to improve performance and security. - -Install `nginx` and allow the 'Nginx Full' rule in your firewall: - -```sh -sudo ufw allow 'Nginx Full' -``` - -Check nginx is running via systemctl: - -```sh -systemctl status nginx -``` - -Which should return: - -```sh -● nginx.service - A high performance web server and a reverse proxy server - Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) - Active: active (running) since Fri 2018-04-20 16:08:19 UTC; 3 days ago - Docs: man:nginx(8) - Main PID: 2369 (nginx) - Tasks: 2 (limit: 1153) - CGroup: /system.slice/nginx.service - ├─2369 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; - └─2380 nginx: worker process -``` - -#### Full Node Configuration - -Proxying various full node services through port 80 can then be done by creating a file with the following at `/etc/nginx/sites-enabled/nyxd-webrequests.conf`: - -Setting up a reverse proxy using a webserver such as Nginx allows you to easily configure SSL certificates for the endpoints. When running on mainnet, it is recommended to encrypt all web traffic to your node. - -```sh -### To expose RPC server -server { - listen 80; - listen [::]:80; - server_name ""; - - location / { - proxy_pass http://127.0.0.1:26657; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - } - - location /websocket { - proxy_pass http://127.0.0.1:26657; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection "Upgrade"; - proxy_set_header Host $host; - } -} - -### To expose Cosmos API server -server { - server_name ""; - location / { - proxy_pass http://127.0.0.1:1317; - proxy_set_header X-Forwarded-For $remote_addr; - proxy_set_header Host $http_host; - proxy_set_header Upgrade websocket; - proxy_set_header Connection Upgrade; - } -} - -### To expose GRPC endpoint -server { - server_name ""; - location / { - grpc_pass 127.0.0.1:9090; - } -} -``` - -#### nym-api Configuration - -```sh -### To expose nym-api webserver -server { - listen 80; - listen [::]:80; - server_name ""; - add_header 'Access-Control-Allow-Origin' '*'; - - location / { - proxy_pass http://127.0.0.1:8000; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - } -} -``` - -Followed by: - -```sh -sudo apt install certbot nginx python3 -certbot --nginx -m --agree-tos -``` - -```admonish caution title="" -If using a VPS running Ubuntu 20: replace `certbot nginx python3` with `python3-certbot-nginx` -``` - -These commands will get you an https encrypted nginx proxy in front of the various endpoints. - -### Configuring Prometheus metrics (optional) - -Configure Prometheus with the following commands (adapted from NodesGuru's [Agoric setup guide](https://nodes.guru/agoric/setup-guide/en)): - -```sh -echo 'export OTEL_EXPORTER_PROMETHEUS_PORT=9464' >> $HOME/.bashrc -source ~/.bashrc -sed -i '/\[telemetry\]/{:a;n;/enabled/s/false/true/;Ta}' $HOME/.nymd/config/app.toml -sed -i "s/prometheus-retention-time = 0/prometheus-retention-time = 60/g" $HOME/.nymd/config/app.toml -sudo ufw allow 9464 -echo 'Metrics URL: http://'$(curl -s ifconfig.me)':26660/metrics' -``` - -Your validator's metrics will be available to you at the returned 'Metrics URL'. - -~~~admonish example collapsible=true title="Console output" -``` -# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles. -# TYPE go_gc_duration_seconds summary -go_gc_duration_seconds{quantile="0"} 6.7969e-05 -go_gc_duration_seconds{quantile="0.25"} 7.864e-05 -go_gc_duration_seconds{quantile="0.5"} 8.4591e-05 -go_gc_duration_seconds{quantile="0.75"} 0.000115919 -go_gc_duration_seconds{quantile="1"} 0.001137591 -go_gc_duration_seconds_sum 0.356555301 -go_gc_duration_seconds_count 2448 -# HELP go_goroutines Number of goroutines that currently exist. -# TYPE go_goroutines gauge -go_goroutines 668 -# HELP go_info Information about the Go environment. -# TYPE go_info gauge -go_info{version="go1.15.7"} 1 -# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use. -# TYPE go_memstats_alloc_bytes gauge -go_memstats_alloc_bytes 1.62622216e+08 -# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed. -# TYPE go_memstats_alloc_bytes_total counter -go_memstats_alloc_bytes_total 2.09341707264e+11 -# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table. -# TYPE go_memstats_buck_hash_sys_bytes gauge -go_memstats_buck_hash_sys_bytes 5.612319e+06 -# HELP go_memstats_frees_total Total number of frees. -# TYPE go_memstats_frees_total counter -go_memstats_frees_total 2.828263344e+09 -# HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started. -# TYPE go_memstats_gc_cpu_fraction gauge -go_memstats_gc_cpu_fraction 0.03357798610671518 -# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata. -# TYPE go_memstats_gc_sys_bytes gauge -go_memstats_gc_sys_bytes 1.3884192e+07 -``` -~~~ ## Ports All ``-specific port configuration can be found in `$HOME/.nym///config/config.toml`. If you do edit any port configs, remember to restart your client and node processes. -### Mix Node port reference +### Nym Node: Minode mode port reference | Default port | Use | | ------------ | ------------------------- | | `1789` | Listen for Mixnet traffic | @@ -824,19 +244,13 @@ All ``-specific port configuration can be found in `$HOME/.nym// The process here is similar for the Mix Node, Gateway and Network Requester binaries. In the following steps we use a placeholder `` in the commands, please change it for the binary name you want to upgrade (e.g.`nym-mixnode`). Any particularities for the given type of node are included. +> Any syntax in `<>` brackets is a user's unique variable. Exchange with a corresponding name without the `<>` brackets. -Upgrading your node is a two-step process: +**Upgrading your node is a straight forward two-step process:** -1. Updating the binary and `~/.nym///config/config.toml` on your VPS -2. Updating the node information in the [mixnet smart contract](https://nymtech.net/docs/nyx/mixnet-contract.html). **This is the information that is present on the [mixnet explorer](https://explorer.nymtech.net)**. +#### 1. Updating the binary and `~/.nym///config/config.toml` on your VPS +#### 2. Updating the node information in the [mixnet smart contract](https://nymtech.net/docs/nyx/mixnet-contract.html). This is the information that is present on the [mixnet explorer](https://explorer.nymtech.net). ## Step 1: Upgrading your binary -Follow these steps to upgrade your Node binary and update its config file: -* Pause your node process. + +Follow these steps to upgrade your node binary and update its config file: +1. Pause your node process. - if you see the terminal window with your node, press `ctrl + c` - - if you run it as `systemd` service, run: `systemctl stop .service` -* Replace the existing `` binary with the newest binary (which you can either [compile yourself](https://nymtech.net/docs/binaries/building-nym.html) or grab from our [releases page](https://github.com/nymtech/nym/releases)). -* Re-run `init` with the same values as you used initially for your `` ([Mix Node](./mix-node-setup.md#initialising-your-mix-node), [Gateway](./gateway-setup.md#initialising-your-gateway)) . **This will just update the config file, it will not overwrite existing keys**. -* Restart your node process with the new binary: - - if your node is *not automated*, just `run` your `` with `./ run --id `. Here are exact guidelines for [Mix Node](./mix-node-setup.md#running-your-mix-node) and [Gateway](./gateway-setup.md#running-your-gateway). + - if you run it as `systemd` service, run: `service stop` +2. Replace the existing `` binary with the newest binary (which you can either [compile yourself](../binaries/building-nym.md) or [download](../binaries/pre-built-binaries.md). +3. [Re-run with the same values](setup.md#initialise--run) as you used initially for your ``. **This will just update the config file, it will not overwrite existing keys**. + - if your node is *not automated*, just `run` your `` with `./ run --id `. - if you *automated* your node with systemd (recommended) run: ```sh systemctl daemon-reload # to pickup the new unit file -systemctl start .service -journalctl -f -u .service # to monitor log of you node + +service start && journalctl -f -u .service # to monitor log of you node ``` -If these steps are too difficult and you prefer to automate the process, try to setup your flow with [Nymvisor](nymvisor-upgrade.md). - -> In case of a Network Requester this is all, the following step is only for Mix Nodes and Gateways. +If you prefer to automate the process, try to setup your flow with [Nymvisor](nymvisor-upgrade.md). ## Step 2: Updating your node information in the smart contract + Follow these steps to update the information about your `` which is publicly available from the [`nym-api`](https://validator.nymtech.net/api/swagger/index.html) and information displayed on the [Mixnet explorer](https://explorer.nymtech.net). You can either do this graphically via the Desktop Wallet, or the CLI. ### Updating node information via the Desktop Wallet (recommended) -* Navigate to the `Bonding` page and click the `Node Settings` link in the top right corner: + +1. Navigate to the `Bonding` page and click the `Node Settings` link in the top right corner: ![Bonding page](../images/wallet-screenshots/bonding.png) -* Update the fields in the `Node Settings` page (usually the field `Version` is the only one to change) and click `Submit changes to the blockchain`. +2. Update the fields in the `Node Settings` page (usually the field `Version` is the only one to change) and click `Submit changes to the blockchain`. ![Node Settings Page](../images/wallet-screenshots/node_settings.png) ### Updating node information via the CLI + If you want to bond your `` via the CLI, then check out the [relevant section in the Nym CLI](https://nymtech.net/docs/tools/nym-cli.html#upgrade-a-mix-node) docs. - -## Upgrading Network Requester to >= v1.1.10 from -``` - -Copy the old keys from your client to the network-requester configuration that was created above: - -```sh -cp -vr ~/.nym/clients/myoldclient/data/* ~/.nym/service-providers/network-requester//data -``` - -Edit the configuration to match what you used on your client. Specifically, edit the configuration file at: - -```sh -~/.nym/service-providers/network-requester//config/config.toml -``` - -Ensure that the fields `gateway_id`, `gateway_owner`, `gateway_listener` in the new config match those in the old client config at: - -```sh -~/.nym/clients/myoldclient/config/config.toml +```admonish info +If you run a Gateway, visit [Nym Harbour Master](https://harbourmaster.nymtech.net/) to get all the probe info about your node directly from API. ``` ## Upgrading your validator diff --git a/documentation/operators/src/nodes/nym-node.md b/documentation/operators/src/nodes/nym-node.md new file mode 100644 index 0000000000..15df4c9562 --- /dev/null +++ b/documentation/operators/src/nodes/nym-node.md @@ -0,0 +1,101 @@ +# Nym Node + +```admonish note +If you are a `nym-mixnode` or `nym-gateway` operator and you are not familiar with the binary changes called *Project Smoosh*, you can read the archived [Smoosh FAQ](../archive/smoosh-faq.md) page. +``` + +NYM NODE is a tool for running a node within the Nym network. Nym Nodes containing functionality such as `mixnode`, `entry-gateway` and `exit-gateway` are fundamental components of Nym Mixnet architecture. Nym Nodes are ran by decentralised node operators. + +To setup any type of Nym Node, start with either building [Nym's platform](../binaries/building-nym.md) from source or download [pre-compiled binaries](../binaries/pre-built-binaries.md) on the [configured server (VPS)](vps-setup.md) where you want to run the node. Nym Node will need to be bond to [Nym's wallet](wallet-preparation.md). Follow [preliminary steps](preliminary-steps.md) page before you initialise and run a node. + +```admonish info +**Migrating an existing node to a new `nym-node` is simple. The steps are documented on the [next page](setup.md#migrate)** +``` + +## Steps for Nym Node Operators + +Once VPS and Nym wallet are configured, binaries ready, the operators of `nym-node` need to: + +1. **[Setup & Run](setup.md) the node** + +2. **[Configure](configuration.md) the node** (and optionally WSS, reversed proxy, automation) + +3. **[Bond](bonding.md) the node to the Nym API, using Nym wallet** + +## Quick `nym-node --mode exit-gateway` Setup + +During our ongoing testing events [Fast and Furious](https://nymtech.net/events/fast-and-furious) we found out, that after introducing IP Packet Router and [Nym exit policy](https://nymtech.net/.wellknown/network-requester/exit-policy.txt) by default, only a fragment of Gateways routes correctly through IPv4 and IPv6. We built a useful monitor to check out your Gateway (`nym-node --mode exit-gateway`) at [harbourmaster.nymtech.net](https://harbourmaster.nymtech.net/). + +Below is a fast - ten commands - deployment to get and setup your node, configure networking and connectivity and verify that it all works as it should by getting two free jokes through the Mixnet. + +```admonish caution +If you are not well familiar with `nym-node` setup, automation, and `nymtun0` configuration, follow the [steps above](#steps-for-nym-node-operators) page by page. You can use this flow as a reference later on. +``` + +1. [Get](../binaries/pre-built-binaries.md) or [build](../binaries/building-nym.md) the latest `nym-node` binary + +2. Get [network_tunnel_manager.sh](https://gist.github.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77) script and grant permissions +```sh +curl -o network_tunnel_manager.sh -L https://gist.githubusercontent.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77/raw/9d785d6ee3aa2970553633eccbd89a827f49fab5/network_tunnel_manager.sh && chmod +x network_tunnel_manager.sh +``` + +3. If you have a running `nym-node` or `nym-gateway` (alone or service), stop the process + - In case your node was a `nym-gateway`, [**migrate to `nym-node`**](setup.md#migrate) now! + +4. Check Nymtun IP tables: +```sh +sudo ./network_tunnel_manager.sh check_nymtun_iptables +``` + - if there's no process running it shouldn't get anything + +5. Display IPv6: +```sh +sudo ./network_tunnel_manager.sh fetch_and_display_ipv6 +``` + - if you have a `global ipv6` address this is good +~~~admonish example collapsible=true title="Correct `./network_tunnel_manager.sh fetch_and_display_ipv6` output:" +```sh +iptables-persistent is already installed. +Using IPv6 address: 2001:db8:a160::1/112 #the address will be different for you +operation fetch_ipv6_address_nym_tun completed successfully. +``` +~~~ + +6. Apply the rules: +```sh +sudo ./network_tunnel_manager.sh apply_iptables_rules +``` + - and check them again like in point 3. + +7. (If you didn't have a `nym-node` service yet) Create `systemd` [automation and configuration file](configuration.md#systemd), reload, enable + +8. Start `nym-node` service: +```sh +sudo service start nym-node && journalctl -u nym-node -f -n 100 +``` + +9. After a minute of running properly, check `nymtun0`: +```sh +ip addr show nymtun0 +``` + +~~~admonish example collapsible=true title="Correct `ip addr show nymtun0` output:" +```sh +# your addresses will be different +8: nymtun0: mtu 1420 qdisc fq_codel state UNKNOWN group default qlen 500 + link/none + inet 10.0.0.1/16 scope global nymtun0 + valid_lft forever preferred_lft forever + inet6 2001:db8:a160::1/112 scope global + valid_lft forever preferred_lft forever + inet6 fe80::ad08:d167:5700:8c7c/64 scope link stable-privacy + valid_lft forever preferred_lft forever` +``` +~~~ + +10. Validate your IPv6 and IPv4 networking by running a joke via Mixnet: +```sh +sudo ./network_tunnel_manager.sh joke_through_the_mixnet +``` + +Make sure that you get the validation of IPv4 and IPv6 connectivity, in case of problems, check [troubleshooting page](../troubleshooting/vps-setup.md#incorrect-gateway-network-check). After proceed to [bonding](bonding.md). diff --git a/documentation/operators/src/nodes/nymvisor-upgrade.md b/documentation/operators/src/nodes/nymvisor-upgrade.md index fcee0a1564..49bb267815 100644 --- a/documentation/operators/src/nodes/nymvisor-upgrade.md +++ b/documentation/operators/src/nodes/nymvisor-upgrade.md @@ -11,9 +11,7 @@ In essence, it tries to mirror the behaviour of [Cosmovisor](https://github.com/ You can use Nymvisor to automate the upgrades of the following binaries: * `nym-api` -* `nym-mixnode` -* `nym-gateway` -* `nym-network-requester` +* `nym-node` * `nym-client` * `nym-socks5-client` @@ -27,9 +25,9 @@ Nymvisor is an early and experimental software. Users should use it at their own ``` ## Preliminary steps -You need to have at least one Mixnet node / client / Nym API instance already set up on the **same VPS** that you wish to run Nymvisor on. +You need to have at least one Nym Node / client / Nym API instance already set up on the **same VPS** that you wish to run Nymvisor on. -> Using Nymvisor presumes your VPS is running an operating system that is compatible with the pre-compiled binaries avaliable on the [Github releases page](https://github.com/nymtech/nym/releases). If you're not, then until we're packaging for a greater variety of operating systems, you're stuck with [manually upgrading your node](manual-upgrade.md). +> Using Nymvisor presumes your VPS is running an operating system that is compatible with the pre-compiled binaries availiable on the [Github releases page](https://github.com/nymtech/nym/releases). If you're not, then until we're packaging for a greater variety of operating systems, you're stuck with [manually upgrading your node](manual-upgrade.md). ## Setup and Usage ### Viewing command help @@ -54,7 +52,7 @@ You can also check the various arguments required for individual commands with: ``` ### Initialising your Nymvisor Instance -> This example will use the Mix Node binary as an example - however replacing `nym-mixnode` with any other supported binary will work the same. +> This example will use the Nym Node binary as an example - however replacing `nym-node` with any other supported binary will work the same. Initialise your Nymvisor instance with the following command. You must initialise Nymvisor with the binary you wish to add upgrades for: @@ -62,11 +60,11 @@ Initialise your Nymvisor instance with the following command. You must initialis ./nymvisor init --daemon-home ~/.nym// ``` -Where the value of `--daemon-home` might be `~/.nym/mixnodes/my-node` and `` might be `/home/my_user/nym/target/release/nym-mixnode`, or wherever your node binary is located. +Where the value of `--daemon-home` might be `~/.nym/nym-nodes/default-nym-node` and `` might be `/home/my_user/nym/target/release/nym-node`, or wherever your node binary is located. ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ @@ -75,7 +73,7 @@ By default this will create config files at `~/.nym/nymvisors/instances/` command is quite simple. Any argument passed after the `run` command will be passed directly to the underlying daemon, for example: `nymvisor run run --id my-mixnode` will run the `$DAEMON_NAME run --id my-mixnode` command (where `DAEMON_NAME` is the name of the binary itself (e.g. `nym-api`, `nym-mixnode`, etc.)). +The interface to the `nymvisor run ` command is quite simple. Any argument passed after the `run` command will be passed directly to the underlying daemon, for example: `nymvisor run run --id default-nym-node` will run the `$DAEMON_NAME run --id default-nym-node` command (where `DAEMON_NAME` is the name of the binary itself (e.g. `nym-api`, `nym-node`, etc.)). `run` Nymvisor and start your node via the following command. Make sure to stop any existing node before running this command. @@ -85,11 +83,11 @@ The interface to the `nymvisor run ` command is quite simple. Any argument ~~~admonish example collapsible=true title="Console output" ``` - + ``` ~~~ -Nymvisor will now manage your node process (for an in-depth overview of this command check the [in-depth command information](./nymvisor-upgrade.md#commands-in-depth) below). It will periodically poll [this endpoint](https://nymtech.net/.wellknown/nym-mixnode/upgrade-info.json) (replace `nym-mixnode` with whatever node you may actually be running via Nymvisor) and check for a new `version` of the binary it is watching. If this exists, it will then, using the information there: +Nymvisor will now manage your node process (for an in-depth overview of this command check the [in-depth command information](./nymvisor-upgrade.md#commands-in-depth) below). It will periodically poll [this endpoint](https://nymtech.net/.wellknown/nym-node/upgrade-info.json) (replace `nym-node` with whatever node you may actually be running via Nymvisor) and check for a new `version` of the binary it is watching. If this exists, it will then, using the information there: * pause your node process * grab the new binary (`version`) * verify it against the provided `checksum` @@ -118,9 +116,9 @@ Similarly to `init`, `add-upgrade` requires a positional argument specifying a v ## Config The output format of `nymvisor config` can be further configured with `--output` argument. By default a human-readable text representation is used: ``` -id: nym-mixnode-default -daemon name: nym-mixnode -daemon home: /home/nym/.nym/mixnodes/my-mixnode +id: nym-node-default +daemon name: nym-node +daemon home: /home/nym/.nym/nym-nodes/default-nym-node upstream base upgrade url: https://nymtech.net/.wellknown/ disable nymvisor logs: false CUSTOM upgrade data directory "" @@ -143,7 +141,7 @@ nymvisor config --output=json ``` outputs: ``` -{"nymvisor":{"id":"nym-mixnode-default","upstream_base_upgrade_url":"https://nymtech.net/.wellknown/","upstream_polling_rate":"1h","disable_logs":false,"upgrade_data_directory":null},"daemon":{"name":"nym-mixnode","home":"/home/nym/.nym/mixnodes/my-mixnode","absolute_upstream_upgrade_url":null,"allow_binaries_download":true,"enforce_download_checksum":true,"restart_after_upgrade":true,"restart_on_failure":false,"failure_restart_delay":"10s","max_startup_failures":10,"startup_period_duration":"2m","shutdown_grace_period":"10s","backup_data_directory":null,"unsafe_skip_backup":false}} +{"nymvisor":{"id":"nym-node-default","upstream_base_upgrade_url":"https://nymtech.net/.wellknown/","upstream_polling_rate":"1h","disable_logs":false,"upgrade_data_directory":null},"daemon":{"name":"nym-node","home":"/home/nym/.nym/nym-nodes/default-nym-nodee","absolute_upstream_upgrade_url":null,"allow_binaries_download":true,"enforce_download_checksum":true,"restart_after_upgrade":true,"restart_on_failure":false,"failure_restart_delay":"10s","max_startup_failures":10,"startup_period_duration":"2m","shutdown_grace_period":"10s","backup_data_directory":null,"unsafe_skip_backup":false}} ``` ## CLI Overview @@ -179,8 +177,8 @@ For any of its commands as described in [CLI Overview section](./nymvisor-upgrad - `NYMVISOR_UPSTREAM_POLLING_RATE` (defaults to 1h) is polling rate the upstream url for upgrade information. - `NYMVISOR_DISABLE_LOGS` (defaults to `false`). If set to `true`, this will disable Nymvisor logs (but not the underlying process) completely. - `NYMVISOR_UPGRADE_DATA_DIRECTORY` is the custom directory for upgrade data - binaries and upgrade plans. If not set, the global Nymvisors' data directory will be used instead. -- `DAEMON_NAME` is the name of the binary itself (e.g. `nym-api`, `nym-mixnode`, etc.). -- `DAEMON_HOME` is the location where the `nymvisor/` directory is kept that contains the auxiliary files associated with the underlying daemon instance, such as any backups or current version information, e.g. `$HOME/.nym/nym-api/my-nym-api`, `$HOME/.nym/mixnodes/my-mixnode`, etc. +- `DAEMON_NAME` is the name of the binary itself (e.g. `nym-api`, `nym-node`, etc.). +- `DAEMON_HOME` is the location where the `nymvisor/` directory is kept that contains the auxiliary files associated with the underlying daemon instance, such as any backups or current version information, e.g. `$HOME/.nym/nym-api/my-nym-api`, `$HOME/.nym/nym-nodes/default-nym-node`, etc. - `DAEMON_ABSOLUTE_UPSTREAM_UPGRADE_URL` is the absolute (i.e. the full url) upstream source for upgrade plans for this daemon. The url has to point to an endpoint containing a valid `UpgradeInfo` json file. If set it takes precedence over `NYMVISOR_UPSTREAM_BASE_UPGRADE_URL`. - `DAEMON_ALLOW_BINARIES_DOWNLOAD` (defaults to `true`), if set to `true`, it will enable auto-downloading of new binaries (as declared by urls in corresponding `upgrade-info.json` files). For security reasons one might wish to disable it and instead manually provide binaries by either placing them in the appropriate directory or by invoking `add-upgrade` command. - `DAEMON_ENFORCE_DOWNLOAD_CHECKSUM` (defaults to `true`), if set to `true` Nymvisor will require that a checksum is provided in the upgrade plan for the upgrade binary to be downloaded. If disabled, Nymvisor will not require a checksum to be provided, but still check the checksum if one is provided. @@ -226,7 +224,7 @@ A sample full structure looks as follows: │ │ │ └── upgrade-info.json │ │ ├── upgrade-history.json │ │ └── upgrade-plan.json -│ ├── nym-mixnode +│ ├── nym-node │ │ └── ... │ └── $DAEMON_NAME │ └── ... @@ -264,7 +262,7 @@ This section outlines what happens under the hood with the following commands: - saves the Nymvisor instance's config file to `$NYMVISOR_CONFIG_PATH` and creates the full directory structure for the file - outputs (to `stdout`) the full configuration used -> `nymvisor init` is specifically for initializing Nymvisor, and should **not** be confused with a daemon's `init` command - such as `nym-mixnode init` (e.g. `cosmovisor run init`). +> `nymvisor init` is specifically for initializing Nymvisor, and should **not** be confused with a daemon's `init` command - such as `nym-node init` (e.g. `cosmovisor run init`). ### Run `nymvisor run` is a lightweight wrapper around the underlying daemon. It uses only a single thread and spawns three simple tasks: diff --git a/documentation/operators/src/nodes/nyx-configuration.md b/documentation/operators/src/nodes/nyx-configuration.md new file mode 100644 index 0000000000..cbad32a3b5 --- /dev/null +++ b/documentation/operators/src/nodes/nyx-configuration.md @@ -0,0 +1,268 @@ +# Nyx Validator & Nym API Configuration + +## Automation + +### Validator `systemd` Automation + +To automate with `systemd` use this init service file by saving it as `/etc/systemd/system/nymd.service` and follow the steps bellow.. + +1. Open text editor +```sh +nano /etc/systemd/system/nymd.service +``` + +2. Paste this file +```ini +[Unit] +Description=Nyxd +StartLimitInterval=350 +StartLimitBurst=10 + +[Service] +User= # change to your user +Type=simple +Environment="LD_LIBRARY_PATH=/home//" # change to correct path +ExecStart=/home///nymd start # change to correct path +Restart=on-failure +RestartSec=30 +LimitNOFILE=infinity + +[Install] +WantedBy=multi-user.target +``` + +Proceed to start it with: + +```sh +systemctl daemon-reload # to pickup the new unit file +systemctl enable nymd # to enable the service +systemctl start nymd # to actually start the service +journalctl -f -u nymd # to monitor system logs showing the service start +``` + +**Note:** if you make any changes to your `systemd` script after you've enabled it, you will need to run: + +```sh +systemctl daemon-reload +``` + +This lets your operating system know it's ok to reload the service configuration. Then restart your ``. + +### Nym API `systemd` Automation + + +To automate with `systemd` use this init service file by saving it as `/etc/systemd/system/nym-api.service` and follow the steps bellow.. + +1. Open text editor +```sh +nano /etc/systemd/system/nym-api.service +``` + +2. Paste this file +```ini +[Unit] +Description=NymAPI +StartLimitInterval=350 +StartLimitBurst=10 + +[Service] +User= # change to your user +Type=simple +ExecStart=/home///nym-api start # change to correct path +Restart=on-failure +RestartSec=30 +LimitNOFILE=infinity + +[Install] +WantedBy=multi-user.target +``` + +Proceed to start it with: + +```sh +systemctl daemon-reload # to pickup the new unit file +systemctl enable nym-api # to enable the service +systemctl start nym-api # to actually start the service +journalctl -f -u nym-api # to monitor system logs showing the service start +``` + +**Note:** if you make any changes to your `systemd` script after you've enabled it, you will need to run: + +```sh +systemctl daemon-reload +``` + +This lets your operating system know it's ok to reload the service configuration. Then restart your ``. + +## Nym API (previously 'Validator API') endpoints + +Numerous API endpoints are documented on the Nym API (previously 'Validator API')'s [Swagger Documentation](https://validator.nymtech.net/api/swagger/index.html). There you can also try out various requests from your browser, and download the response from the API. Swagger will also show you what commands it is running, so that you can run these from an app or from your CLI if you prefer. + +```sh +sudo ufw allow 'Nginx Full' +``` + +Check nginx is running via systemctl: + +```sh +systemctl status nginx +``` + +Which should return: + +```sh +● nginx.service - A high performance web server and a reverse proxy server + Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) + Active: active (running) since Fri 2018-04-20 16:08:19 UTC; 3 days ago + Docs: man:nginx(8) + Main PID: 2369 (nginx) + Tasks: 2 (limit: 1153) + CGroup: /system.slice/nginx.service + ├─2369 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; + └─2380 nginx: worker process +``` + +## Full Node Configuration + +Proxying various full node services through port 80 can then be done by creating a file with the following at `/etc/nginx/sites-enabled/nyxd-webrequests.conf`: + +Setting up a reverse proxy using a webserver such as Nginx allows you to easily configure SSL certificates for the endpoints. When running on mainnet, it is recommended to encrypt all web traffic to your node. + +```sh +### To expose RPC server +server { + listen 80; + listen [::]:80; + server_name ""; + + location / { + proxy_pass http://127.0.0.1:26657; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } + + location /websocket { + proxy_pass http://127.0.0.1:26657; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "Upgrade"; + proxy_set_header Host $host; + } +} + +### To expose Cosmos API server +server { + server_name ""; + location / { + proxy_pass http://127.0.0.1:1317; + proxy_set_header X-Forwarded-For $remote_addr; + proxy_set_header Host $http_host; + proxy_set_header Upgrade websocket; + proxy_set_header Connection Upgrade; + } +} + +### To expose GRPC endpoint +server { + server_name ""; + location / { + grpc_pass 127.0.0.1:9090; + } +} +``` + +## nym-api Configuration + +```sh +### To expose nym-api webserver +server { + listen 80; + listen [::]:80; + server_name ""; + add_header 'Access-Control-Allow-Origin' '*'; + + location / { + proxy_pass http://127.0.0.1:8000; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } +} +``` + +Followed by: + +```sh +sudo apt install certbot nginx python3 +certbot --nginx -m --agree-tos +``` + +```admonish caution title="" +If using a VPS running Ubuntu 20: replace `certbot nginx python3` with `python3-certbot-nginx` +``` + +These commands will get you an https encrypted nginx proxy in front of the various endpoints. + +## Configuring Prometheus metrics (optional) + +Configure Prometheus with the following commands (adapted from NodesGuru's [Agoric setup guide](https://nodes.guru/agoric/setup-guide/en)): + +```sh +echo 'export OTEL_EXPORTER_PROMETHEUS_PORT=9464' >> $HOME/.bashrc +source ~/.bashrc +sed -i '/\[telemetry\]/{:a;n;/enabled/s/false/true/;Ta}' $HOME/.nymd/config/app.toml +sed -i "s/prometheus-retention-time = 0/prometheus-retention-time = 60/g" $HOME/.nymd/config/app.toml +sudo ufw allow 9464 +echo 'Metrics URL: http://'$(curl -s ifconfig.me)':26660/metrics' +``` + +Your validator's metrics will be available to you at the returned 'Metrics URL'. + +~~~admonish example collapsible=true title="Console output" +``` +# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles. +# TYPE go_gc_duration_seconds summary +go_gc_duration_seconds{quantile="0"} 6.7969e-05 +go_gc_duration_seconds{quantile="0.25"} 7.864e-05 +go_gc_duration_seconds{quantile="0.5"} 8.4591e-05 +go_gc_duration_seconds{quantile="0.75"} 0.000115919 +go_gc_duration_seconds{quantile="1"} 0.001137591 +go_gc_duration_seconds_sum 0.356555301 +go_gc_duration_seconds_count 2448 +# HELP go_goroutines Number of goroutines that currently exist. +# TYPE go_goroutines gauge +go_goroutines 668 +# HELP go_info Information about the Go environment. +# TYPE go_info gauge +go_info{version="go1.15.7"} 1 +# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use. +# TYPE go_memstats_alloc_bytes gauge +go_memstats_alloc_bytes 1.62622216e+08 +# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed. +# TYPE go_memstats_alloc_bytes_total counter +go_memstats_alloc_bytes_total 2.09341707264e+11 +# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table. +# TYPE go_memstats_buck_hash_sys_bytes gauge +go_memstats_buck_hash_sys_bytes 5.612319e+06 +# HELP go_memstats_frees_total Total number of frees. +# TYPE go_memstats_frees_total counter +go_memstats_frees_total 2.828263344e+09 +# HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started. +# TYPE go_memstats_gc_cpu_fraction gauge +go_memstats_gc_cpu_fraction 0.03357798610671518 +# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata. +# TYPE go_memstats_gc_sys_bytes gauge +go_memstats_gc_sys_bytes 1.3884192e+07 +``` +~~~ + +## Validator port reference + +All validator-specific port configuration can be found in `$HOME/.nymd/config/config.toml`. If you do edit any port configs, remember to restart your validator. + +| Default port | Use | +|--------------|--------------------------------------| +| 1317 | REST API server endpoint | +| 26656 | Listen for incoming peer connections | +| 26660 | Listen for Prometheus connections | diff --git a/documentation/operators/src/nodes/preliminary-steps.md b/documentation/operators/src/nodes/preliminary-steps.md new file mode 100644 index 0000000000..02b278c90e --- /dev/null +++ b/documentation/operators/src/nodes/preliminary-steps.md @@ -0,0 +1,10 @@ +# Preliminary Steps + +> The `nym-node` binary was built in the [building nym](../binaries/building-nym.md) section. If you haven't yet built Nym and want to run the code, go there first. + +There are a couple of steps that need completing before starting to set up your `nym-node`: + +1. **[Prepare your wallet](wallet-preparation.md):** [desktop](https://nymtech.net/docs/wallet/desktop-wallet.html) or [CLI](https://nymtech.net/docs/wallet/cli-wallet.html). +2. **[Requisition and setup a VPS](vps-setup.md)** (Virtual Private Server) + +Make sure to follow these steps carefully as it prevents a lot of troubleshooting later on. diff --git a/documentation/operators/src/nodes/proxy-configuration.md b/documentation/operators/src/nodes/proxy-configuration.md new file mode 100644 index 0000000000..d765ec51b4 --- /dev/null +++ b/documentation/operators/src/nodes/proxy-configuration.md @@ -0,0 +1,482 @@ +# Reversed Proxy & Web Secure Socket + +It's useful to put your Exit Gateway behind a reversed proxy and have it accessible via `https` domain, where you can host a [landing page](../legal/landing-pages.md). The guide is right [below](#reversed-proxy). + +Another solution is to have a your Gateway behind WSS. With ongoing migration from `nym-gateway` to `nym-node --mode exit-gateway` we are working on a detailed guide for WSS setup. + +## Reversed Proxy: Avril 14th Exit Gateways Guide + +```admonish info +This guide was created by a Nym node operator, Avril 14th as a part of [Nym Operators Community Counsel](../legal/community-counsel.md) + +The entire content of this page is under [Creative Commons Attribution 4.0 International Public License](https://creativecommons.org/licenses/by/4.0/). +``` + +The following snippet needs be modified as described below according to the public identity that you may want to show on this public notice, i.e. your graphics and your email. +It would allow you to serve it as a landing page resembling the one proposed by [Tor](https://gitlab.torproject.org/tpo/core/tor/-/raw/HEAD/contrib/operator-tools/tor-exit-notice.html) but with all the changes needed to adhere to the Nym's operators case. + +[Avril 14th](https://avril14th.org/) is a team of professionals and optimistic realists eager to contribute to a future of privacy, decentralization and connectivity. + +```admonish warning +With a migration from `nym-gateway` to `nym-node --mode exit-gateway` the directory tree has been slightly changed. After the migration the configuration directory is in `~/.nym/nym-nodes/`. **The pathways and exact syntax in this guide is yet to be modified!** +``` + +When done with the customization needs as described below, you'll need to upload the file and reference it as the `landing_page_assets_path` on the `[http]` section of the config.toml file: +``` +landing_page_assets_path = '' +``` + +### HTML file customization + +- add your favicon logo on the line: +``` + +``` + +and your header logo on the line: +``` + +``` + +by either setting the URl to the image (if you're hosting it publicly, i.e. on your web server) +``` +href="" +``` +and +``` +src="" + +``` +**or** by adding the image inline as base64 encoded image +``` +href="href="data:image/x-icon;base64,AAABAAMA...."" +``` +and +``` +src="href="data:image/x-icon;base64,AAABAAMA...."" +``` + + +- add the email address you're willing to use for being contacted. +``` +maintainer +``` + +- If you're running the node within the US check the sections marked as FIXME, add your DNS name and uncomment those. +*** + +```html + + + + +This is a NYM Exit Gateway + + + + + + +
+

This is a NYM Exit Gateway

+

+ +

+ +

+You are most likely accessing this website because you've had some issue with +the traffic coming from this IP. This router is part of the NYM project, which is +dedicated to create outstanding +privacy software that is legally compliant without sacrificing integrity or +having any backdoors. +This router IP should be generating no other traffic, unless it has been +compromised.

+ +

+The Nym mixnet is operated by a decentralised community of node operators +and stakers. The Nym mixnet is trustless, meaning that no parts of the system +nor its operators have access to information that might compromise the privacy +of users. Nym software enacts a strict principle of data minimisation and has +no back doors. The Nym mixnet works by encrypting packets in several layers +and relaying those through a multi-layered network called a mixnet, eventually +letting the traffic exit the Nym mixnet through an exit gateway like this one. +This design makes it very hard for a service to know which user is connecting to it, +since it can only see the IP-address of the Nym exit gateway:

+ +

+ +Illustration showing how a user might connect to a service through the Nym network. The user first sends their data through three daisy-chained encrypted Nym nodes that exist on three different continents. Then the last Nym node in the chain connects to the target service over the normal internet. + + + + + + + + + + + + + + + + + + + + + +The user +This server +Your service +Nym network link +Unencrypted link + + + + + + +

+ +

+Read more about how Nym works.

+ +

+Nym relies on a growing ecosystem of users, developers and researcher partners +aligned with the mission to make sure Nym software is running, remains usable +and solves real problems. While Nym is not designed for malicious computer +users, it is true that they can use the network for malicious ends. This +is largely because criminals and hackers have significantly better access to +privacy and anonymity than do the regular users whom they prey upon. Criminals +can and do build, sell, and trade far larger and more powerful networks than +Nym on a daily basis. Thus, in the mind of this operator, the social need for +easily accessible censorship-resistant private, anonymous communication trumps +the risk of unskilled bad actors, who are almost always more easily uncovered +by traditional police work than by extensive monitoring and surveillance anyway.

+ +

+In terms of applicable law, the best way to understand Nym is to consider it a +network of routers operating as common carriers, much like the Internet +backbone. However, unlike the Internet backbone routers, Nym mixnodes do not +contain identifiable routing information about the source of a packet and do +mix the user internet traffic with that of other users, making communications +private and protecting not just the user content but the metadata +(user's IP address, who the user talks to, when, where, from what device and +more) and no single Nym node can determine both the origin and destination +of a given transmission.

+ +

+As such, there is little the operator of this Exit Gateway can do to help you +track the connection further. This Exit Gateway maintains no logs of any of the +Nym mixnet traffic, so there is little that can be done to trace either legitimate or +illegitimate traffic (or to filter one from the other). Attempts to +seize this router will accomplish nothing.

+ + + + + + +

To decentralise and enable privacy for a broad range of services, this +Exit Gateway adopts an Exit Policy +in accordance with the Tor Null ‘deny’ list +and the Tor reduced policy, +which are two established safeguards. +

+ +

+That being said, if you still have a complaint about the router, you may email the + maintainer. If complaints are related + to a particular service that is being abused, the maintainer will submit that to the + NYM Operators Community in order to add it to the Exit Policy cited above. +If approved, that would prevent this router from allowing that traffic to exit through it. +That can be done only on an IP+destination port basis, however. Common P2P ports are already blocked.

+ +

+You also have the option of blocking this IP address and others on the Nym network if you so desire. + The Nym project provides a + web service to fetch a list of all IP addresses of Nym Gateway Exit nodes that allow exiting to a +specified IP:port combination. Please be considerate when using these options.

+ +
+ + +``` +*** +### Reverse proxy +You may set up a [reverse proxy](https://www.nginx.com/resources/glossary/reverse-proxy-server/) in order to serve this landing page with proper SSL and DNS management, i.e. to resolve it to https://nym-exit. or https://nym-readme.. + +The following assumes that you're owning a domain and that you've already set the Let's Encrypt certificates on your hosting, and you've copied those on your Gateway, i.e. copy the two Let's Encript pem files on your Gateway's home folder. +Else you may obtain a Let's Encrypt certificate using a -[-certonly procedure](https://eff-certbot.readthedocs.io/en/latest/using.html#getting-certificates-and-choosing-plugins). + +``` +sudo apt install nginx +``` +If you're running your Gateway (you should...) exposing only the needed ports as listed [here](https://nymtech.net/operators/nodes/maintenance.html?highlight=port#ports), you need to add the Nginx service to your ufw configuration: +``` +sudo ufw app list +sudo ufw allow 'Nginx Full' +sudo ufw reload +``` +Disable the default Nginx landing page +``` +sudo systemctl status nginx +sudo unlink /etc/nginx/sites-enabled/default +sudo systemctl restart nginx +``` +Add your endpoint configuration to Nginx changing occurrencies below with your domain name, including the top-level domain, i.e. *avril4th.org* + +``` +sudo nano /etc/nginx/sites-available/nym-exit. +``` + +``` +server { + listen 443 ssl http2; + listen [::]:443 ssl http2; + + server_name nym-exit.; + + ssl_certificate /fullchain.pem; + ssl_certificate_key /privkey.pem; + + access_log /var/log/nginx/access.log; + error_log /var/log/nginx/error.log; + + location / { + proxy_pass http://127.0.0.1:8080; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } +} + +server { + listen 80; + listen [::]:80; + + if ($host = nym-exit.) { + return 301 https://$host$request_uri; + } + + server_name www.; + + return 301 https://$request_uri; +} +``` + +``` +sudo ln -s /etc/nginx/sites-available/nym-exit. /etc/nginx/sites-enabled/ +sudo nginx -t +sudo systemctl restart nginx +``` + +When done with the customization needs as described below, you'll need to upload the file and reference it as the `landing_page_assets_path` on the `[http]` section of the config.toml file that you'll find in the `~/.nym/gateways//config/config.toml` file +``` +nano ${HOME}/.nym/gateways//config/config.toml +``` +change +``` +landing_page_assets_path = '' +``` + +If you're running the Gateway as a service as described [here](https://nymtech.net/operators/nodes/maintenance.html#systemd) and [here](https://nymtech.net/operators/nodes/maintenance.html#following-steps-for-nym-nodes-running-as-systemd-service) you may just restart your service +``` +service nym-gateway stop +service nym-gateway start +``` +and check for the page being served either checking the service logs +``` +sudo journalctl -u nym-gateway.service | grep 8080 +``` +where you should read out +``` +... Started NymNodeHTTPServer on 0.0.0.0:8080 +``` +or just point your browser to the URI which you set above, such as https://nym-exit. + + diff --git a/documentation/operators/src/nodes/setup-guides.md b/documentation/operators/src/nodes/setup-guides.md deleted file mode 100644 index 52fdbf2ee8..0000000000 --- a/documentation/operators/src/nodes/setup-guides.md +++ /dev/null @@ -1,11 +0,0 @@ -# Node Setup Guides - -To setup any type of Nym's node, start with building [Nym's platform](../binaries/building-nym.md) on the machine (VPS) where you want to run the node. Nodes will need to be bond to Nym's wallet, setup one [here](https://nymtech.net/docs/wallet/desktop-wallet.html). - -This section contains setup guides for the following node types: -* [Mix Node](./mix-node-setup.md) -* [Gateway](./gateway-setup.md) -* [Network Requester](./network-requester-setup.md) -* [Validator](./validator-setup.md) - - diff --git a/documentation/operators/src/nodes/setup.md b/documentation/operators/src/nodes/setup.md new file mode 100644 index 0000000000..0cb1b34881 --- /dev/null +++ b/documentation/operators/src/nodes/setup.md @@ -0,0 +1,198 @@ +# Nym Node Setup & Run + +This documentation page provides a guide on how to set up and run a [NYM NODE](nym-node.md), along with explanations of available flags, commands, and examples. + +## Current version + +``` + +``` + +```admonish info +**Migrating an existing node to a new `nym-node` is simple. The steps are documented [below](#migrate).** +``` + +```admonish note +If you are a `nym-mixnode` or `nym-gateway` operator and you are not familiar wwith the binary changes called *Project Smoosh*, you can read the archived [Smoosh FAQ](../archive/smoosh-faq.md) page. +``` + +## Summary + +> Any syntax in `<>` brackets is a user's unique variable. Exchange with a corresponding name without the `<>` brackets. + +To run a new node, you can simply execute the `nym-node` command without any flags. By default, the node will set necessary configurations. If you later decide to change a setting, you can use the `-w` flag. + +The most crucial aspect of running the node is specifying the `--mode`, which can be one of three: `mixnode`, `entry-gateway`, and `exit-gateway`. + +Currently `nym-node` binary enables to run only one `--mode` at a time. In the future the operators will be able to specify multiple modes within one `nym-node`. Our goal is to have as many nodes each running all the available modes enabled and let the Nym API to position the node acoording the network needs in the beginning of each epoch. + +Every `exit-gateway` mode is basically an `entry-gateway` with NR (Network Requester) and IPR (IP Packet Router) enabled. This means that every `exit-gateway` is automatically seen as an `entry-gateway` but not the opposite. + +Gateway operators can check out the performance, connectivity and much more in our new tool [harbourmaster.nymtech.net](https://harbourmaster.nymtech.net/). + +To determine which mode your node is running, you can check the `:8080/api/v1/roles` endpoint. For example: +``` +# for http +http://:8080/api/v1/roles + +# for https reversed proxy +https:///api/v1/roles +``` + +Everything necessary will exist on your node by default. For instance, if you're running a mixnode, you'll find that a NR (Network Requester) and IPR (IP Packet Router) address exist, but they will be ignored in `mixnode` mode. + +For more information about available endpoints and their status, you can refer to: +``` +# for http +http://:8080/api/v1/swagger/#/ + +# for https reversed proxy +https:///api/v1/swagger/#/ +``` + +## Usage + +### Help Command + +There are a few changes from the individual binaries used in the past. For example by default `run` command does `init` function as well, local node `--id` will be set by default unless specified otherwise etcetera. + +Run `./nym-node --help` to see all available commands: + +~~~admonish example collapsible=true title="`./nym-node --help` output:" +``` + +``` +~~~ + +To list all available flags for each command, run `./nym-node --help` for example `./nym-node run --help`: + +~~~admonish example collapsible=true title="`./nym-node run --help` output:" +``` + +``` +~~~ + +The Wireguard flags currently have limited functionality, with `--wireguard-enabled` being the most relevant, enabling or disabling wireguard functionality. + +#### Flags Summary + +Some of the most useful flags and their explanation: + +- `--id `: Local identifier of your node. This `` determines your config path located at `~/.nym/nym-nodes//config/config.toml`, default value is `default-nym-node` +- `--config-file `: Used for the migrate command to indicate the location of the existing node config file. Default path is `~/.nym/nym-nodes/default-nym-node/config/config.toml` +- `--deny-init`: Use this flag to prevent a new node from being initialized. It's recommended to use this after the first run to avoid accidental spinning up of a second node. +- `--init-only`: Use this flag if you want to set up a node without starting it. +- `--mode`: Determines the mode of the node and is always required. +- `--write-changes`: Used to change values within the `config.toml` file after the node has been run. +- `--mnemonic`: This is for when gateways are coconut-credentials-enforced, and this mnemonic is used as the `double_spend` prevention. This account needs credit in order for it to work. +- `--expose-system-info `: Sets your system info visibility on the network. +- `--expose-system-hardware `: Sets your system hardware info visibility on the network. +- `--expose-crypto-hardware `: Sets your crypto hardware info visibility on the network. + + +## Commands & Examples + +**`nym-node` introduces a default human readible ID (local only) `default-nym-node`, which is used if there is not an explicit custom `--id ` specified. All configuration is stored in `~/.nym/nym-nodes/default-nym-node/config/config.toml` or `~/.nym/nym-nodes//config/config.toml` erespectively.** + +### Initialise & Run + +When we use `run` command the node will do `init` as well, unless we specify with a flag `--deny-init`. Below are some examples of initialising and running `nym-node` with different modes (`--mode`) like `mixnode`, `entry-gateway`, `exit-gateway`. + +```admonish note +To prevent over-flooding of our documentation we cannot provide with every single command syntax as there is a large combination of possibilities. Please use a common sense and the explanation in `--help` option. +``` + +#### Mode: `exit-gateway` + +As part of the transition, `allowed.list` on Exit Gateway embedded Network Requester was depreciated. + +**Initialise and run:** +```sh +# simple default +./nym-node run --mode exit-gateway + +# with other options +./nym-node run --id --mode exit-gateway --public-ips "$(curl -4 https://ifconfig.me)" --hostname "" --http-bind-address 0.0.0.0:8080 --mixnet-bind-address 0.0.0.0:1789 true --wireguard-enabled true +``` + +Initialise only with a custom `--id` and `--init-only` command : + +```sh +./nym-node run --id --init-only --mode exit-gateway --public-ips "$(curl -4 https://ifconfig.me)" --hostname "" --http-bind-address 0.0.0.0:8080 --mixnet-bind-address 0.0.0.0:1789 true --wireguard-enabled true +``` + +Run the node with custom `--id` without initialising +```sh +./nym-node run --id --deny-init --mode exit-gateway +``` + +#### Mode: `entry-gateway` + +**Initialise and run:** +```sh +./nym-node run --mode entry-gateway +``` + +Initialise only with a custom `--id` and `--init-only` command: +```sh +./nym-node run --id --init-only --mode entry-gateway --public-ips "$(curl -4 https://ifconfig.me)" --hostname "" --http-bind-address 0.0.0.0:8080 --mixnet-bind-address 0.0.0.0:1789 +``` + +Run the node with custom `--id` without initialising: +```sh +./nym-node run --id --deny-init --mode entry-gateway +``` + +#### Mode: `mixnode` + +**Initialise and run:** +```sh +./nym-node run --mode mixnode +``` + +Initialise only with a custom `--id` and `--init-only` command: +```sh +./nym-node run --id --init-only --mode mixnode --verloc-bind-address 0.0.0.0:1790 --public-ips "$(curl -4 https://ifconfig.me)" +``` + +Run the node with custom `--id` without initialising: +```sh +./nym-node run --id --deny-init --mode mixnode +``` + +Run the node with custom `--id` without initialising: +```sh +./nym-node run --id --deny-init --mode entry-gateway +``` + +### Migrate + +```admonish caution +Migration is a must for all deprecated nodes (`nym-mixnode`, `nym-gateway`). For backward compatibility we created an [archive section](../archive/setup-guides.md) with all the guides for individual binaries. However, the binaries from version 1.1.35 (`nym-gateway`) and 1.1.37 (`nym-mixnode`) onwards will no longer have `init` command. +``` + +To migrate a `nym-mixnode` or a `nym-gateway` to `nym-node` is fairly simple, use the `migrate` command with `--config-file` flag pointing to the original `config.toml` file, with a conditional argument defining which type of node this configuration belongs to. Examples are below. + +#### Mode: `mixnode` +```sh +# move relevant infor from config.toml +./nym-node migrate --config-file ~/.nym/mixnodes//config/config.toml mixnode + +# initialise with the new nym-node config +./nym-node run --mode mixnode --id --deny-init +``` + +#### Mode: `entry-gateway` and `exit-gateway` +```sh +# move relevant infor from config.toml +./nym-node migrate --config-file ~/.nym/gateways//config/config.toml gateway + +# initialise with the new nym-node config +./nym-node run --mode exit-gateway --id --deny-init # or change to entry-gateway +``` + +### Next steps + +If there are any problems checkout the troubleshooting section or report an issue. + +Follow up with [configuration](configuration.md) page for automation, reversed proxy setup and other tweaks, then head straight to [bonding](bonding.md) page to finalise your setup. diff --git a/documentation/operators/src/nodes/vps-setup.md b/documentation/operators/src/nodes/vps-setup.md new file mode 100644 index 0000000000..3f25edc3b5 --- /dev/null +++ b/documentation/operators/src/nodes/vps-setup.md @@ -0,0 +1,235 @@ +# VPS Setup & Configuration + +We aim for Nym Mixnet to be reliable and quality base layer of privacy accross the globe, while growing as distributed as possible. It's essential to have a fine tuned machine as a foundation for the nodes to meet the requirements and be rewarded for their work. + +```admonish info +A suboptimally configured VPS often results in a non-functional node. To follow these steps carefully will save you time and money later on. +``` + +## VPS Hardware Specs + +You will need to rent a VPS to run your node on. One key reason for this is that your node **must be able to send TCP data using both IPv4 and IPv6** (as other nodes you talk to may use either protocol). + +Tor community created a very helpful table called [*Good Bad ISPs*](https://community.torproject.org/relay/community-resources/good-bad-isps/), use that one as a guideline for your choice of ISP for your VPS. + +Currently we run [performance testing](../testing/performance.md) events to find out the best optimization. Sphinx packet decryption is CPU-bound, so more fast cores the better throughput. + +### `nym-node` + +Before we conclude the testing with exact results, these are the rough specs: + +| **Hardware** | **Minimum Specification** w +| :--- | ---: | +| CPU Cores | 4 | +| Memory | 4 GB RAM | +| Storage | 40 GB | +| Connectivity | IPv4, IPv6, TCP/IP, UDP | +| Bandwidth | 1Tb | +| Port speed | 1Gbps | + +### Nyx validator + +The specification mentioned below is for running a full node alongside the nym-api. It is recommended to run `nym-api` and a full Nyx node on the same machine for optimum performance. + +Bear in mind that credential signing is primarily CPU-bound, so choose the fastest CPU available to you. + +#### Minimum Requirements + +| Hardware | **Minimum Specification** | +|----------|--------------------------------------------| +| CPU | 8-cores, 2.8GHz base clock speed or higher | +| RAM | 16GB DDR4+ | +| Disk | 500 GiB+ NVMe SSD | + +#### Recommended Requirements + +| Hardware | **Minimum Specification** | +|----------|---------------------------------------------| +| CPU | 16-cores, 2.8GHz base clock speed or higher | +| RAM | 32GB DDR4+ | +| Disk | 1 TiB+ NVMe SSD | + + +#### Full node configuration (validator) + +To install a full node from scratch, refer to the [validator setup guide](validator-setup.md) and follow the steps outlined there. + +## VPS Configuration + +Before node or validator setup, the VPS needs to be configured and tested, to verify your connectivity and make sure that your provider wasn't dishonest with the offered services. + +### Configure your Firewall + +For a `nym-node` or Nyx validator to recieve traffic, you need to open ports on the server. The following commands will allow you to set up a firewall using `ufw`. + +1. Check `ufw`: +```sh +# check if you have ufw installed +ufw version + +# if it is not installed, install with +sudo apt install ufw -y + +# enable ufw +sudo ufw enable + +# check the status of the firewall +sudo ufw status +``` + +2. Open all needed ports to have your firewall working correctly: +```sh +# for nym-node +sudo ufw allow 1789,1790,8000,9000,9001,22/tcp + +# in case of planning to setup a WSS (for Gateway functionality) +sudo ufw allow 9001/tcp + +# in case of reverse proxy for the swagger page (for Gateway optionality) +sudo ufw allow 8080,80,443 + +# for validator +sudo ufw allow 1317,26656,26660,22,80,443/tcp +``` + +3. Check the status of the firewall: +```sh +sudo ufw status +``` + +For more information about your node's port configuration, check the [port reference table](#ports-reference-table) below. + +## Setting the ulimit + +Linux machines limit how many open files a user is allowed to have. This is called a `ulimit`. + +`ulimit` is 1024 by default on most systems. It needs to be set higher, because Nym Nodes make and receive a lot of connections with each others. + +If you see errors such as: + +```sh +Failed to accept incoming connection - Os { code: 24, kind: Other, message: "Too many open files" } +``` + +This means that the operating system is preventing network connections from being made. + +### Set the `ulimit` via `systemd` service file + +> **Replace `` variable with the name of your service, for example `nym-node`** as we migrated from `nym-mixnode`, `nym-gateway` and `nym-network-requester`. + +The ulimit setup is relevant for maintenance of Nym Node only. + +Query the `ulimit` of your `` with: + +```sh +# for nym-node +grep -i "open files" /proc/$(ps -A -o pid,cmd|grep | grep -v grep |head -n 1 | awk '{print $1}')/limits + +# for nyx validator +grep -i "open files" /proc/$(ps -A -o pid,cmd|grep nymd | grep -v grep |head -n 1 | awk '{print $1}')/limits +``` + +You'll get back the hard and soft limits, which looks something like this: + +```sh +Max open files 65536 65536 files +``` + +If your output is **the same as above**, your node will not encounter any `ulimit` related issues. + +However if either value is `1024`, you must raise the limit via the systemd service file. Add the line: + +```sh +LimitNOFILE=65536 +``` + +Reload the daemon: + +```sh +systemctl daemon-reload +``` + +or execute this as root for system-wide setting of `ulimit`: + +```sh +echo "DefaultLimitNOFILE=65535" >> /etc/systemd/system.conf +``` + +Reboot your server, and restart your node. When it comes back, use: +```sh +# for nym-node +cat /proc/$(pidof )/limits | grep "Max open files" + +# for validator +cat /proc/$(pidof nym-validator)/limits | grep "Max open files" +``` +Make sure the limit has changed to `65535`. + +### Set the ulimit on `non-systemd` based distributions + +In case you chose tmux option for Nym Node automation, see your `ulimit` list by running: + +```sh +ulimit -a + +# watch for the output line -n +-n: file descriptors 1024 +``` + +You can change it either by running a command: + +```sh +ulimit -u -n 4096 +``` + +or editing `etc/security/conf` and add the following lines: + +```sh +# Example hard limit for max opened files +username hard nofile 4096 + +# Example soft limit for max opened files +username soft nofile 4096 +``` + +Then reboot your server and restart your node. + + +## Ports reference tables + +All node-specific port configuration can be found in `$HOME/.nym///config/config.toml`. If you do edit any port configs, remember to restart your client and node processes. + +### Nym node port reference + +#### Mix Node functionality ports + +| Default port | Use | +| ------------ | ------------------------- | +| `1789` | Listen for Mixnet traffic | +| `1790` | Listen for VerLoc traffic | +| `8000` | Metrics http API endpoint | + +#### Gateway functionality ports + +| Default port | Use | +|-----------------|-------------------------------| +| `1789` | Listen for Mixnet traffic | +| `9000` | Listen for Client traffic | +| `9001` | WSS | +| `8080, 80, 443` | Reversed Proxy & Swagger page | + +#### Embedded Network Requester functionality ports + +| Default port | Use | +|--------------|---------------------------| +| `9000` | Listen for Client traffic | + +### Validator port reference + +All validator-specific port configuration can be found in `$HOME/.nymd/config/config.toml`. If you do edit any port configs, remember to restart your validator. + +| Default port | Use | +|--------------|--------------------------------------| +| `1317` | REST API server endpoint | +| `26656` | Listen for incoming peer connections | +| `26660` | Listen for Prometheus connections | diff --git a/documentation/operators/src/nodes/wallet-preparation.md b/documentation/operators/src/nodes/wallet-preparation.md new file mode 100644 index 0000000000..e89cebc3f9 --- /dev/null +++ b/documentation/operators/src/nodes/wallet-preparation.md @@ -0,0 +1,17 @@ +# Nym Wallet Preparation + +## Mainnet + +Head to our [website](https://nymtech.net/download/) and download the Nym wallet for your operating system. If pre-compiled binaries for your operating system aren't available, you can build the wallet yourself with instructions [here](https://nymtech.net/docs/wallet/desktop-wallet.html). + +If you don't already have one, please create a Nym address using the wallet, and fund it with NYM tokens. The minimum amount required to bond a node is 100 `NYM`, but make sure you have a bit more to account for gas costs. + +`NYM` can be purchased via Bity from the wallet itself with BTC or fiat, and is currently present on several [exchanges](https://www.coingecko.com/en/coins/nym#markets). + +> Remember that you can **only** use Cosmos `NYM` tokens to bond your node. You **cannot** use ERC20 representations of `NYM` to run a node. + + +## Sandbox testnet + +Make sure to download a wallet and create an account as outlined above. Then head to our [Operators Element channel](https://matrix.to/#/#operators:nymtech.chat) and request testnet tokens. + diff --git a/documentation/operators/src/testing/node-setup.md b/documentation/operators/src/testing/node-setup.md index 3c1a1ca84c..23bb4fe2d5 100644 --- a/documentation/operators/src/testing/node-setup.md +++ b/documentation/operators/src/testing/node-setup.md @@ -19,16 +19,8 @@ After you signed your node (or several) into the Performance testing environment - Before you re-start your node, follow the steps below -#### 2. If you run `nym-gateway` proceed with these steps. If not, go to the next point - - Make sure to have your `nym-gateway` setup as [Nym Exit Gateway](../legal/exit-gateway.md) following the commands [here](../nodes/gateway-setup.md#initialising-exit-gateway) - - Enable `[ip_packet_router]` (IPR) in your `~/.nym/gateways/*/config/config.toml` and IPv4 and IPv6 with [this script](https://gist.github.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77) by running the two command below -```sh -# command to enable IPR -./nym-gateway setup-ip-packet-router --id --enabled true - -# script to enable IPv4 and IPv6 -curl -o enable_networking_for_nym_nodes.sh https://gist.githubusercontent.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77/raw/0840e1d2ee9949716c45655457d198607dfd3107/enable_networking_for_nym_nodes.sh -L && chmod u+x enable_networking_for_nym_nodes.sh && sudo ./enable_networking_for_nym_nodes.sh -``` +#### 2. If you run `gateway` mode proceed with these steps. If not, go to the next point + - Make sure to have your `nym-node --exit-gateway` setup as [Nym Exit Gateway](../legal/exit-gateway.md) following the commands [here](..//nodes/nym-node.md#quick-nym-node---mode-exit-gateway-setup) -#### 3. Restart your node with root privileges +#### 3. Restart your node with root privileges and verify connectivity - Either in a root shell or with `sudo -E` command - - In case you run your node as a [`systemd` service](../nodes/maintenance.md#systemd) make sure to run `systemctl daemon-reload` before the service restart + - In case you run your node as a [`systemd` service](../nodes/maintenance.md#systemd) make sure to run `systemctl daemon-reload` before the `service nym-node restart` + - Verify that it all worked out on [Nym Harbour Master](https://harbourmaster.nymtech.net/) ## Troubleshooting -If you come to any errors during the setup see troubleshooting page related to [Mix Nodes](../nodes/troubleshooting.md#mix-nodes) and [Gateways](../nodes/troubleshooting.md#gateways--network-requesters). In case your issue isn't documented ask in our Element [Node Operators channel](https://matrix.to/#/#operators:nymtech.chat) or raise an [issue](https://github.com/nymtech/nym/issues) on Github. +If you come to any errors during the setup visit [troubleshooting page](../troubleshooting/nym-node.md#gateways-mode). In case your issue isn't documented ask in our Element [Node Operators channel](https://matrix.to/#/#operators:nymtech.chat) or raise an [issue](https://github.com/nymtech/nym/issues) on Github. diff --git a/documentation/operators/src/testing/performance.md b/documentation/operators/src/testing/performance.md index ec4108d313..232da85700 100644 --- a/documentation/operators/src/testing/performance.md +++ b/documentation/operators/src/testing/performance.md @@ -2,7 +2,7 @@ > To configure your node for a testing event, visit [node setup page](node-setup.md). -Nym Mixnet has been running on mainnet for quite some time. There is still work to be done in order for the network to meet its full potential - mass adoption of privacy through fully distributed Mixnet. +Nym Mixnet has been running on mainnet for quite some time. There is still work to be done in order for the network to meet its full potential - mass adoption of privacy through fully distributed Mixnet. Nym asks its decentralised community of operators to join a series of performance testing events in order to **increase the overall quality of the Mixnet**. The main takeaways of such event are: @@ -15,10 +15,12 @@ Nym asks its decentralised community of operators to join a series of performanc 5. Increase quality of Nym Nodes 6. Show each operator a way to monitor their nodes in a distributed fashion +Visit [Fast and Furious web page]({{performance_testing_webpage}}) and [Nym Harbour Master](https://harbourmaster.nymtech.net/) Gateways monitoring page to read more about the performance testing and the results of it. + ## Performance Testing Work Flow * Nym runs a paralel network environment [validator.performance.nymte.ch]({{performance_validator}}) with a chain ID `perf` -* Operators of Nym Nodes (currently `nym-mixnode` and `nym-gateway`) join by following easy steps on [performance testing web page]({{performance_testing_webpage}}), including simplified node authentication signature (while keep running their nodes on the mainnet) +* Operators of Nym Nodes join by following easy steps on [performance testing web page]({{performance_testing_webpage}}), including simplified node authentication signature (while keep running their nodes on the mainnet) * Once signed in, operators will be asked to swap their binary for the modified version with metrics endpoint to be able to connect their own [monitoring system](templates.md) * Core node data will be fed to a unique mixnet contract for the `perf` side chain * Nym starts a new API and start packet transition in high load through these nodes in both settings diff --git a/documentation/operators/src/testing/templates.md b/documentation/operators/src/testing/templates.md index 5bc922163f..d55213cfad 100644 --- a/documentation/operators/src/testing/templates.md +++ b/documentation/operators/src/testing/templates.md @@ -1,4 +1,4 @@ - Metrics of Performance Testing +# Metrics of Performance Testing At Nym as well as several core community operators had setup metrics monitors for a clear overview of node performance. diff --git a/documentation/operators/src/tokenomics/mixnet-rewards.md b/documentation/operators/src/tokenomics/mixnet-rewards.md new file mode 100644 index 0000000000..f0856b6757 --- /dev/null +++ b/documentation/operators/src/tokenomics/mixnet-rewards.md @@ -0,0 +1,46 @@ + + + + +### Mix Node Reward Estimation API endpoint + +The Reward Estimation API endpoint allows Mix Node operators to estimate the rewards they could earn for running a Nym Mix Node with a specific `MIX_ID`. + +> The `` can be found in the "Mix ID" column of the [Network Explorer](https://explorer.nymtech.net/network-components/mixnodes/active). + +The endpoint is a particularly common for Mix Node operators as it can provide an estimate of potential earnings based on factors such as the amount of traffic routed through the Mix Node, the quality of the Mix Node's performance, and the overall demand for Mix Nodes in the network. This information can be useful for Mix Node operators in deciding whether or not to run a Mix Node and in optimizing its operations for maximum profitability. + +Using this API endpoint returns information about the Reward Estimation: + +```sh +/status/mixnode//reward-estimation +``` + +Query Response: + +```sh + "estimation": { + "total_node_reward": "942035.916721770541325331", + "operator": "161666.263307386408152071", + "delegates": "780369.65341438413317326", + "operating_cost": "54444.444444444444444443" + }, +``` + +> The unit of value is measured in `uNYM`. + +- `estimated_total_node_reward` - An estimate of the total amount of rewards that a particular Mix Node can expect to receive during the current epoch. This value is calculated by the Nym Validator based on a number of factors, including the current state of the network, the number of Mix Nodes currently active in the network, and the amount of network traffic being processed by the Mix Node. + +- `estimated_operator_reward` - An estimate of the amount of rewards that a particular Mix Node operator can expect to receive. This value is calculated by the Nym Validator based on a number of factors, including the amount of traffic being processed by the Mix Node, the quality of service provided by the Mix Node, and the operator's stake in the network. + +- `estimated_delegators_reward` - An estimate of the amount of rewards that Mix Node delegators can expect to receive individually. This value is calculated by the Nym Validator based on a number of factors, including the amount of traffic being processed by the Mix Node, the quality of service provided by the Mix Node, and the delegator's stake in the network. + +- `estimated_node_profit` - An estimate of the profit that a particular Mix node operator can expect to earn. This value is calculated by subtracting the Mix Node operator's `operating_costs` from their `estimated_operator_reward` for the current epoch. + +- `estimated_operator_cost` - An estimate of the total cost that a particular Mix Node operator can expect to incur for their participation. This value is calculated by the Nym Validator based on a number of factors, including the cost of running a Mix Node, such as server hosting fees, and other expenses associated with operating the Mix Node. + +### Validator: Installing and configuring nginx for HTTPS +#### Setup +[Nginx](https://www.nginx.com/resources/glossary/nginx) is an open source software used for operating high-performance web servers. It allows us to set up reverse proxying on our validator server to improve performance and security. + +Install `nginx` and allow the 'Nginx Full' rule in your firewall: diff --git a/documentation/operators/src/nodes/troubleshooting.md b/documentation/operators/src/troubleshooting/nym-node.md similarity index 71% rename from documentation/operators/src/nodes/troubleshooting.md rename to documentation/operators/src/troubleshooting/nym-node.md index 96640e45e6..e37a0b72ba 100644 --- a/documentation/operators/src/nodes/troubleshooting.md +++ b/documentation/operators/src/troubleshooting/nym-node.md @@ -1,4 +1,4 @@ -# Troubleshooting +# Nym Node Troubleshooting ## Binary Build Problems @@ -38,22 +38,40 @@ All config and keys files are stored in a directory named after your `id` which The directory structure for each node will be roughly as follows: ``` -bob@nym:~$ tree /home/nym/.nym/mixnodes/ -/home/nym/.nym/mixnodes/ -|-- nym010 -| |-- config -| | `-- config.toml -| `-- data -| |-- private_identity.pem -| |-- private_sphinx.pem -| |-- public_identity.pem -| `-- public_sphinx.pem +bob@desktop:~/nym/target/release# tree ~/.nym/nym-nodes/ +~/.nym/nym-nodes/ +└── default-nym-node + ├── config + │   └── config.toml + └── data + ├── aes128ctr_ipr_ack + ├── aes128ctr_nr_ack + ├── clients.sqlite + ├── cosmos_mnemonic + ├── description.toml + ├── ed25519_identity + ├── ed25519_identity.pub + ├── ed25519_ipr_identity + ├── ed25519_ipr_identity.pub + ├── ed25519_nr_identity + ├── ed25519_nr_identity.pub + ├── ipr_gateways_info_store.sqlite + ├── nr_gateways_info_store.sqlite + ├── nr_persistent_reply_store.sqlite + ├── x25519_ipr_dh + ├── x25519_ipr_dh.pub + ├── x25519_noise + ├── x25519_noise.pub + ├── x25519_nr_dh + ├── x25519_nr_dh.pub + ├── x25519_sphinx + └── x25519_sphinx.pub ``` > If you `cat` the `public_sphinx.pem` key, the output will be different from the public key you will see on Nym [dashboard](https://sandbox-explorer.nymtech.net/). The reason for this is that `.pem` files are encoded in **base64**, however on the web they are in **base58**. Don't be confused if your keys look different. They are the same keys, just with different encoding :) -## Mix Nodes +## Mixnode Mode ### How can I tell my node is up and running and mixing traffic? @@ -97,27 +115,27 @@ sudo lsof -i TCP:1789 # if you have specified a different port in your mixnode c This command should return something like this: ``` -nym-mixno 103349 root 53u IPv6 1333229972 0t0 TCP [2a03:b0c0:3:d0::ff3:f001]:57844->[2a01:4f9:c011:38ae::5]:1789 (ESTABLISHED) -nym-mixno 103349 root 54u IPv4 1333229973 0t0 TCP nym:57104->194.5.78.73:1789 (ESTABLISHED) -nym-mixno 103349 root 55u IPv4 1333229974 0t0 TCP nym:48130->static.236.109.119.168.clients.your-server.de:1789 (ESTABLISHED) -nym-mixno 103349 root 56u IPv4 1333229975 0t0 TCP nym:52548->vmi572614.contaboserver.net:1789 (ESTABLISHED) -nym-mixno 103349 root 57u IPv6 1333229976 0t0 TCP [2a03:b0c0:3:d0::ff3:f001]:43244->[2600:1f18:1031:2401:c04b:2f25:ca79:fef3]:1789 (ESTABLISHED) +nym-node 103349 root 53u IPv6 1333229972 0t0 TCP [2a03:b0c0:3:d0::ff3:f001]:57844->[2a01:4f9:c011:38ae::5]:1789 (ESTABLISHED) +nym-node 103349 root 54u IPv4 1333229973 0t0 TCP nym:57104->194.5.78.73:1789 (ESTABLISHED) +nym-node 103349 root 55u IPv4 1333229974 0t0 TCP nym:48130->static.236.109.119.168.clients.your-server.de:1789 (ESTABLISHED) +nym-node 103349 root 56u IPv4 1333229975 0t0 TCP nym:52548->vmi572614.contaboserver.net:1789 (ESTABLISHED) +nym-node 103349 root 57u IPv6 1333229976 0t0 TCP [2a03:b0c0:3:d0::ff3:f001]:43244->[2600:1f18:1031:2401:c04b:2f25:ca79:fef3]:1789 (ESTABLISHED) ``` ##### Query `systemd` journal with `journalctl` ``` -sudo journalctl -u nym-mixnode -o cat | grep "Since startup mixed" +sudo journalctl -u nym-node -o cat | grep "Since startup mixed" ``` -If you have created `nym-mixnode.service` file (i.e. you are running your Mix Node via `systemd`) then this command shows you how many packets have you mixed so far, and should return a list of messages like this: +If you have created `nym-node.service` file (i.e. you are running your Ny, Node via `systemd`) then this command shows you how many packets have you mixed so far, and should return a list of messages like this: ``` -2021-05-18T12:35:24.057Z INFO nym_mixnode::node::metrics > Since startup mixed 233639 packets! -2021-05-18T12:38:02.178Z INFO nym_mixnode::node::metrics > Since startup mixed 233739 packets! -2021-05-18T12:40:32.344Z INFO nym_mixnode::node::metrics > Since startup mixed 233837 packets! -2021-05-18T12:46:08.549Z INFO nym_mixnode::node::metrics > Since startup mixed 234081 packets! -2021-05-18T12:56:57.129Z INFO nym_mixnode::node::metrics > Since startup mixed 234491 packets! +2021-05-18T12:35:24.057Z INFO nym_node::node::metrics > Since startup mixed 233639 packets! +2021-05-18T12:38:02.178Z INFO nym_node::node::metrics > Since startup mixed 233739 packets! +2021-05-18T12:40:32.344Z INFO nym_node::node::metrics > Since startup mixed 233837 packets! +2021-05-18T12:46:08.549Z INFO nym_node::node::metrics > Since startup mixed 234081 packets! +2021-05-18T12:56:57.129Z INFO nym_node::node::metrics > Since startup mixed 234491 packets! ``` You can add ` | tail` to the end of the command to watch for new entries in real time if needed. @@ -126,7 +144,7 @@ You can add ` | tail` to the end of the command to watch for new entries in real A `build-info` command prints the build information like commit hash, rust version, binary version just like what command `--version` does. However, you can also specify an `--output=json` flag that will format the whole output as a json, making it an order of magnitude easier to parse. -For example `./target/debug/nym-network-requester --no-banner build-info --output json` will return: +For example `./target/debug/nym-node --no-banner build-info --output json` will return: ``` {"binary_name":"nym-network-requester","build_timestamp":"2023-07-24T15:38:37.00657Z","build_version":"1.1.23","commit_sha":"c70149400206dce24cf20babb1e64f22202672dd","commit_timestamp":"2023-07-24T14:45:45Z","commit_branch":"feature/simplify-cli-parsing","rustc_version":"1.71.0","rustc_channel":"stable","cargo_profile":"debug"} @@ -140,7 +158,7 @@ For example `./target/debug/nym-network-requester --no-banner build-info --outpu nmap -p 1789 -Pn ``` -If your Mix Node is configured properly it should output something like this: +If your Nym Node is configured properly it should output something like this: ``` bob@desktop:~$ nmap -p 1789 95.296.134.220 -Pn @@ -164,28 +182,28 @@ You can query Gateways by replacing `nym-mixnodes` with `nym-gateways` in the ab #### Check with Network API -We currently have an API set up returning our metrics tests of the network. There are two endpoints to ping for information about your Mix Node, `report` and `history`. Find more information about this in the [Mixnodes metrics documentation](./maintenance.md#metrics--api-endpoints). +We currently have an API set up returning our metrics tests of the network. There are two endpoints to ping for information about your Mix Node, `report` and `history`. Find more information about this in the [Mixnodes metrics documentation](../nodes/maintenance.md#metrics--api-endpoints). ### Why is my node not mixing any packets? If you are still unable to see your node on the dashboard, or your node is declaring it has not mixed any packets, there are several potential issues: -- The firewall on your host machine is not configured properly. Checkout the [instructions](./maintenance.md#configure-your-firewall). +- The firewall on your host machine is not configured properly. Checkout the [instructions](../nodes/vps-setup.md#configure-your-firewall). - You provided incorrect information when bonding your node. -- You are running your Mix Node from a VPS without IPv6 support. -- You did not use the `--announce-host` flag while running the Mix Node from your local machine behind NAT. -- You did not configure your router firewall while running the Mix Node from your local machine behind NAT, or you are lacking IPv6 support. -- Your Mix Node is not running at all, it has either exited / panicked or you closed the session without making the node persistent. Check out the [instructions](./maintenance.md#automating-your-node-with-tmux-and-systemd). +- You are running your node from a VPS without IPv6 support. + +- You did not configure your router firewall while running the node from your local machine behind NAT, or you are lacking IPv6 support +- Your Mix Node is not running at all, it has either exited / panicked or you closed the session without making the node persistent. Check out the [instructions](../nodes/configuration.md#automating-your-node-with-tmux-and-systemd). ```admonish caution -Your Mix Node **must speak both IPv4 and IPv6** in order to cooperate with other nodes and route traffic. This is a common reason behind many errors we are seeing among node operators, so check with your provider that your VPS is able to do this! +Your Nym Node **must speak both IPv4 and IPv6** in order to cooperate with other nodes and route traffic. This is a common reason behind many errors we are seeing among node operators, so check with your provider that your VPS is able to do this! ``` #### Incorrect bonding information -Check that you have provided the correct information when bonding your Mix Node in the web wallet interface. When in doubt, un-bond and then re-bond your node! +Check that you have provided the correct information when bonding your Nym Node in the web wallet interface. When in doubt, un-bond and then re-bond your node! -> All delegated stake will be lost when un-bonding! However the Mix Node must be operational in the first place for the delegation to have any effect. +> All delegated stake will be lost when un-bonding! However the Nym Node must be operational in the first place for the delegation to have any effect. #### Missing `announce-host` flag @@ -222,19 +240,14 @@ bob@nym:~$ hostname -I ### Running on a local machine behind NAT with no fixed IP address -Your ISP has to be IPv6 ready if you want to run a Mix Node on your local machine. Sadly, in 2020, most of them are not and you won't get an IPv6 address by default from your ISP. Usually it is an extra paid service or they simply don't offer it. +Your ISP has to be IPv6 ready if you want to run a Nym Node on your local machine. Sadly, in 2020, most of them are not and you won't get an IPv6 address by default from your ISP. Usually it is an extra paid service or they simply don't offer it. -Before you begin, check if you have IPv6 [here](https://test-ipv6.cz/) or by running command explained in the [section above](./troubleshooting.md#no-ipv6-connectivity). If not, then don't waste your time to run a node which won't ever be able to mix any packet due to this limitation. Call your ISP and ask for IPv6, there is a plenty of it for everyone! +Before you begin, check if you have IPv6 [here](https://test-ipv6.cz/) or by running command explained in the [section above](#no-ipv6-connectivity). If not, then don't waste your time to run a node which won't ever be able to mix any packet due to this limitation. Call your ISP and ask for IPv6, there is a plenty of it for everyone! -If all goes well and you have IPv6 available, then you will need to `init` the Mix Node with an extra flag, `--announce-host`. You will also need to edit your `config.toml` file each time your IPv4 address changes, that could be a few days or a few weeks. Check the your IPv4 in the [section above](./troubleshooting.md#no-ipv6-connectivity). +If all goes well and you have IPv6 available, then you will need to `init` the Nym Node with an extra flag, `--announce-host`. You will also need to edit your `config.toml` file each time your IPv4 address changes, that could be a few days or a few weeks. Check the your IPv4 in the [section above](#no-ipv6-connectivity). Additional configuration on your router might also be needed to allow traffic in and out to port 1789 and IPv6 support. -Here is a sample of the `init` command example to create the Mix Node config. - -``` -./nym-mixnode init --id --host 0.0.0.0 --announce-host 85.160.12.13 -``` - `--host 0.0.0.0` should work every time even if your local machine IPv4 address changes. For example on Monday your router gives your machine an address `192.168.0.13` and on Wednesday, the [DHCP](https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol) lease will end and you will be assigned `192.168.0.14`. Using `0.0.0.0` should avoid this without having to set any static IP in your router's configuration. @@ -244,11 +257,11 @@ Make sure you check if your node is really mixing. We are aiming to improve the ### Accidentally killing your node process on exiting session -When you close your current terminal session, you need to make sure you don't kill the Mix Node process! There are multiple ways on how to make it persistent even after exiting your ssh session, the easiest solution is to use `tmux` or `nohup`, and the more elegant solution is to run the node with `systemd`. Read the automation manual [here](./maintenance.md#automating-your-node-with-tmux-and-systemd). +When you close your current terminal session, you need to make sure you don't kill the Mix Node process! There are multiple ways on how to make it persistent even after exiting your ssh session, the easiest solution is to use `tmux` or `nohup`, and the more elegant solution is to run the node with `systemd`. Read the automation manual [here](../nodes/configuration.md#automating-your-node-with-tmux-and-systemd). ### Common errors and warnings -Most of the `ERROR` and `WARN` messages in your node logs are benign - as long as your node outputs `since startup mixed X packets!` (`X` bust be > 0) in your logs (and this number increases over time), your node is mixing packets. If you want to be sure, check the Nym [dashboard](https://sandbox-explorer.nymtech.net/) or see other ways on how to check if your node is mixing properly as outlined in the section [**How can I tell my node is up and running and mixing traffic?**](./troubleshooting.md#how-can-i-tell-my-node-is-up-and-running-and-mixing-traffic?) above. +Most of the `ERROR` and `WARN` messages in your node logs are benign - as long as your node outputs `since startup mixed X packets!` (`X` bust be > 0) in your logs (and this number increases over time), your node is mixing packets. If you want to be sure, check the Nym [dashboard](https://sandbox-explorer.nymtech.net/) or see other ways on how to check if your node is mixing properly as outlined in the section [**How can I tell my node is up and running and mixing traffic?**](#how-can-i-tell-my-node-is-up-and-running-and-mixing-traffic?) above. More specific errors and warnings are covered below. @@ -260,48 +273,48 @@ If you are running into issues with an error including the following: thread 'tokio-runtime-worker' panicked at 'Failed to create TCP listener: Os { code: 99, kind: AddrNotAvailable, message: "Cannot assign requested address" }' ``` -Then you need to `--announce-host ` and `--host ` on startup. This issue is addressed [above](./troubleshooting.md#missing-`announce-host`-flag) +Then you need to `--announce-host ` and `--host ` on startup. This issue is addressed [above](#missing-`announce-host`-flag) ### Can I use a port other than 1789? Yes! Here is what you will need to do: -Assuming you would like to use port `1337` for your Mix Node, you need to open the new port (and close the old one): +Assuming you would like to use port `1337` for your Nym Node, you need to open the new port (and close the old one): ``` sudo ufw allow 1337 sudo ufw deny 1789 ``` -And then edit the Mix Node's config. +And then edit the node's `config.toml`. > If you want to change the port for an already running node, you need to stop the process before editing your config file. -The config file is located at `~/.nym/mixnodes//config/config.toml`. +The config file is located at `~/.nym/nym-nodes//config/config.toml`. For example, assuming `` was chosen to be `alice-node`: ``` -nano ~/.nym/mixnodes/alice-node/config/config.toml +nano ~/.nym/nym-nodes/alice-node/config/config.toml ``` -You will need to edit two parts of the file. `announce_address` and `listening_address` in the config.toml file. Simply replace `:1789` (the default port) with `:1337` (your new port) after your IP address. +You will need to edit two parts of the file. `announce_address` and `listening_address` in the `config.toml` file. Simply replace `:1789` (the default port) with `:1337` (your new port) after your IP address. Finally, restart your node. You should see if the Mix Node is using the port you have changed in the config.toml file right after you run the node. -### What is `verloc` and do I have to configure my Mix Node to implement it? +### What is `verloc` and do I have to configure my Nym Node to implement it? `verloc` is short for _verifiable location_. Mix Nodes and Gateways now measure speed-of-light distances to each other, in an attempt to verify how far apart they are. In later releases, this will allow us to algorithmically verify node locations in a non-fake-able and trustworthy manner. You don't have to do any additional configuration for your node to implement this, it is a passive process that runs in the background of the mixnet from version `0.10.1` onward. -## Gateways & Network Requesters +## Gateways Mode -### My Gateway is running but appears offline in the explorer +### My `exit-gateway` is running but appears offline in the explorer Let your Gateway run and follow these steps: -1. Check if your [firewall configuration](./maintenance.md#configure-your-firewall) is active and if the necessary ports are open / allowed, including the ones for Swagger page and Reversed proxy/WSS if this is your case. +1. Check if your [firewall configuration](../nodes/vps-setup.md#configure-your-firewall) is active and if the necessary ports are open / allowed, including the ones for Swagger page and Reversed proxy/WSS if this is your case. 2. See if the Gateway is not on the [list of blacklisted Gateways](https://validator.nymtech.net/api/v1/gateways/blacklisted) 3. If it's blacklisted, check out the [point below](#my-gateway-is-blacklisted) @@ -312,8 +325,8 @@ Nym API measures performance by routing traffic through the Mixnet. If the avera In case your Gateway appeared on the [blacklist](https://validator.nymtech.net/api/v1/gateways/blacklisted), it's because there is some flaw in the configuration. The most common sources of problems are: - Bonding before starting the node/service -- Bonding before opening [the needed ports](maintenance.md#configure-your-firewall) -- VPS restarted without operator having a [systemd automation](maintenance.md#systemd) or some alert notification flow setup +- Bonding before opening [the needed ports](../nodes/vps-setup.md#configure-your-firewall) +- VPS restarted without operator having a [systemd automation](../nodes/configuration.md#systemd) or some alert notification flow setup What to do: @@ -321,29 +334,29 @@ What to do: - Open all needed ports - Wait until your node gets above 50% of performance (average of last 24h) - this will likely take several hours, up to a day. During this time your node is tested by `nym-api` and every positive response picks up your Gateway's routing score. -**Do not restart your Gateway without reason, your routing score will only get worse!** +**Do not restart your Nym Node without reason, your routing score will only get worse!** ### My exit Gateway "is still not online..." The Nyx chain epoch takes up to 60 min. To prevent the Gateway getting blacklisted, it's essential to start it before the bonding process and let it running. In case it already got [blacklisted](#my-gateway-is-backlisted) check the steps above. - + diff --git a/documentation/operators/src/troubleshooting/validators.md b/documentation/operators/src/troubleshooting/validators.md new file mode 100644 index 0000000000..3deba04386 --- /dev/null +++ b/documentation/operators/src/troubleshooting/validators.md @@ -0,0 +1,16 @@ +# Validators Troubleshooting + +### Common reasons for your validator being jailed + +The most common reason for your validator being jailed is that your validator is out of memory because of bloated syslogs. + +Running the command `df -H` will return the size of the various partitions of your VPS. + +If the `/dev/sda` partition is almost full, try pruning some of the `.gz` syslog archives and restart your validator process. + + +## Where can I get more help? + +The fastest way to reach one of us or get a help from the community, visit our [Telegram Node Setup Help Chat](https://t.me/nymchan_help_chat) or head to our [Discord](https://Discord.gg/nym). + +For more tech heavy question join our [Matrix core community channel](https://matrix.to/#/#general:nymtech.chat), where you can meet other builders and Nym core team members. diff --git a/documentation/operators/src/troubleshooting/vps-setup.md b/documentation/operators/src/troubleshooting/vps-setup.md new file mode 100644 index 0000000000..5a58f92cab --- /dev/null +++ b/documentation/operators/src/troubleshooting/vps-setup.md @@ -0,0 +1,34 @@ +# Troubleshooting VPS Setup + +## Incorrect Gateway Network Check + +If you followed all the steps listed in [Connectivity Test and Configuration](../nodes/vps-setup.md#connectivity-test-and-configuration) chapter of VPS Setup and you still have a problem with a correct connectivity for page in + +1. Tor community created a helpful [table of ISPs](https://community.torproject.org/relay/community-resources/good-bad-isps/). Make sure your one is listed there as a *"good ISP"*. If not, consider migrating! +2. Checkout your VPS dashboard and make sure your IPv6-public enabled. +3. If you are able to add IPv6 address `/64` range, do it. + +![](../images/ipv6_64.png) + + +## Virtual IPs and hosting via Google & AWS + +For true internet decentralization we encourage operators to use diverse VPS providers instead of the largest companies offering such services. If for some reasons you have already running AWS or Google and want to setup a `` there, please read the following. + +On some services (AWS, Google, etc) the machine's available bind address is not the same as the public IP address. In this case, bind `--host` to the local machine address returned by `$(curl -4 https://ifconfig.me)`, but that may not the public IP address to bond your `` in the wallet. + +You can run `ifconfig` command. For example, on a Google machine, you may see the following output: + +```sh +ens4: flags=4163 mtu 1460 + inet 10.126.5.7 netmask 255.255.255.255 broadcast 0.0.0.0 + ... +``` + +The `ens4` interface has the IP `10.126.5.7`. But this isn't the public IP of the machine, it's the IP of the machine on Google's internal network. Google uses virtual routing, so the public IP of this machine is something else, maybe `36.68.243.18`. + +To find the right IP configuration, contact your VPS provider for support to find the right public IP and use it to bond your `` with the `nym-api` via Nym wallet. + +On self-hosted machine it's a bit more tricky. In that case as an operator you must be sure that your ISP allows for public IPv4 and IPv6 and then it may be a bit of playing around to find the right configuration. One way may be to bind your binary with the `--host` flag to local address `127.0.0.1` and run `echo "$(curl -4 https://ifconfig.me)"` to get a public address which you use to bond your Mix Node to `nym-api` via Nym wallet. + +It's up to you as a node operator to ensure that your public and private IPs match up properly.