Compare commits

..

36 Commits

Author SHA1 Message Date
durch 77c4acf602 Add RESET_CONFIG option to entrypoint 2025-06-06 09:38:28 +02:00
durch f4d0ac855c Add periodic route data cleanup to epoch operations
Implements automatic cleanup of route monitoring results to prevent
unbounded storage growth while maintaining data for performance analysis.

- Add purge_old_routes() method to StorageManager following existing patterns
- Integrate route cleanup into purge_old_statuses() wrapper function
- Route data now purged every epoch with 48-hour retention, to facilitate comparisons with legacy data
- Update logging to reflect cleanup of both node statuses and routes
2025-06-06 09:12:21 +02:00
durch eb1c7d649e Client per request 2025-06-05 12:06:15 +02:00
durch 75f34ef51b Add timeout to locust 2025-06-05 11:43:04 +02:00
durch 4f7fa557d5 Optimize database queries by eliminating N+1 patterns in simulation system
This commit addresses critical N+1 query performance issues identified in
the reward simulation system. The optimizations significantly reduce database
round trips and improve performance when processing large datasets.

**Key Optimizations:**

1. **Batch Identity Key Lookups**
   - Added `get_mixnode_identity_keys_batch()` and `get_gateway_identity_keys_batch()`
   - Updated simulation performance conversion to use batch operations
   - Reduced from N individual queries to 2 batch queries

2. **Batch Node Classification**
   - Added `classify_nodes_batch()` method for mixnode/gateway determination
   - Updated reliability calculation methods to use batch classification
   - Reduced from N individual lookups to 2 batch queries

3. **Batch Epoch Metadata Enhancement**
   - Added `count_simulated_node_performance_for_epochs_batch()`
   - Added `get_available_calculation_methods_for_epochs_batch()`
   - Updated API handlers to use batch operations for metadata enhancement
   - Reduced from 2N queries to 2 batch queries for epoch data

4. **Bulk Insert Optimizations**
   - Converted individual INSERT operations to use `sqlx::QueryBuilder::push_values()`
   - Optimized simulation data insertion methods
   - Eliminated transaction overhead from individual inserts

**Performance Impact:**
- Before: N+2N database queries for N nodes/epochs
- After: 2+2 batch queries regardless of dataset size
- Significant performance improvement for large simulation datasets

All changes maintain backward compatibility while providing substantial
performance benefits for the reward simulation system.
2025-06-05 10:45:08 +02:00
durch a96fb098c2 Locust sleep if no clients are available 2025-06-05 09:54:39 +02:00
durch ad5c6ab829 Enhance simulation system with performance comparison framework
Refactors the simulation system to focus on performance methodology comparison
rather than reward amounts, enabling robust analysis of old vs new calculation
methods. Key improvements:

- Replace simulated_rewards table with performance_comparisons for better metrics
- Add performance_rankings table for ranking analysis across methodologies
- Enhance database schema with additional performance tracking fields
- Update simulation coordinator to use performance-focused data structures
- Add comprehensive performance ranking calculations
- Improve API models and handlers for performance comparison workflows
- Update SQLx query cache with new database schema changes

This provides a foundation for data-driven performance methodology evaluation
while maintaining separation from actual reward calculations.
2025-06-04 16:59:42 +02:00
durch b3d07e8832 Tests 2025-06-04 11:28:47 +02:00
durch e761255174 Add complete simulation API layer for reward method comparison
This completes Phase 3 of the simulation system implementation:

- Add comprehensive REST API endpoints for simulation data access
- Implement /v1/simulation/* routes with full CRUD operations
- Support JSON/CSV export for external analysis
- Add statistical comparison between old vs new methods
- Provide node performance history tracking
- Include proper error handling and response formatting
- Simplify simulation coordinator to remove unused complex return types
- Clean up dead code while maintaining all functionality
- Pass clippy with no warnings

The simulation API provides complete access to:
- Simulation epoch listing and details
- Method comparison analytics (old 24h vs new 1h)
- Node performance analysis across epochs
- Route reliability statistics
- Export capabilities for further analysis

All simulation data is persisted and accessible via REST endpoints.
2025-06-03 15:38:58 +02:00
durch e4a20f9cf5 Implement core simulation logic for dual reward calculations
Add complete simulation engine that compares old (24h cache-based) vs new (1h route-based)
reward calculation methodologies with full integration into epoch operations.

Core Simulation Engine:
- Add SimulationCoordinator with configurable time windows and comparison settings
- Implement dual calculation methods with proper Performance type conversions
- Add comprehensive error handling with DatabaseError variant in RewardingError
- Store simulation results in database with proper relationship constraints

Old Method Implementation (24h Cache-Based):
- Wrap existing reliability calculation using get_all_avg_mix_reliability_in_last_24hr()
- Convert reliability percentages to Performance types using from_percentage_value()
- Maintain exact same logic as production for accurate baseline comparison
- Generate simulation data structures with proper metadata

New Method Implementation (1h Route-Based):
- Leverage calculate_corrected_node_reliabilities_for_interval() for route analysis
- Support configurable time windows (default 1 hour vs 24 hours)
- Provide detailed route statistics including success rates and failure analysis
- Convert route reliability data to Performance types with naive_try_from_f64()

Epoch Operations Integration:
- Extend EpochAdvancer struct with optional SimulationConfig field
- Update constructor and start method to accept simulation configuration
- Add simulation trigger in perform_epoch_operations() before real rewarding
- Ensure simulation failures don't break epoch advancement process

CLI Integration:
- Update run.rs to handle both --enable-rewarding and --simulate-rewarding modes
- Create SimulationConfig from rewarding.debug configuration settings
- Implement mutual exclusivity between real rewarding and simulation mode
- Skip permission checks for simulation-only mode (no blockchain transactions)

The simulation system runs in parallel with epoch operations, storing comparative
data for analysis without affecting production reward distribution.
2025-06-03 14:52:13 +02:00
durch 1eefe8a579 Add simulated rewarding system foundation
Implement foundation for simulated reward calculations to compare old (24h cache-based)
vs new (1h route-based) methodologies without blockchain transactions.

Database Changes:
- Add migration with 4 new tables for simulation system
- simulated_reward_epochs: tracks each simulation run
- simulated_node_performance: stores performance calculations
- simulated_rewards: stores reward calculation results
- simulated_route_analysis: metadata for route analysis
- Add comprehensive indexes for efficient querying

Configuration Changes:
- Add simulation_mode flag to rewarding configuration
- Add CLI flag --simulate-rewarding with proper dependencies/conflicts
- Add validation for simulation-specific settings
- Add time window configuration for new method (default: 1 hour)

Storage Layer:
- Add model structs with SQLx FromRow derives for all simulation tables
- Add comprehensive CRUD methods for simulation data management
- Add proper type annotations to fix SQLx compile issues
- Maintain separation between simulation and real rewarding logic

The simulation mode is mutually exclusive with real rewarding and does not
require mnemonic since no blockchain transactions are performed.
2025-06-03 12:39:44 +02:00
durch e9dc848950 Bump nym-api 2025-06-02 12:49:13 +02:00
durch 81162fba7e Allow nym-api init fail 2025-06-02 12:45:21 +02:00
durch be36da68b1 Clap value_delimiter 2025-06-02 11:58:12 +02:00
durch 21a56e307f Bump NNM 2025-06-02 09:31:09 +02:00
durch bd966383be Towards untangling nym-api client 2025-06-02 09:30:36 +02:00
durch 7626785ce4 Bump NM version 2025-05-27 15:40:59 +02:00
durch 6f79d39d48 Filter out non mixnodes 2025-05-27 15:40:33 +02:00
durch 014b5f767a Log APIs used 2025-05-27 15:23:18 +02:00
durch e0966565e6 Mnemonic to run, bump 2025-05-27 13:38:13 +02:00
durch c6aec663b7 Bump nym-api version 2025-05-27 13:18:31 +02:00
durch 7d041ddd44 Explicit mnemonic to entrypoint 2025-05-27 13:02:38 +02:00
durch 5d8bdc6570 Bunch of new query files 2025-05-27 10:28:00 +02:00
durch 06c412b3ba Remove debug logging 2025-05-27 10:25:27 +02:00
durch 356cf00106 Put the monitoring back properly 2025-05-27 10:25:27 +02:00
durch 58493a69aa Fix submission URLs 2025-05-27 10:25:27 +02:00
durch e881da834b More NM logging 2025-05-27 10:25:27 +02:00
durch eee9d8ab0c DEBUG: disable epoch operations, less noisy logging 2025-05-27 10:25:27 +02:00
durch 09026307f4 Debug logging for nym-api 2025-05-27 10:25:27 +02:00
durch 507ddf246c Stagger out route sending 2025-05-27 10:25:26 +02:00
durch 8d8ce29113 Update NM readme, fmt 2025-05-27 10:25:26 +02:00
durch 3be9e06bef sqlx prepare, bunch of nits 2025-05-27 10:25:26 +02:00
durch 770078a9ed Delete test script 2025-05-27 10:25:26 +02:00
durch fcffebfe45 Raw route handling and reliability corrections 2025-05-27 10:25:19 +02:00
durch 9c7d79683b Force routing through all nodes 2025-05-27 10:21:02 +02:00
durch c7f34d04c0 Support submitting to multiple APIs 2025-05-27 10:21:02 +02:00
2209 changed files with 152431 additions and 114818 deletions
+3 -3
View File
@@ -415,9 +415,9 @@
} }
}, },
"node_modules/undici": { "node_modules/undici": {
"version": "5.29.0", "version": "5.28.5",
"resolved": "https://registry.npmjs.org/undici/-/undici-5.29.0.tgz", "resolved": "https://registry.npmjs.org/undici/-/undici-5.28.5.tgz",
"integrity": "sha512-raqeBD6NQK4SkWhQzeYKd1KmIG6dllBOTt55Rmkt4HtI9mwdWtJljnrXjAFUBLTSN67HWrOIZ3EPF4kjUw80Bg==", "integrity": "sha512-zICwjrDrcrUE0pyyJc1I2QzBkLM8FINsgOrt6WjA+BgajVq9Nxu2PbFFXUrAggLfDXlZGZBVZYw7WNV5KiBiBA==",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@fastify/busboy": "^2.0.0" "@fastify/busboy": "^2.0.0"
@@ -21,7 +21,7 @@ jobs:
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
platform: [ arc-linux-latest ] platform: [ arc-ubuntu-22.04 ]
runs-on: ${{ matrix.platform }} runs-on: ${{ matrix.platform }}
env: env:
@@ -38,14 +38,15 @@ jobs:
rm -rf ci-builds || true rm -rf ci-builds || true
mkdir -p $OUTPUT_DIR mkdir -p $OUTPUT_DIR
echo $OUTPUT_DIR echo $OUTPUT_DIR
- name: Install Dependencies (Linux) - name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install libudev-dev run: sudo apt-get update && sudo apt-get -y install libudev-dev
- name: Sets env vars for tokio if set in manual dispatch inputs - name: Sets env vars for tokio if set in manual dispatch inputs
if: github.event_name == 'workflow_dispatch' && inputs.add_tokio_unstable == true
run: | run: |
echo "RUSTFLAGS=--cfg tokio_unstable" >> $GITHUB_ENV echo 'RUSTFLAGS="--cfg tokio_unstable"' >> $GITHUB_ENV
echo "CARGO_FEATURES=--features tokio-console" >> $GITHUB_ENV if: github.event_name == 'workflow_dispatch' && inputs.add_tokio_unstable == true
- name: Install Rust stable - name: Install Rust stable
uses: actions-rs/toolchain@v1 uses: actions-rs/toolchain@v1
with: with:
@@ -102,6 +103,7 @@ jobs:
if [ ${{ github.event_name == 'workflow_dispatch' && inputs.enable_deb == true }} = true ]; then if [ ${{ github.event_name == 'workflow_dispatch' && inputs.enable_deb == true }} = true ]; then
cp target/debian/*.deb $OUTPUT_DIR cp target/debian/*.deb $OUTPUT_DIR
fi fi
- name: Deploy branch to CI www - name: Deploy branch to CI www
continue-on-error: true continue-on-error: true
uses: easingthemes/ssh-deploy@main uses: easingthemes/ssh-deploy@main
+1 -1
View File
@@ -9,7 +9,7 @@ on:
jobs: jobs:
wasm: wasm:
runs-on: arc-linux-latest runs-on: arc-ubuntu-22.04
env: env:
CARGO_TERM_COLOR: always CARGO_TERM_COLOR: always
RUSTUP_PERMIT_COPY_RENAME: 1 RUSTUP_PERMIT_COPY_RENAME: 1
+9 -18
View File
@@ -5,6 +5,7 @@ on:
paths: paths:
- 'clients/**' - 'clients/**'
- 'common/**' - 'common/**'
- 'explorer-api/**'
- 'gateway/**' - 'gateway/**'
- 'integrations/**' - 'integrations/**'
- 'nym-api/**' - 'nym-api/**'
@@ -12,7 +13,6 @@ on:
- 'nym-network-monitor/**' - 'nym-network-monitor/**'
- 'nym-node/**' - 'nym-node/**'
- 'nym-node-status-api/**' - 'nym-node-status-api/**'
- 'nym-statistics-api/**'
- 'nym-outfox/**' - 'nym-outfox/**'
- 'nym-validator-rewarder/**' - 'nym-validator-rewarder/**'
- 'nyx-chain-watcher/**' - 'nyx-chain-watcher/**'
@@ -38,7 +38,7 @@ jobs:
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
os: [ arc-linux-latest, custom-windows-11, custom-macos-15 ] os: [ arc-ubuntu-22.04, custom-windows-11, custom-runner-mac-m1 ]
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
env: env:
CARGO_TERM_COLOR: always CARGO_TERM_COLOR: always
@@ -46,9 +46,9 @@ jobs:
RUSTUP_PERMIT_COPY_RENAME: 1 RUSTUP_PERMIT_COPY_RENAME: 1
steps: steps:
- name: Install Dependencies (Linux) - name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools protobuf-compiler cmake run: sudo apt-get update && sudo apt-get -y install libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools protobuf-compiler
continue-on-error: true continue-on-error: true
if: contains(matrix.os, 'linux') if: contains(matrix.os, 'ubuntu')
- name: Check out repository code - name: Check out repository code
uses: actions/checkout@v4 uses: actions/checkout@v4
@@ -63,7 +63,7 @@ jobs:
# To avoid running out of disk space, skip generating debug symbols # To avoid running out of disk space, skip generating debug symbols
- name: Set debug to false (unix) - name: Set debug to false (unix)
if: contains(matrix.os, 'linux') || contains(matrix.os, 'mac') if: contains(matrix.os, 'ubuntu') || contains(matrix.os, 'mac')
run: | run: |
sed -i.bak 's/\[profile.dev\]/\[profile.dev\]\ndebug = false/' Cargo.toml sed -i.bak 's/\[profile.dev\]/\[profile.dev\]\ndebug = false/' Cargo.toml
git diff git diff
@@ -81,35 +81,26 @@ jobs:
command: fmt command: fmt
args: --all -- --check args: --all -- --check
- name: Clippy (macos) - name: Clippy
if: contains(matrix.os, 'mac')
uses: actions-rs/cargo@v1
with:
command: clippy
args: --workspace --all-targets --exclude nym-gateway-probe -- -D warnings
- name: Clippy (non-macos)
if: contains(matrix.os, 'linux') || contains(matrix.os, 'windows')
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: clippy command: clippy
args: --workspace --all-targets -- -D warnings args: --workspace --all-targets -- -D warnings
- name: Build all binaries - name: Build all binaries
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: build command: build
- name: Build all examples - name: Build all examples
if: contains(matrix.os, 'linux') if: contains(matrix.os, 'ubuntu')
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: build command: build
args: --workspace --examples args: --workspace --examples
- name: Run all tests - name: Run all tests
if: contains(matrix.os, 'linux') if: contains(matrix.os, 'ubuntu')
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
env: env:
NYM_API: https://sandbox-nym-api1.nymtech.net/api NYM_API: https://sandbox-nym-api1.nymtech.net/api
@@ -118,7 +109,7 @@ jobs:
args: --workspace args: --workspace
- name: Run expensive tests - name: Run expensive tests
if: (github.ref == 'refs/heads/develop' || github.event.pull_request.base.ref == 'develop' || github.event.pull_request.base.ref == 'master') && contains(matrix.os, 'linux') if: (github.ref == 'refs/heads/develop' || github.event.pull_request.base.ref == 'develop' || github.event.pull_request.base.ref == 'master') && contains(matrix.os, 'ubuntu')
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: test command: test
@@ -10,13 +10,13 @@ env:
jobs: jobs:
check-if-tag-exists: check-if-tag-exists:
runs-on: arc-linux-latest-dind runs-on: arc-ubuntu-22.04-dind
steps: steps:
- name: Checkout repo - name: Checkout repo
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Get version from cargo.toml - name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1 uses: mikefarah/yq@v4.45.4
id: get_version id: get_version
with: with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
@@ -44,10 +44,8 @@ jobs:
echo "Tag is empty" echo "Tag is empty"
exit 1 exit 1
fi fi
# first, list all tags for logging purposes
curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq
# check if there's a matching tag exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq --arg tag $TAG '.tags | contains([$tag])' )
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq -r --arg tag "$TAG" 'any(.tags[]; . == $tag)' )
if [[ $exists = "true" ]]; then if [[ $exists = "true" ]]; then
echo "Version '$TAG' defined in Cargo.toml ALREADY EXISTS as tag in harbor repo" echo "Version '$TAG' defined in Cargo.toml ALREADY EXISTS as tag in harbor repo"
exit 1 exit 1
@@ -55,5 +53,5 @@ jobs:
echo "Version '$TAG' doesn't exist on the remote" echo "Version '$TAG' doesn't exist on the remote"
else else
echo "Unknown output '$exists'" echo "Unknown output '$exists'"
exit 2 exit 1
fi fi
@@ -1,59 +0,0 @@
name: ci-check-nym-stats-api-version
on:
pull_request:
paths:
- "nym-statistics-api/**"
env:
WORKING_DIRECTORY: "nym-statistics-api"
jobs:
check-if-tag-exists:
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Checkout repo
uses: actions/checkout@v4
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Check if git tag exists
run: |
TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
if [[ -z "$TAG" ]]; then
echo "Tag is empty"
exit 1
fi
git ls-remote --tags origin | awk '{print $2}'
if git ls-remote --tags origin | awk '{print $2}' | grep -q "refs/tags/$TAG$" ; then
echo "Tag '$TAG' ALREADY EXISTS on the remote"
exit 1
else
echo "Tag '$TAG' does not exist on the remote"
fi
- name: Check if harbor tag exists
run: |
TAG=${{ steps.get_version.outputs.result }}
registry=https://harbor.nymte.ch
repo_name=nym/nym-statistics-api
if [[ -z $TAG ]]; then
echo "Tag is empty"
exit 1
fi
# first, list all tags for logging purposes
curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq
# check if there's a matching tag
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq -r --arg tag "$TAG" 'any(.tags[]; . == $tag)' )
if [[ $exists = "true" ]]; then
echo "Version '$TAG' defined in Cargo.toml ALREADY EXISTS as tag in harbor repo"
exit 1
elif [[ $exists = "false" ]]; then
echo "Version '$TAG' doesn't exist on the remote"
else
echo "Unknown output '$exists'"
exit 2
fi
@@ -11,7 +11,7 @@ jobs:
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
platform: [ arc-linux-latest-dind ] platform: [ arc-ubuntu-22.04 ]
runs-on: ${{ matrix.platform }} runs-on: ${{ matrix.platform }}
env: env:
@@ -28,22 +28,34 @@ jobs:
mkdir -p $OUTPUT_DIR mkdir -p $OUTPUT_DIR
echo $OUTPUT_DIR echo $OUTPUT_DIR
- name: Build contracts - name: Install Rust stable
run: make optimize-contracts uses: actions-rs/toolchain@v1
with:
target: wasm32-unknown-unknown
override: true
- name: Check optimized contracts - name: Install wasm-opt
run: make docker-check-contracts uses: ./.github/actions/install-wasm-opt
with:
version: '114'
- name: Install cosmwasm-check
run: cargo install cosmwasm-check
- name: Build release contracts
run: make contracts
- name: Prepare build output - name: Prepare build output
shell: bash shell: bash
env: env:
OUTPUT_DIR: ci-contract-builds/${{ github.ref_name }} OUTPUT_DIR: ci-contract-builds/${{ github.ref_name }}
run: | run: |
find contracts/artifacts -maxdepth 1 -type f -name '*.wasm' -exec cp {} $OUTPUT_DIR \; cp contracts/target/wasm32-unknown-unknown/release/mixnet_contract.wasm $OUTPUT_DIR
# Also include the optimizer-generated checksums if present cp contracts/target/wasm32-unknown-unknown/release/vesting_contract.wasm $OUTPUT_DIR
if [ -f contracts/artifacts/checksums.txt ]; then cp contracts/target/wasm32-unknown-unknown/release/nym_coconut_dkg.wasm $OUTPUT_DIR
cp contracts/artifacts/checksums.txt $OUTPUT_DIR cp contracts/target/wasm32-unknown-unknown/release/cw3_flex_multisig.wasm $OUTPUT_DIR
fi cp contracts/target/wasm32-unknown-unknown/release/cw4_group.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_ecash.wasm $OUTPUT_DIR
- name: Deploy branch to CI www - name: Deploy branch to CI www
continue-on-error: true continue-on-error: true
+3 -5
View File
@@ -17,10 +17,9 @@ jobs:
build: build:
# since it's going to be compiled into wasm, there's absolutely # since it's going to be compiled into wasm, there's absolutely
# no point in running CI on different OS-es # no point in running CI on different OS-es
runs-on: arc-linux-latest runs-on: ubuntu-22.04
env: env:
CARGO_TERM_COLOR: always CARGO_TERM_COLOR: always
RUSTUP_PERMIT_COPY_RENAME: 1
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
@@ -28,8 +27,7 @@ jobs:
uses: actions-rs/toolchain@v1 uses: actions-rs/toolchain@v1
with: with:
profile: minimal profile: minimal
# pinned due to issues building contracts toolchain: stable
toolchain: 1.86.0
target: wasm32-unknown-unknown target: wasm32-unknown-unknown
override: true override: true
components: rustfmt, clippy components: rustfmt, clippy
@@ -54,7 +52,7 @@ jobs:
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
with: with:
command: test command: test
args: --lib --manifest-path contracts/Cargo.toml --all-features args: --lib --manifest-path contracts/Cargo.toml
- name: Check formatting - name: Check formatting
uses: actions-rs/cargo@v1 uses: actions-rs/cargo@v1
+1 -1
View File
@@ -10,7 +10,7 @@ on:
jobs: jobs:
build: build:
runs-on: arc-linux-latest runs-on: arc-ubuntu-22.04
env: env:
RUSTUP_PERMIT_COPY_RENAME: 1 RUSTUP_PERMIT_COPY_RENAME: 1
defaults: defaults:
+1 -1
View File
@@ -11,7 +11,7 @@ on:
jobs: jobs:
build: build:
runs-on: arc-linux-latest runs-on: arc-ubuntu-22.04
env: env:
CARGO_TERM_COLOR: always CARGO_TERM_COLOR: always
RUSTUP_PERMIT_COPY_RENAME: 1 RUSTUP_PERMIT_COPY_RENAME: 1
+7 -7
View File
@@ -4,14 +4,14 @@ on:
workflow_dispatch: workflow_dispatch:
pull_request: pull_request:
paths: paths:
- "wasm/**" - 'wasm/**'
- "clients/client-core/**" - 'clients/client-core/**'
- "common/**" - 'common/**'
- ".github/workflows/ci-sdk-wasm.yml" - '.github/workflows/ci-sdk-wasm.yml'
jobs: jobs:
wasm: wasm:
runs-on: arc-linux-latest runs-on: arc-ubuntu-22.04
env: env:
CARGO_TERM_COLOR: always CARGO_TERM_COLOR: always
RUSTUP_PERMIT_COPY_RENAME: 1 RUSTUP_PERMIT_COPY_RENAME: 1
@@ -33,7 +33,7 @@ jobs:
- name: Set up Go - name: Set up Go
uses: actions/setup-go@v5 uses: actions/setup-go@v5
with: with:
go-version: "1.24.6" go-version: "1.23.7"
- name: Install wasm-pack - name: Install wasm-pack
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
@@ -41,7 +41,7 @@ jobs:
- name: Install wasm-opt - name: Install wasm-opt
uses: ./.github/actions/install-wasm-opt uses: ./.github/actions/install-wasm-opt
with: with:
version: "116" version: '116'
- name: Install wasm-bindgen-cli - name: Install wasm-bindgen-cli
run: cargo install wasm-bindgen-cli run: cargo install wasm-bindgen-cli
-19
View File
@@ -1,19 +0,0 @@
name: Run SonarQube Scan
on:
push:
branches:
- develop
# pull_request:
# types: [opened, synchronize, reopened]
jobs:
sonarqube:
name: SonarQube
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: SonarQube Scan
uses: SonarSource/sonarqube-scan-action@v5
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
+1 -1
View File
@@ -6,7 +6,7 @@ jobs:
greeting: greeting:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/first-interaction@v3 - uses: actions/first-interaction@v1
with: with:
repo-token: ${{ secrets.GITHUB_TOKEN }} repo-token: ${{ secrets.GITHUB_TOKEN }}
issue-message: 'Thank you for raising this issue' issue-message: 'Thank you for raising this issue'
+1 -1
View File
@@ -31,7 +31,7 @@ jobs:
- name: Check out repository code - name: Check out repository code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Download report from previous job - name: Download report from previous job
uses: actions/download-artifact@v5 uses: actions/download-artifact@v4
with: with:
name: report name: report
path: .github/workflows/support-files/notifications path: .github/workflows/support-files/notifications
+6 -8
View File
@@ -19,11 +19,7 @@ jobs:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-binaries-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }} if: ${{ (startsWith(github.ref, 'refs/tags/nym-binaries-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: runs-on: arc-ubuntu-22.04
include:
- os: arc-linux-latest
target: x86_64-unknown-linux-gnu
runs-on: ${{ matrix.os }}
outputs: outputs:
release_id: ${{ steps.create-release.outputs.id }} release_id: ${{ steps.create-release.outputs.id }}
@@ -56,7 +52,7 @@ jobs:
- name: Install Rust stable - name: Install Rust stable
uses: actions-rs/toolchain@v1 uses: actions-rs/toolchain@v1
with: with:
toolchain: 1.88.0 toolchain: stable
override: true override: true
- name: Build all binaries - name: Build all binaries
@@ -70,6 +66,7 @@ jobs:
with: with:
name: my-artifact name: my-artifact
path: | path: |
target/release/explorer-api
target/release/nym-client target/release/nym-client
target/release/nym-socks5-client target/release/nym-socks5-client
target/release/nym-api target/release/nym-api
@@ -78,13 +75,14 @@ jobs:
target/release/nymvisor target/release/nymvisor
target/release/nym-node target/release/nym-node
retention-days: 30 retention-days: 30
- id: create-release - id: create-release
name: Upload to release based on tag name name: Upload to release based on tag name
uses: softprops/action-gh-release@da05d552573ad5aba039eaac05058a918a7bf631 uses: softprops/action-gh-release@v2
if: github.event_name == 'release' if: github.event_name == 'release'
with: with:
files: | files: |
target/release/explorer-api
target/release/nym-client target/release/nym-client
target/release/nym-socks5-client target/release/nym-socks5-client
target/release/nym-api target/release/nym-api
@@ -25,7 +25,7 @@ jobs:
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Install Java - name: Install Java
uses: actions/setup-java@v5 uses: actions/setup-java@v4
with: with:
distribution: "temurin" distribution: "temurin"
java-version: "17" java-version: "17"
@@ -91,7 +91,7 @@ jobs:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Download binary artifact - name: Download binary artifact
uses: actions/download-artifact@v5 uses: actions/download-artifact@v4
with: with:
name: nyms5-apk-arch64 name: nyms5-apk-arch64
path: apk path: apk
+2 -2
View File
@@ -8,7 +8,7 @@ env:
jobs: jobs:
build-container: build-container:
runs-on: arc-linux-latest-dind runs-on: arc-ubuntu-22.04-dind
steps: steps:
- name: Login to Harbor - name: Login to Harbor
uses: docker/login-action@v3 uses: docker/login-action@v3
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder" git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml - name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1 uses: mikefarah/yq@v4.45.4
id: get_version id: get_version
with: with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-credential-proxy/Cargo.toml cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-credential-proxy/Cargo.toml
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder" git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml - name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1 uses: mikefarah/yq@v4.45.4
id: get_version id: get_version
with: with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder" git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml - name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1 uses: mikefarah/yq@v4.45.4
id: get_version id: get_version
with: with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-network-monitor/Cargo.toml cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-network-monitor/Cargo.toml
+16 -39
View File
@@ -5,22 +5,15 @@ on:
inputs: inputs:
gateway_probe_git_ref: gateway_probe_git_ref:
type: string type: string
default: nym-vpn-core-v1.4.0
required: true
description: Which gateway probe git ref to build the image with description: Which gateway probe git ref to build the image with
release_image:
description: 'Tag image as a release'
required: true
default: false
type: boolean
env: env:
WORKING_DIRECTORY: "nym-node-status-api/nym-node-status-agent" WORKING_DIRECTORY: "nym-node-status-api/nym-node-status-agent"
CONTAINER_NAME: "node-status-agent" CONTAINER_NAME: "node-status-agent"
jobs: jobs:
build-container: build-container:
runs-on: arc-linux-latest-dind runs-on: arc-ubuntu-22.04-dind
steps: steps:
- name: Login to Harbor - name: Login to Harbor
uses: docker/login-action@v3 uses: docker/login-action@v3
@@ -38,10 +31,10 @@ jobs:
git config --global user.name "Lawrence Stalder" git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml - name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version id: get_version
run: | with:
VERSION=$(yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml) cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
echo "result=$VERSION" >> $GITHUB_OUTPUT
- name: cleanup-gateway-probe-ref - name: cleanup-gateway-probe-ref
id: cleanup_gateway_probe_ref id: cleanup_gateway_probe_ref
@@ -50,35 +43,19 @@ jobs:
GIT_REF_SLUG="${GATEWAY_PROBE_GIT_REF//\//-}" GIT_REF_SLUG="${GATEWAY_PROBE_GIT_REF//\//-}"
echo "git_ref=${GIT_REF_SLUG}" >> $GITHUB_OUTPUT echo "git_ref=${GIT_REF_SLUG}" >> $GITHUB_OUTPUT
- name: Set GIT_TAG variable - name: Remove existing tag if exists
run: echo "GIT_TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}" >> $GITHUB_ENV run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
fi
- name: Initialize RELEASE_TAG - name: Create tag
run: echo "RELEASE_TAG=" >> $GITHUB_ENV run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }} -m "Version ${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}"
- name: Set RELEASE_TAG for release git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
if: github.event.inputs.release_image == 'true'
run: echo "RELEASE_TAG=golden-" >> $GITHUB_ENV
- name: Set IMAGE_NAME_AND_TAGS variable
run: echo "IMAGE_NAME_AND_TAGS=${{ env.CONTAINER_NAME }}:${{ env.RELEASE_TAG }}${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}" >> $GITHUB_ENV
- name: New env vars
run: echo "RELEASE_TAG='$RELEASE_TAG' GIT_TAG='$GIT_TAG' IMAGE_NAME_AND_TAGS='$IMAGE_NAME_AND_TAGS'"
# - name: Remove existing tag if exists
# run: |
# if git rev-parse $${{ env.GIT_TAG }} >/dev/null 2>&1; then
# git push --delete origin $${{ env.GIT_TAG }}
# git tag -d $${{ env.GIT_TAG }}
# fi
# - name: Create tag
# run: |
# git tag -a $${{ env.GIT_TAG }} -m "Version ${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}"
# git push origin $${{ env.GIT_TAG }}
- name: BuildAndPushImageOnHarbor - name: BuildAndPushImageOnHarbor
run: | run: |
docker build --build-arg GIT_REF=${{ github.event.inputs.gateway_probe_git_ref }} -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.IMAGE_NAME_AND_TAGS }} docker build --build-arg GIT_REF=${{ github.event.inputs.gateway_probe_git_ref }} -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+21 -40
View File
@@ -1,20 +1,14 @@
name: Build and upload Node Status API container to harbor.nymte.ch name: Build and upload Node Status API container to harbor.nymte.ch
on: on:
workflow_dispatch: workflow_dispatch:
inputs:
release_image:
description: 'Tag image as a release'
required: true
default: false
type: boolean
env: env:
WORKING_DIRECTORY: "nym-node-status-api/nym-node-status-api" WORKING_DIRECTORY: "nym-node-status-api/nym-node-status-api"
CONTAINER_NAME: "node-status-api" CONTAINER_NAME: "node-status-api"
jobs: jobs:
build-container: build-container:
runs-on: arc-linux-latest-dind runs-on: arc-ubuntu-22.04-dind
steps: steps:
- name: Login to Harbor - name: Login to Harbor
uses: docker/login-action@v3 uses: docker/login-action@v3
@@ -32,43 +26,30 @@ jobs:
git config --global user.name "Lawrence Stalder" git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml - name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Check if tag exists
run: | run: |
VERSION=$(yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml) if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
echo "result=$VERSION" >> $GITHUB_OUTPUT echo "Tag ${{ steps.get_version.outputs.result }} already exists"
fi
- name: Set GIT_TAG variable - name: Remove existing tag if exists
run: echo "GIT_TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}" >> $GITHUB_ENV run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
fi
- name: Initialise RELEASE_TAG - name: Create tag
run: echo "RELEASE_TAG=" >> $GITHUB_ENV run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
- name: Set RELEASE_TAG for release git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
if: github.event.inputs.release_image == 'true'
run: echo "RELEASE_TAG=golden-" >> $GITHUB_ENV
- name: Set IMAGE_NAME_AND_TAGS variable
run: echo "IMAGE_NAME_AND_TAGS=${{ env.CONTAINER_NAME }}:${{ env.RELEASE_TAG }}${{ steps.get_version.outputs.result }}" >> $GITHUB_ENV
- name: New env vars
run: echo "RELEASE_TAG='$RELEASE_TAG' GIT_TAG='$GIT_TAG' IMAGE_NAME_AND_TAGS='$IMAGE_NAME_AND_TAGS'"
# - name: Remove existing tag if exists, then create
# run: |
# if git rev-parse "$GIT_TAG" >/dev/null 2>&1; then
# echo "Tag '$GIT_TAG' already exists, deleting"
# git push --delete origin "$GIT_TAG"
# git tag -d "$GIT_TAG"
# echo "Tag '$GIT_TAG' deleted"
# else
# echo "Tag '$GIT_TAG' does not exist, creating it"
# git tag -a $GIT_TAG -m "Version ${{ steps.get_version.outputs.result }}"
# git push origin $GIT_TAG
# echo "Tag '$GIT_TAG' created"
# fi
- name: BuildAndPushImageOnHarbor - name: BuildAndPushImageOnHarbor
run: | run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.IMAGE_NAME_AND_TAGS }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder" git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml - name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1 uses: mikefarah/yq@v4.45.4
id: get_version id: get_version
with: with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-api/Cargo.toml cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-api/Cargo.toml
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder" git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml - name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1 uses: mikefarah/yq@v4.45.4
id: get_version id: get_version
with: with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
@@ -1,42 +0,0 @@
name: Build and upload Nym Statistics API container to harbor.nymte.ch
on:
workflow_dispatch:
env:
WORKING_DIRECTORY: "nym-statistics-api"
CONTAINER_NAME: "nym-statistics-api"
jobs:
build-container:
runs-on: arc-linux-latest-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: harbor.nymte.ch
username: ${{ secrets.HARBOR_ROBOT_USERNAME }}
password: ${{ secrets.HARBOR_ROBOT_SECRET }}
- name: Checkout repo
uses: actions/checkout@v4
- name: Configure git identity
run: |
git config --global user.email "lawrence@nymtech.net"
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder" git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml - name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1 uses: mikefarah/yq@v4.45.4
id: get_version id: get_version
with: with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder" git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml - name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1 uses: mikefarah/yq@v4.45.4
id: get_version id: get_version
with: with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
+2 -2
View File
@@ -35,13 +35,12 @@ validator-api/keypair
contracts/mixnet/code_id contracts/mixnet/code_id
contracts/mixnet/Justfile contracts/mixnet/Justfile
contracts/mixnet/Makefile contracts/mixnet/Makefile
artifacts
contracts/artifacts
validator-config validator-config
*.patch *.patch
validator-api-config.toml validator-api-config.toml
dist dist
storybook-static storybook-static
envs/qwerty.env
.parcel-cache .parcel-cache
**/.DS_Store **/.DS_Store
cpu-cycles/libcpucycles/build cpu-cycles/libcpucycles/build
@@ -63,3 +62,4 @@ nym-api/redocly/formatted-openapi.json
**/settings.sql **/settings.sql
**/enter_db.sh **/enter_db.sh
CLAUDE.md
-392
View File
@@ -4,398 +4,6 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
## [Unreleased] ## [Unreleased]
## [2025.18-jarlsberg] (2025-10-14)
- ns-api: add descriptions to dVPN gateway responses ([#6102])
- NS API: use new probe download filesize and milliseconds field ([#6097])
- ns-api: use download files size from probes instead of parsing filenames ([#6095])
- ns-api: add new fields for probe output for query_metadata and download file size and duration in ms ([#6091])
- Bugfix/bloomfilters purge ([#6089])
- Hotfix: Update API source in node ping tester script ([#6082])
- Get wireguard keypair as arg instead of reading it from disk ([#6078])
- Feature: Ping probe all nodes /described nodes from a server ([#6074])
- Node Status API: add bridge information to dVPN endpoint ([#6069])
- frontdoor typo fix ([#6067])
- Feature: Node rewards tracker ([#6064])
- [chore] Clippy fix ([#6060])
- Registration Client ([#6059])
- Bugfix: Nym node CLI download nym-node exception ([#6058])
- Feature: Nym node html landing page ([#6053])
- feat: DKG contract method for updating announce address ([#6050])
- feat: NS ticket faucet ([#6047])
- Bridge proto client params in Self-Described ([#6035])
- Node Status API: remove sqlite support ([#6004])
- Benny/ci contract fix ([#5962])
[#6102]: https://github.com/nymtech/nym/pull/6102
[#6097]: https://github.com/nymtech/nym/pull/6097
[#6095]: https://github.com/nymtech/nym/pull/6095
[#6091]: https://github.com/nymtech/nym/pull/6091
[#6089]: https://github.com/nymtech/nym/pull/6089
[#6082]: https://github.com/nymtech/nym/pull/6082
[#6078]: https://github.com/nymtech/nym/pull/6078
[#6074]: https://github.com/nymtech/nym/pull/6074
[#6069]: https://github.com/nymtech/nym/pull/6069
[#6067]: https://github.com/nymtech/nym/pull/6067
[#6064]: https://github.com/nymtech/nym/pull/6064
[#6060]: https://github.com/nymtech/nym/pull/6060
[#6059]: https://github.com/nymtech/nym/pull/6059
[#6058]: https://github.com/nymtech/nym/pull/6058
[#6053]: https://github.com/nymtech/nym/pull/6053
[#6050]: https://github.com/nymtech/nym/pull/6050
[#6047]: https://github.com/nymtech/nym/pull/6047
[#6035]: https://github.com/nymtech/nym/pull/6035
[#6004]: https://github.com/nymtech/nym/pull/6004
[#5962]: https://github.com/nymtech/nym/pull/5962
## [2025.17-isabirra] (2025-09-29)
- Bugfix | Fix the registration handshake ([#6062])
- Convenience for ShutdownTracker ([#6038])
- chore: made http-api-client-macro doctest compile ([#6037])
- feat: refresh mixnet contract on epoch progression ([#6023])
- chore: remove legacy nodes from nym api [and kinda-ish from node status api] ([#6021])
- Feature/credential proxy crate ([#6018])
- Moving clients crate from vpn-client repo to here ([#6015])
- Feature/cancellation migration ([#6014])
- Use default value for the ports until api is deployed ([#6007])
- bugfix: return from MixTrafficController if client request channel has closed ([#6002])
- Revert "Create an axum_test client for more integrated unit testing (… ([#5999])
- chore: upgraded syn to 2.0 and removed nym-execute ([#5998])
- feat: use `ShutdownToken` (`CancellationToken` inside) for nym-api ([#5997])
- bugfix: Recipient deserialisation for deserialisers missing bytes specialisation ([#5991])
- chore: use updated version of simulate endpoint ([#5988])
- chore: purge temp databases on build ([#5984])
- Bump sha.js from 2.4.11 to 2.4.12 ([#5983])
- Feature: Delegation program stake checker and adjuster ([#5980])
- build(deps): bump actions/setup-java from 4 to 5 ([#5975])
- Domain fronting integration ([#5974])
- chore: internal hidden command to force advance nyx epoch ([#5964])
- Create an axum_test client for more integrated unit testing ([#5956])
- feat: shared library for attempting to retrieve update mode attestation ([#5954])
- Bump slab from 0.4.10 to 0.4.11 ([#5952])
- build(deps): bump actions/first-interaction from 1 to 3 ([#5950])
- fix: use WASM compatible time API in client ([#5948])
- feat: credential proxy deposit pool ([#5945])
- build(deps): bump actions/download-artifact from 4 to 5 ([#5939])
- feat: nym signers monitor ([#5933])
- Bump console from 0.15.11 to 0.16.0 ([#5931])
- Bump mock_instant from 0.5.3 to 0.6.0 ([#5930])
- Bump tokio from 1.46.1 to 1.47.1 ([#5929])
- Bump defguard_wireguard_rs from v0.4.7 to v0.7.5 ([#5928])
- Bump indicatif from 0.17.11 to 0.18.0 ([#5924])
- Feature: Nym node autorun CLI ([#5916])
- build(deps): bump mikefarah/yq from 4.45.4 to 4.47.1 ([#5911])
- build(deps): bump pbkdf2 from 3.1.2 to 3.1.3 ([#5869])
[#6062]: https://github.com/nymtech/nym/pull/6062
[#6038]: https://github.com/nymtech/nym/pull/6038
[#6037]: https://github.com/nymtech/nym/pull/6037
[#6023]: https://github.com/nymtech/nym/pull/6023
[#6021]: https://github.com/nymtech/nym/pull/6021
[#6018]: https://github.com/nymtech/nym/pull/6018
[#6015]: https://github.com/nymtech/nym/pull/6015
[#6014]: https://github.com/nymtech/nym/pull/6014
[#6007]: https://github.com/nymtech/nym/pull/6007
[#6002]: https://github.com/nymtech/nym/pull/6002
[#5999]: https://github.com/nymtech/nym/pull/5999
[#5998]: https://github.com/nymtech/nym/pull/5998
[#5997]: https://github.com/nymtech/nym/pull/5997
[#5991]: https://github.com/nymtech/nym/pull/5991
[#5988]: https://github.com/nymtech/nym/pull/5988
[#5984]: https://github.com/nymtech/nym/pull/5984
[#5983]: https://github.com/nymtech/nym/pull/5983
[#5980]: https://github.com/nymtech/nym/pull/5980
[#5975]: https://github.com/nymtech/nym/pull/5975
[#5974]: https://github.com/nymtech/nym/pull/5974
[#5964]: https://github.com/nymtech/nym/pull/5964
[#5956]: https://github.com/nymtech/nym/pull/5956
[#5954]: https://github.com/nymtech/nym/pull/5954
[#5952]: https://github.com/nymtech/nym/pull/5952
[#5950]: https://github.com/nymtech/nym/pull/5950
[#5948]: https://github.com/nymtech/nym/pull/5948
[#5945]: https://github.com/nymtech/nym/pull/5945
[#5939]: https://github.com/nymtech/nym/pull/5939
[#5933]: https://github.com/nymtech/nym/pull/5933
[#5931]: https://github.com/nymtech/nym/pull/5931
[#5930]: https://github.com/nymtech/nym/pull/5930
[#5929]: https://github.com/nymtech/nym/pull/5929
[#5928]: https://github.com/nymtech/nym/pull/5928
[#5924]: https://github.com/nymtech/nym/pull/5924
[#5916]: https://github.com/nymtech/nym/pull/5916
[#5911]: https://github.com/nymtech/nym/pull/5911
[#5869]: https://github.com/nymtech/nym/pull/5869
## [2025.16-halloumi] (2025-09-16)
- Backport metadata endpoint ([#6010])
- bugfix: make sure tables are removed in correct order to not trigger FK constraint issue ([#5987])
- chore: move authenticator into gateway crate ([#5982])
- Fix the ns api ci workflow ([#5981])
- Remove freshness check on testrun submit ([#5977])
- Update sysinfo to the latest ([#5976])
- bugfix: manually calculate per node work on rewarded set changes ([#5972])
- fixing the ci for ns agent ([#5965])
- Feature/testing utils ([#5963])
- bugfix: fix ci-build for linux (and use updated runner) ([#5958])
- chore: updated refs to cheddar rev of nym repo ([#5955])
- http api client adjustment ([#5953])
- chore: fix rust 1.89 clippy issues ([#5944])
- Wireguard metadata client library ([#5943])
- chore: remove unused import ([#5942])
- feat: introduce additional checks when attempting to send to bounded channels ([#5941])
- Move credential verifier in peer controller ([#5938])
- change PK/FK on expiration date signatures tables ([#5934])
- Wireguard private metadata ([#5915])
[#6010]: https://github.com/nymtech/nym/pull/6010
[#5987]: https://github.com/nymtech/nym/pull/5987
[#5982]: https://github.com/nymtech/nym/pull/5982
[#5981]: https://github.com/nymtech/nym/pull/5981
[#5977]: https://github.com/nymtech/nym/pull/5977
[#5976]: https://github.com/nymtech/nym/pull/5976
[#5972]: https://github.com/nymtech/nym/pull/5972
[#5965]: https://github.com/nymtech/nym/pull/5965
[#5963]: https://github.com/nymtech/nym/pull/5963
[#5958]: https://github.com/nymtech/nym/pull/5958
[#5955]: https://github.com/nymtech/nym/pull/5955
[#5953]: https://github.com/nymtech/nym/pull/5953
[#5944]: https://github.com/nymtech/nym/pull/5944
[#5943]: https://github.com/nymtech/nym/pull/5943
[#5942]: https://github.com/nymtech/nym/pull/5942
[#5941]: https://github.com/nymtech/nym/pull/5941
[#5938]: https://github.com/nymtech/nym/pull/5938
[#5934]: https://github.com/nymtech/nym/pull/5934
[#5915]: https://github.com/nymtech/nym/pull/5915
## [2025.15-gruyere] (2025-08-20)
- Migrate strum to 0.27.2 ([#5960])
- WG exit policy scripts update ([#5921])
- Make DNS Resolver fallback optional ([#5920])
- nym-node debug command to reset providers db ([#5914])
- basic zulip client for sending messages ([#5913])
- chore: allow compatibility with 'CDLA-Permissive-2.0' ([#5910])
- feat: ecash liveness check ([#5890])
- Remove old free credential handle ([#5864])
[#5960]: https://github.com/nymtech/nym/pull/5960
[#5921]: https://github.com/nymtech/nym/pull/5921
[#5920]: https://github.com/nymtech/nym/pull/5920
[#5914]: https://github.com/nymtech/nym/pull/5914
[#5913]: https://github.com/nymtech/nym/pull/5913
[#5910]: https://github.com/nymtech/nym/pull/5910
[#5890]: https://github.com/nymtech/nym/pull/5890
[#5864]: https://github.com/nymtech/nym/pull/5864
## [2025.14-feta] (2025-08-05)
- chore: nym node tokio console ([#5909])
- Feature/dkg snapshot epoch ([#5900])
- Feature/dkg epoch dealers query ([#5899])
- sqlx-pool-guard: allocate more memory on windows ([#5896])
- Support mnemonic in the NS agent ([#5883])
- Allow PG database backend ([#5880])
[#5909]: https://github.com/nymtech/nym/pull/5909
[#5900]: https://github.com/nymtech/nym/pull/5900
[#5899]: https://github.com/nymtech/nym/pull/5899
[#5896]: https://github.com/nymtech/nym/pull/5896
[#5883]: https://github.com/nymtech/nym/pull/5883
[#5880]: https://github.com/nymtech/nym/pull/5880
## [2025.13-emmental] (2025-07-22)
- fix: don't allow mixnode running in exit mode ([#5898])
- fix contract build process in Makefile ([#5892])
- bugfix: ignore 'Send' responses when claiming bandwidth ([#5884])
- Update push-node-status-agent.yaml ([#5882])
- listen for shutdown signals during nym-node startup ([#5879])
- feat: forbid running mixnode + entry on the same node ([#5878])
- chore: 1.88 clippy ([#5877])
- Batch SQL writes for packet stats ([#5874])
- fix the broken link ([#5873])
- Set busy_timeout in sqlx ([#5872])
- feat: basic performance contract integration [within Nym API] ([#5871])
- scraper bugfix: ignore precommits from missing validators ([#5867])
- Return true remaining ([#5866])
- Make Mix hops optional for Mixnet Client SURBs ([#5861])
- Check gateway supported versions ([#5860])
- Add build info endpoints ([#5857])
- Clear out screaming logs ([#5856])
- fix removal of qa env ([#5855])
- Use display when printing paths ([#5853])
- feat: initial performance contract ([#5833])
- Security patches for the `dkg` crate ([#5828])
- HTTP Discovery objects & network defaults ([#5814])
[#5898]: https://github.com/nymtech/nym/pull/5898
[#5892]: https://github.com/nymtech/nym/pull/5892
[#5884]: https://github.com/nymtech/nym/pull/5884
[#5882]: https://github.com/nymtech/nym/pull/5882
[#5879]: https://github.com/nymtech/nym/pull/5879
[#5878]: https://github.com/nymtech/nym/pull/5878
[#5877]: https://github.com/nymtech/nym/pull/5877
[#5874]: https://github.com/nymtech/nym/pull/5874
[#5873]: https://github.com/nymtech/nym/pull/5873
[#5872]: https://github.com/nymtech/nym/pull/5872
[#5871]: https://github.com/nymtech/nym/pull/5871
[#5867]: https://github.com/nymtech/nym/pull/5867
[#5866]: https://github.com/nymtech/nym/pull/5866
[#5861]: https://github.com/nymtech/nym/pull/5861
[#5860]: https://github.com/nymtech/nym/pull/5860
[#5857]: https://github.com/nymtech/nym/pull/5857
[#5856]: https://github.com/nymtech/nym/pull/5856
[#5855]: https://github.com/nymtech/nym/pull/5855
[#5853]: https://github.com/nymtech/nym/pull/5853
[#5833]: https://github.com/nymtech/nym/pull/5833
[#5828]: https://github.com/nymtech/nym/pull/5828
[#5814]: https://github.com/nymtech/nym/pull/5814
## [2025.12-dolcelatte] (2025-07-07)
- bugfix: key-rotation + reply SURBs ([#5876])
- Bugfix/backwards compat ([#5865])
- bugfix: allow gateways to permit authentication from v4 clients ([#5862])
- fixed client route for obtaining v2 list of gateways ([#5859])
- Updated browser extension piece removal ([#5849])
- Remove/old env references ([#5848])
- Remove qa env ([#5847])
- remove not used old mock-api ([#5845])
- remove bity dir ([#5844])
- build(deps-dev): bump webpack-dev-server from 4.13.2 to 5.2.1 in /wasm/mix-fetch/internal-dev ([#5843])
- Amended the buy section ([#5841])
- Removing test-net faucet ([#5840])
- Feature/node status dvpn directory ([#5829])
- build(deps-dev): bump webpack-dev-server from 4.15.2 to 5.2.1 in /nym-credential-proxy/vpn-api-lib-wasm/internal-dev ([#5826])
- bugfix: fix swapped total and circulating supplies ([#5822])
- build(deps): bump tar-fs from 3.0.8 to 3.0.9 in /sdk/typescript/tests/integration-tests/mix-fetch ([#5821])
- Url scheme warning log ([#5819])
- chore: adjust heuristic for wireguard peer activity ([#5818])
- Use the same client bandwidth for top up ([#5813])
- Replace chrono with time in NS API ([#5811])
- build(deps-dev): bump http-proxy-middleware from 2.0.4 to 2.0.9 in /clients/native/examples/js-examples/websocket ([#5810])
- build(deps): bump tokio from 1.44.2 to 1.45.1 ([#5798])
- Close sqlite pool before moving or reopening databases ([#5796])
- HTTP Client Retries, Fallbacks, and Redirects ([#5789])
- feat: key rotation ([#5777])
- build(deps): bump next from 14.2.15 to 14.2.26 in /documentation/docs ([#5772])
- build(deps): bump undici from 5.28.5 to 5.29.0 in /.github/actions/nym-hash-releases/src ([#5771])
- build(deps): bump cargo_metadata from 0.18.1 to 0.19.2 ([#5765])
- build(deps): bump tempfile from 3.19.1 to 3.20.0 ([#5764])
- [Feature] Noise XKpsk3 integration (2025 version) ([#5692])
- feature: nympool contract ([#5464])
- chore: fixed typo in API endpoint parameter ([#5449])
[#5876]: https://github.com/nymtech/nym/pull/5876
[#5865]: https://github.com/nymtech/nym/pull/5865
[#5862]: https://github.com/nymtech/nym/pull/5862
[#5859]: https://github.com/nymtech/nym/pull/5859
[#5849]: https://github.com/nymtech/nym/pull/5849
[#5848]: https://github.com/nymtech/nym/pull/5848
[#5847]: https://github.com/nymtech/nym/pull/5847
[#5845]: https://github.com/nymtech/nym/pull/5845
[#5844]: https://github.com/nymtech/nym/pull/5844
[#5843]: https://github.com/nymtech/nym/pull/5843
[#5841]: https://github.com/nymtech/nym/pull/5841
[#5840]: https://github.com/nymtech/nym/pull/5840
[#5829]: https://github.com/nymtech/nym/pull/5829
[#5826]: https://github.com/nymtech/nym/pull/5826
[#5822]: https://github.com/nymtech/nym/pull/5822
[#5821]: https://github.com/nymtech/nym/pull/5821
[#5819]: https://github.com/nymtech/nym/pull/5819
[#5818]: https://github.com/nymtech/nym/pull/5818
[#5813]: https://github.com/nymtech/nym/pull/5813
[#5811]: https://github.com/nymtech/nym/pull/5811
[#5810]: https://github.com/nymtech/nym/pull/5810
[#5798]: https://github.com/nymtech/nym/pull/5798
[#5796]: https://github.com/nymtech/nym/pull/5796
[#5789]: https://github.com/nymtech/nym/pull/5789
[#5777]: https://github.com/nymtech/nym/pull/5777
[#5772]: https://github.com/nymtech/nym/pull/5772
[#5771]: https://github.com/nymtech/nym/pull/5771
[#5765]: https://github.com/nymtech/nym/pull/5765
[#5764]: https://github.com/nymtech/nym/pull/5764
[#5692]: https://github.com/nymtech/nym/pull/5692
[#5464]: https://github.com/nymtech/nym/pull/5464
[#5449]: https://github.com/nymtech/nym/pull/5449
## [2025.11-cheddar] (2025-06-10)
- No autoremoval of peers ([#5831])
- Set cached storage counters to 0 ([#5812])
- hack: temporarily use next.config.js instead of next.config.ts ([#5805])
- chore: resolve 1.87 clippy warnings ([#5802])
- Nym Statistics API ([#5800])
- QoL: RequestPath trait for http-api-client ([#5788])
- Fix contains ticketbook function that always returned true ([#5787])
- swap a decode into a fromrow to please future postgres feature ([#5785])
- Make address cache configurable ([#5784])
- Track wireguard credential retries ([#5783])
[#5831]: https://github.com/nymtech/nym/pull/5831
[#5812]: https://github.com/nymtech/nym/pull/5812
[#5805]: https://github.com/nymtech/nym/pull/5805
[#5802]: https://github.com/nymtech/nym/pull/5802
[#5800]: https://github.com/nymtech/nym/pull/5800
[#5788]: https://github.com/nymtech/nym/pull/5788
[#5787]: https://github.com/nymtech/nym/pull/5787
[#5785]: https://github.com/nymtech/nym/pull/5785
[#5784]: https://github.com/nymtech/nym/pull/5784
[#5783]: https://github.com/nymtech/nym/pull/5783
## [2025.10-brie] (2025-05-27)
- Backport PR 5779 ([#5801])
- Expanded Accept Encoding for `reqwest` ([#5779])
- Teach HttpClientError how to report its status code and timeout ([#5770])
- Skip refreshing the topology on startup as we already have an initial set ([#5768])
- Fetch the topology from the nym-api concurrently ([#5767])
- feat: use bincode by default in NymApiClient + remove feature-lock ([#5761])
- Instrument create_request ([#5760])
- Add node_bonded field to delegations ([#5759])
- build(deps): bump mikefarah/yq from 4.45.1 to 4.45.4 ([#5758])
- Raw route submissions ([#5756])
- feat: expires header for `/active` nym-api responses ([#5755])
- Decrease default average packet delay to 15 ms ([#5754])
- build(deps): bump the patch-updates group across 1 directory with 12 updates ([#5753])
- Remove pretty_env_logger and switch remaining crates to use tracing ([#5749])
- Update pretty_env_logger to latest to not depend on unmaintained crate atty ([#5748])
- Upgrade prometheus crate to fix security warning ([#5747])
- Downgrade deranged crate to 0.4.0 ([#5746])
- feat: nym-api bincode + yaml support ([#5745])
- fix parallel feature in ecash crate with send + sync ([#5744])
- Remove old test directory - Update validator docker ([#5743])
- [Feature] `RememberMe` is the new don't `ForgetMe` ([#5742])
- build(deps): bump ammonia from 4.0.0 to 4.1.0 ([#5739])
- build(deps): bump base-x from 3.0.9 to 3.0.11 in /testnet-faucet ([#5737])
- build(deps): bump http-proxy-middleware from 2.0.8 to 2.0.9 ([#5730])
[#5801]: https://github.com/nymtech/nym/pull/5801
[#5779]: https://github.com/nymtech/nym/pull/5779
[#5770]: https://github.com/nymtech/nym/pull/5770
[#5768]: https://github.com/nymtech/nym/pull/5768
[#5767]: https://github.com/nymtech/nym/pull/5767
[#5761]: https://github.com/nymtech/nym/pull/5761
[#5760]: https://github.com/nymtech/nym/pull/5760
[#5759]: https://github.com/nymtech/nym/pull/5759
[#5758]: https://github.com/nymtech/nym/pull/5758
[#5756]: https://github.com/nymtech/nym/pull/5756
[#5755]: https://github.com/nymtech/nym/pull/5755
[#5754]: https://github.com/nymtech/nym/pull/5754
[#5753]: https://github.com/nymtech/nym/pull/5753
[#5749]: https://github.com/nymtech/nym/pull/5749
[#5748]: https://github.com/nymtech/nym/pull/5748
[#5747]: https://github.com/nymtech/nym/pull/5747
[#5746]: https://github.com/nymtech/nym/pull/5746
[#5745]: https://github.com/nymtech/nym/pull/5745
[#5744]: https://github.com/nymtech/nym/pull/5744
[#5743]: https://github.com/nymtech/nym/pull/5743
[#5742]: https://github.com/nymtech/nym/pull/5742
[#5739]: https://github.com/nymtech/nym/pull/5739
[#5737]: https://github.com/nymtech/nym/pull/5737
[#5730]: https://github.com/nymtech/nym/pull/5730
## [2025.9-appenzeller] (2025-05-13) ## [2025.9-appenzeller] (2025-05-13)
- build(deps): bump clap from 4.5.36 to 4.5.37 in the patch-updates group ([#5722]) - build(deps): bump clap from 4.5.36 to 4.5.37 in the patch-updates group ([#5722])
-686
View File
@@ -1,686 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Nym is a privacy platform that uses mixnet technology to protect against metadata surveillance. The platform consists of several key components:
- Mixnet nodes (mixnodes) for packet mixing
- Gateways (entry/exit points for the network)
- Clients for interacting with the network
- Network monitoring tools
- Validators for network consensus
- Various service providers and integrations
## Build Commands
### Rust Components
```bash
# Default build (debug)
cargo build
# Release build
cargo build --release
# Build a specific package
cargo build -p <package-name>
# Build main components
make build
# Build release versions of main binaries and contracts
make build-release
# Build specific binaries
make build-nym-cli
cargo build -p nym-node --release
cargo build -p nym-api --release
```
### Testing
```bash
# Run clippy, unit tests, and formatting
make test
# Run all tests including slow tests
make test-all
# Run clippy on all workspaces
make clippy
# Run unit tests for a specific package
cargo test -p <package-name>
# Run only expensive/ignored tests
cargo test --workspace -- --ignored
# Run API tests
dotenv -f envs/sandbox.env -- cargo test --test public-api-tests
# Run tests with specific log level
RUST_LOG=debug cargo test -p <package-name>
# Run specific test scripts
./nym-node/tests/test_apis.sh
./scripts/wireguard-exit-policy/exit-policy-tests.sh
```
### Linting and Formatting
```bash
# Run rustfmt on all code
make fmt
# Check formatting without modifying
cargo fmt --all -- --check
# Run clippy with all targets
cargo clippy --workspace --all-targets -- -D warnings
# TypeScript linting
yarn lint
yarn lint:fix
yarn types:lint:fix
# Check dependencies for security/licensing issues
cargo deny check
```
### WASM Components
```bash
# Build all WASM components
make sdk-wasm-build
# Build TypeScript SDK
yarn build:sdk
npx lerna run --scope @nymproject/sdk build --stream
# Build and test WASM components
make sdk-wasm
# Build specific WASM packages
cd wasm/client && make
cd wasm/mix-fetch && make
cd wasm/node-tester && make
```
### Contract Development
```bash
# Build all contracts
make contracts
# Build contracts in release mode
make build-release-contracts
# Generate contract schemas
make contract-schema
# Run wasm-opt on contracts
make wasm-opt-contracts
# Check contracts with cosmwasm-check
make cosmwasm-check-contracts
```
### Running Components
```bash
# Run nym-node as a mixnode
cargo run -p nym-node -- run --mode mixnode
# Run nym-node as a gateway
cargo run -p nym-node -- run --mode gateway
# Run the network monitor
cargo run -p nym-network-monitor
# Run the API server
cargo run -p nym-api
# Run with specific environment
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Start a local network
./scripts/localnet_start.sh
```
## Architecture
The Nym platform consists of various components organized as a monorepo:
1. **Core Mixnet Infrastructure**:
- `nym-node`: Core binary supporting mixnode and gateway modes
- `common/nymsphinx`: Implementation of the Sphinx packet format
- `common/topology`: Network topology management
- `common/types`: Shared data types across components
2. **Network Monitoring**:
- `nym-network-monitor`: Monitors the network's reliability and performance
- `nym-api`: API server for network stats and monitoring data
- Metrics tracking for nodes, routes, and overall network health
3. **Client Implementations**:
- `clients/native`: Native Rust client implementation
- `clients/socks5`: SOCKS5 proxy client for standard applications
- `wasm`: WebAssembly client implementations (for browsers)
- `nym-connect`: Desktop and mobile clients
4. **Blockchain & Smart Contracts**:
- `common/cosmwasm-smart-contracts`: Smart contract implementations
- `contracts`: CosmWasm contracts for the Nym network
- `common/ledger`: Blockchain integration
5. **Utilities & Tools**:
- `tools`: Various CLI tools and utilities
- `sdk`: SDKs for different languages and platforms
- `documentation`: Documentation generation and management
## Packet System
Nym uses a modified Sphinx packet format for its mixnet:
1. **Message Chunking**:
- Messages are divided into "sets" and "fragments"
- Each fragment fits in a single Sphinx packet
- The `common/nymsphinx/chunking` module handles message fragmentation
2. **Routing**:
- Packets traverse through 3 layers of mixnodes
- Routing information is encrypted in layers (onion routing)
- The final gateway receives and processes the messages
3. **Monitoring**:
- Monitoring system tracks packet delivery through the network
- Routes are analyzed for reliability statistics
- Node performance metrics are collected
## Network Protocol
Nym implements the Loopix mixnet design with several key privacy features:
1. **Continuous-time Mixing**:
- Each mixnode delays messages independently with an exponential distribution
- This creates random reordering of packets, destroying timing correlations
- Offers better anonymity properties than batch mixing approaches
2. **Cover Traffic**:
- Clients and nodes generate dummy "loop" packets that circulate through the network
- These packets are indistinguishable from real traffic
- Creates a baseline level of traffic that hides actual communication patterns
- Provides unobservability (hiding when and how much real traffic is being sent)
3. **Stratified Network Architecture**:
- Traffic flows through Entry Gateway → 3 Mixnode Layers → Exit Gateway
- Path selection is independent per-message (unlike Tor)
- Each node connects only to adjacent layers
4. **Anonymous Replies**:
- Single-Use Reply Blocks (SURBs) allow receiving messages without revealing identity
- Enables bidirectional communication while maintaining privacy
## Network Monitoring Architecture
The network monitoring system is a core component that measures mixnet reliability:
1. The `nym-network-monitor` sends test packets through the network
2. These packets follow predefined routes through multiple mixnodes
3. Metrics are collected about:
- Successful and failed packet deliveries
- Node reliability (percentage of successful packet handling)
- Route reliability (which specific route combinations work best)
4. Results are stored in the database and used by `nym-api` to:
- Present node performance statistics
- Determine network rewards
- Provide route selection guidance to clients
In the current branch, metrics collection is being enhanced with a fanout approach to submit to multiple API endpoints.
## Development Environment
### Required Dependencies
- Rust toolchain (stable, 1.80+)
- Node.js (v20+) and yarn for TypeScript components
- SQLite for local database development
- PostgreSQL for API database (optional, for full API functionality)
- CosmWasm tools for contract development
- For building contracts: `wasm-opt` tool from `binaryen`
- Python 3.8+ for some scripts
- Docker (optional, for containerized development)
- protoc (Protocol Buffers compiler) for some components
### Environment Configurations
The `envs/` directory contains pre-configured environments:
#### Available Environments
- **`local.env`**: Local development environment
- Points to local services (localhost)
- Uses test mnemonics and keys
- Ideal for testing without external dependencies
- **`sandbox.env`**: Sandbox test network
- Public test network with real nodes
- Test tokens available from faucet
- Contract addresses for sandbox deployment
- API: https://sandbox-nym-api1.nymtech.net
- **`mainnet.env`**: Production mainnet
- Real network with real tokens
- Production contract addresses
- API: https://validator.nymtech.net
- Use with caution!
- **`canary.env`**: Canary deployment
- Pre-release testing environment
- Tests new features before mainnet
- **`mainnet-local-api.env`**: Hybrid environment
- Uses mainnet contracts but local API
- Useful for API development against mainnet data
#### Key Environment Variables
```bash
# Network configuration
NETWORK_NAME=sandbox # Network identifier
BECH32_PREFIX=n # Address prefix (n for sandbox, n for mainnet)
NYM_API=https://sandbox-nym-api1.nymtech.net/api
NYXD=https://rpc.sandbox.nymtech.net
NYM_API_NETWORK=sandbox
# Contract addresses (network-specific)
MIXNET_CONTRACT_ADDRESS=n1xr3rq8yvd7qplsw5yx90ftsr2zdhg4e9z60h5duusgxpv72hud3sjkxkav
VESTING_CONTRACT_ADDRESS=n1unyuj8qnmygvzuex3dwmg9yzt9alhvyeat0uu0jedg2wj33efl5qackslz
# ... other contract addresses
# Mnemonic for testing (NEVER use in production)
MNEMONIC="clutch captain shoe salt awake harvest setup primary inmate ugly among become"
# API Keys and tokens
IPINFO_API_TOKEN=your_token_here
AUTHENTICATOR_PASSWORD=password_here
# Logging
RUST_LOG=info # Options: error, warn, info, debug, trace
RUST_BACKTRACE=1 # Enable backtraces
# Database
DATABASE_URL=postgresql://user:pass@localhost/nym_api
```
#### Using Environment Files
```bash
# Load environment and run command
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Export to shell
source envs/sandbox.env
# Use with make targets
dotenv -f envs/sandbox.env -- make run-api-tests
```
## Initial Setup
### First Time Setup
1. **Install Prerequisites**
```bash
# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install Node.js and yarn
# Via nvm (recommended):
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
nvm install 20
npm install -g yarn
# Install build tools
# Ubuntu/Debian:
sudo apt-get install build-essential pkg-config libssl-dev protobuf-compiler libpq-dev
# macOS:
brew install protobuf postgresql
# Install wasm-opt for contract builds
npm install -g wasm-opt
# Add wasm target for Rust
rustup target add wasm32-unknown-unknown
```
2. **Clone and Setup Repository**
```bash
git clone https://github.com/nymtech/nym.git
cd nym/nym
# Install JavaScript dependencies
yarn install
# Build the project
make build
```
3. **Database Setup (Optional, for API development)**
```bash
# Install PostgreSQL
# Create database
createdb nym_api
# Run migrations (from nym-api directory)
cd nym-api
sqlx migrate run
```
### Quick Start
```bash
# Run a mixnode locally
dotenv -f envs/sandbox.env -- cargo run -p nym-node -- run --mode mixnode --id my-mixnode
# Run a gateway locally
dotenv -f envs/sandbox.env -- cargo run -p nym-node -- run --mode gateway --id my-gateway
# Run the API server
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Run a client
cargo run -p nym-client -- init --id my-client
cargo run -p nym-client -- run --id my-client
```
## CI/CD Pipeline
The project uses GitHub Actions for CI/CD with several key workflows:
1. **Build and Test**:
- `ci-build.yml`: Main build workflow for Rust components
- Tests are run on multiple platforms (Linux, Windows, macOS)
- Includes formatting check (rustfmt) and linting (clippy)
2. **Release Process**:
- Binary artifacts are published on release tags
- Multiple platform builds are created
3. **Documentation**:
- Documentation is automatically built and deployed
## Database Structure
The system uses SQLite databases with tables like:
- `mixnode_status`: Status information about mixnodes
- `gateway_status`: Status information about gateways
- `routes`: Route performance information (success/failure of specific paths)
- `monitor_run`: Information about monitoring test runs
## Development Workflows
### Running a Node
To run the mixnode or gateway:
```bash
# Run nym-node as a mixnode with specified identity
cargo run -p nym-node -- run --mode mixnode --id my-mixnode
# Run nym-node as a gateway
cargo run -p nym-node -- run --mode gateway --id my-gateway
```
### Configuration
Nodes can be configured with files in various locations:
- Command-line arguments
- Environment variables
- `.env` files specified with `--config-env-file`
### Monitoring
To monitor the health of your node:
- View logs for real-time information
- Use the node's HTTP API for status information
- Check the explorer for public node statistics
## Common Libraries
- `common/types`: Shared data types across all components
- `common/crypto`: Cryptographic primitives and wrappers
- `common/client-core`: Core client functionality
- `common/gateway-client`: Client-gateway communication
- `common/task`: Task management and concurrency utilities
- `common/nymsphinx`: Sphinx packet implementation for mixnet
- `common/topology`: Network topology management
- `common/credentials`: Credential system for privacy-preserving authentication
- `common/bandwidth-controller`: Bandwidth management and accounting
## Code Conventions
- Error handling: Use anyhow/thiserror for structured error handling
- Logging: Use the tracing framework for logging and diagnostics
- State management: Generally use Tokio/futures for async code
- Configuration: Use the config crate and env vars with defaults
- Database: Use sqlx for type-safe database queries
- Follow clippy recommendations and rustfmt formatting
- Use semantic commit messages: feat, fix, docs, refactor, test, chore
## When Making Changes
- Run `make test` before submitting PRs
- Follow Rust naming conventions
- Use `clippy` to check for common issues
- Update SQLx query caches when modifying DB queries: `cargo sqlx prepare`
- Consider backward compatibility for protocol changes
- Use lefthook pre-commit hooks for TypeScript formatting
- Run `cargo deny check` to verify dependency compliance
- Test against both sandbox and local environments when possible
- Update relevant documentation and CHANGELOG.md
## Development Tools
### Useful Cargo Commands
```bash
# Check for outdated dependencies
cargo outdated
# Analyze binary size
cargo bloat --release -p nym-node
# Generate dependency graph
cargo tree -p nym-api
# Run with instrumentation
cargo run --features profiling -p nym-node
# Check for security advisories
cargo audit
```
### Database Tools
```bash
# SQLx CLI for migrations
cargo install sqlx-cli
# Create new migration
cd nym-api && sqlx migrate add <migration_name>
# Prepare query metadata for offline compilation
cargo sqlx prepare --workspace
# View database schema
./nym-api/enter_db.sh
```
### Development Scripts
- `scripts/build_topology.py`: Generate network topology files
- `scripts/node_api_check.py`: Verify node API endpoints
- `scripts/network_tunnel_manager.sh`: Manage network tunnels
- `scripts/localnet_start.sh`: Start a local test network
- Various deployment scripts in `deployment/` for different environments
## Debugging
- Enable more verbose logging with the RUST_LOG environment variable:
```
RUST_LOG=debug,nym_node=trace cargo run -p nym-node -- run --mode mixnode
```
- Use the HTTP API endpoints for status information
- Check monitoring data in the database for network performance metrics
- For complex issues, use tracing tools to follow packet flow
- Enable backtraces: `RUST_BACKTRACE=full`
- For WASM debugging: Use browser developer tools with source maps
## Deployment and Advanced Configurations
### Deployment Structure
The `deployment/` directory contains Ansible playbooks and configurations for various deployment scenarios:
- **`aws/`**: AWS-specific deployment configurations
- **`mixnode/`**: Mixnode deployment playbooks
- **`gateway/`**: Gateway deployment playbooks
- **`validator/`**: Validator node deployment
- **`sandbox-v2/`**: Complete sandbox environment setup
- **`big-dipper-2/`**: Block explorer deployment
### Sandbox V2 Deployment
The sandbox-v2 deployment (`deployment/sandbox-v2/`) provides a complete test environment:
```bash
# Key playbooks:
- deploy.yaml # Main deployment orchestrator
- deploy-mixnodes.yaml # Deploy mixnodes
- deploy-gateways.yaml # Deploy gateways
- deploy-validators.yaml # Deploy validator nodes
- deploy-nym-api.yaml # Deploy API services
```
### Custom Environment Setup
To create a custom environment:
1. Copy an existing env file: `cp envs/sandbox.env envs/custom.env`
2. Modify the network endpoints and contract addresses
3. Update the `NETWORK_NAME` to your identifier
4. Set appropriate mnemonics and keys (use fresh ones for production!)
### Contract Addresses
Contract addresses are network-specific and defined in environment files:
- Mixnet contract: Manages mixnode/gateway registry
- Vesting contract: Handles token vesting schedules
- Coconut contracts: Privacy-preserving credentials
- Name service: Human-readable address mapping
- Ecash contract: Electronic cash functionality
### Local Network Setup
For a completely local network:
```bash
# Start local chain
./scripts/localnet_start.sh
# Deploy contracts
cd contracts
make deploy-local
# Start nodes with local config
dotenv -f envs/local.env -- cargo run -p nym-node -- run --mode mixnode
```
## Common Issues and Troubleshooting
### Database Issues
- When modifying database queries, you must update SQLx query caches:
```bash
cargo sqlx prepare
```
- If you see SQLx errors about missing query files, this is likely the cause
- For "database is locked" errors with SQLite, ensure only one process accesses the DB
- For PostgreSQL connection issues, verify DATABASE_URL and that the server is running
### API Connection Issues
- Check the environment variables pointing to the APIs (NYM_API, NYXD)
- Verify network connectivity and API health endpoints
- For authentication issues, check node keys and credentials
- Common endpoints to verify:
- API health: `$NYM_API/health`
- Chain status: `$NYXD/status`
- Contract info: `$NYXD/cosmwasm/wasm/v1/contract/$CONTRACT_ADDRESS`
### Build Problems
- Clean dependencies with `cargo clean` for a fresh build
- Check for compatible Rust version (1.80+ recommended)
- For smart contract builds, ensure wasm-opt is installed: `npm install -g wasm-opt`
- For cross-compilation issues, check target-specific dependencies
- WASM build issues: Ensure wasm32-unknown-unknown target is installed:
```bash
rustup target add wasm32-unknown-unknown
```
- For "cannot find -lpq" errors, install PostgreSQL development files:
```bash
# Ubuntu/Debian
sudo apt-get install libpq-dev
# macOS
brew install postgresql
```
### Environment Issues
- Contract address mismatches: Ensure you're using the correct environment file
- "Account sequence mismatch": The account nonce is out of sync, wait and retry
- Token decimal issues: Sandbox uses different decimal places than mainnet
- API version mismatches: Ensure your local API version matches the network
- "Insufficient funds": Get test tokens from faucet (sandbox) or check balance
- Gateway/mixnode bonding issues: Verify minimum stake requirements
## Working with Routes and Monitoring
1. Route monitoring metrics are stored in a `routes` table with:
- Layer node IDs (layer1, layer2, layer3, gw)
- Success flag (boolean)
- Timestamp
2. To analyze routes:
- Check `NetworkAccount` and `AccountingRoute` in `nym-network-monitor/src/accounting.rs`
- View monitoring logic in `common/nymsphinx/chunking/monitoring.rs`
- Observe how routes are submitted to the database in the `submit_accounting_routes_to_db` function
## Performance Optimization
### Profiling and Benchmarking
```bash
# Run benchmarks
cargo bench -p nym-node
# Profile with perf (Linux)
cargo build --release --features profiling
perf record --call-graph=dwarf ./target/release/nym-node run --mode mixnode
perf report
# Generate flamegraph
cargo install flamegraph
cargo flamegraph --bin nym-node -- run --mode mixnode
```
### Common Performance Considerations
- Use bounded channels for backpressure
- Batch database operations where possible
- Monitor memory usage with `RUST_LOG=nym_node::metrics=debug`
- Use connection pooling for database connections
- Consider using `jemalloc` for better memory allocation performance
Generated
+900 -1664
View File
File diff suppressed because it is too large Load Diff
+39 -64
View File
@@ -31,20 +31,14 @@ members = [
"common/client-libs/mixnet-client", "common/client-libs/mixnet-client",
"common/client-libs/validator-client", "common/client-libs/validator-client",
"common/commands", "common/commands",
"common/nym-common",
"common/config", "common/config",
"common/cosmwasm-smart-contracts/coconut-dkg", "common/cosmwasm-smart-contracts/coconut-dkg",
"common/cosmwasm-smart-contracts/contracts-common", "common/cosmwasm-smart-contracts/contracts-common", "common/cosmwasm-smart-contracts/easy_addr",
"common/cosmwasm-smart-contracts/contracts-common-testing",
"common/cosmwasm-smart-contracts/easy_addr",
"common/cosmwasm-smart-contracts/ecash-contract", "common/cosmwasm-smart-contracts/ecash-contract",
"common/cosmwasm-smart-contracts/group-contract", "common/cosmwasm-smart-contracts/group-contract",
"common/cosmwasm-smart-contracts/mixnet-contract", "common/cosmwasm-smart-contracts/mixnet-contract",
"common/cosmwasm-smart-contracts/multisig-contract", "common/cosmwasm-smart-contracts/multisig-contract",
"common/cosmwasm-smart-contracts/nym-performance-contract",
"common/cosmwasm-smart-contracts/nym-pool-contract",
"common/cosmwasm-smart-contracts/vesting-contract", "common/cosmwasm-smart-contracts/vesting-contract",
"common/credential-proxy",
"common/credential-storage", "common/credential-storage",
"common/credential-utils", "common/credential-utils",
"common/credential-verification", "common/credential-verification",
@@ -52,15 +46,13 @@ members = [
"common/credentials-interface", "common/credentials-interface",
"common/crypto", "common/crypto",
"common/dkg", "common/dkg",
"common/ecash-signer-check",
"common/ecash-signer-check-types",
"common/ecash-time", "common/ecash-time",
"common/execute",
"common/exit-policy", "common/exit-policy",
"common/gateway-requests", "common/gateway-requests",
"common/gateway-stats-storage", "common/gateway-stats-storage",
"common/gateway-storage", "common/gateway-storage",
"common/http-api-client", "common/http-api-client",
"common/http-api-client-macro",
"common/http-api-common", "common/http-api-common",
"common/inclusion-probability", "common/inclusion-probability",
"common/ip-packet-requests", "common/ip-packet-requests",
@@ -69,13 +61,9 @@ members = [
"common/network-defaults", "common/network-defaults",
"common/node-tester-utils", "common/node-tester-utils",
"common/nonexhaustive-delayqueue", "common/nonexhaustive-delayqueue",
"common/nym-cache",
"common/nym-connection-monitor",
"common/nym-id", "common/nym-id",
"common/nym-metrics", "common/nym-metrics",
"common/nym_offline_compact_ecash", "common/nym_offline_compact_ecash",
"common/nymnoise",
"common/nymnoise/keys",
"common/nymsphinx", "common/nymsphinx",
"common/nymsphinx/acknowledgements", "common/nymsphinx/acknowledgements",
"common/nymsphinx/addressing", "common/nymsphinx/addressing",
@@ -89,7 +77,6 @@ members = [
"common/nymsphinx/types", "common/nymsphinx/types",
"common/nyxd-scraper", "common/nyxd-scraper",
"common/pemstore", "common/pemstore",
"common/registration",
"common/serde-helpers", "common/serde-helpers",
"common/service-provider-requests-common", "common/service-provider-requests-common",
"common/socks5-client-core", "common/socks5-client-core",
@@ -98,33 +85,25 @@ members = [
"common/statistics", "common/statistics",
"common/store-cipher", "common/store-cipher",
"common/task", "common/task",
"common/test-utils",
"common/ticketbooks-merkle", "common/ticketbooks-merkle",
"common/topology", "common/topology",
"common/tun", "common/tun",
"common/types", "common/types",
"common/upgrade-mode-check",
"common/verloc", "common/verloc",
"common/wasm/client-core", "common/wasm/client-core",
"common/wasm/storage", "common/wasm/storage",
"common/wasm/utils", "common/wasm/utils",
"common/wireguard", "common/wireguard",
"common/wireguard-private-metadata/client",
"common/wireguard-private-metadata/server",
"common/wireguard-private-metadata/shared",
"common/wireguard-private-metadata/tests",
"common/wireguard-types", "common/wireguard-types",
"common/zulip-client",
"documentation/autodoc", "documentation/autodoc",
"gateway", "gateway",
"integrations/bity",
"nym-api", "nym-api",
"nym-api/nym-api-requests", "nym-api/nym-api-requests",
"nym-authenticator-client",
"nym-browser-extension/storage", "nym-browser-extension/storage",
"nym-credential-proxy/nym-credential-proxy", "nym-credential-proxy/nym-credential-proxy",
"nym-credential-proxy/nym-credential-proxy-requests", "nym-credential-proxy/nym-credential-proxy-requests",
"nym-credential-proxy/vpn-api-lib-wasm", "nym-credential-proxy/vpn-api-lib-wasm",
"nym-ip-packet-client",
"nym-network-monitor", "nym-network-monitor",
"nym-node", "nym-node",
"nym-node-status-api/nym-node-status-agent", "nym-node-status-api/nym-node-status-agent",
@@ -133,19 +112,16 @@ members = [
"nym-node/nym-node-metrics", "nym-node/nym-node-metrics",
"nym-node/nym-node-requests", "nym-node/nym-node-requests",
"nym-outfox", "nym-outfox",
"nym-registration-client",
"nym-signers-monitor",
"nym-statistics-api",
"nym-validator-rewarder", "nym-validator-rewarder",
"nyx-chain-watcher", "nyx-chain-watcher",
"sdk/ffi/cpp", "sdk/ffi/cpp",
"sdk/ffi/go", "sdk/ffi/go",
"sdk/ffi/shared", "sdk/ffi/shared",
"sdk/rust/nym-sdk", "sdk/rust/nym-sdk",
"service-providers/authenticator",
"service-providers/common", "service-providers/common",
"service-providers/ip-packet-router", "service-providers/ip-packet-router",
"service-providers/network-requester", "service-providers/network-requester",
"sqlx-pool-guard",
"tools/echo-server", "tools/echo-server",
"tools/internal/contract-state-importer/importer-cli", "tools/internal/contract-state-importer/importer-cli",
"tools/internal/contract-state-importer/importer-contract", "tools/internal/contract-state-importer/importer-contract",
@@ -153,8 +129,9 @@ members = [
# "tools/internal/sdk-version-bump", # "tools/internal/sdk-version-bump",
"tools/internal/ssl-inject", "tools/internal/ssl-inject",
"tools/internal/testnet-manager", "tools/internal/testnet-manager",
"tools/internal/testnet-manager",
"tools/internal/testnet-manager/dkg-bypass-contract", "tools/internal/testnet-manager/dkg-bypass-contract",
"tools/internal/validator-status-check", "tools/internal/testnet-manager/dkg-bypass-contract", "tools/internal/validator-status-check",
"tools/nym-cli", "tools/nym-cli",
"tools/nym-id-cli", "tools/nym-id-cli",
"tools/nym-nr-query", "tools/nym-nr-query",
@@ -165,7 +142,6 @@ members = [
"wasm/mix-fetch", "wasm/mix-fetch",
"wasm/node-tester", "wasm/node-tester",
"wasm/zknym-lib", "wasm/zknym-lib",
"nym-gateway-probe"
] ]
default-members = [ default-members = [
@@ -176,24 +152,24 @@ default-members = [
"nym-node", "nym-node",
"nym-node-status-api/nym-node-status-agent", "nym-node-status-api/nym-node-status-agent",
"nym-node-status-api/nym-node-status-api", "nym-node-status-api/nym-node-status-api",
"nym-statistics-api",
"nym-validator-rewarder", "nym-validator-rewarder",
"nyx-chain-watcher", "nyx-chain-watcher",
"service-providers/authenticator",
"service-providers/ip-packet-router", "service-providers/ip-packet-router",
"service-providers/network-requester", "service-providers/network-requester",
"tools/nymvisor", "tools/nymvisor",
] ]
exclude = ["contracts", "nym-wallet", "cpu-cycles"] exclude = ["explorer", "contracts", "nym-wallet", "cpu-cycles"]
[workspace.package] [workspace.package]
authors = ["Nym Technologies SA"] authors = ["Nym Technologies SA"]
repository = "https://github.com/nymtech/nym" repository = "https://github.com/nymtech/nym"
homepage = "https://nymtech.net" homepage = "https://nymtech.net"
documentation = "https://nymtech.net" documentation = "https://nymtech.net"
edition = "2024" edition = "2021"
license = "Apache-2.0" license = "Apache-2.0"
rust-version = "1.85" rust-version = "1.80"
readme = "README.md" readme = "README.md"
[workspace.dependencies] [workspace.dependencies]
@@ -215,13 +191,14 @@ base64 = "0.22.1"
base85rs = "0.1.3" base85rs = "0.1.3"
bincode = "1.3.3" bincode = "1.3.3"
bip39 = { version = "2.0.0", features = ["zeroize"] } bip39 = { version = "2.0.0", features = ["zeroize"] }
bit-vec = "0.7.0" # can we unify those?
bitvec = "1.0.0" bitvec = "1.0.0"
blake3 = "1.7.0" blake3 = "1.7.0"
bloomfilter = "3.0.1" bloomfilter = "3.0.1"
bs58 = "0.5.1" bs58 = "0.5.1"
bytecodec = "0.4.15" bytecodec = "0.4.15"
bytes = "1.10.1" bytes = "1.10.1"
cargo_metadata = "0.19.2" cargo_metadata = "0.18.1"
celes = "2.6.0" celes = "2.6.0"
cfg-if = "1.0.0" cfg-if = "1.0.0"
chacha20 = "0.9.0" chacha20 = "0.9.0"
@@ -233,8 +210,8 @@ clap_complete = "4.5"
clap_complete_fig = "4.5" clap_complete_fig = "4.5"
colored = "2.2" colored = "2.2"
comfy-table = "7.1.4" comfy-table = "7.1.4"
console = "0.16.0" console = "0.15.11"
console-subscriber = "0.4.1" console-subscriber = "0.1.1"
console_error_panic_hook = "0.1" console_error_panic_hook = "0.1"
const-str = "0.5.6" const-str = "0.5.6"
const_format = "0.2.34" const_format = "0.2.34"
@@ -242,13 +219,14 @@ criterion = "0.5"
csv = "1.3.1" csv = "1.3.1"
ctr = "0.9.1" ctr = "0.9.1"
cupid = "0.6.1" cupid = "0.6.1"
curve25519-dalek = "4.1"
dashmap = "5.5.3" dashmap = "5.5.3"
# We want https://github.com/DefGuard/wireguard-rs/pull/64 , but there's no crates.io release being pushed out anymore # We want https://github.com/DefGuard/wireguard-rs/pull/64 , but there's no crates.io release being pushed out anymore
defguard_wireguard_rs = { git = "https://github.com/DefGuard/wireguard-rs.git", rev = "v0.4.7" } defguard_wireguard_rs = { git = "https://github.com/DefGuard/wireguard-rs.git", rev = "v0.4.7" }
digest = "0.10.7" digest = "0.10.7"
dirs = "6.0" dirs = "5.0"
doc-comment = "0.3"
dotenvy = "0.15.6" dotenvy = "0.15.6"
dyn-clone = "1.0.19"
ecdsa = "0.16" ecdsa = "0.16"
ed25519-dalek = "2.1" ed25519-dalek = "2.1"
encoding_rs = "0.8.35" encoding_rs = "0.8.35"
@@ -262,8 +240,11 @@ futures = "0.3.31"
futures-util = "0.3" futures-util = "0.3"
generic-array = "0.14.7" generic-array = "0.14.7"
getrandom = "0.2.10" getrandom = "0.2.10"
getset = "0.1.5"
handlebars = "3.5.5" handlebars = "3.5.5"
headers = "0.4.0"
hex = "0.4.3" hex = "0.4.3"
hex-literal = "0.3.3"
hickory-resolver = "0.25" hickory-resolver = "0.25"
hkdf = "0.12.3" hkdf = "0.12.3"
hmac = "0.12.1" hmac = "0.12.1"
@@ -275,22 +256,22 @@ humantime = "2.2.0"
humantime-serde = "1.1.1" humantime-serde = "1.1.1"
hyper = "1.6.0" hyper = "1.6.0"
hyper-util = "0.1" hyper-util = "0.1"
indicatif = "0.18.0" indicatif = "0.17.11"
inquire = "0.6.2" inquire = "0.6.2"
inventory = "0.3.21"
ip_network = "0.4.1" ip_network = "0.4.1"
ipnetwork = "0.20" ipnetwork = "0.20"
itertools = "0.14.0" itertools = "0.14.0"
jwt-simple = { version = "0.12.12", default-features = false, features = ["pure-rust"] }
k256 = "0.13" k256 = "0.13"
lazy_static = "1.5.0" lazy_static = "1.5.0"
ledger-transport = "0.10.0" ledger-transport = "0.10.0"
ledger-transport-hid = "0.10.0" ledger-transport-hid = "0.10.0"
log = "0.4" log = "0.4"
maxminddb = "0.23.0"
mime = "0.3.17" mime = "0.3.17"
moka = { version = "0.12", features = ["future"] } moka = { version = "0.12", features = ["future"] }
nix = "0.27.1" nix = "0.27.1"
notify = "5.1.0" notify = "5.1.0"
okapi = "0.7.0"
once_cell = "1.21.3" once_cell = "1.21.3"
opentelemetry = "0.19.0" opentelemetry = "0.19.0"
opentelemetry-jaeger = "0.18.0" opentelemetry-jaeger = "0.18.0"
@@ -298,18 +279,20 @@ parking_lot = "0.12.3"
pem = "0.8" pem = "0.8"
petgraph = "0.6.5" petgraph = "0.6.5"
pin-project = "1.1" pin-project = "1.1"
pnet_packet = "0.35.0" pin-project-lite = "0.2.16"
publicsuffix = "2.3.0" publicsuffix = "2.3.0"
proc_pidinfo = "0.1.3"
quote = "1" quote = "1"
rand = "0.8.5" rand = "0.8.5"
rand_chacha = "0.3" rand_chacha = "0.3"
rand_core = "0.6.3" rand_core = "0.6.3"
rand_distr = "0.4" rand_distr = "0.4"
rand_pcg = "0.3.1"
rand_seeder = "0.2.3"
rayon = "1.5.1" rayon = "1.5.1"
regex = "1.10.6" regex = "1.10.6"
reqwest = { version = "0.12.15", default-features = false } reqwest = { version = "0.12.15", default-features = false }
rs_merkle = "1.5.0" rs_merkle = "1.5.0"
safer-ffi = "0.1.13"
schemars = "0.8.22" schemars = "0.8.22"
semver = "1.0.26" semver = "1.0.26"
serde = "1.0.219" serde = "1.0.219"
@@ -320,24 +303,21 @@ serde_json_path = "0.7.2"
serde_repr = "0.1" serde_repr = "0.1"
serde_with = "3.9.0" serde_with = "3.9.0"
serde_yaml = "0.9.25" serde_yaml = "0.9.25"
serde_plain = "1.0.2"
sha2 = "0.10.9" sha2 = "0.10.9"
si-scale = "0.2.3" si-scale = "0.2.3"
snow = "0.9.6"
sphinx-packet = "=0.6.0" sphinx-packet = "=0.6.0"
sqlx = "0.8.6" sqlx = "0.7.4"
strum = "0.27.2" strum = "0.26"
strum_macros = "0.27.2" strum_macros = "0.26"
subtle-encoding = "0.5" subtle-encoding = "0.5"
syn = "2" syn = "1"
sysinfo = "0.37.0" sysinfo = "0.33.0"
tap = "1.0.1" tap = "1.0.1"
tar = "0.4.44" tar = "0.4.44"
test-with = { version = "0.15.4", default-features = false } tempfile = "3.19"
tempfile = "3.20"
thiserror = "2.0" thiserror = "2.0"
time = "0.3.41" time = "0.3.41"
tokio = "1.47" tokio = "1.44"
tokio-postgres = "0.7" tokio-postgres = "0.7"
tokio-stream = "0.1.17" tokio-stream = "0.1.17"
tokio-test = "0.4.4" tokio-test = "0.4.4"
@@ -353,10 +333,8 @@ tracing-opentelemetry = "0.19.0"
tracing-subscriber = "0.3.19" tracing-subscriber = "0.3.19"
tracing-tree = "0.2.2" tracing-tree = "0.2.2"
tracing-indicatif = "0.3.9" tracing-indicatif = "0.3.9"
tracing-test = "0.2.5"
ts-rs = "10.1.0" ts-rs = "10.1.0"
tungstenite = { version = "0.20.1", default-features = false } tungstenite = { version = "0.20.1", default-features = false }
typed-builder = "0.23.0"
uniffi = "0.29.2" uniffi = "0.29.2"
uniffi_build = "0.29.0" uniffi_build = "0.29.0"
url = "2.5" url = "2.5"
@@ -365,8 +343,8 @@ utoipa-swagger-ui = "8.1"
utoipauto = "0.2" utoipauto = "0.2"
uuid = "*" uuid = "*"
vergen = { version = "=8.3.1", default-features = false } vergen = { version = "=8.3.1", default-features = false }
vergen-gitcl = { version = "1.0.8", default-features = false }
walkdir = "2" walkdir = "2"
wasm-bindgen-test = "0.3.49"
x25519-dalek = "2.0.0" x25519-dalek = "2.0.0"
zeroize = "1.7.0" zeroize = "1.7.0"
@@ -384,6 +362,9 @@ subtle = "2.5.0"
# cosmwasm-related # cosmwasm-related
cosmwasm-schema = "=2.2.2" cosmwasm-schema = "=2.2.2"
cosmwasm-std = "=2.2.2" cosmwasm-std = "=2.2.2"
# use 1.0.1 as that's the version used by cosmwasm-std 2.2.1
# (and ideally we don't want to pull the same dependency twice)
serde-json-wasm = "=1.0.1"
# same version as used by cosmwasm # same version as used by cosmwasm
cw-utils = "=2.0.0" cw-utils = "=2.0.0"
cw-storage-plus = "=2.0.0" cw-storage-plus = "=2.0.0"
@@ -391,7 +372,6 @@ cw2 = { version = "=2.0.0" }
cw3 = { version = "=2.0.0" } cw3 = { version = "=2.0.0" }
cw4 = { version = "=2.0.0" } cw4 = { version = "=2.0.0" }
cw-controllers = { version = "=2.0.0" } cw-controllers = { version = "=2.0.0" }
cw-multi-test = "=2.3.2"
# cosmrs-related # cosmrs-related
bip32 = { version = "0.5.3", default-features = false } bip32 = { version = "0.5.3", default-features = false }
@@ -405,19 +385,17 @@ prost = { version = "0.13", default-features = false }
# wasm-related dependencies # wasm-related dependencies
gloo-utils = "0.2.0" gloo-utils = "0.2.0"
gloo-net = "0.6.0" gloo-net = "0.6.0"
gloo-timers = "0.3.0"
indexed_db_futures = "0.6.4" indexed_db_futures = "0.6.4"
js-sys = "0.3.76" js-sys = "0.3.76"
serde-wasm-bindgen = "0.6.5" serde-wasm-bindgen = "0.6.5"
tsify = "0.4.5" tsify = "0.4.5"
tokio_with_wasm = { version = "0.8.7" }
wasm-bindgen = "0.2.99" wasm-bindgen = "0.2.99"
wasm-bindgen-futures = "0.4.49" wasm-bindgen-futures = "0.4.49"
wasm-bindgen-test = "0.3.49"
wasmtimer = "0.4.1" wasmtimer = "0.4.1"
web-sys = "0.3.76" web-sys = "0.3.76"
# for local development: # for local development:
#[patch.crates-io] #[patch.crates-io]
#sphinx-packet = { path = "../sphinx" } #sphinx-packet = { path = "../sphinx" }
@@ -449,9 +427,6 @@ opt-level = 'z'
# lto = true # lto = true
opt-level = 'z' opt-level = 'z'
[workspace.lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tokio_unstable)'] }
[workspace.lints.clippy] [workspace.lints.clippy]
unwrap_used = "deny" unwrap_used = "deny"
expect_used = "deny" expect_used = "deny"
+15 -71
View File
@@ -12,11 +12,7 @@ help:
@echo " clippy: run clippy for all workspaces" @echo " clippy: run clippy for all workspaces"
@echo " test: run clippy, unit tests, and formatting." @echo " test: run clippy, unit tests, and formatting."
@echo " test-all: like test, but also includes the expensive tests" @echo " test-all: like test, but also includes the expensive tests"
@echo " deb: build debian packages" @echo " deb: build debian packages
@echo ""
@echo "Contract building targets:"
@echo " contracts: build contracts for development (includes wasm-opt)"
@echo " publish-contracts: build contracts using Docker optimizer (deterministic)"
# ----------------------------------------------------------------------------- # -----------------------------------------------------------------------------
# Meta targets # Meta targets
@@ -107,16 +103,16 @@ sdk-wasm-build:
$(MAKE) -C nym-browser-extension/storage wasm-pack $(MAKE) -C nym-browser-extension/storage wasm-pack
$(MAKE) -C wasm/client $(MAKE) -C wasm/client
$(MAKE) -C wasm/node-tester $(MAKE) -C wasm/node-tester
# $(MAKE) -C wasm/mix-fetch $(MAKE) -C wasm/mix-fetch
$(MAKE) -C wasm/zknym-lib $(MAKE) -C wasm/zknym-lib
# $(MAKE) -C wasm/full-nym-wasm #$(MAKE) -C wasm/full-nym-wasm
# run this from npm/yarn to ensure tools are in the path, e.g. yarn build:sdk from root of repo # run this from npm/yarn to ensure tools are in the path, e.g. yarn build:sdk from root of repo
sdk-typescript-build: sdk-typescript-build:
npx lerna run --scope @nymproject/sdk build --stream npx lerna run --scope @nymproject/sdk build --stream
# npx lerna run --scope @nymproject/mix-fetch build --stream npx lerna run --scope @nymproject/mix-fetch build --stream
# npx lerna run --scope @nymproject/node-tester build --stream npx lerna run --scope @nymproject/node-tester build --stream
# yarn --cwd sdk/typescript/codegen/contract-clients build yarn --cwd sdk/typescript/codegen/contract-clients build
# NOTE: These targets are part of the main workspace (but not as wasm32-unknown-unknown) # NOTE: These targets are part of the main workspace (but not as wasm32-unknown-unknown)
WASM_CRATES = extension-storage nym-client-wasm nym-node-tester-wasm zknym-lib WASM_CRATES = extension-storage nym-client-wasm nym-node-tester-wasm zknym-lib
@@ -134,77 +130,25 @@ cargo-test: sdk-wasm-test
clippy: sdk-wasm-lint clippy: sdk-wasm-lint
# ----------------------------------------------------------------------------- # -----------------------------------------------------------------------------
# Build CosmWasm contracts (deterministic docker build) # Build contracts ready for deploy
# ----------------------------------------------------------------------------- # -----------------------------------------------------------------------------
CONTRACTS=vesting_contract mixnet_contract nym_ecash cw3_flex_multisig cw4_group nym_coconut_dkg
CONTRACTS_WASM=$(addsuffix .wasm, $(CONTRACTS))
CONTRACTS_OUT_DIR=contracts/target/wasm32-unknown-unknown/release
WASM_CONTRACT_DIR := contracts/target/wasm32-unknown-unknown/release contracts: build-release-contracts wasm-opt-contracts cosmwasm-check-contracts
# Find every direct contract folder that contains a Cargo.toml
CONTRACT_DIRS := $(shell find contracts -type f -name Cargo.toml \( ! -path "contracts/Cargo.toml" \) | grep -v integration-tests | xargs -n1 dirname | sort -u)
CONTRACTS_OUT_DIR = contracts/artifacts
# Build all contracts via the official CosmWasm optimizer image (one invocation per contract)
# See : https://github.com/CosmWasm/optimizer?tab=readme-ov-file#contracts-excluded-from-workspace
# The optimizer ships separate multi-arch images. ARM builds are *not* bit-for-bit identical to the
# canonical x86_64 build (see README notice in CosmWasm/optimizer). For reproducible artefacts we
# therefore always run the amd64 variant by default.
# Override with :
# $ COSMWASM_OPTIMIZER_IMAGE=cosmwasm/optimizer-arm64:0.17.0 make contracts-publish
#
COSMWASM_OPTIMIZER_IMAGE ?= cosmwasm/optimizer:0.17.0
COSMWASM_OPTIMIZER_PLATFORM ?= linux/amd64
COSMWASM_CHECK_IMAGE ?= rust:1.88
# Ensure clean build environment and run the optimizer
optimize-contracts:
@rm -rf artifacts 2>/dev/null || true
@echo "=== Ensuring clean build environment"
docker volume rm nym_contracts_cache 2>/dev/null || true
docker volume rm registry_cache 2>/dev/null || true
@for DIR in $(CONTRACT_DIRS); do \
echo "=== Optimizing $${DIR}"; \
docker run --rm --platform $(COSMWASM_OPTIMIZER_PLATFORM) \
-v $(CURDIR):/code \
--mount type=volume,source=nym_contracts_cache,target=/target \
--mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
-e CARGO_BUILD_INCREMENTAL=false \
-e RUSTFLAGS="-C target-cpu=generic -C debuginfo=0" \
-e SOURCE_DATE_EPOCH=1 \
$(COSMWASM_OPTIMIZER_IMAGE) $${DIR}; \
done
@mkdir -p $(CONTRACTS_OUT_DIR)
@cp artifacts/*.wasm $(CONTRACTS_OUT_DIR)/ 2>/dev/null || true
@cd $(CONTRACTS_OUT_DIR) && sha256sum *.wasm > checksums.txt
# Cleanup temporary artefacts directory
@rm -rf artifacts 2>/dev/null || true
# Check artifacts with cosmwasm-check inside the optimizer image
docker-check-contracts:
@docker run --rm --platform $(COSMWASM_OPTIMIZER_PLATFORM) \
-v $(CURDIR):/code --workdir /code \
--entrypoint /bin/sh \
$(COSMWASM_CHECK_IMAGE) -lc 'apt-get update && apt-get install -y --no-install-recommends llvm-dev libclang-dev pkg-config && export PATH="/usr/local/cargo/bin:/usr/local/rustup/bin:$$PATH" && cargo install cosmwasm-check --locked && WASMER_ENGINE=universal WASMER_COMPILER=singlepass cosmwasm-check contracts/artifacts/*.wasm'
wasm-opt-contracts: wasm-opt-contracts:
@for WASM in $(WASM_CONTRACT_DIR)/*.wasm; do \ for contract in $(CONTRACTS_WASM); do \
echo "Running wasm-opt on $$WASM"; \ wasm-opt --signext-lowering -Os $(CONTRACTS_OUT_DIR)/$$contract -o $(CONTRACTS_OUT_DIR)/$$contract; \
wasm-opt --signext-lowering -Os $$WASM -o $$WASM ; \
done done
cosmwasm-check-contracts: cosmwasm-check-contracts:
@for WASM in $(WASM_CONTRACT_DIR)/*.wasm; do \ for contract in $(CONTRACTS_WASM); do \
echo "Checking $$WASM"; \ cosmwasm-check $(CONTRACTS_OUT_DIR)/$$contract; \
cosmwasm-check $$WASM ; \
done done
# Default development build
contracts: build-release-contracts wasm-opt-contracts cosmwasm-check-contracts
# Publishing build used by CI deterministic Docker optimiser
publish-contracts: optimize-contracts cosmwasm-check-contracts
# Consider adding 's' to make plural consistent (beware: used in github workflow) # Consider adding 's' to make plural consistent (beware: used in github workflow)
contract-schema: contract-schema:
$(MAKE) -C contracts schema $(MAKE) -C contracts schema
+2 -2
View File
@@ -1,10 +1,10 @@
[package] [package]
name = "nym-client" name = "nym-client"
version = "1.1.64" version = "1.1.55"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"] authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
description = "Implementation of the Nym Client" description = "Implementation of the Nym Client"
edition = "2021" edition = "2021"
rust-version = "1.85" rust-version = "1.70"
license.workspace = true license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -2048,11 +2048,10 @@
} }
}, },
"node_modules/http-proxy-middleware": { "node_modules/http-proxy-middleware": {
"version": "2.0.9", "version": "2.0.4",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.9.tgz", "resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.4.tgz",
"integrity": "sha512-c1IyJYLYppU574+YI7R4QyX2ystMtVXZwIdzazUIPIJsHuWNd+mho2j+bKoHftndicGj9yh+xjd+l0yj7VeT1Q==", "integrity": "sha512-m/4FxX17SUvz4lJ5WPXOHDUuCwIqXLfLHs1s0uZ3oYjhoXlx9csYxaOa0ElDEJ+h8Q4iJ1s+lTMbiCa4EXIJqg==",
"dev": true, "dev": true,
"license": "MIT",
"dependencies": { "dependencies": {
"@types/http-proxy": "^1.17.8", "@types/http-proxy": "^1.17.8",
"http-proxy": "^1.18.1", "http-proxy": "^1.18.1",
@@ -6096,9 +6095,9 @@
} }
}, },
"http-proxy-middleware": { "http-proxy-middleware": {
"version": "2.0.9", "version": "2.0.4",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.9.tgz", "resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.4.tgz",
"integrity": "sha512-c1IyJYLYppU574+YI7R4QyX2ystMtVXZwIdzazUIPIJsHuWNd+mho2j+bKoHftndicGj9yh+xjd+l0yj7VeT1Q==", "integrity": "sha512-m/4FxX17SUvz4lJ5WPXOHDUuCwIqXLfLHs1s0uZ3oYjhoXlx9csYxaOa0ElDEJ+h8Q4iJ1s+lTMbiCa4EXIJqg==",
"dev": true, "dev": true,
"requires": { "requires": {
"@types/http-proxy": "^1.17.8", "@types/http-proxy": "^1.17.8",
+11 -16
View File
@@ -11,7 +11,7 @@ use nym_client_core::client::base_client::{
BaseClientBuilder, ClientInput, ClientOutput, ClientState, BaseClientBuilder, ClientInput, ClientOutput, ClientState,
}; };
use nym_sphinx::params::PacketType; use nym_sphinx::params::PacketType;
use nym_task::ShutdownManager; use nym_task::TaskHandle;
use nym_validator_client::QueryHttpRpcNyxdClient; use nym_validator_client::QueryHttpRpcNyxdClient;
use std::error::Error; use std::error::Error;
use std::path::PathBuf; use std::path::PathBuf;
@@ -29,8 +29,6 @@ pub struct SocketClient {
/// Optional path to a .json file containing standalone network details. /// Optional path to a .json file containing standalone network details.
custom_mixnet: Option<PathBuf>, custom_mixnet: Option<PathBuf>,
shutdown_manager: ShutdownManager,
} }
impl SocketClient { impl SocketClient {
@@ -42,7 +40,6 @@ impl SocketClient {
SocketClient { SocketClient {
config, config,
custom_mixnet, custom_mixnet,
shutdown_manager: Default::default(),
} }
} }
@@ -52,7 +49,7 @@ impl SocketClient {
client_output: ClientOutput, client_output: ClientOutput,
client_state: ClientState, client_state: ClientState,
self_address: &Recipient, self_address: &Recipient,
shutdown_token: nym_task::ShutdownToken, task_client: nym_task::TaskClient,
packet_type: PacketType, packet_type: PacketType,
) { ) {
info!("Starting websocket listener..."); info!("Starting websocket listener...");
@@ -60,7 +57,6 @@ impl SocketClient {
let ClientInput { let ClientInput {
connection_command_sender, connection_command_sender,
input_sender, input_sender,
..
} = client_input; } = client_input;
let ClientOutput { let ClientOutput {
@@ -81,24 +77,24 @@ impl SocketClient {
shared_lane_queue_lengths, shared_lane_queue_lengths,
reply_controller_sender, reply_controller_sender,
Some(packet_type), Some(packet_type),
shutdown_token.clone(), task_client.fork("websocket_handler"),
); );
websocket::Listener::new( websocket::Listener::new(
config.socket.host, config.socket.host,
config.socket.listening_port, config.socket.listening_port,
shutdown_token.child_token(), task_client.with_suffix("websocket_listener"),
) )
.start(websocket_handler); .start(websocket_handler);
} }
/// blocking version of `start_socket` method. Will run forever (or until SIGINT is sent) /// blocking version of `start_socket` method. Will run forever (or until SIGINT is sent)
pub async fn run_socket_forever(self) -> Result<(), Box<dyn Error + Send + Sync>> { pub async fn run_socket_forever(self) -> Result<(), Box<dyn Error + Send + Sync>> {
let mut shutdown = self.start_socket().await?; let shutdown = self.start_socket().await?;
shutdown.run_until_shutdown().await; let res = shutdown.wait_for_shutdown().await;
log::info!("Stopping nym-client"); log::info!("Stopping nym-client");
Ok(()) res
} }
async fn initialise_storage(&self) -> Result<OnDiskPersistent, ClientError> { async fn initialise_storage(&self) -> Result<OnDiskPersistent, ClientError> {
@@ -115,7 +111,7 @@ impl SocketClient {
let dkg_query_client = if self.config.base.client.disabled_credentials_mode { let dkg_query_client = if self.config.base.client.disabled_credentials_mode {
None None
} else { } else {
Some(default_query_dkg_client_from_config(&self.config.base)?) Some(default_query_dkg_client_from_config(&self.config.base))
}; };
let storage = self.initialise_storage().await?; let storage = self.initialise_storage().await?;
@@ -123,7 +119,6 @@ impl SocketClient {
let mut base_client = let mut base_client =
BaseClientBuilder::new(self.config().base(), storage, dkg_query_client) BaseClientBuilder::new(self.config().base(), storage, dkg_query_client)
.with_shutdown(self.shutdown_manager.shutdown_tracker_owned())
.with_user_agent(user_agent); .with_user_agent(user_agent);
if let Some(custom_mixnet) = &self.custom_mixnet { if let Some(custom_mixnet) = &self.custom_mixnet {
@@ -133,7 +128,7 @@ impl SocketClient {
Ok(base_client) Ok(base_client)
} }
pub async fn start_socket(self) -> Result<ShutdownManager, ClientError> { pub async fn start_socket(self) -> Result<TaskHandle, ClientError> {
if !self.config.socket.socket_type.is_websocket() { if !self.config.socket.socket_type.is_websocket() {
return Err(ClientError::InvalidSocketMode); return Err(ClientError::InvalidSocketMode);
} }
@@ -152,13 +147,13 @@ impl SocketClient {
client_output, client_output,
client_state, client_state,
&self_address, &self_address,
self.shutdown_manager.child_shutdown_token(), started_client.task_handle.get_handle(),
packet_type, packet_type,
); );
info!("Client startup finished!"); info!("Client startup finished!");
info!("The address of this client is: {self_address}"); info!("The address of this client is: {self_address}");
Ok(self.shutdown_manager) Ok(started_client.task_handle)
} }
} }
+28 -22
View File
@@ -19,7 +19,7 @@ use nym_sphinx::receiver::ReconstructedMessage;
use nym_task::connections::{ use nym_task::connections::{
ConnectionCommand, ConnectionCommandSender, ConnectionId, LaneQueueLengths, TransmissionLane, ConnectionCommand, ConnectionCommandSender, ConnectionId, LaneQueueLengths, TransmissionLane,
}; };
use nym_task::ShutdownToken; use nym_task::TaskClient;
use std::time::Duration; use std::time::Duration;
use tokio::net::TcpStream; use tokio::net::TcpStream;
use tokio::time::Instant; use tokio::time::Instant;
@@ -44,7 +44,7 @@ pub(crate) struct HandlerBuilder {
lane_queue_lengths: LaneQueueLengths, lane_queue_lengths: LaneQueueLengths,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
packet_type: Option<PacketType>, packet_type: Option<PacketType>,
shutdown_token: ShutdownToken, task_client: TaskClient,
} }
impl HandlerBuilder { impl HandlerBuilder {
@@ -57,7 +57,7 @@ impl HandlerBuilder {
lane_queue_lengths: LaneQueueLengths, lane_queue_lengths: LaneQueueLengths,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
packet_type: Option<PacketType>, packet_type: Option<PacketType>,
shutdown_token: ShutdownToken, task_client: TaskClient,
) -> Self { ) -> Self {
Self { Self {
msg_input, msg_input,
@@ -67,13 +67,14 @@ impl HandlerBuilder {
lane_queue_lengths, lane_queue_lengths,
reply_controller_sender, reply_controller_sender,
packet_type, packet_type,
shutdown_token, task_client,
} }
} }
// TODO: make sure we only ever have one active handler // TODO: make sure we only ever have one active handler
pub fn create_active_handler(&self) -> Handler { pub fn create_active_handler(&self) -> Handler {
let shutdown_token = self.shutdown_token.clone(); let mut task_client = self.task_client.fork("active_handler");
task_client.disarm();
Handler { Handler {
msg_input: self.msg_input.clone(), msg_input: self.msg_input.clone(),
client_connection_tx: self.client_connection_tx.clone(), client_connection_tx: self.client_connection_tx.clone(),
@@ -84,7 +85,7 @@ impl HandlerBuilder {
lane_queue_lengths: self.lane_queue_lengths.clone(), lane_queue_lengths: self.lane_queue_lengths.clone(),
reply_controller_sender: self.reply_controller_sender.clone(), reply_controller_sender: self.reply_controller_sender.clone(),
packet_type: self.packet_type, packet_type: self.packet_type,
shutdown_token, task_client,
} }
} }
} }
@@ -99,14 +100,19 @@ pub(crate) struct Handler {
lane_queue_lengths: LaneQueueLengths, lane_queue_lengths: LaneQueueLengths,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
packet_type: Option<PacketType>, packet_type: Option<PacketType>,
shutdown_token: ShutdownToken, task_client: TaskClient,
} }
impl Drop for Handler { impl Drop for Handler {
fn drop(&mut self) { fn drop(&mut self) {
let _ = self if let Err(err) = self
.buffer_requester .buffer_requester
.unbounded_send(ReceivedBufferMessage::ReceiverDisconnect); .unbounded_send(ReceivedBufferMessage::ReceiverDisconnect)
{
if !self.task_client.is_shutdown_poll() {
error!("failed to disconnect the receiver from the buffer: {err}");
}
}
} }
} }
@@ -136,7 +142,7 @@ impl Handler {
{ {
Ok(length) => length, Ok(length) => length,
Err(err) => { Err(err) => {
if !self.shutdown_token.is_cancelled() { if !self.task_client.is_shutdown_poll() {
error!( error!(
"Failed to get reply queue length for connection {connection_id}: {err}" "Failed to get reply queue length for connection {connection_id}: {err}"
); );
@@ -186,7 +192,7 @@ impl Handler {
// the ack control is now responsible for chunking, etc. // the ack control is now responsible for chunking, etc.
let input_msg = InputMessage::new_regular(recipient, message, lane, self.packet_type); let input_msg = InputMessage::new_regular(recipient, message, lane, self.packet_type);
if let Err(err) = self.msg_input.send(input_msg).await { if let Err(err) = self.msg_input.send(input_msg).await {
if !self.shutdown_token.is_cancelled() { if !self.task_client.is_shutdown_poll() {
error!("Failed to send message to the input buffer: {err}"); error!("Failed to send message to the input buffer: {err}");
} }
} }
@@ -219,7 +225,7 @@ impl Handler {
let input_msg = let input_msg =
InputMessage::new_anonymous(recipient, message, reply_surbs, lane, self.packet_type); InputMessage::new_anonymous(recipient, message, reply_surbs, lane, self.packet_type);
if let Err(err) = self.msg_input.send(input_msg).await { if let Err(err) = self.msg_input.send(input_msg).await {
if !self.shutdown_token.is_cancelled() { if !self.task_client.is_shutdown_poll() {
error!("Failed to send anonymous message to the input buffer: {err}"); error!("Failed to send anonymous message to the input buffer: {err}");
} }
} }
@@ -247,7 +253,7 @@ impl Handler {
let input_msg = InputMessage::new_reply(recipient_tag, message, lane, self.packet_type); let input_msg = InputMessage::new_reply(recipient_tag, message, lane, self.packet_type);
if let Err(err) = self.msg_input.send(input_msg).await { if let Err(err) = self.msg_input.send(input_msg).await {
if !self.shutdown_token.is_cancelled() { if !self.task_client.is_shutdown_poll() {
error!("Failed to send reply message to the input buffer: {err}"); error!("Failed to send reply message to the input buffer: {err}");
} }
} }
@@ -269,7 +275,7 @@ impl Handler {
.client_connection_tx .client_connection_tx
.unbounded_send(ConnectionCommand::Close(connection_id)) .unbounded_send(ConnectionCommand::Close(connection_id))
{ {
if !self.shutdown_token.is_cancelled() { if !self.task_client.is_shutdown_poll() {
error!("Failed to send close connection command: {err}"); error!("Failed to send close connection command: {err}");
} }
} }
@@ -312,7 +318,7 @@ impl Handler {
async fn handle_text_message(&mut self, msg: String) -> Option<WsMessage> { async fn handle_text_message(&mut self, msg: String) -> Option<WsMessage> {
debug!("Handling text message request"); debug!("Handling text message request");
trace!("Content: {msg:?}"); trace!("Content: {:?}", msg);
self.received_response_type = ReceivedResponseType::Text; self.received_response_type = ReceivedResponseType::Text;
let client_request = ClientRequest::try_from_text(msg); let client_request = ClientRequest::try_from_text(msg);
@@ -388,14 +394,11 @@ impl Handler {
} }
async fn listen_for_requests(&mut self, mut msg_receiver: ReconstructedMessagesReceiver) { async fn listen_for_requests(&mut self, mut msg_receiver: ReconstructedMessagesReceiver) {
let shutdown_token = self.shutdown_token.clone(); let mut task_client = self.task_client.fork("select");
task_client.disarm();
loop { while !task_client.is_shutdown() {
tokio::select! { tokio::select! {
_ = shutdown_token.cancelled() => {
log::trace!("Websocket handler: Received shutdown");
break;
}
// we can either get a client request from the websocket // we can either get a client request from the websocket
socket_msg = self.next_websocket_request() => { socket_msg = self.next_websocket_request() => {
if socket_msg.is_none() { if socket_msg.is_none() {
@@ -433,6 +436,9 @@ impl Handler {
break; break;
} }
} }
_ = task_client.recv() => {
log::trace!("Websocket handler: Received shutdown");
}
} }
} }
log::debug!("Websocket handler: Exiting"); log::debug!("Websocket handler: Exiting");
@@ -458,7 +464,7 @@ impl Handler {
reconstructed_sender, reconstructed_sender,
)) ))
{ {
if !self.shutdown_token.is_cancelled() { if !self.task_client.is_shutdown_poll() {
error!("failed to announce the receiver to the buffer: {err}"); error!("failed to announce the receiver to the buffer: {err}");
} }
} }
+9 -9
View File
@@ -3,7 +3,7 @@
use super::handler::HandlerBuilder; use super::handler::HandlerBuilder;
use log::*; use log::*;
use nym_task::ShutdownToken; use nym_task::TaskClient;
use std::net::IpAddr; use std::net::IpAddr;
use std::{net::SocketAddr, process, sync::Arc}; use std::{net::SocketAddr, process, sync::Arc};
use tokio::io::AsyncWriteExt; use tokio::io::AsyncWriteExt;
@@ -23,15 +23,15 @@ impl State {
pub(crate) struct Listener { pub(crate) struct Listener {
address: SocketAddr, address: SocketAddr,
state: State, state: State,
shutdown_token: ShutdownToken, task_client: TaskClient,
} }
impl Listener { impl Listener {
pub(crate) fn new(host: IpAddr, port: u16, shutdown_token: ShutdownToken) -> Self { pub(crate) fn new(host: IpAddr, port: u16, task_client: TaskClient) -> Self {
Listener { Listener {
address: SocketAddr::new(host, port), address: SocketAddr::new(host, port),
state: State::AwaitingConnection, state: State::AwaitingConnection,
shutdown_token, task_client,
} }
} }
@@ -46,11 +46,11 @@ impl Listener {
let notify = Arc::new(Notify::new()); let notify = Arc::new(Notify::new());
while !self.shutdown_token.is_cancelled() { while !self.task_client.is_shutdown() {
tokio::select! { tokio::select! {
// When the handler finishes we check if shutdown is signalled // When the handler finishes we check if shutdown is signalled
_ = notify.notified() => { _ = notify.notified() => {
if self.shutdown_token.is_cancelled() { if self.task_client.is_shutdown() {
log::trace!("Websocket listener: detected shutdown after connection closed"); log::trace!("Websocket listener: detected shutdown after connection closed");
break; break;
} }
@@ -59,7 +59,7 @@ impl Listener {
} }
// ... but when there is no connected client at the time of shutdown being // ... but when there is no connected client at the time of shutdown being
// signalled, we handle it here. // signalled, we handle it here.
_ = self.shutdown_token.cancelled() => { _ = self.task_client.recv() => {
if !self.state.is_connected() { if !self.state.is_connected() {
log::trace!("Not connected: shutting down"); log::trace!("Not connected: shutting down");
break; break;
@@ -68,9 +68,9 @@ impl Listener {
new_conn = tcp_listener.accept() => { new_conn = tcp_listener.accept() => {
match new_conn { match new_conn {
Ok((mut socket, remote_addr)) => { Ok((mut socket, remote_addr)) => {
debug!("Received connection from {remote_addr:?}"); debug!("Received connection from {:?}", remote_addr);
if self.state.is_connected() { if self.state.is_connected() {
warn!("Tried to open a duplicate websocket connection. The request came from {remote_addr}"); warn!("Tried to open a duplicate websocket connection. The request came from {}", remote_addr);
// if we've already got a connection, don't allow another one // if we've already got a connection, don't allow another one
// while we only ever want to accept a single connection, we don't want // while we only ever want to accept a single connection, we don't want
// to leave clients hanging (and also allow for reconnection if it somehow // to leave clients hanging (and also allow for reconnection if it somehow
+2 -2
View File
@@ -1,10 +1,10 @@
[package] [package]
name = "nym-socks5-client" name = "nym-socks5-client"
version = "1.1.64" version = "1.1.55"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"] authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
description = "A SOCKS5 localhost proxy that converts incoming messages to Sphinx and sends them to a Nym address" description = "A SOCKS5 localhost proxy that converts incoming messages to Sphinx and sends them to a Nym address"
edition = "2021" edition = "2021"
rust-version = "1.85" rust-version = "1.70"
license.workspace = true license.workspace = true
[dependencies] [dependencies]
+6 -6
View File
@@ -1,8 +1,8 @@
// Copyright 2023 - Nym Technologies SA <contact@nymtech.net> // Copyright 2023 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use futures::StreamExt;
use futures::channel::mpsc; use futures::channel::mpsc;
use futures::StreamExt;
use notify::event::{DataChange, MetadataKind, ModifyKind}; use notify::event::{DataChange, MetadataKind, ModifyKind};
use notify::{Config, Event, EventKind, RecommendedWatcher, RecursiveMode, Watcher}; use notify::{Config, Event, EventKind, RecommendedWatcher, RecursiveMode, Watcher};
use std::collections::HashMap; use std::collections::HashMap;
@@ -96,10 +96,10 @@ impl AsyncFileWatcher {
// when testing I was consistently getting two `Modify(Data(Any))` events in quick succession // when testing I was consistently getting two `Modify(Data(Any))` events in quick succession
// (probably to modify content and metadata). // (probably to modify content and metadata).
// we really only want to propagate one of them // we really only want to propagate one of them
if let Some(previous) = self.last_received.get(&event.kind) if let Some(previous) = self.last_received.get(&event.kind) {
&& now.duration_since(*previous) < self.tick_duration if now.duration_since(*previous) < self.tick_duration {
{ return false;
return false; }
} }
let Some(filters) = &self.filters else { let Some(filters) = &self.filters else {
@@ -137,7 +137,7 @@ impl AsyncFileWatcher {
log::error!("the file watcher receiver has been dropped!"); log::error!("the file watcher receiver has been dropped!");
} }
} else { } else {
log::debug!("will not propagate information about {event:?}"); log::debug!("will not propagate information about {:?}", event);
} }
} }
Err(err) => { Err(err) => {
-2
View File
@@ -13,8 +13,6 @@ base64 = { workspace = true }
bincode = { workspace = true } bincode = { workspace = true }
rand = { workspace = true } rand = { workspace = true }
serde = { workspace = true, features = ["derive"] } serde = { workspace = true, features = ["derive"] }
semver = { workspace = true }
strum_macros = { workspace = true }
thiserror = { workspace = true } thiserror = { workspace = true }
nym-credentials-interface = { path = "../credentials-interface" } nym-credentials-interface = { path = "../credentials-interface" }
@@ -1,273 +0,0 @@
// Copyright 2025 Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use nym_sphinx::addressing::Recipient;
use nym_wireguard_types::PeerPublicKey;
use crate::{
AuthenticatorVersion, Error,
latest::registration::IpPair,
traits::{FinalMessage, InitMessage, QueryBandwidthMessage, TopUpMessage, Versionable},
v2, v3, v4, v5,
};
// This is very redundant with AuthenticatorRequest and I reckon they could be smooshed.
// It is a bit out of scope for me at the moment though
#[derive(Debug)]
pub enum ClientMessage {
Initial(Box<dyn InitMessage + Send + Sync + 'static>),
Final(Box<dyn FinalMessage + Send + Sync + 'static>),
Query(Box<dyn QueryBandwidthMessage + Send + Sync + 'static>),
TopUp(Box<dyn TopUpMessage + Send + Sync + 'static>),
}
impl ClientMessage {
// check if message is wasteful e.g. contains a credential
pub fn is_wasteful(&self) -> bool {
match self {
Self::Final(msg) => msg.credential().is_some(),
Self::TopUp(_) => true,
Self::Initial(_) | Self::Query(_) => false,
}
}
fn version(&self) -> AuthenticatorVersion {
match self {
ClientMessage::Initial(msg) => msg.version(),
ClientMessage::Final(msg) => msg.version(),
ClientMessage::Query(msg) => msg.version(),
ClientMessage::TopUp(msg) => msg.version(),
}
}
pub fn bytes(&self, reply_to: Recipient) -> Result<(Vec<u8>, u64), Error> {
match self.version() {
AuthenticatorVersion::V1 => Err(Error::UnsupportedVersion),
AuthenticatorVersion::V2 => {
use v2::{
registration::{ClientMac, FinalMessage, GatewayClient, InitMessage},
request::AuthenticatorRequest,
};
match self {
ClientMessage::Initial(init_message) => {
let (req, id) = AuthenticatorRequest::new_initial_request(
InitMessage {
pub_key: init_message.pub_key(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Final(final_message) => {
let (req, id) = AuthenticatorRequest::new_final_request(
FinalMessage {
gateway_client: GatewayClient {
pub_key: final_message.gateway_client_pub_key(),
private_ip: final_message
.gateway_client_ipv4()
.ok_or(Error::UnsupportedMessage)?
.into(),
mac: ClientMac::new(final_message.gateway_client_mac()),
},
credential: final_message.credential(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Query(query_message) => {
let (req, id) = AuthenticatorRequest::new_query_request(
query_message.pub_key(),
reply_to,
);
Ok((req.to_bytes()?, id))
}
_ => Err(Error::UnsupportedMessage),
}
}
AuthenticatorVersion::V3 => {
use v3::{
registration::{ClientMac, FinalMessage, GatewayClient, InitMessage},
request::AuthenticatorRequest,
topup::TopUpMessage,
};
match self {
ClientMessage::Initial(init_message) => {
let (req, id) = AuthenticatorRequest::new_initial_request(
InitMessage {
pub_key: init_message.pub_key(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Final(final_message) => {
let (req, id) = AuthenticatorRequest::new_final_request(
FinalMessage {
gateway_client: GatewayClient {
pub_key: final_message.gateway_client_pub_key(),
private_ip: final_message
.gateway_client_ipv4()
.ok_or(Error::UnsupportedMessage)?
.into(),
mac: ClientMac::new(final_message.gateway_client_mac()),
},
credential: final_message.credential(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Query(query_message) => {
let (req, id) = AuthenticatorRequest::new_query_request(
query_message.pub_key(),
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::TopUp(top_up_message) => {
let (req, id) = AuthenticatorRequest::new_topup_request(
TopUpMessage {
pub_key: top_up_message.pub_key(),
credential: top_up_message.credential(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
}
}
AuthenticatorVersion::V4 => {
use v4::{
registration::{ClientMac, FinalMessage, GatewayClient, InitMessage},
request::AuthenticatorRequest,
topup::TopUpMessage,
};
match self {
ClientMessage::Initial(init_message) => {
let (req, id) = AuthenticatorRequest::new_initial_request(
InitMessage {
pub_key: init_message.pub_key(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Final(final_message) => {
let (req, id) = AuthenticatorRequest::new_final_request(
FinalMessage {
gateway_client: GatewayClient {
pub_key: final_message.gateway_client_pub_key(),
private_ips: IpPair {
ipv4: final_message
.gateway_client_ipv4()
.ok_or(Error::UnsupportedMessage)?,
ipv6: final_message
.gateway_client_ipv6()
.ok_or(Error::UnsupportedMessage)?,
}
.into(),
mac: ClientMac::new(final_message.gateway_client_mac()),
},
credential: final_message.credential(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Query(query_message) => {
let (req, id) = AuthenticatorRequest::new_query_request(
query_message.pub_key(),
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::TopUp(top_up_message) => {
let (req, id) = AuthenticatorRequest::new_topup_request(
TopUpMessage {
pub_key: top_up_message.pub_key(),
credential: top_up_message.credential(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
}
}
AuthenticatorVersion::V5 => {
use v5::{
registration::{ClientMac, FinalMessage, GatewayClient, InitMessage},
request::AuthenticatorRequest,
topup::TopUpMessage,
};
match self {
ClientMessage::Initial(init_message) => {
let (req, id) = AuthenticatorRequest::new_initial_request(InitMessage {
pub_key: init_message.pub_key(),
});
Ok((req.to_bytes()?, id))
}
ClientMessage::Final(final_message) => {
let (req, id) = AuthenticatorRequest::new_final_request(FinalMessage {
gateway_client: GatewayClient {
pub_key: final_message.gateway_client_pub_key(),
private_ips: IpPair {
ipv4: final_message
.gateway_client_ipv4()
.ok_or(Error::UnsupportedMessage)?,
ipv6: final_message
.gateway_client_ipv6()
.ok_or(Error::UnsupportedMessage)?,
},
mac: ClientMac::new(final_message.gateway_client_mac()),
},
credential: final_message.credential(),
});
Ok((req.to_bytes()?, id))
}
ClientMessage::Query(query_message) => {
let (req, id) =
AuthenticatorRequest::new_query_request(query_message.pub_key());
Ok((req.to_bytes()?, id))
}
ClientMessage::TopUp(top_up_message) => {
let (req, id) = AuthenticatorRequest::new_topup_request(TopUpMessage {
pub_key: top_up_message.pub_key(),
credential: top_up_message.credential(),
});
Ok((req.to_bytes()?, id))
}
}
}
AuthenticatorVersion::UNKNOWN => Err(Error::UnknownVersion),
}
}
pub fn use_surbs(&self) -> bool {
use AuthenticatorVersion::*;
match self.version() {
V1 | V2 | V3 | V4 => false,
V5 => true,
UNKNOWN => true,
}
}
}
// Same comment as above struct
#[derive(Debug)]
pub struct QueryMessageImpl {
pub pub_key: PeerPublicKey,
pub version: AuthenticatorVersion,
}
impl Versionable for QueryMessageImpl {
fn version(&self) -> AuthenticatorVersion {
self.version
}
}
impl QueryBandwidthMessage for QueryMessageImpl {
fn pub_key(&self) -> PeerPublicKey {
self.pub_key
}
}
+2 -13
View File
@@ -23,17 +23,6 @@ pub enum Error {
#[error("conversion: {0}")] #[error("conversion: {0}")]
Conversion(String), Conversion(String),
// TODO add version number for debugging #[error("failed to serialize response packet: {source}")]
#[error("unknown version number")] FailedToSerializeResponsePacket { source: Box<bincode::ErrorKind> },
UnknownVersion,
// TODO add version number for debugging
#[error("unsupported request version")]
UnsupportedVersion,
#[error("gateway doesn't support this type of message")]
UnsupportedMessage,
#[error(transparent)]
Bincode(#[from] bincode::Error),
} }
+1 -6
View File
@@ -1,9 +1,6 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net> // Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
pub mod client_message;
pub mod request;
pub mod response;
pub mod traits; pub mod traits;
pub mod v1; pub mod v1;
pub mod v2; pub mod v2;
@@ -13,13 +10,11 @@ pub mod v5;
mod error; mod error;
mod util; mod util;
mod version;
pub use error::Error; pub use error::Error;
pub use v5 as latest; pub use v5 as latest;
pub use version::AuthenticatorVersion;
pub const CURRENT_VERSION: u8 = latest::VERSION; pub const CURRENT_VERSION: u8 = 5;
fn make_bincode_serializer() -> impl bincode::Options { fn make_bincode_serializer() -> impl bincode::Options {
use bincode::Options; use bincode::Options;
@@ -1,204 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use nym_service_provider_requests_common::{Protocol, ServiceProviderType};
use nym_sphinx::addressing::Recipient;
use crate::traits::{FinalMessage, InitMessage, QueryBandwidthMessage, TopUpMessage};
use crate::{v1, v2, v3, v4, v5};
#[derive(Debug)]
pub enum AuthenticatorRequest {
Initial {
msg: Box<dyn InitMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
Final {
msg: Box<dyn FinalMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
QueryBandwidth {
msg: Box<dyn QueryBandwidthMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
TopUpBandwidth {
msg: Box<dyn TopUpMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
}
impl From<v1::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v1::request::AuthenticatorRequest) -> Self {
match value.data {
v1::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v1::request::AuthenticatorRequestData::Final(gateway_client) => Self::Final {
msg: Box::new(gateway_client),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v1::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl From<v2::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v2::request::AuthenticatorRequest) -> Self {
match value.data {
v2::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v2::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v2::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl From<v3::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v3::request::AuthenticatorRequest) -> Self {
match value.data {
v3::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v3::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v3::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
v3::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl From<v4::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v4::request::AuthenticatorRequest) -> Self {
match value.data {
v4::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v4::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v4::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
v4::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl From<v5::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v5::request::AuthenticatorRequest) -> Self {
match value.data {
v5::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
},
v5::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
},
v5::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
}
}
v5::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
}
}
}
}
}
@@ -1,106 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::traits::{
Id, PendingRegistrationResponse, RegisteredResponse, RemainingBandwidthResponse,
TopUpBandwidthResponse,
};
use crate::{v2, v3, v4, v5};
#[derive(Debug)]
pub enum AuthenticatorResponse {
PendingRegistration(Box<dyn PendingRegistrationResponse + Send + Sync + 'static>),
Registered(Box<dyn RegisteredResponse + Send + Sync + 'static>),
RemainingBandwidth(Box<dyn RemainingBandwidthResponse + Send + Sync + 'static>),
TopUpBandwidth(Box<dyn TopUpBandwidthResponse + Send + Sync + 'static>),
}
impl Id for AuthenticatorResponse {
fn id(&self) -> u64 {
match self {
AuthenticatorResponse::PendingRegistration(pending_registration_response) => {
pending_registration_response.id()
}
AuthenticatorResponse::Registered(registered_response) => registered_response.id(),
AuthenticatorResponse::RemainingBandwidth(remaining_bandwidth_response) => {
remaining_bandwidth_response.id()
}
AuthenticatorResponse::TopUpBandwidth(top_up_bandwidth_response) => {
top_up_bandwidth_response.id()
}
}
}
}
impl From<v2::response::AuthenticatorResponse> for AuthenticatorResponse {
fn from(value: v2::response::AuthenticatorResponse) -> Self {
match value.data {
v2::response::AuthenticatorResponseData::PendingRegistration(
pending_registration_response,
) => Self::PendingRegistration(Box::new(pending_registration_response)),
v2::response::AuthenticatorResponseData::Registered(registered_response) => {
Self::Registered(Box::new(registered_response))
}
v2::response::AuthenticatorResponseData::RemainingBandwidth(
remaining_bandwidth_response,
) => Self::RemainingBandwidth(Box::new(remaining_bandwidth_response)),
}
}
}
impl From<v3::response::AuthenticatorResponse> for AuthenticatorResponse {
fn from(value: v3::response::AuthenticatorResponse) -> Self {
match value.data {
v3::response::AuthenticatorResponseData::PendingRegistration(
pending_registration_response,
) => Self::PendingRegistration(Box::new(pending_registration_response)),
v3::response::AuthenticatorResponseData::Registered(registered_response) => {
Self::Registered(Box::new(registered_response))
}
v3::response::AuthenticatorResponseData::RemainingBandwidth(
remaining_bandwidth_response,
) => Self::RemainingBandwidth(Box::new(remaining_bandwidth_response)),
v3::response::AuthenticatorResponseData::TopUpBandwidth(top_up_bandwidth_response) => {
Self::TopUpBandwidth(Box::new(top_up_bandwidth_response))
}
}
}
}
impl From<v4::response::AuthenticatorResponse> for AuthenticatorResponse {
fn from(value: v4::response::AuthenticatorResponse) -> Self {
match value.data {
v4::response::AuthenticatorResponseData::PendingRegistration(
pending_registration_response,
) => Self::PendingRegistration(Box::new(pending_registration_response)),
v4::response::AuthenticatorResponseData::Registered(registered_response) => {
Self::Registered(Box::new(registered_response))
}
v4::response::AuthenticatorResponseData::RemainingBandwidth(
remaining_bandwidth_response,
) => Self::RemainingBandwidth(Box::new(remaining_bandwidth_response)),
v4::response::AuthenticatorResponseData::TopUpBandwidth(top_up_bandwidth_response) => {
Self::TopUpBandwidth(Box::new(top_up_bandwidth_response))
}
}
}
}
impl From<v5::response::AuthenticatorResponse> for AuthenticatorResponse {
fn from(value: v5::response::AuthenticatorResponse) -> Self {
match value.data {
v5::response::AuthenticatorResponseData::PendingRegistration(
pending_registration_response,
) => Self::PendingRegistration(Box::new(pending_registration_response)),
v5::response::AuthenticatorResponseData::Registered(registered_response) => {
Self::Registered(Box::new(registered_response))
}
v5::response::AuthenticatorResponseData::RemainingBandwidth(
remaining_bandwidth_response,
) => Self::RemainingBandwidth(Box::new(remaining_bandwidth_response)),
v5::response::AuthenticatorResponseData::TopUpBandwidth(top_up_bandwidth_response) => {
Self::TopUpBandwidth(Box::new(top_up_bandwidth_response))
}
}
}
}
+220 -437
View File
@@ -1,105 +1,49 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net> // Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use std::fmt;
use std::net::{Ipv4Addr, Ipv6Addr};
use nym_credentials_interface::CredentialSpendingData; use nym_credentials_interface::CredentialSpendingData;
use nym_crypto::asymmetric::x25519::PrivateKey; use nym_crypto::asymmetric::x25519::PrivateKey;
use nym_service_provider_requests_common::{Protocol, ServiceProviderType};
use nym_sphinx::addressing::clients::Recipient;
use nym_wireguard_types::PeerPublicKey; use nym_wireguard_types::PeerPublicKey;
use crate::latest::registration::IpPair; use crate::{
use crate::{AuthenticatorVersion, Error, v1, v2, v3, v4, v5}; v1, v2, v3, v4,
v5::{self, registration::IpPair},
Error,
};
pub trait Versionable { #[derive(Copy, Clone, Debug)]
fn version(&self) -> AuthenticatorVersion; pub enum AuthenticatorVersion {
V1,
V2,
V3,
V4,
V5,
UNKNOWN,
} }
impl Versionable for v1::GatewayClient { impl From<Protocol> for AuthenticatorVersion {
fn version(&self) -> AuthenticatorVersion { fn from(value: Protocol) -> Self {
AuthenticatorVersion::V1 if value.service_provider_type != ServiceProviderType::Authenticator {
AuthenticatorVersion::UNKNOWN
} else if value.version == v1::VERSION {
AuthenticatorVersion::V1
} else if value.version == v2::VERSION {
AuthenticatorVersion::V2
} else if value.version == v3::VERSION {
AuthenticatorVersion::V3
} else if value.version == v4::VERSION {
AuthenticatorVersion::V4
} else if value.version == v5::VERSION {
AuthenticatorVersion::V5
} else {
AuthenticatorVersion::UNKNOWN
}
} }
} }
impl Versionable for v1::registration::InitMessage { pub trait InitMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V1
}
}
impl Versionable for v2::registration::InitMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V2
}
}
impl Versionable for v3::registration::InitMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V3
}
}
impl Versionable for v4::registration::InitMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V4
}
}
impl Versionable for v5::registration::InitMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V5
}
}
impl Versionable for v2::registration::FinalMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V2
}
}
impl Versionable for v3::registration::FinalMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V3
}
}
impl Versionable for v4::registration::FinalMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V4
}
}
impl Versionable for v5::registration::FinalMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V5
}
}
impl Versionable for PeerPublicKey {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V3
}
}
impl Versionable for v3::topup::TopUpMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V3
}
}
impl Versionable for v4::topup::TopUpMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V4
}
}
impl Versionable for v5::topup::TopUpMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V5
}
}
pub trait InitMessage: Versionable + fmt::Debug {
fn pub_key(&self) -> PeerPublicKey; fn pub_key(&self) -> PeerPublicKey;
} }
@@ -133,18 +77,15 @@ impl InitMessage for v5::registration::InitMessage {
} }
} }
pub trait FinalMessage: Versionable + fmt::Debug { pub trait FinalMessage {
fn gateway_client_pub_key(&self) -> PeerPublicKey; fn pub_key(&self) -> PeerPublicKey;
fn verify(&self, private_key: &PrivateKey, nonce: u64) -> Result<(), Error>; fn verify(&self, private_key: &PrivateKey, nonce: u64) -> Result<(), Error>;
fn private_ips(&self) -> IpPair; fn private_ips(&self) -> IpPair;
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr>;
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr>;
fn gateway_client_mac(&self) -> Vec<u8>;
fn credential(&self) -> Option<CredentialSpendingData>; fn credential(&self) -> Option<CredentialSpendingData>;
} }
impl FinalMessage for v1::GatewayClient { impl FinalMessage for v1::GatewayClient {
fn gateway_client_pub_key(&self) -> PeerPublicKey { fn pub_key(&self) -> PeerPublicKey {
self.pub_key self.pub_key
} }
@@ -156,28 +97,13 @@ impl FinalMessage for v1::GatewayClient {
self.private_ip.into() self.private_ip.into()
} }
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr> {
match self.private_ip {
std::net::IpAddr::V4(ipv4_addr) => Some(ipv4_addr),
std::net::IpAddr::V6(_) => None,
}
}
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr> {
None
}
fn gateway_client_mac(&self) -> Vec<u8> {
self.mac.to_vec()
}
fn credential(&self) -> Option<CredentialSpendingData> { fn credential(&self) -> Option<CredentialSpendingData> {
None None
} }
} }
impl FinalMessage for v2::registration::FinalMessage { impl FinalMessage for v2::registration::FinalMessage {
fn gateway_client_pub_key(&self) -> PeerPublicKey { fn pub_key(&self) -> PeerPublicKey {
self.gateway_client.pub_key self.gateway_client.pub_key
} }
@@ -189,28 +115,13 @@ impl FinalMessage for v2::registration::FinalMessage {
self.gateway_client.private_ip.into() self.gateway_client.private_ip.into()
} }
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr> {
match self.gateway_client.private_ip {
std::net::IpAddr::V4(ipv4_addr) => Some(ipv4_addr),
std::net::IpAddr::V6(_) => None,
}
}
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr> {
None
}
fn gateway_client_mac(&self) -> Vec<u8> {
self.gateway_client.mac.to_vec()
}
fn credential(&self) -> Option<CredentialSpendingData> { fn credential(&self) -> Option<CredentialSpendingData> {
self.credential.clone() self.credential.clone()
} }
} }
impl FinalMessage for v3::registration::FinalMessage { impl FinalMessage for v3::registration::FinalMessage {
fn gateway_client_pub_key(&self) -> PeerPublicKey { fn pub_key(&self) -> PeerPublicKey {
self.gateway_client.pub_key self.gateway_client.pub_key
} }
@@ -222,28 +133,13 @@ impl FinalMessage for v3::registration::FinalMessage {
self.gateway_client.private_ip.into() self.gateway_client.private_ip.into()
} }
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr> {
match self.gateway_client.private_ip {
std::net::IpAddr::V4(ipv4_addr) => Some(ipv4_addr),
std::net::IpAddr::V6(_) => None,
}
}
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr> {
None
}
fn gateway_client_mac(&self) -> Vec<u8> {
self.gateway_client.mac.to_vec()
}
fn credential(&self) -> Option<CredentialSpendingData> { fn credential(&self) -> Option<CredentialSpendingData> {
self.credential.clone() self.credential.clone()
} }
} }
impl FinalMessage for v4::registration::FinalMessage { impl FinalMessage for v4::registration::FinalMessage {
fn gateway_client_pub_key(&self) -> PeerPublicKey { fn pub_key(&self) -> PeerPublicKey {
self.gateway_client.pub_key self.gateway_client.pub_key
} }
@@ -255,25 +151,13 @@ impl FinalMessage for v4::registration::FinalMessage {
self.gateway_client.private_ips.into() self.gateway_client.private_ips.into()
} }
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr> {
Some(self.gateway_client.private_ips.ipv4)
}
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr> {
Some(self.gateway_client.private_ips.ipv6)
}
fn gateway_client_mac(&self) -> Vec<u8> {
self.gateway_client.mac.to_vec()
}
fn credential(&self) -> Option<CredentialSpendingData> { fn credential(&self) -> Option<CredentialSpendingData> {
self.credential.clone() self.credential.clone()
} }
} }
impl FinalMessage for v5::registration::FinalMessage { impl FinalMessage for v5::registration::FinalMessage {
fn gateway_client_pub_key(&self) -> PeerPublicKey { fn pub_key(&self) -> PeerPublicKey {
self.gateway_client.pub_key self.gateway_client.pub_key
} }
@@ -285,24 +169,12 @@ impl FinalMessage for v5::registration::FinalMessage {
self.gateway_client.private_ips self.gateway_client.private_ips
} }
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr> {
Some(self.gateway_client.private_ips.ipv4)
}
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr> {
Some(self.gateway_client.private_ips.ipv6)
}
fn gateway_client_mac(&self) -> Vec<u8> {
self.gateway_client.mac.to_vec()
}
fn credential(&self) -> Option<CredentialSpendingData> { fn credential(&self) -> Option<CredentialSpendingData> {
self.credential.clone() self.credential.clone()
} }
} }
pub trait QueryBandwidthMessage: Versionable + fmt::Debug { pub trait QueryBandwidthMessage {
fn pub_key(&self) -> PeerPublicKey; fn pub_key(&self) -> PeerPublicKey;
} }
@@ -312,7 +184,7 @@ impl QueryBandwidthMessage for PeerPublicKey {
} }
} }
pub trait TopUpMessage: Versionable + fmt::Debug { pub trait TopUpMessage {
fn pub_key(&self) -> PeerPublicKey; fn pub_key(&self) -> PeerPublicKey;
fn credential(&self) -> CredentialSpendingData; fn credential(&self) -> CredentialSpendingData;
} }
@@ -347,286 +219,197 @@ impl TopUpMessage for v5::topup::TopUpMessage {
} }
} }
pub trait Id { pub enum AuthenticatorRequest {
fn id(&self) -> u64; Initial {
msg: Box<dyn InitMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
Final {
msg: Box<dyn FinalMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
QueryBandwidth {
msg: Box<dyn QueryBandwidthMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
TopUpBandwidth {
msg: Box<dyn TopUpMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
} }
impl Id for v2::response::PendingRegistrationResponse { impl From<v1::request::AuthenticatorRequest> for AuthenticatorRequest {
fn id(&self) -> u64 { fn from(value: v1::request::AuthenticatorRequest) -> Self {
self.request_id match value.data {
v1::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v1::request::AuthenticatorRequestData::Final(gateway_client) => Self::Final {
msg: Box::new(gateway_client),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v1::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
} }
} }
impl Id for v3::response::PendingRegistrationResponse { impl From<v2::request::AuthenticatorRequest> for AuthenticatorRequest {
fn id(&self) -> u64 { fn from(value: v2::request::AuthenticatorRequest) -> Self {
self.request_id match value.data {
v2::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v2::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v2::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
} }
} }
impl Id for v4::response::PendingRegistrationResponse { impl From<v3::request::AuthenticatorRequest> for AuthenticatorRequest {
fn id(&self) -> u64 { fn from(value: v3::request::AuthenticatorRequest) -> Self {
self.request_id match value.data {
v3::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v3::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v3::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
v3::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
} }
} }
impl Id for v5::response::PendingRegistrationResponse { impl From<v4::request::AuthenticatorRequest> for AuthenticatorRequest {
fn id(&self) -> u64 { fn from(value: v4::request::AuthenticatorRequest) -> Self {
self.request_id match value.data {
v4::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v4::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v4::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
v4::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
} }
} }
impl Id for v2::response::RegisteredResponse { impl From<v5::request::AuthenticatorRequest> for AuthenticatorRequest {
fn id(&self) -> u64 { fn from(value: v5::request::AuthenticatorRequest) -> Self {
self.request_id match value.data {
} v5::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
} msg: Box::new(init_message),
protocol: value.protocol,
impl Id for v3::response::RegisteredResponse { reply_to: None,
fn id(&self) -> u64 { request_id: value.request_id,
self.request_id },
} v5::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
} msg: final_message,
protocol: value.protocol,
impl Id for v4::response::RegisteredResponse { reply_to: None,
fn id(&self) -> u64 { request_id: value.request_id,
self.request_id },
} v5::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
} Self::QueryBandwidth {
msg: Box::new(peer_public_key),
impl Id for v5::response::RegisteredResponse { protocol: value.protocol,
fn id(&self) -> u64 { reply_to: None,
self.request_id request_id: value.request_id,
} }
} }
v5::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
impl Id for v2::response::RemainingBandwidthResponse { Self::TopUpBandwidth {
fn id(&self) -> u64 { msg: top_up_message,
self.request_id protocol: value.protocol,
} reply_to: None,
} request_id: value.request_id,
}
impl Id for v3::response::RemainingBandwidthResponse { }
fn id(&self) -> u64 { }
self.request_id
}
}
impl Id for v4::response::RemainingBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v5::response::RemainingBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v3::response::TopUpBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v4::response::TopUpBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v5::response::TopUpBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
pub trait PendingRegistrationResponse: Id + fmt::Debug {
fn nonce(&self) -> u64;
fn verify(&self, gateway_key: &PrivateKey) -> std::result::Result<(), Error>;
fn pub_key(&self) -> PeerPublicKey;
fn private_ips(&self) -> IpPair;
}
impl PendingRegistrationResponse for v2::response::PendingRegistrationResponse {
fn nonce(&self) -> u64 {
self.reply.nonce
}
fn verify(&self, gateway_key: &PrivateKey) -> std::result::Result<(), Error> {
self.reply.gateway_data.verify(gateway_key, self.nonce())
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.gateway_data.pub_key
}
fn private_ips(&self) -> IpPair {
self.reply.gateway_data.private_ip.into()
}
}
impl PendingRegistrationResponse for v3::response::PendingRegistrationResponse {
fn nonce(&self) -> u64 {
self.reply.nonce
}
fn verify(&self, gateway_key: &PrivateKey) -> std::result::Result<(), Error> {
self.reply.gateway_data.verify(gateway_key, self.nonce())
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.gateway_data.pub_key
}
fn private_ips(&self) -> IpPair {
self.reply.gateway_data.private_ip.into()
}
}
impl PendingRegistrationResponse for v4::response::PendingRegistrationResponse {
fn nonce(&self) -> u64 {
self.reply.nonce
}
fn verify(&self, gateway_key: &PrivateKey) -> std::result::Result<(), Error> {
self.reply.gateway_data.verify(gateway_key, self.nonce())
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.gateway_data.pub_key
}
fn private_ips(&self) -> IpPair {
self.reply.gateway_data.private_ips.into()
}
}
impl PendingRegistrationResponse for v5::response::PendingRegistrationResponse {
fn nonce(&self) -> u64 {
self.reply.nonce
}
fn verify(&self, gateway_key: &PrivateKey) -> std::result::Result<(), Error> {
self.reply.gateway_data.verify(gateway_key, self.nonce())
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.gateway_data.pub_key
}
fn private_ips(&self) -> IpPair {
self.reply.gateway_data.private_ips
}
}
pub trait RegisteredResponse: Id + fmt::Debug {
fn private_ips(&self) -> IpPair;
fn pub_key(&self) -> PeerPublicKey;
fn wg_port(&self) -> u16;
}
impl RegisteredResponse for v2::response::RegisteredResponse {
fn private_ips(&self) -> IpPair {
self.reply.private_ip.into()
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.pub_key
}
fn wg_port(&self) -> u16 {
self.reply.wg_port
}
}
impl RegisteredResponse for v3::response::RegisteredResponse {
fn private_ips(&self) -> IpPair {
self.reply.private_ip.into()
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.pub_key
}
fn wg_port(&self) -> u16 {
self.reply.wg_port
}
}
impl RegisteredResponse for v4::response::RegisteredResponse {
fn private_ips(&self) -> IpPair {
self.reply.private_ips.into()
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.pub_key
}
fn wg_port(&self) -> u16 {
self.reply.wg_port
}
}
impl RegisteredResponse for v5::response::RegisteredResponse {
fn private_ips(&self) -> IpPair {
self.reply.private_ips
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.pub_key
}
fn wg_port(&self) -> u16 {
self.reply.wg_port
}
}
pub trait RemainingBandwidthResponse: Id + fmt::Debug {
fn available_bandwidth(&self) -> Option<i64>;
}
impl RemainingBandwidthResponse for v2::response::RemainingBandwidthResponse {
fn available_bandwidth(&self) -> Option<i64> {
self.reply.as_ref().map(|r| r.available_bandwidth)
}
}
impl RemainingBandwidthResponse for v3::response::RemainingBandwidthResponse {
fn available_bandwidth(&self) -> Option<i64> {
self.reply.as_ref().map(|r| r.available_bandwidth)
}
}
impl RemainingBandwidthResponse for v4::response::RemainingBandwidthResponse {
fn available_bandwidth(&self) -> Option<i64> {
self.reply.as_ref().map(|r| r.available_bandwidth)
}
}
impl RemainingBandwidthResponse for v5::response::RemainingBandwidthResponse {
fn available_bandwidth(&self) -> Option<i64> {
self.reply.as_ref().map(|r| r.available_bandwidth)
}
}
pub trait TopUpBandwidthResponse: Id + fmt::Debug {
fn available_bandwidth(&self) -> i64;
}
impl TopUpBandwidthResponse for v3::response::TopUpBandwidthResponse {
fn available_bandwidth(&self) -> i64 {
self.reply.available_bandwidth
}
}
impl TopUpBandwidthResponse for v4::response::TopUpBandwidthResponse {
fn available_bandwidth(&self) -> i64 {
self.reply.available_bandwidth
}
}
impl TopUpBandwidthResponse for v5::response::TopUpBandwidthResponse {
fn available_bandwidth(&self) -> i64 {
self.reply.available_bandwidth
} }
} }
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::error::Error; use crate::error::Error;
use base64::{Engine, engine::general_purpose}; use base64::{engine::general_purpose, Engine};
use nym_wireguard_types::PeerPublicKey; use nym_wireguard_types::PeerPublicKey;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::collections::HashMap; use std::collections::HashMap;
@@ -108,7 +108,7 @@ impl GatewayClient {
#[cfg(feature = "verify")] #[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> { pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key // use gateways key as a ref to an x25519_dalek key
let dh = gateway_key.inner().diffie_hellman(&self.pub_key); let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead // TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)] #[allow(clippy::expect_used)]
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::error::Error; use crate::error::Error;
use base64::{Engine, engine::general_purpose}; use base64::{engine::general_purpose, Engine};
use nym_credentials_interface::CredentialSpendingData; use nym_credentials_interface::CredentialSpendingData;
use nym_wireguard_types::PeerPublicKey; use nym_wireguard_types::PeerPublicKey;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@@ -117,7 +117,7 @@ impl GatewayClient {
#[cfg(feature = "verify")] #[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> { pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key // use gateways key as a ref to an x25519_dalek key
let dh = gateway_key.inner().diffie_hellman(&self.pub_key); let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead // TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)] #[allow(clippy::expect_used)]
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::error::Error; use crate::error::Error;
use base64::{Engine, engine::general_purpose}; use base64::{engine::general_purpose, Engine};
use nym_credentials_interface::CredentialSpendingData; use nym_credentials_interface::CredentialSpendingData;
use nym_wireguard_types::PeerPublicKey; use nym_wireguard_types::PeerPublicKey;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@@ -117,7 +117,7 @@ impl GatewayClient {
#[cfg(feature = "verify")] #[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> { pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key // use gateways key as a ref to an x25519_dalek key
let dh = gateway_key.inner().diffie_hellman(&self.pub_key); let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead // TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)] #[allow(clippy::expect_used)]
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::error::Error; use crate::error::Error;
use base64::{Engine, engine::general_purpose}; use base64::{engine::general_purpose, Engine};
use nym_credentials_interface::CredentialSpendingData; use nym_credentials_interface::CredentialSpendingData;
use nym_network_defaults::constants::{WG_TUN_DEVICE_IP_ADDRESS_V4, WG_TUN_DEVICE_IP_ADDRESS_V6}; use nym_network_defaults::constants::{WG_TUN_DEVICE_IP_ADDRESS_V4, WG_TUN_DEVICE_IP_ADDRESS_V6};
use nym_wireguard_types::PeerPublicKey; use nym_wireguard_types::PeerPublicKey;
@@ -169,7 +169,7 @@ impl GatewayClient {
#[cfg(feature = "verify")] #[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> { pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key // use gateways key as a ref to an x25519_dalek key
let dh = gateway_key.inner().diffie_hellman(&self.pub_key); let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead // TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)] #[allow(clippy::expect_used)]
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::error::Error; use crate::error::Error;
use base64::{Engine, engine::general_purpose}; use base64::{engine::general_purpose, Engine};
use nym_credentials_interface::CredentialSpendingData; use nym_credentials_interface::CredentialSpendingData;
use nym_network_defaults::constants::{WG_TUN_DEVICE_IP_ADDRESS_V4, WG_TUN_DEVICE_IP_ADDRESS_V6}; use nym_network_defaults::constants::{WG_TUN_DEVICE_IP_ADDRESS_V4, WG_TUN_DEVICE_IP_ADDRESS_V6};
use nym_wireguard_types::PeerPublicKey; use nym_wireguard_types::PeerPublicKey;
@@ -28,6 +28,8 @@ pub type HmacSha256 = Hmac<Sha256>;
pub type Nonce = u64; pub type Nonce = u64;
pub type Taken = Option<SystemTime>; pub type Taken = Option<SystemTime>;
pub const BANDWIDTH_CAP_PER_DAY: u64 = 250 * 1024 * 1024 * 1024; // 250 GB
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)] #[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct IpPair { pub struct IpPair {
pub ipv4: Ipv4Addr, pub ipv4: Ipv4Addr,
@@ -167,7 +169,7 @@ impl GatewayClient {
#[cfg(feature = "verify")] #[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> { pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key // use gateways key as a ref to an x25519_dalek key
let dh = gateway_key.inner().diffie_hellman(&self.pub_key); let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead // TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)] #[allow(clippy::expect_used)]
@@ -1,195 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use super::{v1, v2, v3, v4, v5};
use nym_service_provider_requests_common::{Protocol, ServiceProviderType};
#[derive(Copy, Clone, Debug, PartialEq, strum_macros::Display)]
#[strum(serialize_all = "snake_case")]
pub enum AuthenticatorVersion {
/// introduced in wispa release (1.1.5)
V1,
/// introduced in aero release (1.1.9)
V2,
/// introduced in magura release (1.1.10)
V3,
/// introduced in crunch release (1.2.0)
V4,
/// introduced in dorina-patched release (1.6.1)
V5,
UNKNOWN,
}
impl AuthenticatorVersion {
pub const LATEST: Self = Self::V5;
pub const fn release_version(&self) -> semver::Version {
match self {
AuthenticatorVersion::V1 => semver::Version::new(1, 1, 5),
AuthenticatorVersion::V2 => semver::Version::new(1, 1, 9),
AuthenticatorVersion::V3 => semver::Version::new(1, 1, 10),
AuthenticatorVersion::V4 => semver::Version::new(1, 2, 0),
AuthenticatorVersion::V5 => semver::Version::new(1, 6, 1),
AuthenticatorVersion::UNKNOWN => semver::Version::new(0, 0, 0),
}
}
}
impl From<Protocol> for AuthenticatorVersion {
fn from(value: Protocol) -> Self {
if value.service_provider_type != ServiceProviderType::Authenticator {
AuthenticatorVersion::UNKNOWN
} else if value.version == v1::VERSION {
AuthenticatorVersion::V1
} else if value.version == v2::VERSION {
AuthenticatorVersion::V2
} else if value.version == v3::VERSION {
AuthenticatorVersion::V3
} else if value.version == v4::VERSION {
AuthenticatorVersion::V4
} else if value.version == v5::VERSION {
AuthenticatorVersion::V5
} else {
AuthenticatorVersion::UNKNOWN
}
}
}
impl From<u8> for AuthenticatorVersion {
fn from(value: u8) -> Self {
if value == v1::VERSION {
AuthenticatorVersion::V1
} else if value == v2::VERSION {
AuthenticatorVersion::V2
} else if value == v3::VERSION {
AuthenticatorVersion::V3
} else if value == v4::VERSION {
AuthenticatorVersion::V4
} else if value == v5::VERSION {
AuthenticatorVersion::V5
} else {
AuthenticatorVersion::UNKNOWN
}
}
}
impl From<&str> for AuthenticatorVersion {
fn from(value: &str) -> Self {
let Ok(semver) = semver::Version::parse(value) else {
return Self::UNKNOWN;
};
semver.into()
}
}
impl From<Option<&String>> for AuthenticatorVersion {
fn from(value: Option<&String>) -> Self {
match value {
None => Self::UNKNOWN,
Some(value) => value.as_str().into(),
}
}
}
impl From<String> for AuthenticatorVersion {
fn from(value: String) -> Self {
Self::from(value.as_str())
}
}
impl From<Option<String>> for AuthenticatorVersion {
fn from(value: Option<String>) -> Self {
value.as_ref().into()
}
}
impl From<semver::Version> for AuthenticatorVersion {
fn from(semver: semver::Version) -> Self {
if semver < AuthenticatorVersion::V1.release_version() {
return Self::UNKNOWN;
}
if semver < AuthenticatorVersion::V2.release_version() {
return Self::V1;
}
if semver < AuthenticatorVersion::V3.release_version() {
return Self::V2;
}
if semver < AuthenticatorVersion::V4.release_version() {
return Self::V3;
}
if semver < AuthenticatorVersion::V5.release_version() {
return Self::V4;
}
// if provided version is higher (or equal) to release version of V5,
// we return the latest (i.e. v5)
debug_assert_eq!(
Self::V5,
Self::LATEST,
"a new AuthenticatorVersion variant has been introduced without adjusting the `From<semver::Version>` trait"
);
Self::LATEST
}
}
#[cfg(test)]
mod tests {
use super::super::latest;
use super::*;
#[test]
fn strum_display() {
// sanity check on formatting and casing
assert_eq!("v1", AuthenticatorVersion::V1.to_string());
assert_eq!("v2", AuthenticatorVersion::V2.to_string());
assert_eq!("unknown", AuthenticatorVersion::UNKNOWN.to_string());
}
#[test]
fn u8_conversion() {
assert_eq!(AuthenticatorVersion::V1, AuthenticatorVersion::from(1u8));
assert_eq!(AuthenticatorVersion::V2, AuthenticatorVersion::from(2u8));
assert_eq!(
AuthenticatorVersion::UNKNOWN,
AuthenticatorVersion::from(latest::VERSION + 1)
);
assert_eq!(
AuthenticatorVersion::UNKNOWN,
AuthenticatorVersion::from(0u8)
);
assert_eq!(
AuthenticatorVersion::UNKNOWN,
AuthenticatorVersion::from(255u8)
);
}
#[test]
fn semver_checks() {
assert_eq!(AuthenticatorVersion::UNKNOWN, "1.1.4".into());
assert_eq!(AuthenticatorVersion::UNKNOWN, "0.1.0".into());
assert_eq!(AuthenticatorVersion::UNKNOWN, "1.0.4".into());
assert_eq!(AuthenticatorVersion::V1, "1.1.5".into());
assert_eq!(AuthenticatorVersion::V1, "1.1.6".into());
assert_eq!(AuthenticatorVersion::V1, "1.1.8".into());
assert_eq!(AuthenticatorVersion::V2, "1.1.9".into());
assert_eq!(AuthenticatorVersion::V3, "1.1.10".into());
assert_eq!(AuthenticatorVersion::V3, "1.1.11".into());
assert_eq!(AuthenticatorVersion::V3, "1.1.60".into());
assert_eq!(AuthenticatorVersion::V4, "1.2.0".into());
assert_eq!(AuthenticatorVersion::V4, "1.2.1".into());
assert_eq!(AuthenticatorVersion::V4, "1.5.1".into());
assert_eq!(AuthenticatorVersion::V4, "1.6.0".into());
assert_eq!(AuthenticatorVersion::V5, "1.6.1".into());
assert_eq!(AuthenticatorVersion::V5, "1.6.11".into());
assert_eq!(AuthenticatorVersion::V5, "1.7.0".into());
assert_eq!(AuthenticatorVersion::V5, "1.16.11".into());
assert_eq!(AuthenticatorVersion::V5, "1.17.0".into());
}
}
-1
View File
@@ -7,7 +7,6 @@ license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies] [dependencies]
async-trait = { workspace = true }
bip39 = { workspace = true } bip39 = { workspace = true }
log = { workspace = true } log = { workspace = true }
rand = { workspace = true } rand = { workspace = true }
+1 -1
View File
@@ -11,7 +11,7 @@ impl std::fmt::Display for BandwidthStatusMessage {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self { match self {
BandwidthStatusMessage::RemainingBandwidth(b) => { BandwidthStatusMessage::RemainingBandwidth(b) => {
write!(f, "remaining bandwidth: {b}") write!(f, "remaining bandwidth: {}", b)
} }
BandwidthStatusMessage::NoBandwidth => write!(f, "no bandwidth left"), BandwidthStatusMessage::NoBandwidth => write!(f, "no bandwidth left"),
} }
-2
View File
@@ -23,12 +23,10 @@ use nym_validator_client::nym_api::EpochId;
use nym_validator_client::nyxd::contract_traits::DkgQueryClient; use nym_validator_client::nyxd::contract_traits::DkgQueryClient;
pub use event::BandwidthStatusMessage; pub use event::BandwidthStatusMessage;
pub use traits::{BandwidthTicketProvider, DEFAULT_TICKETS_TO_SPEND};
pub mod acquire; pub mod acquire;
pub mod error; pub mod error;
mod event; mod event;
mod traits;
mod utils; mod utils;
#[derive(Debug)] #[derive(Debug)]
-42
View File
@@ -1,42 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use async_trait::async_trait;
use nym_credential_storage::storage::Storage;
use nym_credentials_interface::TicketType;
use nym_crypto::asymmetric::ed25519;
use nym_validator_client::nyxd::contract_traits::DkgQueryClient;
use crate::{error::BandwidthControllerError, BandwidthController, PreparedCredential};
pub const DEFAULT_TICKETS_TO_SPEND: u32 = 1;
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait BandwidthTicketProvider: Send + Sync {
async fn get_ecash_ticket(
&self,
ticket_type: TicketType,
gateway_id: ed25519::PublicKey,
tickets_to_spend: u32,
) -> Result<PreparedCredential, BandwidthControllerError>;
}
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
impl<C, St> BandwidthTicketProvider for BandwidthController<C, St>
where
C: DkgQueryClient + Sync + Send,
St: nym_credential_storage::storage::Storage,
<St as Storage>::StorageError: Send + Sync + 'static,
{
async fn get_ecash_ticket(
&self,
ticket_type: TicketType,
gateway_id: ed25519::PublicKey,
tickets_to_spend: u32,
) -> Result<PreparedCredential, BandwidthControllerError> {
self.prepare_ecash_ticket(ticket_type, gateway_id.to_bytes(), tickets_to_spend)
.await
}
}
+3 -3
View File
@@ -13,7 +13,7 @@ use nym_credentials_interface::{
}; };
use nym_ecash_time::Date; use nym_ecash_time::Date;
use nym_validator_client::coconut::all_ecash_api_clients; use nym_validator_client::coconut::all_ecash_api_clients;
use nym_validator_client::nym_api::{EpochId, NymApiClientExt}; use nym_validator_client::nym_api::EpochId;
use nym_validator_client::nyxd::contract_traits::DkgQueryClient; use nym_validator_client::nyxd::contract_traits::DkgQueryClient;
use nym_validator_client::EcashApiClient; use nym_validator_client::EcashApiClient;
use rand::prelude::SliceRandom; use rand::prelude::SliceRandom;
@@ -207,7 +207,7 @@ where
<St as Storage>::StorageError: Send + Sync + 'static, <St as Storage>::StorageError: Send + Sync + 'static,
{ {
if let Some(stored) = storage if let Some(stored) = storage
.get_expiration_date_signatures(expiration_date, epoch_id) .get_expiration_date_signatures(expiration_date)
.await .await
.map_err(BandwidthControllerError::credential_storage_error)? .map_err(BandwidthControllerError::credential_storage_error)?
{ {
@@ -220,7 +220,7 @@ where
ecash_apis, ecash_apis,
|api| async move { |api| async move {
api.api_client api.api_client
.global_expiration_date_signatures(Some(expiration_date), Some(epoch_id)) .global_expiration_date_signatures(Some(expiration_date))
.await .await
}, },
format!("aggregated coin index signatures for date {expiration_date}"), format!("aggregated coin index signatures for date {expiration_date}"),
+2 -2
View File
@@ -1,8 +1,8 @@
use clap::Args;
use clap::builder::Command; use clap::builder::Command;
use clap::clap_derive::ValueEnum; use clap::clap_derive::ValueEnum;
use clap_complete::Shell as ClapShell; use clap::Args;
use clap_complete::generator::generate; use clap_complete::generator::generate;
use clap_complete::Shell as ClapShell;
use std::io; use std::io;
pub fn fig_generate(command: &mut Command, name: &str) { pub fn fig_generate(command: &mut Command, name: &str) {
+10 -17
View File
@@ -3,7 +3,7 @@ name = "nym-client-core"
version = "1.1.15" version = "1.1.15"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"] authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
edition = "2021" edition = "2021"
rust-version = "1.85" rust-version = "1.76"
license.workspace = true license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -13,10 +13,10 @@ async-trait = { workspace = true }
base64 = { workspace = true } base64 = { workspace = true }
bs58 = { workspace = true } bs58 = { workspace = true }
clap = { workspace = true, optional = true } clap = { workspace = true, optional = true }
cfg-if = { workspace = true }
comfy-table = { workspace = true, optional = true } comfy-table = { workspace = true, optional = true }
futures = { workspace = true } futures = { workspace = true }
humantime = { workspace = true } humantime-serde = { workspace = true }
log = { workspace = true }
rand = { workspace = true } rand = { workspace = true }
rand_chacha = { workspace = true } rand_chacha = { workspace = true }
serde = { workspace = true, features = ["derive"] } serde = { workspace = true, features = ["derive"] }
@@ -25,23 +25,26 @@ sha2 = { workspace = true }
si-scale = { workspace = true } si-scale = { workspace = true }
thiserror = { workspace = true } thiserror = { workspace = true }
url = { workspace = true, features = ["serde"] } url = { workspace = true, features = ["serde"] }
tokio = { workspace = true, features = ["macros"] }
time = { workspace = true } time = { workspace = true }
tokio = { workspace = true, features = ["sync", "macros"] }
tracing = { workspace = true } tracing = { workspace = true }
zeroize = { workspace = true } zeroize = { workspace = true }
# internal # internal
nym-id = { path = "../nym-id" } nym-id = { path = "../nym-id" }
nym-bandwidth-controller = { path = "../bandwidth-controller" } nym-bandwidth-controller = { path = "../bandwidth-controller" }
nym-config = { path = "../config" }
nym-crypto = { path = "../crypto" } nym-crypto = { path = "../crypto" }
nym-gateway-client = { path = "../client-libs/gateway-client" } nym-gateway-client = { path = "../client-libs/gateway-client" }
nym-gateway-requests = { path = "../gateway-requests" } nym-gateway-requests = { path = "../gateway-requests" }
nym-http-api-client = { path = "../http-api-client", features = ["network-defaults"] } nym-http-api-client = { path = "../http-api-client" }
nym-metrics = { path = "../nym-metrics" }
nym-nonexhaustive-delayqueue = { path = "../nonexhaustive-delayqueue" } nym-nonexhaustive-delayqueue = { path = "../nonexhaustive-delayqueue" }
nym-sphinx = { path = "../nymsphinx" } nym-sphinx = { path = "../nymsphinx" }
nym-statistics-common = { path = "../statistics" } nym-statistics-common = { path = "../statistics" }
nym-pemstore = { path = "../pemstore" } nym-pemstore = { path = "../pemstore" }
nym-topology = { path = "../topology", features = ["persistence"] } nym-topology = { path = "../topology", features = ["persistence"] }
nym-mixnet-client = { path = "../client-libs/mixnet-client", default-features = false }
nym-validator-client = { path = "../client-libs/validator-client", default-features = false } nym-validator-client = { path = "../client-libs/validator-client", default-features = false }
nym-task = { path = "../task" } nym-task = { path = "../task" }
nym-credentials-interface = { path = "../credentials-interface" } nym-credentials-interface = { path = "../credentials-interface" }
@@ -53,10 +56,6 @@ nym-client-core-config-types = { path = "./config-types", features = [
nym-client-core-surb-storage = { path = "./surb-storage" } nym-client-core-surb-storage = { path = "./surb-storage" }
nym-client-core-gateways-storage = { path = "./gateways-storage" } nym-client-core-gateways-storage = { path = "./gateways-storage" }
nym-ecash-time = { path = "../ecash-time" } nym-ecash-time = { path = "../ecash-time" }
nym-mixnet-contract-common = { path = "../cosmwasm-smart-contracts/mixnet-contract" }
[target."cfg(not(target_arch = \"wasm32\"))".dependencies]
nym-mixnet-client = { path = "../client-libs/mixnet-client", default-features = false }
### For serving prometheus metrics ### For serving prometheus metrics
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.hyper] [target."cfg(not(target_arch = \"wasm32\"))".dependencies.hyper]
@@ -69,6 +68,7 @@ workspace = true
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.hyper-util] [target."cfg(not(target_arch = \"wasm32\"))".dependencies.hyper-util]
workspace = true workspace = true
features = ["tokio"] features = ["tokio"]
###
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio-stream] [target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio-stream]
workspace = true workspace = true
@@ -102,7 +102,7 @@ workspace = true
features = ["tokio"] features = ["tokio"]
[target."cfg(target_arch = \"wasm32\")".dependencies.gloo-timers] [target."cfg(target_arch = \"wasm32\")".dependencies.gloo-timers]
workspace = true version = "0.3.0"
features = ["futures"] features = ["futures"]
[target."cfg(target_arch = \"wasm32\")".dependencies.wasm-utils] [target."cfg(target_arch = \"wasm32\")".dependencies.wasm-utils]
@@ -113,10 +113,6 @@ features = ["websocket"]
workspace = true workspace = true
features = ["wasm-bindgen"] features = ["wasm-bindgen"]
[target."cfg(target_arch = \"wasm32\")".dependencies.tokio_with_wasm]
workspace = true
features = ["full"]
[dev-dependencies] [dev-dependencies]
tempfile = { workspace = true } tempfile = { workspace = true }
@@ -128,6 +124,3 @@ fs-surb-storage = ["nym-client-core-surb-storage/fs-surb-storage"]
fs-gateways-storage = ["nym-client-core-gateways-storage/fs-gateways-storage"] fs-gateways-storage = ["nym-client-core-gateways-storage/fs-gateways-storage"]
wasm = ["nym-gateway-client/wasm"] wasm = ["nym-gateway-client/wasm"]
metrics-server = [] metrics-server = []
[lints]
workspace = true
+15 -8
View File
@@ -57,7 +57,9 @@ const DEFAULT_MAXIMUM_ALLOWED_SURB_REQUEST_SIZE: u32 = 500;
const DEFAULT_MAXIMUM_REPLY_SURB_REREQUEST_WAITING_PERIOD: Duration = Duration::from_secs(10); const DEFAULT_MAXIMUM_REPLY_SURB_REREQUEST_WAITING_PERIOD: Duration = Duration::from_secs(10);
const DEFAULT_MAXIMUM_REPLY_SURB_DROP_WAITING_PERIOD: Duration = Duration::from_secs(5 * 60); const DEFAULT_MAXIMUM_REPLY_SURB_DROP_WAITING_PERIOD: Duration = Duration::from_secs(5 * 60);
const DEFAULT_MAXIMUM_REPLY_SURB_REREQUESTS: usize = 5;
// 12 hours
const DEFAULT_MAXIMUM_REPLY_SURB_AGE: Duration = Duration::from_secs(12 * 60 * 60);
// 24 hours // 24 hours
const DEFAULT_MAXIMUM_REPLY_KEY_AGE: Duration = Duration::from_secs(24 * 60 * 60); const DEFAULT_MAXIMUM_REPLY_KEY_AGE: Duration = Duration::from_secs(24 * 60 * 60);
@@ -416,9 +418,6 @@ pub struct Traffic {
/// will be routed as usual, to the entry gateway, through three mix nodes, egressing /// will be routed as usual, to the entry gateway, through three mix nodes, egressing
/// through the exit gateway. If mix hops are disabled, traffic will be routed directly /// through the exit gateway. If mix hops are disabled, traffic will be routed directly
/// from the entry gateway to the exit gateway, bypassing the mix nodes. /// from the entry gateway to the exit gateway, bypassing the mix nodes.
///
/// This overrides the `use_legacy_sphinx_format` setting as reduced mix hops
/// requires use of the updated SURB packet format.
pub disable_mix_hops: bool, pub disable_mix_hops: bool,
} }
@@ -626,9 +625,10 @@ pub struct ReplySurbs {
#[serde(with = "humantime_serde")] #[serde(with = "humantime_serde")]
pub maximum_reply_surb_drop_waiting_period: Duration, pub maximum_reply_surb_drop_waiting_period: Duration,
/// Defines maximum number of times the client is going to re-request reply surbs /// Defines maximum amount of time given reply surb is going to be valid for.
/// for clearing pending messages before giving up after making no progress. /// This is going to be superseded by key rotation once implemented.
pub maximum_reply_surbs_rerequests: usize, #[serde(with = "humantime_serde")]
pub maximum_reply_surb_age: Duration,
/// Defines maximum amount of time given reply key is going to be valid for. /// Defines maximum amount of time given reply key is going to be valid for.
/// This is going to be superseded by key rotation once implemented. /// This is going to be superseded by key rotation once implemented.
@@ -638,6 +638,9 @@ pub struct ReplySurbs {
/// Specifies the number of mixnet hops the packet should go through. If not specified, then /// Specifies the number of mixnet hops the packet should go through. If not specified, then
/// the default value is used. /// the default value is used.
pub surb_mix_hops: Option<u8>, pub surb_mix_hops: Option<u8>,
/// Specifies if we should reset all the sender tags on startup
pub fresh_sender_tags: bool,
} }
impl Default for ReplySurbs { impl Default for ReplySurbs {
@@ -652,9 +655,10 @@ impl Default for ReplySurbs {
maximum_reply_surb_rerequest_waiting_period: maximum_reply_surb_rerequest_waiting_period:
DEFAULT_MAXIMUM_REPLY_SURB_REREQUEST_WAITING_PERIOD, DEFAULT_MAXIMUM_REPLY_SURB_REREQUEST_WAITING_PERIOD,
maximum_reply_surb_drop_waiting_period: DEFAULT_MAXIMUM_REPLY_SURB_DROP_WAITING_PERIOD, maximum_reply_surb_drop_waiting_period: DEFAULT_MAXIMUM_REPLY_SURB_DROP_WAITING_PERIOD,
maximum_reply_surbs_rerequests: DEFAULT_MAXIMUM_REPLY_SURB_REREQUESTS, maximum_reply_surb_age: DEFAULT_MAXIMUM_REPLY_SURB_AGE,
maximum_reply_key_age: DEFAULT_MAXIMUM_REPLY_KEY_AGE, maximum_reply_key_age: DEFAULT_MAXIMUM_REPLY_KEY_AGE,
surb_mix_hops: None, surb_mix_hops: None,
fresh_sender_tags: false,
} }
} }
} }
@@ -707,10 +711,13 @@ pub struct DebugConfig {
/// Defines all configuration options related to reply SURBs. /// Defines all configuration options related to reply SURBs.
pub reply_surbs: ReplySurbs, pub reply_surbs: ReplySurbs,
/// Defines all configuration options related to stats reporting. /// Defines all configuration options related to stats reporting.
pub stats_reporting: StatsReporting, pub stats_reporting: StatsReporting,
/// Defines all configuration options related to the forget me flag. /// Defines all configuration options related to the forget me flag.
pub forget_me: ForgetMe, pub forget_me: ForgetMe,
/// Defines all configuration options related to the remember me flag. /// Defines all configuration options related to the remember me flag.
pub remember_me: RememberMe, pub remember_me: RememberMe,
} }
@@ -189,13 +189,14 @@ impl From<ConfigV6> for Config {
.debug .debug
.reply_surbs .reply_surbs
.maximum_reply_surb_drop_waiting_period, .maximum_reply_surb_drop_waiting_period,
maximum_reply_surb_age: value.debug.reply_surbs.maximum_reply_surb_age,
maximum_reply_key_age: value.debug.reply_surbs.maximum_reply_key_age, maximum_reply_key_age: value.debug.reply_surbs.maximum_reply_key_age,
surb_mix_hops: value.debug.reply_surbs.surb_mix_hops, surb_mix_hops: value.debug.reply_surbs.surb_mix_hops,
minimum_reply_surb_threshold_buffer: value minimum_reply_surb_threshold_buffer: value
.debug .debug
.reply_surbs .reply_surbs
.minimum_reply_surb_threshold_buffer, .minimum_reply_surb_threshold_buffer,
..Default::default() fresh_sender_tags: value.debug.reply_surbs.fresh_sender_tags,
}, },
stats_reporting: StatsReporting { stats_reporting: StatsReporting {
enabled: value.debug.stats_reporting.enabled, enabled: value.debug.stats_reporting.enabled,
@@ -543,8 +544,10 @@ pub struct DebugConfigV6 {
/// Defines all configuration options related to reply SURBs. /// Defines all configuration options related to reply SURBs.
pub reply_surbs: ReplySurbsV6, pub reply_surbs: ReplySurbsV6,
/// Defines all configuration options related to stats reporting. /// Defines all configuration options related to stats reporting.
pub stats_reporting: StatsReportingV6, pub stats_reporting: StatsReportingV6,
/// Defines all configuration options related to the forget me flag. /// Defines all configuration options related to the forget me flag.
pub forget_me: ForgetMeV6, pub forget_me: ForgetMeV6,
@@ -3,18 +3,17 @@ name = "nym-client-core-gateways-storage"
version = "0.1.0" version = "0.1.0"
edition = "2021" edition = "2021"
license.workspace = true license.workspace = true
rust-version.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies] [dependencies]
async-trait.workspace = true async-trait.workspace = true
cosmrs.workspace = true cosmrs.workspace = true
log.workspace = true
serde = { workspace = true, features = ["derive"] } serde = { workspace = true, features = ["derive"] }
thiserror.workspace = true thiserror.workspace = true
time.workspace = true time.workspace = true
tokio = { workspace = true, features = ["sync"] } tokio = { workspace = true, features = ["sync"] }
tracing.workspace = true
url.workspace = true url.workspace = true
zeroize = { workspace = true, features = ["zeroize_derive"] } zeroize = { workspace = true, features = ["zeroize_derive"] }
@@ -27,7 +26,6 @@ features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate", "time"]
optional = true optional = true
[build-dependencies] [build-dependencies]
anyhow = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] } tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
sqlx = { workspace = true, features = [ sqlx = { workspace = true, features = [
"runtime-tokio-rustls", "runtime-tokio-rustls",
+4 -13
View File
@@ -2,30 +2,23 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
#[tokio::main] #[tokio::main]
async fn main() -> anyhow::Result<()> { async fn main() {
#[cfg(feature = "fs-gateways-storage")] #[cfg(feature = "fs-gateways-storage")]
{ {
use anyhow::Context;
use sqlx::{Connection, SqliteConnection}; use sqlx::{Connection, SqliteConnection};
use std::env; use std::env;
let out_dir = env::var("OUT_DIR")?; let out_dir = env::var("OUT_DIR").unwrap();
let database_path = format!("{out_dir}/gateways-storage-example.sqlite"); let database_path = format!("{out_dir}/gateways-storage-example.sqlite");
// remove the db file if it already existed from previous build
// in case it was from a different branch
if std::fs::exists(&database_path)? {
std::fs::remove_file(&database_path)?;
}
let mut conn = SqliteConnection::connect(&format!("sqlite://{database_path}?mode=rwc")) let mut conn = SqliteConnection::connect(&format!("sqlite://{database_path}?mode=rwc"))
.await .await
.context("Failed to create SQLx database connection")?; .expect("Failed to create SQLx database connection");
sqlx::migrate!("./fs_gateways_migrations") sqlx::migrate!("./fs_gateways_migrations")
.run(&mut conn) .run(&mut conn)
.await .await
.context("Failed to perform SQLx migrations")?; .expect("Failed to perform SQLx migrations");
#[cfg(target_family = "unix")] #[cfg(target_family = "unix")]
println!("cargo:rustc-env=DATABASE_URL=sqlite://{}", &database_path); println!("cargo:rustc-env=DATABASE_URL=sqlite://{}", &database_path);
@@ -35,6 +28,4 @@ async fn main() -> anyhow::Result<()> {
// not a valid windows path... but hey, it works... // not a valid windows path... but hey, it works...
println!("cargo:rustc-env=DATABASE_URL=sqlite:///{}", &database_path); println!("cargo:rustc-env=DATABASE_URL=sqlite:///{}", &database_path);
} }
Ok(())
} }
@@ -2,7 +2,8 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::BadGateway; use crate::BadGateway;
use std::{io, path::PathBuf}; use std::io;
use std::path::PathBuf;
use thiserror::Error; use thiserror::Error;
#[derive(Debug, Error)] #[derive(Debug, Error)]
@@ -18,6 +19,7 @@ pub enum StorageError {
#[error("failed to perform sqlx migration: {source}")] #[error("failed to perform sqlx migration: {source}")]
MigrationError { MigrationError {
#[source]
#[from] #[from]
source: sqlx::migrate::MigrateError, source: sqlx::migrate::MigrateError,
}, },
@@ -30,6 +32,7 @@ pub enum StorageError {
#[error("failed to run the SQL query: {source}")] #[error("failed to run the SQL query: {source}")]
QueryError { QueryError {
#[source]
#[from] #[from]
source: sqlx::error::Error, source: sqlx::error::Error,
}, },
@@ -7,12 +7,12 @@ use crate::{
RawActiveGateway, RawCustomGatewayDetails, RawRegisteredGateway, RawRemoteGatewayDetails, RawActiveGateway, RawCustomGatewayDetails, RawRegisteredGateway, RawRemoteGatewayDetails,
}, },
}; };
use log::{debug, error};
use sqlx::{ use sqlx::{
sqlite::{SqliteAutoVacuum, SqliteSynchronous}, sqlite::{SqliteAutoVacuum, SqliteSynchronous},
ConnectOptions, ConnectOptions,
}; };
use std::path::Path; use std::path::Path;
use tracing::{debug, error};
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct StorageManager { pub struct StorageManager {
@@ -31,6 +31,7 @@ impl StorageManager {
} }
})?; })?;
} }
let opts = sqlx::sqlite::SqliteConnectOptions::new() let opts = sqlx::sqlite::SqliteConnectOptions::new()
.journal_mode(sqlx::sqlite::SqliteJournalMode::Wal) .journal_mode(sqlx::sqlite::SqliteJournalMode::Wal)
.synchronous(SqliteSynchronous::Normal) .synchronous(SqliteSynchronous::Normal)
@@ -86,7 +87,7 @@ impl StorageManager {
sqlx::query!("SELECT EXISTS (SELECT 1 FROM registered_gateway WHERE gateway_id_bs58 = ?) AS 'exists'", gateway_id) sqlx::query!("SELECT EXISTS (SELECT 1 FROM registered_gateway WHERE gateway_id_bs58 = ?) AS 'exists'", gateway_id)
.fetch_one(&self.connection_pool) .fetch_one(&self.connection_pool)
.await .await
.map(|result| result.exists == 1) .map(|result| result.exists == Some(1))
} }
pub(crate) async fn maybe_get_registered_gateway( pub(crate) async fn maybe_get_registered_gateway(
@@ -12,12 +12,12 @@ use crate::{
error::ClientCoreError, error::ClientCoreError,
init::types::{GatewaySelectionSpecification, GatewaySetup}, init::types::{GatewaySelectionSpecification, GatewaySetup},
}; };
use log::info;
use nym_client_core_gateways_storage::GatewayDetails; use nym_client_core_gateways_storage::GatewayDetails;
use nym_crypto::asymmetric::ed25519; use nym_crypto::asymmetric::ed25519;
use nym_topology::NymTopology; use nym_topology::NymTopology;
use nym_validator_client::UserAgent; use nym_validator_client::UserAgent;
use std::path::PathBuf; use std::path::PathBuf;
use tracing::info;
#[cfg_attr(feature = "cli", derive(clap::Args))] #[cfg_attr(feature = "cli", derive(clap::Args))]
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
@@ -81,14 +81,14 @@ where
// Attempt to use a user-provided gateway, if possible // Attempt to use a user-provided gateway, if possible
let user_chosen_gateway_id = common_args.gateway_id; let user_chosen_gateway_id = common_args.gateway_id;
tracing::debug!("User chosen gateway id: {user_chosen_gateway_id:?}"); log::debug!("User chosen gateway id: {user_chosen_gateway_id:?}");
let selection_spec = GatewaySelectionSpecification::new( let selection_spec = GatewaySelectionSpecification::new(
user_chosen_gateway_id.map(|id| id.to_base58_string()), user_chosen_gateway_id.map(|id| id.to_base58_string()),
Some(common_args.latency_based_selection), Some(common_args.latency_based_selection),
common_args.force_tls_gateway, common_args.force_tls_gateway,
); );
tracing::debug!("Gateway selection specification: {selection_spec:?}"); log::debug!("Gateway selection specification: {selection_spec:?}");
let registered_gateways = get_all_registered_identities(&details_store).await?; let registered_gateways = get_all_registered_identities(&details_store).await?;
@@ -114,12 +114,13 @@ where
})?; })?;
hardcoded_topology.entry_capable_nodes().cloned().collect() hardcoded_topology.entry_capable_nodes().cloned().collect()
} else { } else {
let mut rng = rand::thread_rng();
crate::init::helpers::gateways_for_init( crate::init::helpers::gateways_for_init(
&mut rng,
&core.client.nym_api_urls, &core.client.nym_api_urls,
user_agent, user_agent,
core.debug.topology.minimum_gateway_performance, core.debug.topology.minimum_gateway_performance,
core.debug.topology.ignore_ingress_epoch_role, core.debug.topology.ignore_ingress_epoch_role,
None,
) )
.await? .await?
}; };
@@ -58,7 +58,6 @@ where
Some(data) => data, Some(data) => data,
None => { None => {
// SAFETY: one of those arguments must have been set // SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.signatures_path.unwrap())? fs::read(common_args.signatures_path.unwrap())?
} }
}; };
@@ -64,7 +64,6 @@ where
Some(data) => data, Some(data) => data,
None => { None => {
// SAFETY: one of those arguments must have been set // SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.credential_path.unwrap())? fs::read(common_args.credential_path.unwrap())?
} }
}; };
@@ -58,7 +58,6 @@ where
Some(data) => data, Some(data) => data,
None => { None => {
// SAFETY: one of those arguments must have been set // SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.signatures_path.unwrap())? fs::read(common_args.signatures_path.unwrap())?
} }
}; };
@@ -58,7 +58,6 @@ where
Some(data) => data, Some(data) => data,
None => { None => {
// SAFETY: one of those arguments must have been set // SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.key_path.unwrap())? fs::read(common_args.key_path.unwrap())?
} }
}; };
@@ -12,6 +12,7 @@ use crate::{
}, },
init::types::{GatewaySelectionSpecification, GatewaySetup, InitResults}, init::types::{GatewaySelectionSpecification, GatewaySetup, InitResults},
}; };
use log::info;
use nym_client_core_gateways_storage::GatewayDetails; use nym_client_core_gateways_storage::GatewayDetails;
use nym_crypto::asymmetric::ed25519; use nym_crypto::asymmetric::ed25519;
use nym_sphinx::addressing::Recipient; use nym_sphinx::addressing::Recipient;
@@ -19,7 +20,6 @@ use nym_topology::NymTopology;
use nym_validator_client::UserAgent; use nym_validator_client::UserAgent;
use rand::rngs::OsRng; use rand::rngs::OsRng;
use std::path::PathBuf; use std::path::PathBuf;
use tracing::info;
// we can suppress this warning (as suggested by linter itself) since we're only using it in our own code // we can suppress this warning (as suggested by linter itself) since we're only using it in our own code
#[allow(async_fn_in_trait)] #[allow(async_fn_in_trait)]
@@ -130,23 +130,23 @@ where
// Attempt to use a user-provided gateway, if possible // Attempt to use a user-provided gateway, if possible
let user_chosen_gateway_id = common_args.gateway; let user_chosen_gateway_id = common_args.gateway;
tracing::debug!("User chosen gateway id: {user_chosen_gateway_id:?}"); log::debug!("User chosen gateway id: {user_chosen_gateway_id:?}");
let selection_spec = GatewaySelectionSpecification::new( let selection_spec = GatewaySelectionSpecification::new(
user_chosen_gateway_id.map(|id| id.to_base58_string()), user_chosen_gateway_id.map(|id| id.to_base58_string()),
Some(common_args.latency_based_selection), Some(common_args.latency_based_selection),
common_args.force_tls_gateway, common_args.force_tls_gateway,
); );
tracing::debug!("Gateway selection specification: {selection_spec:?}"); log::debug!("Gateway selection specification: {selection_spec:?}");
// Load and potentially override config // Load and potentially override config
tracing::debug!("Init arguments: {init_args:#?}"); log::debug!("Init arguments: {init_args:#?}");
let config = C::construct_config(&init_args); let config = C::construct_config(&init_args);
tracing::debug!("Constructed config: {config:#?}"); log::debug!("Constructed config: {config:#?}");
let paths = config.common_paths(); let paths = config.common_paths();
let core = config.core_config(); let core = config.core_config();
tracing::info!( log::info!(
"Using nym-api: {}", "Using nym-api: {}",
core.client core.client
.nym_api_urls .nym_api_urls
@@ -173,12 +173,13 @@ where
})?; })?;
hardcoded_topology.entry_capable_nodes().cloned().collect() hardcoded_topology.entry_capable_nodes().cloned().collect()
} else { } else {
let mut rng = rand::thread_rng();
crate::init::helpers::gateways_for_init( crate::init::helpers::gateways_for_init(
&mut rng,
&core.client.nym_api_urls, &core.client.nym_api_urls,
user_agent, user_agent,
core.debug.topology.minimum_gateway_performance, core.debug.topology.minimum_gateway_performance,
core.debug.topology.ignore_ingress_epoch_role, core.debug.topology.ignore_ingress_epoch_role,
None,
) )
.await? .await?
}; };
+97 -398
View File
@@ -7,19 +7,17 @@ use super::statistics_control::StatisticsControl;
use crate::client::base_client::storage::helpers::store_client_keys; use crate::client::base_client::storage::helpers::store_client_keys;
use crate::client::base_client::storage::MixnetClientStorage; use crate::client::base_client::storage::MixnetClientStorage;
use crate::client::cover_traffic_stream::LoopCoverTrafficStream; use crate::client::cover_traffic_stream::LoopCoverTrafficStream;
use crate::client::event_control::EventControl;
use crate::client::inbound_messages::{InputMessage, InputMessageReceiver, InputMessageSender}; use crate::client::inbound_messages::{InputMessage, InputMessageReceiver, InputMessageSender};
use crate::client::key_manager::persistence::KeyStore; use crate::client::key_manager::persistence::KeyStore;
use crate::client::key_manager::ClientKeys; use crate::client::key_manager::ClientKeys;
use crate::client::mix_traffic::transceiver::{GatewayReceiver, GatewayTransceiver, RemoteGateway}; use crate::client::mix_traffic::transceiver::{GatewayReceiver, GatewayTransceiver, RemoteGateway};
use crate::client::mix_traffic::{BatchMixMessageSender, MixTrafficController, MixTrafficEvent}; use crate::client::mix_traffic::{BatchMixMessageSender, MixTrafficController};
use crate::client::real_messages_control; use crate::client::real_messages_control;
use crate::client::real_messages_control::RealMessagesController; use crate::client::real_messages_control::RealMessagesController;
use crate::client::received_buffer::{ use crate::client::received_buffer::{
ReceivedBufferRequestReceiver, ReceivedBufferRequestSender, ReceivedMessagesBufferController, ReceivedBufferRequestReceiver, ReceivedBufferRequestSender, ReceivedMessagesBufferController,
}; };
use crate::client::replies::reply_controller; use crate::client::replies::reply_controller;
use crate::client::replies::reply_controller::key_rotation_helpers::KeyRotationConfig;
use crate::client::replies::reply_controller::{ReplyControllerReceiver, ReplyControllerSender}; use crate::client::replies::reply_controller::{ReplyControllerReceiver, ReplyControllerSender};
use crate::client::replies::reply_storage::{ use crate::client::replies::reply_storage::{
CombinedReplyStorage, PersistentReplyStorage, ReplyStorageBackend, SentReplyKeys, CombinedReplyStorage, PersistentReplyStorage, ReplyStorageBackend, SentReplyKeys,
@@ -28,14 +26,15 @@ use crate::client::topology_control::nym_api_provider::NymApiTopologyProvider;
use crate::client::topology_control::{ use crate::client::topology_control::{
TopologyAccessor, TopologyRefresher, TopologyRefresherConfig, TopologyAccessor, TopologyRefresher, TopologyRefresherConfig,
}; };
use crate::config;
use crate::config::{Config, DebugConfig}; use crate::config::{Config, DebugConfig};
use crate::error::ClientCoreError; use crate::error::ClientCoreError;
use crate::init::{ use crate::init::{
setup_gateway, setup_gateway,
types::{GatewaySetup, InitialisationResult}, types::{GatewaySetup, InitialisationResult},
}; };
use crate::{config, spawn_future};
use futures::channel::mpsc; use futures::channel::mpsc;
use log::*;
use nym_bandwidth_controller::BandwidthController; use nym_bandwidth_controller::BandwidthController;
use nym_client_core_config_types::{ForgetMe, RememberMe}; use nym_client_core_config_types::{ForgetMe, RememberMe};
use nym_client_core_gateways_storage::{GatewayDetails, GatewaysDetailsStore}; use nym_client_core_gateways_storage::{GatewayDetails, GatewaysDetailsStore};
@@ -49,34 +48,23 @@ use nym_gateway_client::{
use nym_sphinx::acknowledgements::AckKey; use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient; use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::addressing::nodes::NodeIdentity; use nym_sphinx::addressing::nodes::NodeIdentity;
use nym_sphinx::params::PacketType;
use nym_sphinx::receiver::{ReconstructedMessage, SphinxMessageReceiver}; use nym_sphinx::receiver::{ReconstructedMessage, SphinxMessageReceiver};
use nym_statistics_common::clients::ClientStatsSender; use nym_statistics_common::clients::ClientStatsSender;
use nym_statistics_common::generate_client_stats_id; use nym_statistics_common::generate_client_stats_id;
use nym_task::connections::{ConnectionCommandReceiver, ConnectionCommandSender, LaneQueueLengths}; use nym_task::connections::{ConnectionCommandReceiver, ConnectionCommandSender, LaneQueueLengths};
use nym_task::ShutdownTracker; use nym_task::{TaskClient, TaskHandle};
use nym_topology::provider_trait::TopologyProvider; use nym_topology::provider_trait::TopologyProvider;
use nym_topology::HardcodedTopologyProvider; use nym_topology::HardcodedTopologyProvider;
use nym_validator_client::nym_api::NymApiClientExt;
use nym_validator_client::{nyxd::contract_traits::DkgQueryClient, UserAgent}; use nym_validator_client::{nyxd::contract_traits::DkgQueryClient, UserAgent};
use rand::prelude::SliceRandom;
use rand::rngs::OsRng; use rand::rngs::OsRng;
use rand::thread_rng;
use std::fmt::Debug; use std::fmt::Debug;
use std::os::raw::c_int as RawFd; use std::os::raw::c_int as RawFd;
use std::path::Path; use std::path::Path;
use std::sync::Arc; use std::sync::Arc;
use time::OffsetDateTime;
use tokio::sync::mpsc::Sender; use tokio::sync::mpsc::Sender;
use url::Url; use url::Url;
#[cfg(target_arch = "wasm32")]
#[cfg(debug_assertions)]
use wasm_utils::console_log;
/// Default number of retries for Nym API requests when using network details with domain fronting.
/// This allows the client to try alternative URLs if the primary endpoint is unavailable.
const DEFAULT_NYM_API_RETRIES: usize = 3;
#[cfg(all( #[cfg(all(
not(target_arch = "wasm32"), not(target_arch = "wasm32"),
feature = "fs-surb-storage", feature = "fs-surb-storage",
@@ -87,28 +75,10 @@ pub mod non_wasm_helpers;
pub mod helpers; pub mod helpers;
pub mod storage; pub mod storage;
#[derive(Clone, Copy, Debug)]
pub enum MixnetClientEvent {
Traffic(MixTrafficEvent),
}
pub type EventReceiver = mpsc::UnboundedReceiver<MixnetClientEvent>;
#[derive(Clone)]
pub struct EventSender(pub mpsc::UnboundedSender<MixnetClientEvent>);
impl EventSender {
pub fn send(&self, event: MixnetClientEvent) {
if let Err(err) = self.0.unbounded_send(event) {
tracing::warn!("Failed to send error event. The caller event reader was closed: {err}");
}
}
}
#[derive(Clone)] #[derive(Clone)]
pub struct ClientInput { pub struct ClientInput {
pub connection_command_sender: ConnectionCommandSender, pub connection_command_sender: ConnectionCommandSender,
pub input_sender: InputMessageSender, pub input_sender: InputMessageSender,
pub client_request_sender: ClientRequestSender,
} }
impl ClientInput { impl ClientInput {
@@ -120,6 +90,7 @@ impl ClientInput {
} }
} }
#[derive(Clone)]
pub struct ClientOutput { pub struct ClientOutput {
pub received_buffer_request_sender: ReceivedBufferRequestSender, pub received_buffer_request_sender: ReceivedBufferRequestSender,
} }
@@ -159,11 +130,9 @@ pub enum ClientInputStatus {
} }
impl ClientInputStatus { impl ClientInputStatus {
#[allow(clippy::panic)]
pub fn register_producer(&mut self) -> ClientInput { pub fn register_producer(&mut self) -> ClientInput {
match std::mem::replace(self, ClientInputStatus::Connected) { match std::mem::replace(self, ClientInputStatus::Connected) {
ClientInputStatus::AwaitingProducer { client_input } => client_input, ClientInputStatus::AwaitingProducer { client_input } => client_input,
// critical failure implying misuse of software
ClientInputStatus::Connected => panic!("producer was already registered before"), ClientInputStatus::Connected => panic!("producer was already registered before"),
} }
} }
@@ -175,11 +144,9 @@ pub enum ClientOutputStatus {
} }
impl ClientOutputStatus { impl ClientOutputStatus {
#[allow(clippy::panic)]
pub fn register_consumer(&mut self) -> ClientOutput { pub fn register_consumer(&mut self) -> ClientOutput {
match std::mem::replace(self, ClientOutputStatus::Connected) { match std::mem::replace(self, ClientOutputStatus::Connected) {
ClientOutputStatus::AwaitingConsumer { client_output } => client_output, ClientOutputStatus::AwaitingConsumer { client_output } => client_output,
// critical failure implying misuse of software
ClientOutputStatus::Connected => panic!("consumer was already registered before"), ClientOutputStatus::Connected => panic!("consumer was already registered before"),
} }
} }
@@ -216,14 +183,10 @@ pub struct BaseClientBuilder<C, S: MixnetClientStorage> {
client_store: S, client_store: S,
dkg_query_client: Option<C>, dkg_query_client: Option<C>,
// Optional API URLs for domain fronting support
nym_api_urls: Option<Vec<nym_network_defaults::ApiUrl>>,
wait_for_gateway: bool, wait_for_gateway: bool,
custom_topology_provider: Option<Box<dyn TopologyProvider + Send + Sync>>, custom_topology_provider: Option<Box<dyn TopologyProvider + Send + Sync>>,
custom_gateway_transceiver: Option<Box<dyn GatewayTransceiver + Send>>, custom_gateway_transceiver: Option<Box<dyn GatewayTransceiver + Send>>,
shutdown: Option<ShutdownTracker>, shutdown: Option<TaskClient>,
event_tx: Option<EventSender>,
user_agent: Option<UserAgent>, user_agent: Option<UserAgent>,
setup_method: GatewaySetup, setup_method: GatewaySetup,
@@ -248,12 +211,10 @@ where
config: base_config, config: base_config,
client_store, client_store,
dkg_query_client, dkg_query_client,
nym_api_urls: None,
wait_for_gateway: false, wait_for_gateway: false,
custom_topology_provider: None, custom_topology_provider: None,
custom_gateway_transceiver: None, custom_gateway_transceiver: None,
shutdown: None, shutdown: None,
event_tx: None,
user_agent: None, user_agent: None,
setup_method: GatewaySetup::MustLoad { gateway_id: None }, setup_method: GatewaySetup::MustLoad { gateway_id: None },
#[cfg(unix)] #[cfg(unix)]
@@ -271,16 +232,6 @@ where
self self
} }
/// Set Nym API URLs for domain fronting support.
///
/// When provided, the client will use these API URLs (which include front_hosts)
/// to construct HTTP clients with domain fronting enabled.
#[must_use]
pub fn with_nym_api_urls(mut self, nym_api_urls: Vec<nym_network_defaults::ApiUrl>) -> Self {
self.nym_api_urls = Some(nym_api_urls);
self
}
#[must_use] #[must_use]
pub fn with_forget_me(mut self, forget_me: &ForgetMe) -> Self { pub fn with_forget_me(mut self, forget_me: &ForgetMe) -> Self {
self.config.debug.forget_me = *forget_me; self.config.debug.forget_me = *forget_me;
@@ -321,17 +272,11 @@ where
} }
#[must_use] #[must_use]
pub fn with_shutdown(mut self, shutdown: ShutdownTracker) -> Self { pub fn with_shutdown(mut self, shutdown: TaskClient) -> Self {
self.shutdown = Some(shutdown); self.shutdown = Some(shutdown);
self self
} }
#[must_use]
pub fn with_event_tx(mut self, event_tx: EventSender) -> Self {
self.event_tx = Some(event_tx);
self
}
#[must_use] #[must_use]
pub fn with_user_agent(mut self, user_agent: UserAgent) -> Self { pub fn with_user_agent(mut self, user_agent: UserAgent) -> Self {
self.user_agent = Some(user_agent); self.user_agent = Some(user_agent);
@@ -362,18 +307,6 @@ where
details.client_address() details.client_address()
} }
fn start_event_control(
parent_event_tx: Option<EventSender>,
children_event_rx: EventReceiver,
shutdown_tracker: &ShutdownTracker,
) {
let event_control = EventControl::new(parent_event_tx, children_event_rx);
shutdown_tracker.try_spawn_named_with_shutdown(
async move { event_control.run().await },
"EventControl",
);
}
// future constantly pumping loop cover traffic at some specified average rate // future constantly pumping loop cover traffic at some specified average rate
// the pumped traffic goes to the MixTrafficController // the pumped traffic goes to the MixTrafficController
fn start_cover_traffic_stream( fn start_cover_traffic_stream(
@@ -383,11 +316,11 @@ where
topology_accessor: TopologyAccessor, topology_accessor: TopologyAccessor,
mix_tx: BatchMixMessageSender, mix_tx: BatchMixMessageSender,
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
shutdown_tracker: &ShutdownTracker, task_client: TaskClient,
) { ) {
tracing::info!("Starting loop cover traffic stream..."); info!("Starting loop cover traffic stream...");
let mut stream = LoopCoverTrafficStream::new( let stream = LoopCoverTrafficStream::new(
ack_key, ack_key,
debug_config.acknowledgements.average_ack_delay, debug_config.acknowledgements.average_ack_delay,
mix_tx, mix_tx,
@@ -396,15 +329,15 @@ where
debug_config.traffic, debug_config.traffic,
debug_config.cover_traffic, debug_config.cover_traffic,
stats_tx, stats_tx,
task_client,
); );
shutdown_tracker
.try_spawn_named_with_shutdown(async move { stream.run().await }, "CoverTrafficStream"); stream.start();
} }
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
fn start_real_traffic_controller( fn start_real_traffic_controller(
controller_config: real_messages_control::Config, controller_config: real_messages_control::Config,
key_rotation_config: KeyRotationConfig,
topology_accessor: TopologyAccessor, topology_accessor: TopologyAccessor,
ack_receiver: AcknowledgementReceiver, ack_receiver: AcknowledgementReceiver,
input_receiver: InputMessageReceiver, input_receiver: InputMessageReceiver,
@@ -414,14 +347,14 @@ where
reply_controller_receiver: ReplyControllerReceiver, reply_controller_receiver: ReplyControllerReceiver,
lane_queue_lengths: LaneQueueLengths, lane_queue_lengths: LaneQueueLengths,
client_connection_rx: ConnectionCommandReceiver, client_connection_rx: ConnectionCommandReceiver,
task_client: TaskClient,
packet_type: PacketType,
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
shutdown_tracker: &ShutdownTracker,
) { ) {
tracing::info!("Starting real traffic stream..."); info!("Starting real traffic stream...");
let real_messages_controller = RealMessagesController::new( RealMessagesController::new(
controller_config, controller_config,
key_rotation_config,
ack_receiver, ack_receiver,
input_receiver, input_receiver,
mix_sender, mix_sender,
@@ -432,63 +365,9 @@ where
lane_queue_lengths, lane_queue_lengths,
client_connection_rx, client_connection_rx,
stats_tx, stats_tx,
shutdown_tracker.clone_shutdown_token(), task_client,
); )
.start(packet_type);
// break out all the subtasks
let (mut out_queue_control, mut reply_controller, ack_controller) =
real_messages_controller.into_tasks();
let (
mut ack_listener,
mut input_listener,
mut retransmission_listener,
mut sent_notification_listener,
mut ack_action_controller,
) = ack_controller.into_tasks();
shutdown_tracker.try_spawn_named(
async move { out_queue_control.run().await },
"RealMessagesController::OutQueueControl",
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move { reply_controller.run(shutdown_token).await },
"RealMessagesController::ReplyController",
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move { ack_listener.run(shutdown_token).await },
"AcknowledgementController::AcknowledgementListener",
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move { input_listener.run(shutdown_token).await },
"AcknowledgementController::InputMessageListener",
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move { retransmission_listener.run(shutdown_token).await },
"AcknowledgementController::RetransmissionRequestListener",
);
shutdown_tracker.try_spawn_named_with_shutdown(
async move {
sent_notification_listener.run().await;
},
"AcknowledgementController::SentNotificationListener",
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move { ack_action_controller.run(shutdown_token).await },
"AcknowledgementController::ActionController",
);
// .start(packet_type);
} }
// buffer controlling all messages fetched from provider // buffer controlling all messages fetched from provider
@@ -499,29 +378,21 @@ where
mixnet_receiver: MixnetMessageReceiver, mixnet_receiver: MixnetMessageReceiver,
reply_key_storage: SentReplyKeys, reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
shutdown: TaskClient,
metrics_reporter: ClientStatsSender, metrics_reporter: ClientStatsSender,
shutdown_tracker: &ShutdownTracker,
) { ) {
tracing::info!("Starting received messages buffer controller..."); info!("Starting received messages buffer controller...");
let controller = ReceivedMessagesBufferController::<SphinxMessageReceiver>::new( let controller: ReceivedMessagesBufferController<SphinxMessageReceiver> =
local_encryption_keypair, ReceivedMessagesBufferController::new(
query_receiver, local_encryption_keypair,
mixnet_receiver, query_receiver,
reply_key_storage, mixnet_receiver,
reply_controller_sender, reply_key_storage,
metrics_reporter, reply_controller_sender,
shutdown_tracker.clone_shutdown_token(), metrics_reporter,
); shutdown,
let (mut msg_receiver, mut req_receiver) = controller.into_tasks(); );
controller.start()
shutdown_tracker.try_spawn_named(
async move { msg_receiver.run().await },
"ReceivedMessagesBufferController::FragmentedMessageReceiver",
);
shutdown_tracker.try_spawn_named(
async move { req_receiver.run().await },
"ReceivedMessagesBufferController::RequestReceiver",
);
} }
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
@@ -533,7 +404,7 @@ where
packet_router: PacketRouter, packet_router: PacketRouter,
stats_reporter: ClientStatsSender, stats_reporter: ClientStatsSender,
#[cfg(unix)] connection_fd_callback: Option<Arc<dyn Fn(RawFd) + Send + Sync>>, #[cfg(unix)] connection_fd_callback: Option<Arc<dyn Fn(RawFd) + Send + Sync>>,
shutdown_tracker: &ShutdownTracker, shutdown: TaskClient,
) -> Result<GatewayClient<C, S::CredentialStore>, ClientCoreError> ) -> Result<GatewayClient<C, S::CredentialStore>, ClientCoreError>
where where
<S::KeyStore as KeyStore>::StorageError: Send + Sync + 'static, <S::KeyStore as KeyStore>::StorageError: Send + Sync + 'static,
@@ -552,7 +423,7 @@ where
packet_router, packet_router,
bandwidth_controller, bandwidth_controller,
stats_reporter, stats_reporter,
shutdown_tracker.clone_shutdown_token(), shutdown,
) )
} else { } else {
let cfg = GatewayConfig::new( let cfg = GatewayConfig::new(
@@ -577,15 +448,15 @@ where
stats_reporter, stats_reporter,
#[cfg(unix)] #[cfg(unix)]
connection_fd_callback, connection_fd_callback,
shutdown_tracker.clone_shutdown_token(), shutdown,
) )
}; };
let gateway_failure = |err| { let gateway_failure = |err| {
tracing::error!("Could not authenticate and start up the gateway connection - {err}"); log::error!("Could not authenticate and start up the gateway connection - {err}");
ClientCoreError::GatewayClientError { ClientCoreError::GatewayClientError {
gateway_id: details.gateway_id.to_base58_string(), gateway_id: details.gateway_id.to_base58_string(),
source: Box::new(err), source: err,
} }
}; };
@@ -613,7 +484,7 @@ where
details_store details_store
.upgrade_stored_remote_gateway_key(gateway_client.gateway_identity(), &updated_key) .upgrade_stored_remote_gateway_key(gateway_client.gateway_identity(), &updated_key)
.await.map_err(|err| { .await.map_err(|err| {
tracing::error!("failed to store upgraded gateway key! this connection might be forever broken now: {err}"); error!("failed to store upgraded gateway key! this connection might be forever broken now: {err}");
ClientCoreError::GatewaysDetailsStoreError { source: Box::new(err) } ClientCoreError::GatewaysDetailsStoreError { source: Box::new(err) }
})? })?
} }
@@ -640,7 +511,7 @@ where
packet_router: PacketRouter, packet_router: PacketRouter,
stats_reporter: ClientStatsSender, stats_reporter: ClientStatsSender,
#[cfg(unix)] connection_fd_callback: Option<Arc<dyn Fn(RawFd) + Send + Sync>>, #[cfg(unix)] connection_fd_callback: Option<Arc<dyn Fn(RawFd) + Send + Sync>>,
shutdown_tracker: &ShutdownTracker, mut shutdown: TaskClient,
) -> Result<Box<dyn GatewayTransceiver + Send>, ClientCoreError> ) -> Result<Box<dyn GatewayTransceiver + Send>, ClientCoreError>
where where
<S::KeyStore as KeyStore>::StorageError: Send + Sync + 'static, <S::KeyStore as KeyStore>::StorageError: Send + Sync + 'static,
@@ -657,6 +528,7 @@ where
Err(ClientCoreError::CustomGatewaySelectionExpected) Err(ClientCoreError::CustomGatewaySelectionExpected)
} else { } else {
// and make sure to invalidate the task client, so we wouldn't cause premature shutdown // and make sure to invalidate the task client, so we wouldn't cause premature shutdown
shutdown.disarm();
custom_gateway_transceiver.set_packet_router(packet_router)?; custom_gateway_transceiver.set_packet_router(packet_router)?;
Ok(custom_gateway_transceiver) Ok(custom_gateway_transceiver)
}; };
@@ -672,7 +544,7 @@ where
stats_reporter, stats_reporter,
#[cfg(unix)] #[cfg(unix)]
connection_fd_callback, connection_fd_callback,
shutdown_tracker, shutdown,
) )
.await?; .await?;
@@ -683,14 +555,14 @@ where
custom_provider: Option<Box<dyn TopologyProvider + Send + Sync>>, custom_provider: Option<Box<dyn TopologyProvider + Send + Sync>>,
config_topology: config::Topology, config_topology: config::Topology,
nym_api_urls: Vec<Url>, nym_api_urls: Vec<Url>,
nym_api_client: nym_http_api_client::Client, user_agent: Option<UserAgent>,
) -> Box<dyn TopologyProvider + Send + Sync> { ) -> Box<dyn TopologyProvider + Send + Sync> {
// if no custom provider was ... provided ..., create one using nym-api // if no custom provider was ... provided ..., create one using nym-api
custom_provider.unwrap_or_else(|| { custom_provider.unwrap_or_else(|| {
Box::new(NymApiTopologyProvider::new( Box::new(NymApiTopologyProvider::new(
config_topology, config_topology,
nym_api_urls, nym_api_urls,
nym_api_client, user_agent,
)) ))
}) })
} }
@@ -703,28 +575,30 @@ where
topology_accessor: TopologyAccessor, topology_accessor: TopologyAccessor,
local_gateway: NodeIdentity, local_gateway: NodeIdentity,
wait_for_gateway: bool, wait_for_gateway: bool,
shutdown_tracker: &ShutdownTracker, mut task_client: TaskClient,
) -> Result<(), ClientCoreError> { ) -> Result<(), ClientCoreError> {
let topology_refresher_config = let topology_refresher_config =
TopologyRefresherConfig::new(topology_config.topology_refresh_rate); TopologyRefresherConfig::new(topology_config.topology_refresh_rate);
if topology_config.disable_refreshing { if topology_config.disable_refreshing {
// if we're not spawning the refresher, don't cause shutdown immediately // if we're not spawning the refresher, don't cause shutdown immediately
tracing::info!("The background topology refresher is not going to be started"); info!("The background topology refesher is not going to be started");
task_client.disarm();
} }
let mut topology_refresher = TopologyRefresher::new( let mut topology_refresher = TopologyRefresher::new(
topology_refresher_config, topology_refresher_config,
topology_accessor, topology_accessor,
topology_provider, topology_provider,
task_client,
); );
// before returning, block entire runtime to refresh the current network view so that any // before returning, block entire runtime to refresh the current network view so that any
// components depending on topology would see a non-empty view // components depending on topology would see a non-empty view
tracing::info!("Obtaining initial network topology"); info!("Obtaining initial network topology");
topology_refresher.try_refresh().await; topology_refresher.try_refresh().await;
if let Err(err) = topology_refresher.ensure_topology_is_routable().await { if let Err(err) = topology_refresher.ensure_topology_is_routable().await {
tracing::error!( log::error!(
"The current network topology seem to be insufficient to route any packets through \ "The current network topology seem to be insufficient to route any packets through \
- check if enough nodes and a gateway are online - source: {err}" - check if enough nodes and a gateway are online - source: {err}"
); );
@@ -746,13 +620,13 @@ where
.wait_for_gateway(local_gateway, waiting_timeout) .wait_for_gateway(local_gateway, waiting_timeout)
.await .await
{ {
tracing::error!( error!(
"the gateway did not come back online within the specified timeout: {err}" "the gateway did not come back online within the specified timeout: {err}"
); );
return Err(err.into()); return Err(err.into());
} }
} else { } else {
tracing::error!("the gateway we're supposedly connected to does not exist. We'll not be able to send any packets to ourselves: {err}"); error!("the gateway we're supposedly connected to does not exist. We'll not be able to send any packets to ourselves: {err}");
return Err(err.into()); return Err(err.into());
} }
} }
@@ -760,11 +634,8 @@ where
if !topology_config.disable_refreshing { if !topology_config.disable_refreshing {
// don't spawn the refresher if we don't want to be refreshing the topology. // don't spawn the refresher if we don't want to be refreshing the topology.
// only use the initial values obtained // only use the initial values obtained
tracing::info!("Starting topology refresher..."); info!("Starting topology refresher...");
shutdown_tracker.try_spawn_named_with_shutdown( topology_refresher.start();
async move { topology_refresher.run().await },
"TopologyRefresher",
);
} }
Ok(()) Ok(())
@@ -775,9 +646,9 @@ where
user_agent: Option<UserAgent>, user_agent: Option<UserAgent>,
client_stats_id: String, client_stats_id: String,
input_sender: Sender<InputMessage>, input_sender: Sender<InputMessage>,
shutdown_tracker: &ShutdownTracker, task_client: TaskClient,
) -> ClientStatsSender { ) -> ClientStatsSender {
tracing::info!("Starting statistics control..."); info!("Starting statistics control...");
StatisticsControl::create_and_start( StatisticsControl::create_and_start(
config.debug.stats_reporting, config.debug.stats_reporting,
user_agent user_agent
@@ -785,71 +656,45 @@ where
.unwrap_or("unknown".to_string()), .unwrap_or("unknown".to_string()),
client_stats_id, client_stats_id,
input_sender.clone(), input_sender.clone(),
shutdown_tracker, task_client,
) )
} }
fn start_mix_traffic_controller( fn start_mix_traffic_controller(
gateway_transceiver: Box<dyn GatewayTransceiver + Send>, gateway_transceiver: Box<dyn GatewayTransceiver + Send>,
shutdown_tracker: &ShutdownTracker, shutdown: TaskClient,
event_tx: EventSender,
) -> (BatchMixMessageSender, ClientRequestSender) { ) -> (BatchMixMessageSender, ClientRequestSender) {
tracing::info!("Starting mix traffic controller..."); info!("Starting mix traffic controller...");
let mut mix_traffic_controller = MixTrafficController::new( let (mix_traffic_controller, mix_tx, client_tx) =
gateway_transceiver, MixTrafficController::new(gateway_transceiver, shutdown);
shutdown_tracker.clone_shutdown_token(), mix_traffic_controller.start();
event_tx,
);
let mix_tx = mix_traffic_controller.mix_tx();
let client_tx = mix_traffic_controller.client_tx();
shutdown_tracker.try_spawn_named(
async move { mix_traffic_controller.run().await },
"MixTrafficController",
);
(mix_tx, client_tx) (mix_tx, client_tx)
} }
// TODO: rename it as it implies the data is persistent whilst one can use InMemBackend // TODO: rename it as it implies the data is persistent whilst one can use InMemBackend
async fn setup_persistent_reply_storage( async fn setup_persistent_reply_storage(
backend: S::ReplyStore, backend: S::ReplyStore,
key_rotation_config: KeyRotationConfig, shutdown: TaskClient,
shutdown_tracker: &ShutdownTracker,
) -> Result<CombinedReplyStorage, ClientCoreError> ) -> Result<CombinedReplyStorage, ClientCoreError>
where where
<S::ReplyStore as ReplyStorageBackend>::StorageError: Sync + Send, <S::ReplyStore as ReplyStorageBackend>::StorageError: Sync + Send,
S::ReplyStore: Send + Sync, S::ReplyStore: Send + Sync,
{ {
tracing::trace!("Setup persistent reply storage"); log::trace!("Setup persistent reply storage");
let now = OffsetDateTime::now_utc();
let expected_current_key_rotation_start =
key_rotation_config.expected_current_key_rotation_start(now);
// time of the start of one epoch BEFORE the CURRENT rotation has begun
// this indicates the starting time of when packets with the current keys might have been constructed
// (i.e. any surbs OLDER than that MUST BE invalid)
let prior_epoch_start =
expected_current_key_rotation_start - key_rotation_config.epoch_duration;
let persistent_storage = PersistentReplyStorage::new(backend); let persistent_storage = PersistentReplyStorage::new(backend);
let mem_store = persistent_storage let mem_store = persistent_storage
.load_state_from_backend(prior_epoch_start) .load_state_from_backend()
.await .await
.map_err(|err| ClientCoreError::SurbStorageError { .map_err(|err| ClientCoreError::SurbStorageError {
source: Box::new(err), source: Box::new(err),
})?; })?;
let store_clone = mem_store.clone(); let store_clone = mem_store.clone();
let shutdown_token = shutdown_tracker.clone_shutdown_token(); spawn_future(async move {
shutdown_tracker.try_spawn_named( persistent_storage
async move { .flush_on_shutdown(store_clone, shutdown)
persistent_storage .await
.flush_on_shutdown(store_clone, shutdown_token) });
.await
},
"PersistentReplyStorage::flush_on_shutdown",
);
Ok(mem_store) Ok(mem_store)
} }
@@ -866,11 +711,11 @@ where
{ {
// if client keys do not exist already, create and persist them // if client keys do not exist already, create and persist them
if key_store.load_keys().await.is_err() { if key_store.load_keys().await.is_err() {
tracing::info!("could not find valid client keys - a new set will be generated"); info!("could not find valid client keys - a new set will be generated");
let mut rng = OsRng; let mut rng = OsRng;
let keys = if let Some(derivation_material) = derivation_material { let keys = if let Some(derivation_material) = derivation_material {
ClientKeys::from_master_key(&mut rng, &derivation_material) ClientKeys::from_master_key(&mut rng, &derivation_material)
.map_err(|_| ClientCoreError::HkdfDerivationError)? .map_err(|_| ClientCoreError::HkdfDerivationError {})?
} else { } else {
ClientKeys::generate_new(&mut rng) ClientKeys::generate_new(&mut rng)
}; };
@@ -880,77 +725,6 @@ where
setup_gateway(setup_method, key_store, details_store).await setup_gateway(setup_method, key_store, details_store).await
} }
fn construct_nym_api_client(
nym_api_urls: Option<&Vec<nym_network_defaults::ApiUrl>>,
config: &Config,
user_agent: Option<UserAgent>,
) -> Result<nym_http_api_client::Client, ClientCoreError> {
tracing::debug!(
"construct_nym_api_client called with nym_api_urls: {}",
nym_api_urls.is_some()
);
// If API URLs are provided, use new_with_fronted_urls() which handles domain fronting
if let Some(nym_api_urls) = nym_api_urls {
if nym_api_urls.is_empty() {
tracing::warn!("Provided nym_api_urls is empty, falling back to config endpoints");
} else {
tracing::info!(
"Building nym-api client from provided URLs (with domain fronting support): {} URLs",
nym_api_urls.len()
);
let mut builder =
nym_http_api_client::ClientBuilder::new_with_fronted_urls(nym_api_urls.clone())
.map_err(ClientCoreError::from)?
.with_retries(DEFAULT_NYM_API_RETRIES);
if let Some(user_agent) = user_agent {
builder = builder.with_user_agent(user_agent);
}
return builder.build().map_err(ClientCoreError::from);
}
}
// Fallback to basic client for backwards compatibility
tracing::debug!("Building basic nym-api HTTP client from config endpoints");
let mut nym_api_urls = config.get_nym_api_endpoints();
if nym_api_urls.is_empty() {
tracing::warn!("No API endpoints configured in config, this may cause issues");
}
nym_api_urls.shuffle(&mut thread_rng());
// Convert config URLs to ApiUrl format for consistency
let api_urls: Vec<nym_network_defaults::ApiUrl> = nym_api_urls
.into_iter()
.map(|url| nym_network_defaults::ApiUrl {
url: url.to_string(),
front_hosts: None,
})
.collect();
tracing::debug!("Using {} config API endpoints", api_urls.len());
let mut builder = nym_http_api_client::ClientBuilder::new_with_fronted_urls(api_urls)
.map_err(ClientCoreError::from)?
.with_retries(DEFAULT_NYM_API_RETRIES)
.with_bincode();
if let Some(user_agent) = user_agent {
builder = builder.with_user_agent(user_agent);
}
builder.build().map_err(ClientCoreError::from)
}
async fn determine_key_rotation_state(
client: &nym_http_api_client::Client,
) -> Result<KeyRotationConfig, ClientCoreError> {
Ok(client.get_key_rotation_info().await?.into())
}
pub async fn start_base(mut self) -> Result<BaseClient, ClientCoreError> pub async fn start_base(mut self) -> Result<BaseClient, ClientCoreError>
where where
S::ReplyStore: Send + Sync, S::ReplyStore: Send + Sync,
@@ -959,12 +733,7 @@ where
<S::CredentialStore as CredentialStorage>::StorageError: Send + Sync + 'static, <S::CredentialStore as CredentialStorage>::StorageError: Send + Sync + 'static,
<S::GatewaysDetailsStore as GatewaysDetailsStore>::StorageError: Sync + Send, <S::GatewaysDetailsStore as GatewaysDetailsStore>::StorageError: Sync + Send,
{ {
tracing::info!("Starting nym client"); info!("Starting nym client");
#[cfg(debug_assertions)]
#[cfg(target_arch = "wasm32")]
{
console_log!("Starting base Nym Client");
}
// derive (or load) client keys and gateway configuration // derive (or load) client keys and gateway configuration
let init_res = Self::initialise_keys_and_gateway( let init_res = Self::initialise_keys_and_gateway(
@@ -993,22 +762,17 @@ where
// channels responsible for controlling real messages // channels responsible for controlling real messages
let (input_sender, input_receiver) = tokio::sync::mpsc::channel::<InputMessage>(1); let (input_sender, input_receiver) = tokio::sync::mpsc::channel::<InputMessage>(1);
// channels responsible for event management
let (event_sender, event_receiver) = mpsc::unbounded();
// channels responsible for controlling ack messages // channels responsible for controlling ack messages
let (ack_sender, ack_receiver) = mpsc::unbounded(); let (ack_sender, ack_receiver) = mpsc::unbounded();
let shared_topology_accessor = let shared_topology_accessor =
TopologyAccessor::new(self.config.debug.topology.ignore_egress_epoch_role); TopologyAccessor::new(self.config.debug.topology.ignore_egress_epoch_role);
// Create a shutdown tracker for this client - either as a child of provided tracker // Shutdown notifier for signalling tasks to stop
// or get one from the registry let shutdown = self
let shutdown_tracker = match self.shutdown { .shutdown
Some(parent_tracker) => parent_tracker.clone(), .map(Into::<TaskHandle>::into)
None => nym_task::create_sdk_shutdown_tracker()?, .unwrap_or_default()
}; .name_if_unnamed("BaseNymClient");
Self::start_event_control(self.event_tx, event_receiver, &shutdown_tracker);
// channels responsible for dealing with reply-related fun // channels responsible for dealing with reply-related fun
let (reply_controller_sender, reply_controller_receiver) = let (reply_controller_sender, reply_controller_receiver) =
@@ -1025,18 +789,11 @@ where
.dkg_query_client .dkg_query_client
.map(|client| BandwidthController::new(credential_store, client)); .map(|client| BandwidthController::new(credential_store, client));
let nym_api_client = Self::construct_nym_api_client(
self.nym_api_urls.as_ref(),
&self.config,
self.user_agent.clone(),
)?;
let key_rotation_config = Self::determine_key_rotation_state(&nym_api_client).await?;
let topology_provider = Self::setup_topology_provider( let topology_provider = Self::setup_topology_provider(
self.custom_topology_provider.take(), self.custom_topology_provider.take(),
self.config.debug.topology, self.config.debug.topology,
self.config.get_nym_api_endpoints(), self.config.get_nym_api_endpoints(),
nym_api_client, self.user_agent.clone(),
); );
let stats_reporter = Self::start_statistics_control( let stats_reporter = Self::start_statistics_control(
@@ -1044,7 +801,7 @@ where
self.user_agent.clone(), self.user_agent.clone(),
generate_client_stats_id(*self_address.identity()), generate_client_stats_id(*self_address.identity()),
input_sender.clone(), input_sender.clone(),
&shutdown_tracker.clone(), shutdown.fork("statistics_control"),
); );
// needs to be started as the first thing to block if required waiting for the gateway // needs to be started as the first thing to block if required waiting for the gateway
@@ -1054,14 +811,14 @@ where
shared_topology_accessor.clone(), shared_topology_accessor.clone(),
self_address.gateway(), self_address.gateway(),
self.wait_for_gateway, self.wait_for_gateway,
&shutdown_tracker.clone(), shutdown.fork("topology_refresher"),
) )
.await?; .await?;
let gateway_packet_router = PacketRouter::new( let gateway_packet_router = PacketRouter::new(
ack_sender, ack_sender,
mixnet_messages_sender, mixnet_messages_sender,
shutdown_tracker.clone_shutdown_token(), shutdown.get_handle().named("gateway-packet-router"),
); );
let gateway_transceiver = Self::setup_gateway_transceiver( let gateway_transceiver = Self::setup_gateway_transceiver(
@@ -1074,15 +831,14 @@ where
stats_reporter.clone(), stats_reporter.clone(),
#[cfg(unix)] #[cfg(unix)]
self.connection_fd_callback, self.connection_fd_callback,
&shutdown_tracker.clone(), shutdown.fork("gateway_transceiver"),
) )
.await?; .await?;
let gateway_ws_fd = gateway_transceiver.ws_fd(); let gateway_ws_fd = gateway_transceiver.ws_fd();
let reply_storage = Self::setup_persistent_reply_storage( let reply_storage = Self::setup_persistent_reply_storage(
reply_storage_backend, reply_storage_backend,
key_rotation_config, shutdown.fork("persistent_reply_storage"),
&shutdown_tracker.clone(),
) )
.await?; .await?;
@@ -1092,8 +848,8 @@ where
mixnet_messages_receiver, mixnet_messages_receiver,
reply_storage.key_storage(), reply_storage.key_storage(),
reply_controller_sender.clone(), reply_controller_sender.clone(),
shutdown.fork("received_messages_buffer"),
stats_reporter.clone(), stats_reporter.clone(),
&shutdown_tracker.clone(),
); );
// The message_sender is the transmitter for any component generating sphinx packets // The message_sender is the transmitter for any component generating sphinx packets
@@ -1103,8 +859,7 @@ where
let (message_sender, client_request_sender) = Self::start_mix_traffic_controller( let (message_sender, client_request_sender) = Self::start_mix_traffic_controller(
gateway_transceiver, gateway_transceiver,
&shutdown_tracker.clone(), shutdown.fork("mix_traffic_controller"),
EventSender(event_sender),
); );
// Channels that the websocket listener can use to signal downstream to the real traffic // Channels that the websocket listener can use to signal downstream to the real traffic
@@ -1123,7 +878,6 @@ where
Self::start_real_traffic_controller( Self::start_real_traffic_controller(
controller_config, controller_config,
key_rotation_config,
shared_topology_accessor.clone(), shared_topology_accessor.clone(),
ack_receiver, ack_receiver,
input_receiver, input_receiver,
@@ -1133,8 +887,9 @@ where
reply_controller_receiver, reply_controller_receiver,
shared_lane_queue_lengths.clone(), shared_lane_queue_lengths.clone(),
client_connection_rx, client_connection_rx,
shutdown.fork("real_traffic_controller"),
self.config.debug.traffic.packet_type,
stats_reporter.clone(), stats_reporter.clone(),
&shutdown_tracker.clone(),
); );
if !self if !self
@@ -1150,19 +905,12 @@ where
shared_topology_accessor.clone(), shared_topology_accessor.clone(),
message_sender, message_sender,
stats_reporter.clone(), stats_reporter.clone(),
&shutdown_tracker.clone(), shutdown.fork("cover_traffic_stream"),
); );
} }
tracing::debug!("Core client startup finished!"); debug!("Core client startup finished!");
tracing::debug!("The address of this client is: {self_address}"); debug!("The address of this client is: {self_address}");
#[cfg(debug_assertions)]
#[cfg(target_arch = "wasm32")]
{
console_log!("Core client startup finished!");
console_log!("Rust::start_base: the address of this client is: {self_address}");
}
Ok(BaseClient { Ok(BaseClient {
address: self_address, address: self_address,
@@ -1171,7 +919,6 @@ where
client_input: ClientInput { client_input: ClientInput {
connection_command_sender: client_connection_tx, connection_command_sender: client_connection_tx,
input_sender, input_sender,
client_request_sender,
}, },
}, },
client_output: ClientOutputStatus::AwaitingConsumer { client_output: ClientOutputStatus::AwaitingConsumer {
@@ -1186,7 +933,8 @@ where
gateway_connection: GatewayConnection { gateway_ws_fd }, gateway_connection: GatewayConnection { gateway_ws_fd },
}, },
stats_reporter, stats_reporter,
shutdown_handle: shutdown_tracker, // The primary tracker for this client task_handle: shutdown,
client_request_sender,
forget_me: self.config.debug.forget_me, forget_me: self.config.debug.forget_me,
remember_me: self.config.debug.remember_me, remember_me: self.config.debug.remember_me,
}) })
@@ -1200,57 +948,8 @@ pub struct BaseClient {
pub client_output: ClientOutputStatus, pub client_output: ClientOutputStatus,
pub client_state: ClientState, pub client_state: ClientState,
pub stats_reporter: ClientStatsSender, pub stats_reporter: ClientStatsSender,
pub shutdown_handle: ShutdownTracker, pub client_request_sender: ClientRequestSender,
pub task_handle: TaskHandle,
pub forget_me: ForgetMe, pub forget_me: ForgetMe,
pub remember_me: RememberMe, pub remember_me: RememberMe,
} }
#[cfg(test)]
mod tests {
use super::*;
use nym_network_defaults::{ApiUrl, NymNetworkDetails};
#[test]
fn test_network_details_with_multiple_urls() {
// Verify that network details can be configured with multiple API URLs
let mut network_details = NymNetworkDetails::new_empty();
network_details.nym_api_urls = Some(vec![
ApiUrl {
url: "https://validator.nymtech.net/api/".to_string(),
front_hosts: None,
},
ApiUrl {
url: "https://nym-frontdoor.vercel.app/api/".to_string(),
front_hosts: Some(vec!["vercel.app".to_string(), "vercel.com".to_string()]),
},
]);
assert_eq!(network_details.nym_api_urls.as_ref().unwrap().len(), 2);
assert!(network_details.nym_api_urls.as_ref().unwrap()[1]
.front_hosts
.is_some());
}
#[test]
fn test_network_details_with_front_hosts() {
// Verify that ApiUrl can store domain fronting configuration
let api_url = ApiUrl {
url: "https://nym-frontdoor.vercel.app/api/".to_string(),
front_hosts: Some(vec!["vercel.app".to_string(), "vercel.com".to_string()]),
};
assert_eq!(api_url.url, "https://nym-frontdoor.vercel.app/api/");
assert_eq!(api_url.front_hosts.as_ref().unwrap().len(), 2);
assert!(api_url
.front_hosts
.as_ref()
.unwrap()
.contains(&"vercel.app".to_string()));
}
#[test]
fn test_default_nym_api_retries_constant() {
// Verify the retry constant is set correctly
assert_eq!(DEFAULT_NYM_API_RETRIES, 3);
}
}
@@ -1,30 +1,32 @@
// Copyright 2022-2024 - Nym Technologies SA <contact@nymtech.net> // Copyright 2022-2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::{ use crate::client::replies::reply_storage::{
client::replies::reply_storage::{fs_backend, CombinedReplyStorage, ReplyStorageBackend}, fs_backend, CombinedReplyStorage, ReplyStorageBackend,
config,
config::Config,
error::ClientCoreError,
}; };
use crate::config;
use crate::config::Config;
use crate::error::ClientCoreError;
use log::{error, info, trace};
use nym_bandwidth_controller::BandwidthController; use nym_bandwidth_controller::BandwidthController;
use nym_client_core_gateways_storage::OnDiskGatewaysDetails; use nym_client_core_gateways_storage::OnDiskGatewaysDetails;
use nym_credential_storage::storage::Storage as CredentialStorage; use nym_credential_storage::storage::Storage as CredentialStorage;
use nym_validator_client::{nyxd, QueryHttpRpcNyxdClient}; use nym_validator_client::nyxd;
use std::{io, path::Path}; use nym_validator_client::QueryHttpRpcNyxdClient;
use std::path::Path;
use std::{fs, io};
use time::OffsetDateTime; use time::OffsetDateTime;
use tracing::{error, info, trace};
use url::Url; use url::Url;
async fn setup_fresh_backend<P: AsRef<Path>>( async fn setup_fresh_backend<P: AsRef<Path>>(
db_path: P, db_path: P,
surb_config: &config::ReplySurbs, surb_config: &config::ReplySurbs,
) -> Result<fs_backend::Backend, ClientCoreError> { ) -> Result<fs_backend::Backend, ClientCoreError> {
info!("Creating fresh surb database"); info!("creating fresh surb database");
let mut storage_backend = match fs_backend::Backend::init(db_path).await { let mut storage_backend = match fs_backend::Backend::init(db_path).await {
Ok(backend) => backend, Ok(backend) => backend,
Err(err) => { Err(err) => {
error!("setup_fresh_backend: Failed to setup persistent storage backend for our reply needs: {err}"); error!("failed to setup persistent storage backend for our reply needs: {err}");
return Err(ClientCoreError::SurbStorageError { return Err(ClientCoreError::SurbStorageError {
source: Box::new(err), source: Box::new(err),
}); });
@@ -38,15 +40,14 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
surb_config.minimum_reply_surb_storage_threshold, surb_config.minimum_reply_surb_storage_threshold,
surb_config.maximum_reply_surb_storage_threshold, surb_config.maximum_reply_surb_storage_threshold,
); );
match storage_backend.init_fresh(&mem_store).await { storage_backend
Ok(()) => Ok(storage_backend), .init_fresh(&mem_store)
Err(err) => { .await
storage_backend.shutdown().await; .map_err(|err| ClientCoreError::SurbStorageError {
Err(ClientCoreError::SurbStorageError { source: Box::new(err),
source: Box::new(err), })?;
})
} Ok(storage_backend)
}
} }
// fn setup_inactive_backend(surb_config: &config::ReplySurbs) -> fs_backend::Backend { // fn setup_inactive_backend(surb_config: &config::ReplySurbs) -> fs_backend::Backend {
@@ -57,11 +58,12 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
// ) // )
// } // }
async fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> { fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> {
let db_path = db_path.as_ref(); let db_path = db_path.as_ref();
debug_assert!(db_path.exists()); debug_assert!(db_path.exists());
let now = OffsetDateTime::now_utc().unix_timestamp(); let now = OffsetDateTime::now_utc().unix_timestamp();
let suffix = format!("_{now}.corrupted"); let suffix = format!("_{now}.corrupted");
let new_extension = let new_extension =
@@ -70,15 +72,11 @@ async fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()
} else { } else {
suffix suffix
}; };
let renamed = db_path.with_extension(new_extension);
tokio::fs::rename(db_path, &renamed).await.inspect_err(|_| { let mut renamed = db_path.to_owned();
error!( renamed.set_extension(new_extension);
"Failed to rename corrupt database file: {} to {}",
db_path.display(), fs::rename(db_path, renamed)
renamed.display()
);
})
} }
pub async fn setup_fs_reply_surb_backend<P: AsRef<Path>>( pub async fn setup_fs_reply_surb_backend<P: AsRef<Path>>(
@@ -89,12 +87,13 @@ pub async fn setup_fs_reply_surb_backend<P: AsRef<Path>>(
// the existing one // the existing one
let db_path = db_path.as_ref(); let db_path = db_path.as_ref();
if db_path.exists() { if db_path.exists() {
info!("Loading existing surb database"); info!("loading existing surb database");
match fs_backend::Backend::try_load(db_path).await { match fs_backend::Backend::try_load(db_path, surb_config.fresh_sender_tags).await {
Ok(backend) => Ok(backend), Ok(backend) => Ok(backend),
Err(err) => { Err(err) => {
error!("setup_fs_reply_surb_backend: Failed to setup persistent storage backend for our reply needs: {err}. We're going to create a fresh database instead. This behaviour might change in the future"); error!("failed to setup persistent storage backend for our reply needs: {err}. We're going to create a fresh database instead. This behaviour might change in the future");
archive_corrupted_database(db_path).await?;
archive_corrupted_database(db_path)?;
setup_fresh_backend(db_path, surb_config).await setup_fresh_backend(db_path, surb_config).await
} }
} }
@@ -114,32 +113,41 @@ pub async fn setup_fs_gateways_storage<P: AsRef<Path>>(
}) })
} }
pub fn create_bandwidth_controller_with_urls<St: CredentialStorage>( pub fn create_bandwidth_controller<St: CredentialStorage>(
nyxd_url: Url,
storage: St,
) -> Result<BandwidthController<QueryHttpRpcNyxdClient, St>, ClientCoreError> {
let client = default_query_dkg_client(nyxd_url)?;
Ok(BandwidthController::new(storage, client))
}
pub fn default_query_dkg_client_from_config(
config: &Config, config: &Config,
) -> Result<QueryHttpRpcNyxdClient, ClientCoreError> { storage: St,
) -> BandwidthController<QueryHttpRpcNyxdClient, St> {
let nyxd_url = config let nyxd_url = config
.get_validator_endpoints() .get_validator_endpoints()
.pop() .pop()
.ok_or(ClientCoreError::RpcClientMissingUrl)?; .expect("No nyxd validator endpoint provided");
create_bandwidth_controller_with_urls(nyxd_url, storage)
}
pub fn create_bandwidth_controller_with_urls<St: CredentialStorage>(
nyxd_url: Url,
storage: St,
) -> BandwidthController<QueryHttpRpcNyxdClient, St> {
let client = default_query_dkg_client(nyxd_url);
BandwidthController::new(storage, client)
}
pub fn default_query_dkg_client_from_config(config: &Config) -> QueryHttpRpcNyxdClient {
let nyxd_url = config
.get_validator_endpoints()
.pop()
.expect("No nyxd validator endpoint provided");
default_query_dkg_client(nyxd_url) default_query_dkg_client(nyxd_url)
} }
pub fn default_query_dkg_client(nyxd_url: Url) -> Result<QueryHttpRpcNyxdClient, ClientCoreError> { pub fn default_query_dkg_client(nyxd_url: Url) -> QueryHttpRpcNyxdClient {
let details = nym_network_defaults::NymNetworkDetails::new_from_env(); let details = nym_network_defaults::NymNetworkDetails::new_from_env();
let client_config = nyxd::Config::try_from_nym_network_details(&details) let client_config = nyxd::Config::try_from_nym_network_details(&details)
.map_err(|source| ClientCoreError::InvalidNetworkDetails { source })?; .expect("failed to construct validator client config");
// overwrite env configuration with config URLs // overwrite env configuration with config URLs
QueryHttpRpcNyxdClient::connect(client_config, nyxd_url.as_str()) QueryHttpRpcNyxdClient::connect(client_config, nyxd_url.as_str())
.map_err(|source| ClientCoreError::RpcClientCreationFailure { source }) .expect("Could not construct query client")
} }
@@ -3,21 +3,22 @@
use crate::client::mix_traffic::BatchMixMessageSender; use crate::client::mix_traffic::BatchMixMessageSender;
use crate::client::topology_control::TopologyAccessor; use crate::client::topology_control::TopologyAccessor;
use crate::config; use crate::{config, spawn_future};
use futures::task::{Context, Poll}; use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt}; use futures::{Future, Stream, StreamExt};
use log::*;
use nym_sphinx::acknowledgements::AckKey; use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient; use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::cover::generate_loop_cover_packet; use nym_sphinx::cover::generate_loop_cover_packet;
use nym_sphinx::params::{PacketSize, PacketType}; use nym_sphinx::params::{PacketSize, PacketType};
use nym_sphinx::utils::sample_poisson_duration; use nym_sphinx::utils::sample_poisson_duration;
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender}; use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use nym_task::TaskClient;
use rand::{rngs::OsRng, CryptoRng, Rng}; use rand::{rngs::OsRng, CryptoRng, Rng};
use std::pin::Pin; use std::pin::Pin;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::Duration;
use tokio::sync::mpsc::error::TrySendError; use tokio::sync::mpsc::error::TrySendError;
use tracing::*;
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
use tokio::time::{sleep, Sleep}; use tokio::time::{sleep, Sleep};
@@ -68,6 +69,8 @@ where
packet_type: PacketType, packet_type: PacketType,
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
task_client: TaskClient,
} }
impl<R> Stream for LoopCoverTrafficStream<R> impl<R> Stream for LoopCoverTrafficStream<R>
@@ -114,6 +117,7 @@ impl LoopCoverTrafficStream<OsRng> {
traffic_config: config::Traffic, traffic_config: config::Traffic,
cover_config: config::CoverTraffic, cover_config: config::CoverTraffic,
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
task_client: TaskClient,
) -> Self { ) -> Self {
let rng = OsRng; let rng = OsRng;
@@ -133,6 +137,7 @@ impl LoopCoverTrafficStream<OsRng> {
use_legacy_sphinx_format: traffic_config.use_legacy_sphinx_format, use_legacy_sphinx_format: traffic_config.use_legacy_sphinx_format,
packet_type: traffic_config.packet_type, packet_type: traffic_config.packet_type,
stats_tx, stats_tx,
task_client,
} }
} }
@@ -205,10 +210,10 @@ impl LoopCoverTrafficStream<OsRng> {
TrySendError::Full(_) => { TrySendError::Full(_) => {
// This isn't a problem, if the channel is full means we're already sending the // This isn't a problem, if the channel is full means we're already sending the
// max amount of messages downstream can handle. // max amount of messages downstream can handle.
tracing::trace!("Failed to send cover message - channel full"); log::debug!("Failed to send cover message - channel full");
} }
TrySendError::Closed(_) => { TrySendError::Closed(_) => {
tracing::warn!("Failed to send cover message - channel closed"); log::warn!("Failed to send cover message - channel closed");
} }
} }
} else { } else {
@@ -225,24 +230,16 @@ impl LoopCoverTrafficStream<OsRng> {
// JS: due to identical logical structure to OutQueueControl::on_message(), this is also // JS: due to identical logical structure to OutQueueControl::on_message(), this is also
// presumably required to prevent bugs in the future. Exact reason is still unknown to me. // presumably required to prevent bugs in the future. Exact reason is still unknown to me.
// TODO: temporary and BAD workaround for wasm (we should find a way to yield here in wasm)
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
{ tokio::task::yield_now().await;
tokio::task::yield_now().await;
}
#[cfg(target_arch = "wasm32")]
{
tokio_with_wasm::task::yield_now().await;
}
} }
// it's fine if cover traffic stream task gets killed whilst processing next message pub fn start(mut self) {
#[allow(clippy::panic)]
pub async fn run(&mut self) {
if self.cover_traffic.disable_loop_cover_traffic_stream { if self.cover_traffic.disable_loop_cover_traffic_stream {
// we should have never got here in the first place - the task should have never been created to begin with // we should have never got here in the first place - the task should have never been created to begin with
// so panic and review the code that lead to this branch // so panic and review the code that lead to this branch
panic!("attempted to run LoopCoverTrafficStream while config explicitly disabled it.") panic!("attempted to start LoopCoverTrafficStream while config explicitly disabled it.")
} }
// we should set initial delay only when we actually start the stream // we should set initial delay only when we actually start the stream
@@ -252,11 +249,29 @@ impl LoopCoverTrafficStream<OsRng> {
); );
self.set_next_delay(sampled); self.set_next_delay(sampled);
while self.next().await.is_some() { let mut shutdown = self.task_client.fork("select");
self.on_new_message().await;
}
// this should never get triggered spawn_future(async move {
error!("cover traffic stream has been exhausted!") debug!("Started LoopCoverTrafficStream with graceful shutdown support");
while !shutdown.is_shutdown() {
tokio::select! {
biased;
_ = shutdown.recv() => {
log::trace!("LoopCoverTrafficStream: Received shutdown");
}
next = self.next() => {
if next.is_some() {
self.on_new_message().await;
} else {
log::trace!("LoopCoverTrafficStream: Stopping since channel closed");
break;
}
}
}
}
shutdown.recv_timeout().await;
log::debug!("LoopCoverTrafficStream: Exiting");
})
} }
} }
@@ -1,40 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use futures::StreamExt;
use crate::client::base_client::{EventReceiver, EventSender, MixnetClientEvent};
/// Launches and manages task events, propagating upwards what is not strictly internal.
pub(crate) struct EventControl {
parent_event_tx: Option<EventSender>,
children_event_rx: EventReceiver,
}
impl EventControl {
pub(crate) fn new(
parent_event_tx: Option<EventSender>,
children_event_rx: EventReceiver,
) -> Self {
EventControl {
parent_event_tx,
children_event_rx,
}
}
fn is_internal(event: MixnetClientEvent) -> bool {
match event {
MixnetClientEvent::Traffic(_) => false,
}
}
pub(crate) async fn run(mut self) {
while let Some(event) = self.children_event_rx.next().await {
if let Some(parent_event_tx) = &self.parent_event_tx {
if !Self::is_internal(event) {
parent_event_tx.send(event);
}
}
}
}
}
@@ -2,9 +2,9 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
#![allow(unused_imports)] #![allow(unused_imports)]
use std::time::Duration; use std::time::Duration;
#[cfg(target_arch = "wasm32")]
pub use wasmtimer::{std::Instant, tokio::*}; pub use wasmtimer::{std::Instant, tokio::*};
pub type IntervalStream = gloo_timers::future::IntervalStream; pub type IntervalStream = gloo_timers::future::IntervalStream;
@@ -135,9 +135,7 @@ impl InputMessage {
recipient_tag, recipient_tag,
data, data,
lane, lane,
// \/ set it to SOME sane default so that if we run out of surbs and constantly max_retransmissions: None,
// fail to request more, we wouldn't be stuck in limbo
max_retransmissions: Some(10),
}; };
if let Some(packet_type) = packet_type { if let Some(packet_type) = packet_type {
InputMessage::new_wrapper(message, packet_type) InputMessage::new_wrapper(message, packet_type)
+98 -110
View File
@@ -1,14 +1,13 @@
// Copyright 2021 - Nym Technologies SA <contact@nymtech.net> // Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::client::{ use crate::client::mix_traffic::transceiver::GatewayTransceiver;
base_client::{EventSender, MixnetClientEvent}, use crate::error::ClientCoreError;
mix_traffic::transceiver::GatewayTransceiver, use crate::spawn_future;
}; use log::*;
use nym_gateway_requests::ClientRequest; use nym_gateway_requests::ClientRequest;
use nym_sphinx::forwarding::packet::MixPacket; use nym_sphinx::forwarding::packet::MixPacket;
use nym_task::ShutdownToken; use nym_task::TaskClient;
use tracing::*;
use transceiver::ErasedGatewayError; use transceiver::ErasedGatewayError;
pub type BatchMixMessageSender = tokio::sync::mpsc::Sender<Vec<MixPacket>>; pub type BatchMixMessageSender = tokio::sync::mpsc::Sender<Vec<MixPacket>>;
@@ -20,41 +19,33 @@ pub mod transceiver;
// We remind ourselves that 32 x 32kb = 1024kb, a reasonable size for a network buffer. // We remind ourselves that 32 x 32kb = 1024kb, a reasonable size for a network buffer.
pub const MIX_MESSAGE_RECEIVER_BUFFER_SIZE: usize = 32; pub const MIX_MESSAGE_RECEIVER_BUFFER_SIZE: usize = 32;
const MAX_FAILURE_COUNT: usize = 100;
/// Reduced from 100 to 20 to fail fast (~1-2 seconds instead of ~6 seconds).
/// If we can't send 20 packets in a row, the gateway is unreachable.
const MAX_FAILURE_COUNT: usize = 20;
// that's also disgusting. // that's also disgusting.
pub struct Empty; pub struct Empty;
#[derive(Clone, Copy, Debug)]
pub enum MixTrafficEvent {
FailedSendingSphinx,
}
pub struct MixTrafficController { pub struct MixTrafficController {
gateway_transceiver: Box<dyn GatewayTransceiver + Send>, gateway_transceiver: Box<dyn GatewayTransceiver + Send>,
mix_tx: BatchMixMessageSender,
mix_rx: BatchMixMessageReceiver, mix_rx: BatchMixMessageReceiver,
client_rx: ClientRequestReceiver, client_rx: ClientRequestReceiver,
client_tx: ClientRequestSender,
// TODO: this is temporary work-around. // TODO: this is temporary work-around.
// in long run `gateway_client` will be moved away from `MixTrafficController` anyway. // in long run `gateway_client` will be moved away from `MixTrafficController` anyway.
consecutive_gateway_failure_count: usize, consecutive_gateway_failure_count: usize,
shutdown_token: ShutdownToken, task_client: TaskClient,
event_tx: EventSender,
} }
impl MixTrafficController { impl MixTrafficController {
pub fn new<T>( pub fn new<T>(
gateway_transceiver: T, gateway_transceiver: T,
shutdown_token: ShutdownToken, task_client: TaskClient,
event_tx: EventSender, ) -> (
) -> MixTrafficController MixTrafficController,
BatchMixMessageSender,
ClientRequestSender,
)
where where
T: GatewayTransceiver + Send + 'static, T: GatewayTransceiver + Send + 'static,
{ {
@@ -63,32 +54,41 @@ impl MixTrafficController {
let (client_sender, client_receiver) = tokio::sync::mpsc::channel(8); let (client_sender, client_receiver) = tokio::sync::mpsc::channel(8);
MixTrafficController { (
gateway_transceiver: Box::new(gateway_transceiver), MixTrafficController {
mix_tx: message_sender, gateway_transceiver: Box::new(gateway_transceiver),
mix_rx: message_receiver, mix_rx: message_receiver,
client_rx: client_receiver, client_rx: client_receiver,
client_tx: client_sender, consecutive_gateway_failure_count: 0,
consecutive_gateway_failure_count: 0, task_client,
shutdown_token, },
event_tx, message_sender,
} client_sender,
)
} }
pub fn new_dynamic( pub fn new_dynamic(
gateway_transceiver: Box<dyn GatewayTransceiver + Send>, gateway_transceiver: Box<dyn GatewayTransceiver + Send>,
shutdown_token: ShutdownToken, task_client: TaskClient,
event_tx: EventSender, ) -> (
) -> MixTrafficController { MixTrafficController,
Self::new(gateway_transceiver, shutdown_token, event_tx) BatchMixMessageSender,
} ClientRequestSender,
) {
pub fn client_tx(&self) -> ClientRequestSender { let (message_sender, message_receiver) =
self.client_tx.clone() tokio::sync::mpsc::channel(MIX_MESSAGE_RECEIVER_BUFFER_SIZE);
} let (client_sender, client_receiver) = tokio::sync::mpsc::channel(8);
(
pub fn mix_tx(&self) -> BatchMixMessageSender { MixTrafficController {
self.mix_tx.clone() gateway_transceiver,
mix_rx: message_receiver,
client_rx: client_receiver,
consecutive_gateway_failure_count: 0,
task_client,
},
message_sender,
client_sender,
)
} }
async fn on_messages( async fn on_messages(
@@ -96,84 +96,72 @@ impl MixTrafficController {
mut mix_packets: Vec<MixPacket>, mut mix_packets: Vec<MixPacket>,
) -> Result<(), ErasedGatewayError> { ) -> Result<(), ErasedGatewayError> {
debug_assert!(!mix_packets.is_empty()); debug_assert!(!mix_packets.is_empty());
let send_future = if mix_packets.len() == 1 {
// SAFETY: we just checked we have one packet let result = if mix_packets.len() == 1 {
#[allow(clippy::unwrap_used)]
let mix_packet = mix_packets.pop().unwrap(); let mix_packet = mix_packets.pop().unwrap();
self.gateway_transceiver.send_mix_packet(mix_packet) self.gateway_transceiver.send_mix_packet(mix_packet).await
} else { } else {
self.gateway_transceiver.batch_send_mix_packets(mix_packets) self.gateway_transceiver
.batch_send_mix_packets(mix_packets)
.await
}; };
tokio::select! { if result.is_err() {
biased; self.consecutive_gateway_failure_count += 1;
_ = self.shutdown_token.cancelled() => { } else {
trace!("received shutdown while handling messages"); trace!("We *might* have managed to forward sphinx packet(s) to the gateway!");
Ok(()) self.consecutive_gateway_failure_count = 0;
}
result = send_future => {
if result.is_err() {
self.consecutive_gateway_failure_count += 1;
} else {
trace!("We *might* have managed to forward sphinx packet(s) to the gateway!");
self.consecutive_gateway_failure_count = 0;
}
result
}
} }
result
} }
async fn on_client_request(&mut self, client_request: ClientRequest) { pub fn start(mut self) {
tokio::select! { spawn_future(async move {
biased; debug!("Started MixTrafficController with graceful shutdown support");
_ = self.shutdown_token.cancelled() => {
trace!("received shutdown while handling client request");
}
result = self.gateway_transceiver.send_client_request(client_request) => {
if let Err(err) = result {
error!("Failed to send client request: {err}")
}
}
}
}
pub async fn run(&mut self) { while !self.task_client.is_shutdown() {
debug!("Started MixTrafficController with graceful shutdown support"); tokio::select! {
loop { mix_packets = self.mix_rx.recv() => match mix_packets {
tokio::select! { Some(mix_packets) => {
biased; if let Err(err) = self.on_messages(mix_packets).await {
_ = self.shutdown_token.cancelled() => { error!("Failed to send sphinx packet(s) to the gateway: {err}");
trace!("MixTrafficController: Received shutdown"); if self.consecutive_gateway_failure_count == MAX_FAILURE_COUNT {
break; // Disconnect from the gateway. If we should try to re-connect
} // is handled at a higher layer.
// mix_rx should never error out as we're holding one instance of the sender error!("Failed to send sphinx packet to the gateway {MAX_FAILURE_COUNT} times in a row - assuming the gateway is dead");
// Do we need to handle the embedded mixnet client case
Some(mix_packets) = self.mix_rx.recv() => { // separately?
if let Err(err) = self.on_messages(mix_packets).await { self.task_client.send_we_stopped(Box::new(ClientCoreError::GatewayFailedToForwardMessages));
error!("Failed to send sphinx packet(s) to the gateway: {err}"); break;
if self.consecutive_gateway_failure_count == MAX_FAILURE_COUNT { }
// Disconnect from the gateway. If we should try to re-connect }
// is handled at a higher layer. },
error!("Failed to send sphinx packet to the gateway {MAX_FAILURE_COUNT} times in a row - assuming the gateway is dead"); None => {
// Do we need to handle the embedded mixnet client case log::trace!("MixTrafficController: Stopping since channel closed");
// separately?
self.event_tx.send(MixnetClientEvent::Traffic(MixTrafficEvent::FailedSendingSphinx));
// IMO it shouldn't be signalled from there but it is how it is
// TODO : report the failure upwards and shutdown from upwards
// Gateway is dead, we have to shut down currently
error!("Signalling shutdown from the MixTrafficController");
self.shutdown_token.cancel();
break; break;
} }
},
client_request = self.client_rx.recv() => match client_request {
Some(client_request) => {
match self.gateway_transceiver.send_client_request(client_request).await {
Ok(_) => (),
Err(e) => error!("Failed to send client request: {}", e),
};
},
None => {
log::trace!("MixTrafficController, client request channel closed");
}
},
_ = self.task_client.recv() => {
log::trace!("MixTrafficController: Received shutdown");
break;
} }
},
// client_rx should never error out as we're holding one instance of the sender
Some(client_request) = self.client_rx.recv() => {
self.on_client_request(client_request).await;
} }
} }
} self.task_client.recv_timeout().await;
debug!("MixTrafficController: Exiting");
log::debug!("MixTrafficController: Exiting");
});
} }
} }
@@ -2,6 +2,7 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use async_trait::async_trait; use async_trait::async_trait;
use log::{debug, error};
use nym_credential_storage::storage::Storage as CredentialStorage; use nym_credential_storage::storage::Storage as CredentialStorage;
use nym_crypto::asymmetric::ed25519; use nym_crypto::asymmetric::ed25519;
use nym_gateway_client::error::GatewayClientError; use nym_gateway_client::error::GatewayClientError;
@@ -13,7 +14,6 @@ use nym_validator_client::nyxd::contract_traits::DkgQueryClient;
use std::fmt::Debug; use std::fmt::Debug;
use std::os::raw::c_int as RawFd; use std::os::raw::c_int as RawFd;
use thiserror::Error; use thiserror::Error;
use tracing::{debug, error};
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
use futures::channel::oneshot; use futures::channel::oneshot;
@@ -27,7 +27,7 @@ fn erase_err<E: std::error::Error + Send + Sync + 'static>(err: E) -> ErasedGate
ErasedGatewayError(Box::new(err)) ErasedGatewayError(Box::new(err))
} }
/// This combines the functionalities of being able to send and receive mix packets. /// This combines combines the functionalities of being able to send and receive mix packets.
#[async_trait] #[async_trait]
pub trait GatewayTransceiver: GatewaySender + GatewayReceiver { pub trait GatewayTransceiver: GatewaySender + GatewayReceiver {
fn gateway_identity(&self) -> ed25519::PublicKey; fn gateway_identity(&self) -> ed25519::PublicKey;
@@ -87,7 +87,7 @@ impl<G: GatewayTransceiver + ?Sized + Send> GatewayTransceiver for Box<G> {
message: ClientRequest, message: ClientRequest,
) -> Result<(), GatewayClientError> { ) -> Result<(), GatewayClientError> {
let _ = (**self).send_client_request(message.clone()).await?; let _ = (**self).send_client_request(message.clone()).await?;
tracing::debug!("Sent client request: {:?}", message); log::debug!("Sent client request: {:?}", message);
Ok(()) Ok(())
} }
} }
@@ -269,8 +269,6 @@ pub struct MockGateway {
} }
impl Default for MockGateway { impl Default for MockGateway {
// test code
#[allow(clippy::unwrap_used)]
fn default() -> Self { fn default() -> Self {
MockGateway { MockGateway {
dummy_identity: "3ebjp1Fb9hdcS1AR6AZihgeJiMHkB5jjJUsvqNnfQwU7" dummy_identity: "3ebjp1Fb9hdcS1AR6AZihgeJiMHkB5jjJUsvqNnfQwU7"
-1
View File
@@ -3,7 +3,6 @@
pub mod base_client; pub mod base_client;
pub mod cover_traffic_stream; pub mod cover_traffic_stream;
pub(crate) mod event_control;
pub(crate) mod helpers; pub(crate) mod helpers;
pub mod inbound_messages; pub mod inbound_messages;
pub mod key_manager; pub mod key_manager;
@@ -5,22 +5,23 @@ use super::action_controller::{AckActionSender, Action};
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender}; use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use futures::StreamExt; use futures::StreamExt;
use log::*;
use nym_gateway_client::AcknowledgementReceiver; use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::{ use nym_sphinx::{
acknowledgements::{identifier::recover_identifier, AckKey}, acknowledgements::{identifier::recover_identifier, AckKey},
chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID}, chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID},
}; };
use nym_task::ShutdownToken; use nym_task::TaskClient;
use std::sync::Arc; use std::sync::Arc;
use tracing::*;
/// Module responsible for listening for any data resembling acknowledgements from the network /// Module responsible for listening for any data resembling acknowledgements from the network
/// and firing actions to remove them from the 'Pending' state. /// and firing actions to remove them from the 'Pending' state.
pub(crate) struct AcknowledgementListener { pub(super) struct AcknowledgementListener {
ack_key: Arc<AckKey>, ack_key: Arc<AckKey>,
ack_receiver: AcknowledgementReceiver, ack_receiver: AcknowledgementReceiver,
action_sender: AckActionSender, action_sender: AckActionSender,
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
task_client: TaskClient,
} }
impl AcknowledgementListener { impl AcknowledgementListener {
@@ -29,12 +30,14 @@ impl AcknowledgementListener {
ack_receiver: AcknowledgementReceiver, ack_receiver: AcknowledgementReceiver,
action_sender: AckActionSender, action_sender: AckActionSender,
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
task_client: TaskClient,
) -> Self { ) -> Self {
AcknowledgementListener { AcknowledgementListener {
ack_key, ack_key,
ack_receiver, ack_receiver,
action_sender, action_sender,
stats_tx, stats_tx,
task_client,
} }
} }
@@ -62,12 +65,17 @@ impl AcknowledgementListener {
return; return;
} }
trace!("Received {frag_id} from the mix network"); trace!("Received {} from the mix network", frag_id);
self.stats_tx self.stats_tx
.report(PacketStatisticsEvent::RealAckReceived(ack_content.len()).into()); .report(PacketStatisticsEvent::RealAckReceived(ack_content.len()).into());
let _ = self if let Err(err) = self
.action_sender .action_sender
.unbounded_send(Action::new_remove(frag_id)); .unbounded_send(Action::new_remove(frag_id))
{
if !self.task_client.is_shutdown_poll() {
error!("Failed to send remove action to action controller: {err}");
}
}
} }
async fn handle_ack_receiver_item(&mut self, item: Vec<Vec<u8>>) { async fn handle_ack_receiver_item(&mut self, item: Vec<Vec<u8>>) {
@@ -77,26 +85,24 @@ impl AcknowledgementListener {
} }
} }
pub(crate) async fn run(&mut self, shutdown_token: ShutdownToken) { pub(super) async fn run(&mut self) {
debug!("Started AcknowledgementListener with graceful shutdown support"); debug!("Started AcknowledgementListener with graceful shutdown support");
loop { while !self.task_client.is_shutdown() {
tokio::select! { tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("AcknowledgementListener: Received shutdown");
break;
}
acks = self.ack_receiver.next() => match acks { acks = self.ack_receiver.next() => match acks {
Some(acks) => self.handle_ack_receiver_item(acks).await, Some(acks) => self.handle_ack_receiver_item(acks).await,
None => { None => {
tracing::trace!("AcknowledgementListener: Stopping since channel closed"); log::trace!("AcknowledgementListener: Stopping since channel closed");
break; break;
} }
}, },
_ = self.task_client.recv() => {
log::trace!("AcknowledgementListener: Received shutdown");
}
} }
} }
tracing::debug!("AcknowledgementListener: Exiting"); self.task_client.recv_timeout().await;
log::debug!("AcknowledgementListener: Exiting");
} }
} }
@@ -5,14 +5,14 @@ use super::PendingAcknowledgement;
use crate::client::real_messages_control::acknowledgement_control::RetransmissionRequestSender; use crate::client::real_messages_control::acknowledgement_control::RetransmissionRequestSender;
use futures::channel::mpsc; use futures::channel::mpsc;
use futures::StreamExt; use futures::StreamExt;
use log::*;
use nym_nonexhaustive_delayqueue::{Expired, NonExhaustiveDelayQueue, QueueKey}; use nym_nonexhaustive_delayqueue::{Expired, NonExhaustiveDelayQueue, QueueKey};
use nym_sphinx::chunking::fragment::FragmentIdentifier; use nym_sphinx::chunking::fragment::FragmentIdentifier;
use nym_sphinx::Delay as SphinxDelay; use nym_sphinx::Delay as SphinxDelay;
use nym_task::ShutdownToken; use nym_task::TaskClient;
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::Duration;
use tracing::*;
pub(crate) type AckActionSender = mpsc::UnboundedSender<Action>; pub(crate) type AckActionSender = mpsc::UnboundedSender<Action>;
pub(crate) type AckActionReceiver = mpsc::UnboundedReceiver<Action>; pub(crate) type AckActionReceiver = mpsc::UnboundedReceiver<Action>;
@@ -82,7 +82,7 @@ impl Config {
} }
} }
pub(crate) struct ActionController { pub(super) struct ActionController {
/// Configurable parameters of the `ActionController` /// Configurable parameters of the `ActionController`
config: Config, config: Config,
@@ -102,6 +102,8 @@ pub(crate) struct ActionController {
/// Channel for notifying `RetransmissionRequestListener` about expired acknowledgements. /// Channel for notifying `RetransmissionRequestListener` about expired acknowledgements.
retransmission_sender: RetransmissionRequestSender, retransmission_sender: RetransmissionRequestSender,
task_client: TaskClient,
} }
impl ActionController { impl ActionController {
@@ -109,6 +111,7 @@ impl ActionController {
config: Config, config: Config,
retransmission_sender: RetransmissionRequestSender, retransmission_sender: RetransmissionRequestSender,
incoming_actions: AckActionReceiver, incoming_actions: AckActionReceiver,
task_client: TaskClient,
) -> Self { ) -> Self {
ActionController { ActionController {
config, config,
@@ -116,13 +119,14 @@ impl ActionController {
pending_acks_timers: NonExhaustiveDelayQueue::new(), pending_acks_timers: NonExhaustiveDelayQueue::new(),
incoming_actions, incoming_actions,
retransmission_sender, retransmission_sender,
task_client,
} }
} }
fn handle_insert(&mut self, pending_acks: Vec<PendingAcknowledgement>) { fn handle_insert(&mut self, pending_acks: Vec<PendingAcknowledgement>) {
for pending_ack in pending_acks { for pending_ack in pending_acks {
let frag_id = pending_ack.message_chunk.fragment_identifier(); let frag_id = pending_ack.message_chunk.fragment_identifier();
trace!("{frag_id} is inserted"); trace!("{} is inserted", frag_id);
if self if self
.pending_acks_data .pending_acks_data
@@ -157,16 +161,22 @@ impl ActionController {
let new_queue_key = self.pending_acks_timers.insert(frag_id, timeout); let new_queue_key = self.pending_acks_timers.insert(frag_id, timeout);
*queue_key = Some(new_queue_key) *queue_key = Some(new_queue_key)
} else { } else {
debug!("Tried to START TIMER on pending ack that is already gone! - {frag_id}"); debug!(
"Tried to START TIMER on pending ack that is already gone! - {}",
frag_id
);
} }
} }
fn handle_remove(&mut self, frag_id: FragmentIdentifier) { fn handle_remove(&mut self, frag_id: FragmentIdentifier) {
trace!("{frag_id} is getting removed"); trace!("{} is getting removed", frag_id);
match self.pending_acks_data.remove(&frag_id) { match self.pending_acks_data.remove(&frag_id) {
None => { None => {
debug!("Tried to REMOVE pending ack that is already gone! - {frag_id}"); debug!(
"Tried to REMOVE pending ack that is already gone! - {}",
frag_id
);
} }
Some((_, queue_key)) => { Some((_, queue_key)) => {
if let Some(queue_key) = queue_key { if let Some(queue_key) = queue_key {
@@ -178,7 +188,10 @@ impl ActionController {
} else { } else {
// I'm not 100% sure if having a `None` key is even possible here // I'm not 100% sure if having a `None` key is even possible here
// (REMOVE would have to be called before START TIMER), // (REMOVE would have to be called before START TIMER),
debug!("Tried to REMOVE pending ack without TIMER active - {frag_id}"); debug!(
"Tried to REMOVE pending ack without TIMER active - {}",
frag_id
);
} }
} }
} }
@@ -187,26 +200,27 @@ impl ActionController {
// initiated basically as a first step of retransmission. At first data has its delay updated // initiated basically as a first step of retransmission. At first data has its delay updated
// (as new sphinx packet was created with new expected delivery time) // (as new sphinx packet was created with new expected delivery time)
fn handle_update_pending_ack(&mut self, frag_id: FragmentIdentifier, delay: SphinxDelay) { fn handle_update_pending_ack(&mut self, frag_id: FragmentIdentifier, delay: SphinxDelay) {
trace!("{frag_id} is updating its delay"); trace!("{} is updating its delay", frag_id);
// TODO: is it possible to solve this without either locking or temporarily removing the value? // TODO: is it possible to solve this without either locking or temporarily removing the value?
if let Some((pending_ack_data, queue_key)) = self.pending_acks_data.remove(&frag_id) { if let Some((pending_ack_data, queue_key)) = self.pending_acks_data.remove(&frag_id) {
// SAFETY: this Action is triggered by `RetransmissionRequestListener` (for 'normal' packets) // this Action is triggered by `RetransmissionRequestListener` (for 'normal' packets)
// or `ReplyController` (for 'reply' packets) which held the other potential // or `ReplyController` (for 'reply' packets) which held the other potential
// reference to this Arc. HOWEVER, before the Action was pushed onto the queue, the reference // reference to this Arc. HOWEVER, before the Action was pushed onto the queue, the reference
// was dropped hence this unwrap is safe. // was dropped hence this unwrap is safe.
#[allow(clippy::unwrap_used)]
let mut inner_data = Arc::try_unwrap(pending_ack_data).unwrap(); let mut inner_data = Arc::try_unwrap(pending_ack_data).unwrap();
inner_data.update_retransmitted(delay); inner_data.update_retransmitted(delay);
self.pending_acks_data self.pending_acks_data
.insert(frag_id, (Arc::new(inner_data), queue_key)); .insert(frag_id, (Arc::new(inner_data), queue_key));
} else { } else {
debug!("Tried to UPDATE TIMER on pending ack that is already gone! - {frag_id}"); debug!(
"Tried to UPDATE TIMER on pending ack that is already gone! - {}",
frag_id
);
} }
} }
// note: when the entry expires it's automatically removed from pending_acks_timers // note: when the entry expires it's automatically removed from pending_acks_timers
#[allow(clippy::panic)]
fn handle_expired_ack_timer(&mut self, expired_ack: Expired<FragmentIdentifier>) { fn handle_expired_ack_timer(&mut self, expired_ack: Expired<FragmentIdentifier>) {
let frag_id = expired_ack.into_inner(); let frag_id = expired_ack.into_inner();
@@ -222,9 +236,14 @@ impl ActionController {
// downgrading an arc and then upgrading vs cloning is difference of 30ns vs 15ns // downgrading an arc and then upgrading vs cloning is difference of 30ns vs 15ns
// so it's literally a NO difference while it might prevent us from unnecessarily // so it's literally a NO difference while it might prevent us from unnecessarily
// resending data (in maybe 1 in 1 million cases, but it's something) // resending data (in maybe 1 in 1 million cases, but it's something)
let _ = self if let Err(err) = self
.retransmission_sender .retransmission_sender
.unbounded_send(Arc::downgrade(pending_ack_data)); .unbounded_send(Arc::downgrade(pending_ack_data))
{
if !self.task_client.is_shutdown_poll() {
log::error!("Failed to send pending ack for retransmission: {err}");
}
}
} else { } else {
// this shouldn't cause any issues but shouldn't have happened to begin with! // this shouldn't cause any issues but shouldn't have happened to begin with!
error!("An already removed pending ack has expired") error!("An already removed pending ack has expired")
@@ -242,20 +261,15 @@ impl ActionController {
} }
} }
pub(crate) async fn run(&mut self, shutdown_token: ShutdownToken) { pub(super) async fn run(&mut self) {
debug!("Started ActionController with graceful shutdown support"); debug!("Started ActionController with graceful shutdown support");
loop { while !self.task_client.is_shutdown() {
tokio::select! { tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("ActionController: Received shutdown");
break;
}
action = self.incoming_actions.next() => match action { action = self.incoming_actions.next() => match action {
Some(action) => self.process_action(action), Some(action) => self.process_action(action),
None => { None => {
tracing::trace!( log::trace!(
"ActionController: Stopping since incoming actions channel closed" "ActionController: Stopping since incoming actions channel closed"
); );
break; break;
@@ -264,12 +278,17 @@ impl ActionController {
expired_ack = self.pending_acks_timers.next() => match expired_ack { expired_ack = self.pending_acks_timers.next() => match expired_ack {
Some(expired_ack) => self.handle_expired_ack_timer(expired_ack), Some(expired_ack) => self.handle_expired_ack_timer(expired_ack),
None => { None => {
tracing::trace!("ActionController: Stopping since ack channel closed"); log::trace!("ActionController: Stopping since ack channel closed");
break; break;
} }
}, },
_ = self.task_client.recv() => {
log::trace!("ActionController: Received shutdown");
break;
}
} }
} }
tracing::debug!("ActionController: Exiting"); self.task_client.recv_timeout().await;
log::debug!("ActionController: Exiting");
} }
} }
@@ -5,25 +5,26 @@ use crate::client::inbound_messages::{InputMessage, InputMessageReceiver};
use crate::client::real_messages_control::message_handler::MessageHandler; use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::real_messages_control::real_traffic_stream::RealMessage; use crate::client::real_messages_control::real_traffic_stream::RealMessage;
use crate::client::replies::reply_controller::ReplyControllerSender; use crate::client::replies::reply_controller::ReplyControllerSender;
use log::*;
use nym_sphinx::addressing::clients::Recipient; use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag; use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::forwarding::packet::MixPacket; use nym_sphinx::forwarding::packet::MixPacket;
use nym_sphinx::params::PacketType; use nym_sphinx::params::PacketType;
use nym_task::connections::TransmissionLane; use nym_task::connections::TransmissionLane;
use nym_task::ShutdownToken; use nym_task::TaskClient;
use rand::{CryptoRng, Rng}; use rand::{CryptoRng, Rng};
use tracing::*;
/// Module responsible for dealing with the received messages: splitting them, creating acknowledgements, /// Module responsible for dealing with the received messages: splitting them, creating acknowledgements,
/// putting everything into sphinx packets, etc. /// putting everything into sphinx packets, etc.
/// It also makes an initial sending attempt for said messages. /// It also makes an initial sending attempt for said messages.
pub(crate) struct InputMessageListener<R> pub(super) struct InputMessageListener<R>
where where
R: CryptoRng + Rng, R: CryptoRng + Rng,
{ {
input_receiver: InputMessageReceiver, input_receiver: InputMessageReceiver,
message_handler: MessageHandler<R>, message_handler: MessageHandler<R>,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
task_client: TaskClient,
} }
impl<R> InputMessageListener<R> impl<R> InputMessageListener<R>
@@ -37,11 +38,13 @@ where
input_receiver: InputMessageReceiver, input_receiver: InputMessageReceiver,
message_handler: MessageHandler<R>, message_handler: MessageHandler<R>,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
task_client: TaskClient,
) -> Self { ) -> Self {
InputMessageListener { InputMessageListener {
input_receiver, input_receiver,
message_handler, message_handler,
reply_controller_sender, reply_controller_sender,
task_client,
} }
} }
@@ -65,9 +68,14 @@ where
max_retransmissions: Option<u32>, max_retransmissions: Option<u32>,
) { ) {
// offload reply handling to the dedicated task // offload reply handling to the dedicated task
let _ = if let Err(err) =
self.reply_controller_sender self.reply_controller_sender
.send_reply(recipient_tag, data, lane, max_retransmissions); .send_reply(recipient_tag, data, lane, max_retransmissions)
{
if !self.task_client.is_shutdown_poll() {
error!("failed to send a reply - {err}");
}
}
} }
async fn handle_plain_message( async fn handle_plain_message(
@@ -112,7 +120,6 @@ where
} }
} }
#[allow(clippy::panic)]
async fn on_input_message(&mut self, msg: InputMessage) { async fn on_input_message(&mut self, msg: InputMessage) {
match msg { match msg {
InputMessage::Regular { InputMessage::Regular {
@@ -206,35 +213,31 @@ where
self.handle_premade_packets(msgs, lane).await self.handle_premade_packets(msgs, lane).await
} }
// MessageWrappers can't be nested // MessageWrappers can't be nested
InputMessage::MessageWrapper { .. } => { InputMessage::MessageWrapper { .. } => unimplemented!(),
panic!("attempted to use nested MessageWrapper")
}
}, },
}; };
} }
pub(crate) async fn run(&mut self, shutdown_token: ShutdownToken) { pub(super) async fn run(&mut self) {
debug!("Started InputMessageListener with graceful shutdown support"); debug!("Started InputMessageListener with graceful shutdown support");
loop { while !self.task_client.is_shutdown() {
tokio::select! { tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("InputMessageListener: Received shutdown");
break;
}
input_msg = self.input_receiver.recv() => match input_msg { input_msg = self.input_receiver.recv() => match input_msg {
Some(input_msg) => { Some(input_msg) => {
self.on_input_message(input_msg).await; self.on_input_message(input_msg).await;
}, },
None => { None => {
tracing::trace!("InputMessageListener: Stopping since channel closed"); log::trace!("InputMessageListener: Stopping since channel closed");
break; break;
} }
}, },
_ = self.task_client.recv() => {
log::trace!("InputMessageListener: Received shutdown");
}
} }
} }
tracing::debug!("InputMessageListener: Exiting"); self.task_client.recv_timeout().await;
log::debug!("InputMessageListener: Exiting");
} }
} }
@@ -10,8 +10,10 @@ use self::{
use crate::client::inbound_messages::InputMessageReceiver; use crate::client::inbound_messages::InputMessageReceiver;
use crate::client::real_messages_control::message_handler::MessageHandler; use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::replies::reply_controller::ReplyControllerSender; use crate::client::replies::reply_controller::ReplyControllerSender;
use crate::spawn_future;
use action_controller::AckActionReceiver; use action_controller::AckActionReceiver;
use futures::channel::mpsc; use futures::channel::mpsc;
use log::*;
use nym_gateway_client::AcknowledgementReceiver; use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag; use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::params::{PacketSize, PacketType}; use nym_sphinx::params::{PacketSize, PacketType};
@@ -22,6 +24,7 @@ use nym_sphinx::{
Delay as SphinxDelay, Delay as SphinxDelay,
}; };
use nym_statistics_common::clients::ClientStatsSender; use nym_statistics_common::clients::ClientStatsSender;
use nym_task::TaskClient;
use rand::{CryptoRng, Rng}; use rand::{CryptoRng, Rng};
use std::{ use std::{
sync::{Arc, Weak}, sync::{Arc, Weak},
@@ -187,9 +190,6 @@ pub(super) struct Config {
/// Predefined packet size used for the encapsulated messages. /// Predefined packet size used for the encapsulated messages.
packet_size: PacketSize, packet_size: PacketSize,
/// Type of packets used for retransmissions
packet_type: PacketType,
} }
impl Config { impl Config {
@@ -197,14 +197,12 @@ impl Config {
maximum_retransmissions: Option<u32>, maximum_retransmissions: Option<u32>,
ack_wait_addition: Duration, ack_wait_addition: Duration,
ack_wait_multiplier: f64, ack_wait_multiplier: f64,
packet_type: PacketType,
) -> Self { ) -> Self {
Config { Config {
maximum_retransmissions, maximum_retransmissions,
ack_wait_addition, ack_wait_addition,
ack_wait_multiplier, ack_wait_multiplier,
packet_size: Default::default(), packet_size: Default::default(),
packet_type,
} }
} }
@@ -214,7 +212,7 @@ impl Config {
} }
} }
pub(crate) struct AcknowledgementController<R> pub(super) struct AcknowledgementController<R>
where where
R: CryptoRng + Rng, R: CryptoRng + Rng,
{ {
@@ -236,6 +234,7 @@ where
message_handler: MessageHandler<R>, message_handler: MessageHandler<R>,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
task_client: TaskClient,
) -> Self { ) -> Self {
let (retransmission_tx, retransmission_rx) = mpsc::unbounded(); let (retransmission_tx, retransmission_rx) = mpsc::unbounded();
@@ -245,6 +244,7 @@ where
action_config, action_config,
retransmission_tx, retransmission_tx,
connectors.ack_action_receiver, connectors.ack_action_receiver,
task_client.fork("action_controller"),
); );
// will listen for any acks coming from the network // will listen for any acks coming from the network
@@ -253,6 +253,7 @@ where
connectors.ack_receiver, connectors.ack_receiver,
connectors.ack_action_sender.clone(), connectors.ack_action_sender.clone(),
stats_tx, stats_tx,
task_client.fork("acknowledgement_listener"),
); );
// will listen for any new messages from the client // will listen for any new messages from the client
@@ -260,6 +261,7 @@ where
connectors.input_receiver, connectors.input_receiver,
message_handler.clone(), message_handler.clone(),
reply_controller_sender.clone(), reply_controller_sender.clone(),
task_client.fork("input_message_listener"),
); );
// will listen for any ack timeouts and trigger retransmission // will listen for any ack timeouts and trigger retransmission
@@ -269,13 +271,16 @@ where
message_handler, message_handler,
retransmission_rx, retransmission_rx,
reply_controller_sender, reply_controller_sender,
config.packet_type, task_client.fork("retransmission_request_listener"),
); );
// will listen for events indicating the packet was sent through the network so that // will listen for events indicating the packet was sent through the network so that
// the retransmission timer should be started. // the retransmission timer should be started.
let sent_notification_listener = let sent_notification_listener = SentNotificationListener::new(
SentNotificationListener::new(connectors.sent_notifier, connectors.ack_action_sender); connectors.sent_notifier,
connectors.ack_action_sender,
task_client.with_suffix("sent_notification_listener"),
);
AcknowledgementController { AcknowledgementController {
acknowledgement_listener, acknowledgement_listener,
@@ -286,21 +291,36 @@ where
} }
} }
pub(crate) fn into_tasks( pub(super) fn start(self, packet_type: PacketType) {
self, let mut acknowledgement_listener = self.acknowledgement_listener;
) -> ( let mut input_message_listener = self.input_message_listener;
AcknowledgementListener, let mut retransmission_request_listener = self.retransmission_request_listener;
InputMessageListener<R>, let mut sent_notification_listener = self.sent_notification_listener;
RetransmissionRequestListener<R>, let mut action_controller = self.action_controller;
SentNotificationListener,
ActionController, spawn_future(async move {
) { acknowledgement_listener.run().await;
( debug!("The acknowledgement listener has finished execution!");
self.acknowledgement_listener, });
self.input_message_listener,
self.retransmission_request_listener, spawn_future(async move {
self.sent_notification_listener, input_message_listener.run().await;
self.action_controller, debug!("The input listener has finished execution!");
) });
spawn_future(async move {
retransmission_request_listener.run(packet_type).await;
debug!("The retransmission request listener has finished execution!");
});
spawn_future(async move {
sent_notification_listener.run().await;
debug!("The sent notification listener has finished execution!");
});
spawn_future(async move {
action_controller.run().await;
debug!("The controller has finished execution!");
});
} }
} }
@@ -10,22 +10,22 @@ use crate::client::real_messages_control::message_handler::{MessageHandler, Prep
use crate::client::real_messages_control::real_traffic_stream::RealMessage; use crate::client::real_messages_control::real_traffic_stream::RealMessage;
use crate::client::replies::reply_controller::ReplyControllerSender; use crate::client::replies::reply_controller::ReplyControllerSender;
use futures::StreamExt; use futures::StreamExt;
use log::*;
use nym_sphinx::chunking::fragment::Fragment; use nym_sphinx::chunking::fragment::Fragment;
use nym_sphinx::preparer::PreparedFragment; use nym_sphinx::preparer::PreparedFragment;
use nym_sphinx::{addressing::clients::Recipient, params::PacketType}; use nym_sphinx::{addressing::clients::Recipient, params::PacketType};
use nym_task::{connections::TransmissionLane, ShutdownToken}; use nym_task::{connections::TransmissionLane, TaskClient};
use rand::{CryptoRng, Rng}; use rand::{CryptoRng, Rng};
use std::sync::{Arc, Weak}; use std::sync::{Arc, Weak};
use tracing::*;
// responsible for packet retransmission upon fired timer // responsible for packet retransmission upon fired timer
pub(crate) struct RetransmissionRequestListener<R> { pub(super) struct RetransmissionRequestListener<R> {
maximum_retransmissions: Option<u32>, maximum_retransmissions: Option<u32>,
action_sender: AckActionSender, action_sender: AckActionSender,
message_handler: MessageHandler<R>, message_handler: MessageHandler<R>,
request_receiver: RetransmissionRequestReceiver, request_receiver: RetransmissionRequestReceiver,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
packet_type: PacketType, task_client: TaskClient,
} }
impl<R> RetransmissionRequestListener<R> impl<R> RetransmissionRequestListener<R>
@@ -38,7 +38,7 @@ where
message_handler: MessageHandler<R>, message_handler: MessageHandler<R>,
request_receiver: RetransmissionRequestReceiver, request_receiver: RetransmissionRequestReceiver,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
packet_type: PacketType, task_client: TaskClient,
) -> Self { ) -> Self {
RetransmissionRequestListener { RetransmissionRequestListener {
maximum_retransmissions, maximum_retransmissions,
@@ -46,7 +46,7 @@ where
message_handler, message_handler,
request_receiver, request_receiver,
reply_controller_sender, reply_controller_sender,
packet_type, task_client,
} }
} }
@@ -67,6 +67,7 @@ where
async fn on_retransmission_request( async fn on_retransmission_request(
&mut self, &mut self,
weak_timed_out_ack: Weak<PendingAcknowledgement>, weak_timed_out_ack: Weak<PendingAcknowledgement>,
packet_type: PacketType,
) { ) {
let timed_out_ack = match weak_timed_out_ack.upgrade() { let timed_out_ack = match weak_timed_out_ack.upgrade() {
Some(timed_out_ack) => timed_out_ack, Some(timed_out_ack) => timed_out_ack,
@@ -96,18 +97,22 @@ where
} => { } => {
// if this is retransmission for reply, offload it to the dedicated task // if this is retransmission for reply, offload it to the dedicated task
// that deals with all the surbs // that deals with all the surbs
let _ = self.reply_controller_sender.send_retransmission_data( if let Err(err) = self.reply_controller_sender.send_retransmission_data(
*recipient_tag, *recipient_tag,
weak_timed_out_ack, weak_timed_out_ack,
*extra_surb_request, *extra_surb_request,
); ) {
if !self.task_client.is_shutdown_poll() {
error!("Failed to send retransmission data to the reply controller: {err}");
}
}
return; return;
} }
PacketDestination::KnownRecipient(recipient) => { PacketDestination::KnownRecipient(recipient) => {
self.prepare_normal_retransmission_chunk( self.prepare_normal_retransmission_chunk(
**recipient, **recipient,
timed_out_ack.message_chunk.clone(), timed_out_ack.message_chunk.clone(),
self.packet_type, packet_type,
) )
.await .await
} }
@@ -148,9 +153,14 @@ where
// is sent to the `OutQueueControl` and has gone through its internal queue // is sent to the `OutQueueControl` and has gone through its internal queue
// with the additional poisson delay. // with the additional poisson delay.
// And since Actions are executed in order `UpdateTimer` will HAVE TO be executed before `StartTimer` // And since Actions are executed in order `UpdateTimer` will HAVE TO be executed before `StartTimer`
let _ = self if let Err(err) = self
.action_sender .action_sender
.unbounded_send(Action::new_update_pending_ack(frag_id, new_delay)); .unbounded_send(Action::new_update_pending_ack(frag_id, new_delay))
{
if !self.task_client.is_shutdown_poll() {
error!("Failed to send update pending ack action to the controller: {err}");
}
}
// send to `OutQueueControl` to eventually send to the mix network // send to `OutQueueControl` to eventually send to the mix network
self.message_handler self.message_handler
@@ -164,26 +174,24 @@ where
.await .await
} }
pub(crate) async fn run(&mut self, shutdown_token: ShutdownToken) { pub(super) async fn run(&mut self, packet_type: PacketType) {
debug!("Started RetransmissionRequestListener with graceful shutdown support"); debug!("Started RetransmissionRequestListener with graceful shutdown support");
loop { while !self.task_client.is_shutdown() {
tokio::select! { tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("RetransmissionRequestListener: Received shutdown");
break;
}
timed_out_ack = self.request_receiver.next() => match timed_out_ack { timed_out_ack = self.request_receiver.next() => match timed_out_ack {
Some(timed_out_ack) => self.on_retransmission_request(timed_out_ack).await, Some(timed_out_ack) => self.on_retransmission_request(timed_out_ack, packet_type).await,
None => { None => {
tracing::trace!("RetransmissionRequestListener: Stopping since channel closed"); log::trace!("RetransmissionRequestListener: Stopping since channel closed");
break; break;
} }
}, },
_ = self.task_client.recv() => {
log::trace!("RetransmissionRequestListener: Received shutdown");
}
} }
} }
tracing::debug!("RetransmissionRequestListener: Exiting"); self.task_client.recv_timeout().await;
log::debug!("RetransmissionRequestListener: Exiting");
} }
} }
@@ -4,26 +4,30 @@
use super::action_controller::{AckActionSender, Action}; use super::action_controller::{AckActionSender, Action};
use super::SentPacketNotificationReceiver; use super::SentPacketNotificationReceiver;
use futures::StreamExt; use futures::StreamExt;
use log::*;
use nym_sphinx::chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID}; use nym_sphinx::chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID};
use tracing::*; use nym_task::TaskClient;
/// Module responsible for starting up retransmission timers. /// Module responsible for starting up retransmission timers.
/// It is required because when we send our packet to the `real traffic stream` controlled /// It is required because when we send our packet to the `real traffic stream` controlled
/// by a poisson timer, there's no guarantee the message will be sent immediately, so we might /// by a poisson timer, there's no guarantee the message will be sent immediately, so we might
/// accidentally fire retransmission way quicker than we should have. /// accidentally fire retransmission way quicker than we should have.
pub(crate) struct SentNotificationListener { pub(super) struct SentNotificationListener {
sent_notifier: SentPacketNotificationReceiver, sent_notifier: SentPacketNotificationReceiver,
action_sender: AckActionSender, action_sender: AckActionSender,
task_client: TaskClient,
} }
impl SentNotificationListener { impl SentNotificationListener {
pub(super) fn new( pub(super) fn new(
sent_notifier: SentPacketNotificationReceiver, sent_notifier: SentPacketNotificationReceiver,
action_sender: AckActionSender, action_sender: AckActionSender,
task_client: TaskClient,
) -> Self { ) -> Self {
SentNotificationListener { SentNotificationListener {
sent_notifier, sent_notifier,
action_sender, action_sender,
task_client,
} }
} }
@@ -32,18 +36,37 @@ impl SentNotificationListener {
trace!("sent off a cover message - no need to start retransmission timer!"); trace!("sent off a cover message - no need to start retransmission timer!");
return; return;
} }
let _ = self if let Err(err) = self
.action_sender .action_sender
.unbounded_send(Action::new_start_timer(frag_id)); .unbounded_send(Action::new_start_timer(frag_id))
{
if !self.task_client.is_shutdown_poll() {
error!("Failed to send start timer action to action controller: {err}");
}
}
} }
pub(crate) async fn run(&mut self) { pub(super) async fn run(&mut self) {
debug!("Started SentNotificationListener with graceful shutdown support"); debug!("Started SentNotificationListener with graceful shutdown support");
while let Some(frag_id) = self.sent_notifier.next().await { while !self.task_client.is_shutdown() {
self.on_sent_message(frag_id).await; tokio::select! {
frag_id = self.sent_notifier.next() => match frag_id {
Some(frag_id) => {
self.on_sent_message(frag_id).await;
}
None => {
log::trace!("SentNotificationListener: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("SentNotificationListener: Received shutdown");
break;
}
}
} }
assert!(self.task_client.is_shutdown_poll());
tracing::debug!("SentNotificationListener: Exiting"); log::debug!("SentNotificationListener: Exiting");
} }
} }
@@ -9,18 +9,17 @@ use crate::client::real_messages_control::{AckActionSender, Action};
use crate::client::replies::reply_controller::MaxRetransmissions; use crate::client::replies::reply_controller::MaxRetransmissions;
use crate::client::replies::reply_storage::{ReceivedReplySurbsMap, SentReplyKeys, UsedSenderTags}; use crate::client::replies::reply_storage::{ReceivedReplySurbsMap, SentReplyKeys, UsedSenderTags};
use crate::client::topology_control::{TopologyAccessor, TopologyReadPermit}; use crate::client::topology_control::{TopologyAccessor, TopologyReadPermit};
use nym_client_core_surb_storage::RetrievedReplySurb;
use nym_sphinx::acknowledgements::AckKey; use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient; use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::{AnonymousSenderTag, RepliableMessage, ReplyMessage}; use nym_sphinx::anonymous_replies::requests::{AnonymousSenderTag, RepliableMessage, ReplyMessage};
use nym_sphinx::anonymous_replies::ReplySurbWithKeyRotation; use nym_sphinx::anonymous_replies::{ReplySurb, SurbEncryptionKey};
use nym_sphinx::chunking::fragment::{Fragment, FragmentIdentifier}; use nym_sphinx::chunking::fragment::{Fragment, FragmentIdentifier};
use nym_sphinx::message::NymMessage; use nym_sphinx::message::NymMessage;
use nym_sphinx::params::{PacketSize, PacketType}; use nym_sphinx::params::{PacketSize, PacketType};
use nym_sphinx::preparer::{MessagePreparer, PreparedFragment}; use nym_sphinx::preparer::{MessagePreparer, PreparedFragment};
use nym_sphinx::Delay; use nym_sphinx::Delay;
use nym_task::connections::TransmissionLane; use nym_task::connections::TransmissionLane;
use nym_task::ShutdownToken; use nym_task::TaskClient;
use nym_topology::{NymRouteProvider, NymTopologyError}; use nym_topology::{NymRouteProvider, NymTopologyError};
use rand::{CryptoRng, Rng}; use rand::{CryptoRng, Rng};
use std::collections::HashMap; use std::collections::HashMap;
@@ -35,9 +34,6 @@ pub enum PreparationError {
#[error(transparent)] #[error(transparent)]
NymTopologyError(#[from] NymTopologyError), NymTopologyError(#[from] NymTopologyError),
#[error("message wasn't split into any fragments!")]
EmptyFragments,
#[error("message too long for a single SURB, splitting into {fragments} fragments.")] #[error("message too long for a single SURB, splitting into {fragments} fragments.")]
MessageTooLongForSingleSurb { fragments: usize }, MessageTooLongForSingleSurb { fragments: usize },
@@ -48,7 +44,7 @@ pub enum PreparationError {
} }
impl PreparationError { impl PreparationError {
fn return_surbs(self, returned_surbs: Vec<RetrievedReplySurb>) -> SurbWrappedPreparationError { fn return_surbs(self, returned_surbs: Vec<ReplySurb>) -> SurbWrappedPreparationError {
SurbWrappedPreparationError { SurbWrappedPreparationError {
source: self, source: self,
returned_surbs: Some(returned_surbs), returned_surbs: Some(returned_surbs),
@@ -62,7 +58,7 @@ pub struct SurbWrappedPreparationError {
#[source] #[source]
source: PreparationError, source: PreparationError,
returned_surbs: Option<Vec<RetrievedReplySurb>>, returned_surbs: Option<Vec<ReplySurb>>,
} }
impl<T> From<T> for SurbWrappedPreparationError impl<T> From<T> for SurbWrappedPreparationError
@@ -84,7 +80,7 @@ impl SurbWrappedPreparationError {
target: &AnonymousSenderTag, target: &AnonymousSenderTag,
) -> PreparationError { ) -> PreparationError {
if let Some(reply_surbs) = self.returned_surbs { if let Some(reply_surbs) = self.returned_surbs {
surb_storage.re_insert_reply_surbs(target, reply_surbs) surb_storage.insert_surbs(target, reply_surbs)
} }
self.source self.source
} }
@@ -106,9 +102,6 @@ pub(crate) struct Config {
/// will be routed as usual, to the entry gateway, through three mix nodes, egressing /// will be routed as usual, to the entry gateway, through three mix nodes, egressing
/// through the exit gateway. If mix hops are disabled, traffic will be routed directly /// through the exit gateway. If mix hops are disabled, traffic will be routed directly
/// from the entry gateway to the exit gateway, bypassing the mix nodes. /// from the entry gateway to the exit gateway, bypassing the mix nodes.
///
/// This overrides the `use_legacy_sphinx_format` setting as reduced mix hops
/// requires use of the updated SURB packet format.
disable_mix_hops: bool, disable_mix_hops: bool,
/// Average delay a data packet is going to get delay at a single mixnode. /// Average delay a data packet is going to get delay at a single mixnode.
@@ -163,12 +156,8 @@ impl Config {
} }
/// Configure whether messages senders using this config should use mix hops or not when sending messages. /// Configure whether messages senders using this config should use mix hops or not when sending messages.
///
/// This overrides the `use_legacy_sphinx_format` setting as disabled mix hops
/// requires use of the updated SURB packet format.
pub fn disable_mix_hops(mut self, disable_mix_hops: bool) -> Self { pub fn disable_mix_hops(mut self, disable_mix_hops: bool) -> Self {
self.disable_mix_hops = disable_mix_hops; self.disable_mix_hops = disable_mix_hops;
self.use_legacy_sphinx_format = false;
self self
} }
} }
@@ -189,7 +178,7 @@ pub(crate) struct MessageHandler<R> {
topology_access: TopologyAccessor, topology_access: TopologyAccessor,
reply_key_storage: SentReplyKeys, reply_key_storage: SentReplyKeys,
tag_storage: UsedSenderTags, tag_storage: UsedSenderTags,
shutdown_token: ShutdownToken, task_client: TaskClient,
} }
impl<R> MessageHandler<R> impl<R> MessageHandler<R>
@@ -205,7 +194,7 @@ where
topology_access: TopologyAccessor, topology_access: TopologyAccessor,
reply_key_storage: SentReplyKeys, reply_key_storage: SentReplyKeys,
tag_storage: UsedSenderTags, tag_storage: UsedSenderTags,
shutdown_token: ShutdownToken, task_client: TaskClient,
) -> Self ) -> Self
where where
R: Copy, R: Copy,
@@ -228,14 +217,10 @@ where
topology_access, topology_access,
reply_key_storage, reply_key_storage,
tag_storage, tag_storage,
shutdown_token, task_client,
} }
} }
pub(crate) fn topology_access_handle(&self) -> &TopologyAccessor {
&self.topology_access
}
fn get_or_create_sender_tag(&mut self, recipient: &Recipient) -> AnonymousSenderTag { fn get_or_create_sender_tag(&mut self, recipient: &Recipient) -> AnonymousSenderTag {
if let Some(existing) = self.tag_storage.try_get_existing(recipient) { if let Some(existing) = self.tag_storage.try_get_existing(recipient) {
trace!("we already had sender tag for {recipient}"); trace!("we already had sender tag for {recipient}");
@@ -283,10 +268,10 @@ where
} }
} }
async fn generate_reply_surbs( async fn generate_reply_surbs_with_keys(
&mut self, &mut self,
amount: usize, amount: usize,
) -> Result<Vec<ReplySurbWithKeyRotation>, PreparationError> { ) -> Result<(Vec<ReplySurb>, Vec<SurbEncryptionKey>), PreparationError> {
let topology_permit = self.topology_access.get_read_permit().await; let topology_permit = self.topology_access.get_read_permit().await;
let topology = self.get_topology(&topology_permit)?; let topology = self.get_topology(&topology_permit)?;
@@ -296,14 +281,19 @@ where
topology, topology,
)?; )?;
Ok(reply_surbs) let reply_keys = reply_surbs
.iter()
.map(|s| *s.encryption_key())
.collect::<Vec<_>>();
Ok((reply_surbs, reply_keys))
} }
pub(crate) async fn try_send_single_surb_message( pub(crate) async fn try_send_single_surb_message(
&mut self, &mut self,
target: AnonymousSenderTag, target: AnonymousSenderTag,
message: ReplyMessage, message: ReplyMessage,
reply_surb: RetrievedReplySurb, reply_surb: ReplySurb,
is_extra_surb_request: bool, is_extra_surb_request: bool,
) -> Result<(), SurbWrappedPreparationError> { ) -> Result<(), SurbWrappedPreparationError> {
let msg = NymMessage::new_reply(message); let msg = NymMessage::new_reply(message);
@@ -323,16 +313,6 @@ where
}); });
} }
if fragment.is_empty() {
error!("CRITICAL FAILURE: our split message didn't result in any sendable fragments");
return Err(SurbWrappedPreparationError {
source: PreparationError::EmptyFragments,
returned_surbs: Some(vec![reply_surb]),
});
}
// SAFETY: we just checked we have one fragment
#[allow(clippy::unwrap_used)]
let chunk = fragment.pop().unwrap(); let chunk = fragment.pop().unwrap();
let chunk_clone = chunk.clone(); let chunk_clone = chunk.clone();
let prepared_fragment = self let prepared_fragment = self
@@ -344,10 +324,7 @@ where
Some(chunk.fragment_identifier()), Some(chunk.fragment_identifier()),
); );
let delay = prepared_fragment.total_delay; let delay = prepared_fragment.total_delay;
let max_retransmissions = None;
// we have to set a maximum number of retransmissions in case we fail to retrieve
// surbs for a long period of time; we don't want to be stuck constantly resending the data
let max_retransmissions = Some(10);
let pending_ack = PendingAcknowledgement::new_anonymous( let pending_ack = PendingAcknowledgement::new_anonymous(
chunk, chunk,
delay, delay,
@@ -370,7 +347,7 @@ where
pub(crate) async fn try_request_additional_reply_surbs( pub(crate) async fn try_request_additional_reply_surbs(
&mut self, &mut self,
from: AnonymousSenderTag, from: AnonymousSenderTag,
reply_surb: RetrievedReplySurb, reply_surb: ReplySurb,
amount: u32, amount: u32,
) -> Result<(), SurbWrappedPreparationError> { ) -> Result<(), SurbWrappedPreparationError> {
debug!("requesting {amount} reply SURBs from {from}"); debug!("requesting {amount} reply SURBs from {from}");
@@ -410,9 +387,11 @@ where
&mut self, &mut self,
target: AnonymousSenderTag, target: AnonymousSenderTag,
fragments: Vec<FragmentWithMaxRetransmissions>, fragments: Vec<FragmentWithMaxRetransmissions>,
reply_surbs: impl IntoIterator<Item = RetrievedReplySurb>, reply_surbs: Vec<ReplySurb>,
lane: TransmissionLane, lane: TransmissionLane,
) -> Result<(), SurbWrappedPreparationError> { ) -> Result<(), SurbWrappedPreparationError> {
// TODO: technically this is performing an unnecessary cloning, but in the grand scheme of things
// is it really that bad?
self.try_send_reply_chunks( self.try_send_reply_chunks(
target, target,
fragments.into_iter().map(|f| (lane, f)).collect(), fragments.into_iter().map(|f| (lane, f)).collect(),
@@ -425,7 +404,7 @@ where
&mut self, &mut self,
target: AnonymousSenderTag, target: AnonymousSenderTag,
fragments: Vec<(TransmissionLane, FragmentWithMaxRetransmissions)>, fragments: Vec<(TransmissionLane, FragmentWithMaxRetransmissions)>,
reply_surbs: impl IntoIterator<Item = RetrievedReplySurb>, reply_surbs: Vec<ReplySurb>,
) -> Result<(), SurbWrappedPreparationError> { ) -> Result<(), SurbWrappedPreparationError> {
let prepared_fragments = self let prepared_fragments = self
.prepare_reply_chunks_for_sending( .prepare_reply_chunks_for_sending(
@@ -548,7 +527,6 @@ where
pending_acks.push(pending_ack); pending_acks.push(pending_ack);
} }
drop(topology_permit);
self.insert_pending_acks(pending_acks); self.insert_pending_acks(pending_acks);
self.forward_messages(real_messages, lane).await; self.forward_messages(real_messages, lane).await;
@@ -563,12 +541,8 @@ where
) -> Result<(), PreparationError> { ) -> Result<(), PreparationError> {
debug!("Sending additional reply SURBs with packet type {packet_type}"); debug!("Sending additional reply SURBs with packet type {packet_type}");
let sender_tag = self.get_or_create_sender_tag(&recipient); let sender_tag = self.get_or_create_sender_tag(&recipient);
let reply_surbs = self.generate_reply_surbs(amount as usize).await?; let (reply_surbs, reply_keys) =
self.generate_reply_surbs_with_keys(amount as usize).await?;
let reply_keys = reply_surbs
.iter()
.map(|s| *s.encryption_key())
.collect::<Vec<_>>();
let message = NymMessage::new_repliable(RepliableMessage::new_additional_surbs( let message = NymMessage::new_repliable(RepliableMessage::new_additional_surbs(
self.config.use_legacy_sphinx_format, self.config.use_legacy_sphinx_format,
@@ -588,7 +562,7 @@ where
) )
.await?; .await?;
tracing::trace!("storing {} reply keys", reply_keys.len()); log::trace!("storing {} reply keys", reply_keys.len());
self.reply_key_storage.insert_multiple(reply_keys); self.reply_key_storage.insert_multiple(reply_keys);
Ok(()) Ok(())
@@ -605,12 +579,9 @@ where
) -> Result<(), SurbWrappedPreparationError> { ) -> Result<(), SurbWrappedPreparationError> {
debug!("Sending message with reply SURBs with packet type {packet_type}"); debug!("Sending message with reply SURBs with packet type {packet_type}");
let sender_tag = self.get_or_create_sender_tag(&recipient); let sender_tag = self.get_or_create_sender_tag(&recipient);
let reply_surbs = self.generate_reply_surbs(num_reply_surbs as usize).await?; let (reply_surbs, reply_keys) = self
.generate_reply_surbs_with_keys(num_reply_surbs as usize)
let reply_keys = reply_surbs .await?;
.iter()
.map(|s| *s.encryption_key())
.collect::<Vec<_>>();
let message = NymMessage::new_repliable(RepliableMessage::new_data( let message = NymMessage::new_repliable(RepliableMessage::new_data(
self.config.use_legacy_sphinx_format, self.config.use_legacy_sphinx_format,
@@ -628,7 +599,7 @@ where
) )
.await?; .await?;
tracing::trace!("storing {} reply keys", reply_keys.len()); log::trace!("storing {} reply keys", reply_keys.len());
self.reply_key_storage.insert_multiple(reply_keys); self.reply_key_storage.insert_multiple(reply_keys);
Ok(()) Ok(())
@@ -658,12 +629,20 @@ where
pub(crate) async fn prepare_reply_chunks_for_sending( pub(crate) async fn prepare_reply_chunks_for_sending(
&mut self, &mut self,
fragments: Vec<Fragment>, fragments: Vec<Fragment>,
reply_surbs: impl IntoIterator<Item = RetrievedReplySurb>, reply_surbs: Vec<ReplySurb>,
) -> Result<Vec<PreparedFragment>, SurbWrappedPreparationError> { ) -> Result<Vec<PreparedFragment>, SurbWrappedPreparationError> {
debug_assert_eq!(
fragments.len(),
reply_surbs.len(),
"attempted to send {} fragments with {} reply surbs",
fragments.len(),
reply_surbs.len()
);
let topology_permit = self.topology_access.get_read_permit().await; let topology_permit = self.topology_access.get_read_permit().await;
let topology = match self.get_topology(&topology_permit) { let topology = match self.get_topology(&topology_permit) {
Ok(topology) => topology, Ok(topology) => topology,
Err(err) => return Err(err.return_surbs(reply_surbs.into_iter().collect())), Err(err) => return Err(err.return_surbs(reply_surbs)),
}; };
Ok(fragments Ok(fragments
@@ -671,13 +650,12 @@ where
.zip(reply_surbs.into_iter()) .zip(reply_surbs.into_iter())
.map(|(fragment, reply_surb)| { .map(|(fragment, reply_surb)| {
// unwrap here is fine as we know we have a valid topology // unwrap here is fine as we know we have a valid topology
#[allow(clippy::unwrap_used)]
self.message_preparer self.message_preparer
.prepare_reply_chunk_for_sending( .prepare_reply_chunk_for_sending(
fragment, fragment,
topology, topology,
&self.config.ack_key, &self.config.ack_key,
reply_surb.into(), reply_surb,
PacketType::Mix, PacketType::Mix,
) )
.unwrap() .unwrap()
@@ -687,7 +665,7 @@ where
pub(crate) async fn try_prepare_single_reply_chunk_for_sending( pub(crate) async fn try_prepare_single_reply_chunk_for_sending(
&mut self, &mut self,
reply_surb: RetrievedReplySurb, reply_surb: ReplySurb,
chunk: Fragment, chunk: Fragment,
) -> Result<PreparedFragment, SurbWrappedPreparationError> { ) -> Result<PreparedFragment, SurbWrappedPreparationError> {
let topology_permit = self.topology_access.get_read_permit().await; let topology_permit = self.topology_access.get_read_permit().await;
@@ -700,7 +678,7 @@ where
chunk, chunk,
topology, topology,
&self.config.ack_key, &self.config.ack_key,
reply_surb.into(), reply_surb,
PacketType::Mix, PacketType::Mix,
)?; )?;
@@ -712,7 +690,7 @@ where
.action_sender .action_sender
.unbounded_send(Action::UpdatePendingAck(id, new_delay)) .unbounded_send(Action::UpdatePendingAck(id, new_delay))
{ {
if !self.shutdown_token.is_cancelled() { if !self.task_client.is_shutdown_poll() {
error!("Failed to send update action to the controller: {err}"); error!("Failed to send update action to the controller: {err}");
} }
} }
@@ -723,7 +701,7 @@ where
.action_sender .action_sender
.unbounded_send(Action::new_insert(pending_acks)) .unbounded_send(Action::new_insert(pending_acks))
{ {
if !self.shutdown_token.is_cancelled() { if !self.task_client.is_shutdown_poll() {
error!("Failed to send insert action to the controller: {err}"); error!("Failed to send insert action to the controller: {err}");
} }
} }
@@ -731,21 +709,17 @@ where
// tells real message sender (with the poisson timer) to send this to the mix network // tells real message sender (with the poisson timer) to send this to the mix network
pub(crate) async fn forward_messages( pub(crate) async fn forward_messages(
&mut self, &self,
messages: Vec<RealMessage>, messages: Vec<RealMessage>,
transmission_lane: TransmissionLane, transmission_lane: TransmissionLane,
) { ) {
tokio::select! { if let Err(err) = self
biased; .real_message_sender
_ = self.shutdown_token.cancelled() => { .send((messages, transmission_lane))
trace!("received shutdown while attempting to forward mixnet messages"); .await
} {
sending_res = self.real_message_sender.send((messages, transmission_lane)) => { if !self.task_client.is_shutdown_poll() {
if sending_res.is_err() { error!("Failed to forward messages to the real message sender: {err}");
error!(
"failed to forward mixnet messages due to closed channel (outside of shutdown!)"
);
}
} }
} }
} }
@@ -14,23 +14,27 @@ use crate::client::replies::reply_controller::{
ReplyController, ReplyControllerReceiver, ReplyControllerSender, ReplyController, ReplyControllerReceiver, ReplyControllerSender,
}; };
use crate::client::replies::reply_storage::CombinedReplyStorage; use crate::client::replies::reply_storage::CombinedReplyStorage;
use crate::client::{
inbound_messages::InputMessageReceiver, mix_traffic::BatchMixMessageSender,
real_messages_control::acknowledgement_control::AcknowledgementControllerConnectors,
topology_control::TopologyAccessor,
};
use crate::config; use crate::config;
use crate::{
client::{
inbound_messages::InputMessageReceiver, mix_traffic::BatchMixMessageSender,
real_messages_control::acknowledgement_control::AcknowledgementControllerConnectors,
topology_control::TopologyAccessor,
},
spawn_future,
};
use futures::channel::mpsc; use futures::channel::mpsc;
use log::*;
use nym_gateway_client::AcknowledgementReceiver; use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::acknowledgements::AckKey; use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient; use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::params::PacketType;
use nym_statistics_common::clients::ClientStatsSender; use nym_statistics_common::clients::ClientStatsSender;
use nym_task::connections::{ConnectionCommandReceiver, LaneQueueLengths}; use nym_task::connections::{ConnectionCommandReceiver, LaneQueueLengths};
use nym_task::ShutdownToken; use nym_task::TaskClient;
use rand::{rngs::OsRng, CryptoRng, Rng}; use rand::{rngs::OsRng, CryptoRng, Rng};
use std::sync::Arc; use std::sync::Arc;
use crate::client::replies::reply_controller::key_rotation_helpers::KeyRotationConfig;
pub(crate) use acknowledgement_control::{AckActionSender, Action}; pub(crate) use acknowledgement_control::{AckActionSender, Action};
pub(crate) mod acknowledgement_control; pub(crate) mod acknowledgement_control;
@@ -64,7 +68,6 @@ impl<'a> From<&'a Config> for acknowledgement_control::Config {
cfg.traffic.maximum_number_of_retransmissions, cfg.traffic.maximum_number_of_retransmissions,
cfg.acks.ack_wait_addition, cfg.acks.ack_wait_addition,
cfg.acks.ack_wait_multiplier, cfg.acks.ack_wait_multiplier,
cfg.traffic.packet_type,
) )
.with_custom_packet_size(cfg.traffic.primary_packet_size) .with_custom_packet_size(cfg.traffic.primary_packet_size)
} }
@@ -82,6 +85,12 @@ impl<'a> From<&'a Config> for real_traffic_stream::Config {
} }
} }
impl<'a> From<&'a Config> for reply_controller::Config {
fn from(cfg: &'a Config) -> Self {
reply_controller::Config::new(cfg.reply_surbs)
}
}
impl<'a> From<&'a Config> for message_handler::Config { impl<'a> From<&'a Config> for message_handler::Config {
fn from(cfg: &'a Config) -> Self { fn from(cfg: &'a Config) -> Self {
message_handler::Config::new( message_handler::Config::new(
@@ -130,7 +139,6 @@ impl RealMessagesController<OsRng> {
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
pub(crate) fn new( pub(crate) fn new(
config: Config, config: Config,
key_rotation_config: KeyRotationConfig,
ack_receiver: AcknowledgementReceiver, ack_receiver: AcknowledgementReceiver,
input_receiver: InputMessageReceiver, input_receiver: InputMessageReceiver,
mix_sender: BatchMixMessageSender, mix_sender: BatchMixMessageSender,
@@ -142,7 +150,7 @@ impl RealMessagesController<OsRng> {
lane_queue_lengths: LaneQueueLengths, lane_queue_lengths: LaneQueueLengths,
client_connection_rx: ConnectionCommandReceiver, client_connection_rx: ConnectionCommandReceiver,
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
shutdown_token: ShutdownToken, task_client: TaskClient,
) -> Self { ) -> Self {
let rng = OsRng; let rng = OsRng;
@@ -161,8 +169,7 @@ impl RealMessagesController<OsRng> {
// create all configs for the components // create all configs for the components
let ack_control_config = (&config).into(); let ack_control_config = (&config).into();
let out_queue_config = (&config).into(); let out_queue_config = (&config).into();
let reply_controller_config = let reply_controller_config = (&config).into();
reply_controller::Config::new(config.reply_surbs, key_rotation_config);
let message_handler_config = (&config).into(); let message_handler_config = (&config).into();
// create the actual components // create the actual components
@@ -174,7 +181,7 @@ impl RealMessagesController<OsRng> {
topology_access.clone(), topology_access.clone(),
reply_storage.key_storage(), reply_storage.key_storage(),
reply_storage.tags_storage(), reply_storage.tags_storage(),
shutdown_token.clone(), task_client.fork("message_handler"),
); );
let ack_control = AcknowledgementController::new( let ack_control = AcknowledgementController::new(
@@ -184,6 +191,7 @@ impl RealMessagesController<OsRng> {
message_handler.clone(), message_handler.clone(),
reply_controller_sender, reply_controller_sender,
stats_tx.clone(), stats_tx.clone(),
task_client.fork("ack_control"),
); );
let reply_control = ReplyController::new( let reply_control = ReplyController::new(
@@ -191,6 +199,7 @@ impl RealMessagesController<OsRng> {
message_handler, message_handler,
reply_storage, reply_storage,
reply_controller_receiver, reply_controller_receiver,
task_client.fork("reply_controller"),
); );
let out_queue_control = OutQueueControl::new( let out_queue_control = OutQueueControl::new(
@@ -203,7 +212,7 @@ impl RealMessagesController<OsRng> {
lane_queue_lengths, lane_queue_lengths,
client_connection_rx, client_connection_rx,
stats_tx, stats_tx,
shutdown_token.clone(), task_client.with_suffix("out_queue_control"),
); );
RealMessagesController { RealMessagesController {
@@ -213,13 +222,20 @@ impl RealMessagesController<OsRng> {
} }
} }
pub fn into_tasks( pub fn start(self, packet_type: PacketType) {
self, let mut out_queue_control = self.out_queue_control;
) -> ( let ack_control = self.ack_control;
OutQueueControl<OsRng>, let mut reply_control = self.reply_control;
ReplyController<OsRng>,
AcknowledgementController<OsRng>, spawn_future(async move {
) { out_queue_control.run().await;
(self.out_queue_control, self.reply_control, self.ack_control) debug!("The out queue controller has finished execution!");
});
spawn_future(async move {
reply_control.run().await;
debug!("The reply controller has finished execution!");
});
ack_control.start(packet_type);
} }
} }
@@ -9,6 +9,7 @@ use crate::client::transmission_buffer::TransmissionBuffer;
use crate::config; use crate::config;
use futures::task::{Context, Poll}; use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt}; use futures::{Future, Stream, StreamExt};
use log::*;
use nym_sphinx::acknowledgements::AckKey; use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient; use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::chunking::fragment::FragmentIdentifier; use nym_sphinx::chunking::fragment::FragmentIdentifier;
@@ -21,19 +22,18 @@ use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, C
use nym_task::connections::{ use nym_task::connections::{
ConnectionCommand, ConnectionCommandReceiver, ConnectionId, LaneQueueLengths, TransmissionLane, ConnectionCommand, ConnectionCommandReceiver, ConnectionId, LaneQueueLengths, TransmissionLane,
}; };
use nym_task::ShutdownToken; use nym_task::TaskClient;
use rand::{CryptoRng, Rng}; use rand::{CryptoRng, Rng};
use std::pin::Pin; use std::pin::Pin;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::Duration;
use tracing::*;
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
use tokio::time::{sleep, Sleep}; use tokio::time::{sleep, Sleep};
// use wasm_utils::console_log;
#[cfg(target_arch = "wasm32")] #[cfg(target_arch = "wasm32")]
use wasmtimer::tokio::{sleep, Sleep}; use wasmtimer::tokio::{sleep, Sleep};
mod sending_delay_controller; mod sending_delay_controller;
/// Configurable parameters of the `OutQueueControl` /// Configurable parameters of the `OutQueueControl`
@@ -119,7 +119,7 @@ where
/// Channel used for sending metrics events (specifically `PacketStatistics` events) to the metrics tracker. /// Channel used for sending metrics events (specifically `PacketStatistics` events) to the metrics tracker.
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
shutdown_token: ShutdownToken, task_client: TaskClient,
} }
#[derive(Debug)] #[derive(Debug)]
@@ -179,7 +179,7 @@ where
lane_queue_lengths: LaneQueueLengths, lane_queue_lengths: LaneQueueLengths,
client_connection_rx: ConnectionCommandReceiver, client_connection_rx: ConnectionCommandReceiver,
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
shutdown_token: ShutdownToken, task_client: TaskClient,
) -> Self { ) -> Self {
OutQueueControl { OutQueueControl {
config, config,
@@ -194,7 +194,7 @@ where
client_connection_rx, client_connection_rx,
lane_queue_lengths, lane_queue_lengths,
stats_tx, stats_tx,
shutdown_token, task_client,
} }
} }
@@ -202,7 +202,7 @@ where
// well technically the message was not sent just yet, but now it's up to internal // well technically the message was not sent just yet, but now it's up to internal
// queues and client load rather than the required delay. So realistically we can treat // queues and client load rather than the required delay. So realistically we can treat
// whatever is about to happen as negligible additional delay. // whatever is about to happen as negligible additional delay.
trace!("{frag_id} is about to get sent to the mixnet"); trace!("{} is about to get sent to the mixnet", frag_id);
if let Err(err) = self.sent_notifier.unbounded_send(frag_id) { if let Err(err) = self.sent_notifier.unbounded_send(frag_id) {
error!("Failed to notify about sent message: {err}"); error!("Failed to notify about sent message: {err}");
} }
@@ -249,8 +249,6 @@ where
} }
}; };
// SAFETY: our topology must be valid at this point
#[allow(clippy::expect_used)]
( (
generate_loop_cover_packet( generate_loop_cover_packet(
&mut self.rng, &mut self.rng,
@@ -280,35 +278,17 @@ where
} }
}; };
let sending_res = tokio::select! { if let Err(err) = self.mix_tx.send(vec![next_message]).await {
biased; if !self.task_client.is_shutdown_poll() {
_ = self.shutdown_token.cancelled() => { log::error!("Failed to send: {err}");
trace!("received shutdown signal while attempting to send mix message");
return
}
sending_res = self.mix_tx.send(vec![next_message]) => {
sending_res
}
};
match sending_res {
Err(_) => {
if !self.shutdown_token.is_cancelled() {
tracing::error!(
"failed to send mixnet packet due to closed channel (outside of shutdown!)"
);
}
// Early return to avoid further processing when channel is closed
return;
}
Ok(_) => {
let event = if fragment_id.is_some() {
PacketStatisticsEvent::RealPacketSent(packet_size)
} else {
PacketStatisticsEvent::CoverPacketSent(packet_size)
};
self.stats_tx.report(event.into());
} }
} else {
let event = if fragment_id.is_some() {
PacketStatisticsEvent::RealPacketSent(packet_size)
} else {
PacketStatisticsEvent::CoverPacketSent(packet_size)
};
self.stats_tx.report(event.into());
} }
// notify ack controller about sending our message only after we actually managed to push it // notify ack controller about sending our message only after we actually managed to push it
@@ -327,19 +307,13 @@ where
// ready and hence was immediately re-scheduled causing other tasks to be starved; // ready and hence was immediately re-scheduled causing other tasks to be starved;
// yield makes it go back the scheduling queue regardless of its value availability // yield makes it go back the scheduling queue regardless of its value availability
// TODO: temporary and BAD workaround for wasm (we should find a way to yield here in wasm)
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
{ tokio::task::yield_now().await;
tokio::task::yield_now().await;
}
#[cfg(target_arch = "wasm32")]
{
tokio_with_wasm::task::yield_now().await;
}
} }
fn on_close_connection(&mut self, connection_id: ConnectionId) { fn on_close_connection(&mut self, connection_id: ConnectionId) {
tracing::debug!("Removing lane for connection: {connection_id}"); log::debug!("Removing lane for connection: {connection_id}");
self.transmission_buffer self.transmission_buffer
.remove(&TransmissionLane::ConnectionId(connection_id)); .remove(&TransmissionLane::ConnectionId(connection_id));
} }
@@ -351,7 +325,7 @@ where
fn adjust_current_average_message_sending_delay(&mut self) { fn adjust_current_average_message_sending_delay(&mut self) {
let used_slots = self.mix_tx.max_capacity() - self.mix_tx.capacity(); let used_slots = self.mix_tx.max_capacity() - self.mix_tx.capacity();
tracing::trace!( log::trace!(
"used_slots: {used_slots}, current_multiplier: {}", "used_slots: {used_slots}, current_multiplier: {}",
self.sending_delay_controller.current_multiplier() self.sending_delay_controller.current_multiplier()
); );
@@ -360,7 +334,7 @@ where
.sending_delay_controller .sending_delay_controller
.is_backpressure_currently_detected(used_slots) .is_backpressure_currently_detected(used_slots)
{ {
tracing::trace!("Backpressure detected"); log::trace!("Backpressure detected");
self.sending_delay_controller.record_backpressure_detected(); self.sending_delay_controller.record_backpressure_detected();
} }
@@ -462,11 +436,9 @@ where
Poll::Ready(None) => Poll::Ready(None), Poll::Ready(None) => Poll::Ready(None),
Poll::Ready(Some((real_messages, conn_id))) => { Poll::Ready(Some((real_messages, conn_id))) => {
tracing::trace!("handling real_messages: size: {}", real_messages.len()); log::trace!("handling real_messages: size: {}", real_messages.len());
self.transmission_buffer.store(&conn_id, real_messages); self.transmission_buffer.store(&conn_id, real_messages);
// SAFETY: we just stored the message
#[allow(clippy::expect_used)]
let real_next = self.pop_next_message().expect("Just stored one"); let real_next = self.pop_next_message().expect("Just stored one");
Poll::Ready(Some(StreamMessage::Real(Box::new(real_next)))) Poll::Ready(Some(StreamMessage::Real(Box::new(real_next))))
@@ -511,12 +483,10 @@ where
Poll::Ready(None) => Poll::Ready(None), Poll::Ready(None) => Poll::Ready(None),
Poll::Ready(Some((real_messages, conn_id))) => { Poll::Ready(Some((real_messages, conn_id))) => {
tracing::trace!("handling real_messages: size: {}", real_messages.len()); log::trace!("handling real_messages: size: {}", real_messages.len());
// First store what we got for the given connection id // First store what we got for the given connection id
self.transmission_buffer.store(&conn_id, real_messages); self.transmission_buffer.store(&conn_id, real_messages);
// SAFETY: we just stored the message
#[allow(clippy::expect_used)]
let real_next = self.pop_next_message().expect("we just added one"); let real_next = self.pop_next_message().expect("we just added one");
Poll::Ready(Some(StreamMessage::Real(Box::new(real_next)))) Poll::Ready(Some(StreamMessage::Real(Box::new(real_next))))
@@ -544,7 +514,9 @@ where
} }
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
fn log_status(&self) { fn log_status(&self, shutdown: &mut TaskClient) {
use crate::error::ClientCoreStatusMessage;
let packets = self.transmission_buffer.total_size(); let packets = self.transmission_buffer.total_size();
let lanes = self.transmission_buffer.lanes(); let lanes = self.transmission_buffer.lanes();
let mult = self.sending_delay_controller.current_multiplier(); let mult = self.sending_delay_controller.current_multiplier();
@@ -566,70 +538,69 @@ where
}; };
if packets > 1000 { if packets > 1000 {
tracing::warn!("{status_str}"); log::warn!("{status_str}");
} else if packets > 0 { } else if packets > 0 {
tracing::info!("{status_str}"); log::info!("{status_str}");
} else { } else {
tracing::debug!("{status_str}"); log::debug!("{status_str}");
} }
// leave the code commented in case somebody wanted to restore this logic with a different channel // Send status message to whoever is listening (possibly UI)
// // Send status message to whoever is listening (possibly UI) if mult == self.sending_delay_controller.max_multiplier() {
// if mult == self.sending_delay_controller.max_multiplier() { shutdown.send_status_msg(Box::new(ClientCoreStatusMessage::GatewayIsVerySlow));
// shutdown.send_status_msg(Box::new(ClientCoreStatusMessage::GatewayIsVerySlow)); } else if mult > self.sending_delay_controller.min_multiplier() {
// } else if mult > self.sending_delay_controller.min_multiplier() { shutdown.send_status_msg(Box::new(ClientCoreStatusMessage::GatewayIsSlow));
// shutdown.send_status_msg(Box::new(ClientCoreStatusMessage::GatewayIsSlow)); }
// }
} }
pub(crate) async fn run(&mut self) { pub(super) async fn run(&mut self) {
debug!("Started OutQueueControl with graceful shutdown support"); debug!("Started OutQueueControl with graceful shutdown support");
// avoid borrow on self let mut shutdown = self.task_client.fork("select");
let shutdown_token = self.shutdown_token.clone();
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
{ {
let mut status_timer = tokio::time::interval(Duration::from_secs(5)); let mut status_timer = tokio::time::interval(Duration::from_secs(5));
loop { while !shutdown.is_shutdown() {
tokio::select! { tokio::select! {
biased; biased;
_ = shutdown_token.cancelled() => { _ = shutdown.recv() => {
tracing::trace!("OutQueueControl: Received shutdown"); log::trace!("OutQueueControl: Received shutdown");
break; break;
} }
_ = status_timer.tick() => { _ = status_timer.tick() => {
self.log_status(); self.log_status(&mut shutdown);
} }
next_message = self.next() => if let Some(next_message) = next_message { next_message = self.next() => if let Some(next_message) = next_message {
self.on_message(next_message).await; self.on_message(next_message).await;
} else { } else {
tracing::trace!("OutQueueControl: Stopping since channel closed"); log::trace!("OutQueueControl: Stopping since channel closed");
break; break;
} }
} }
} }
shutdown.recv_timeout().await;
} }
#[cfg(target_arch = "wasm32")] #[cfg(target_arch = "wasm32")]
{ {
loop { while !shutdown.is_shutdown() {
tokio::select! { tokio::select! {
biased; biased;
_ = shutdown_token.cancelled() => { _ = shutdown.recv() => {
tracing::trace!("OutQueueControl: Received shutdown"); log::trace!("OutQueueControl: Received shutdown");
break;
} }
next_message = self.next() => if let Some(next_message) = next_message { next_message = self.next() => if let Some(next_message) = next_message {
self.on_message(next_message).await; self.on_message(next_message).await;
} else { } else {
tracing::trace!("OutQueueControl: Stopping since channel closed"); log::trace!("OutQueueControl: Stopping since channel closed");
break; break;
} }
} }
} }
} }
tracing::debug!("OutQueueControl: Exiting"); log::debug!("OutQueueControl: Exiting");
} }
} }
@@ -83,13 +83,11 @@ impl SendingDelayController {
self.current_multiplier self.current_multiplier
} }
#[allow(dead_code)]
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
pub(crate) fn min_multiplier(&self) -> u32 { pub(crate) fn min_multiplier(&self) -> u32 {
self.lower_bound self.lower_bound
} }
#[allow(dead_code)]
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
pub(crate) fn max_multiplier(&self) -> u32 { pub(crate) fn max_multiplier(&self) -> u32 {
self.upper_bound self.upper_bound
@@ -100,12 +98,12 @@ impl SendingDelayController {
self.current_multiplier = self.current_multiplier =
(self.current_multiplier + 1).clamp(self.lower_bound, self.upper_bound); (self.current_multiplier + 1).clamp(self.lower_bound, self.upper_bound);
self.time_when_changed = get_time_now(); self.time_when_changed = get_time_now();
tracing::debug!( log::debug!(
"Increasing sending delay multiplier to: {}", "Increasing sending delay multiplier to: {}",
self.current_multiplier self.current_multiplier
); );
} else { } else {
tracing::warn!("Trying to increase delay multipler higher than allowed"); log::warn!("Trying to increase delay multipler higher than allowed");
} }
} }
@@ -114,7 +112,7 @@ impl SendingDelayController {
self.current_multiplier = self.current_multiplier =
(self.current_multiplier - 1).clamp(self.lower_bound, self.upper_bound); (self.current_multiplier - 1).clamp(self.lower_bound, self.upper_bound);
self.time_when_changed = get_time_now(); self.time_when_changed = get_time_now();
tracing::debug!( log::debug!(
"Decreasing sending delay multiplier to: {}", "Decreasing sending delay multiplier to: {}",
self.current_multiplier self.current_multiplier
); );
@@ -166,11 +164,11 @@ impl SendingDelayController {
self.current_multiplier() self.current_multiplier()
); );
if self.current_multiplier() > 0 { if self.current_multiplier() > 0 {
tracing::debug!("{status_str}"); log::debug!("{}", status_str);
} else if self.current_multiplier() > 1 { } else if self.current_multiplier() > 1 {
tracing::info!("{status_str}"); log::info!("{}", status_str);
} else if self.current_multiplier() > 2 { } else if self.current_multiplier() > 2 {
tracing::warn!("{status_str}"); log::warn!("{}", status_str);
} }
self.time_when_logged_about_elevated_multiplier = now; self.time_when_logged_about_elevated_multiplier = now;
} }
@@ -1,13 +1,14 @@
// Copyright 2021 - Nym Technologies SA <contact@nymtech.net> // Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::client::helpers::get_time_now;
use crate::client::replies::{ use crate::client::replies::{
reply_controller::ReplyControllerSender, reply_storage::SentReplyKeys, reply_controller::ReplyControllerSender, reply_storage::SentReplyKeys,
}; };
use crate::spawn_future;
use futures::channel::mpsc; use futures::channel::mpsc;
use futures::lock::Mutex; use futures::lock::Mutex;
use futures::StreamExt; use futures::StreamExt;
use log::*;
use nym_crypto::asymmetric::x25519; use nym_crypto::asymmetric::x25519;
use nym_crypto::Digest; use nym_crypto::Digest;
use nym_gateway_client::MixnetMessageReceiver; use nym_gateway_client::MixnetMessageReceiver;
@@ -19,11 +20,10 @@ use nym_sphinx::message::{NymMessage, PlainMessage};
use nym_sphinx::params::ReplySurbKeyDigestAlgorithm; use nym_sphinx::params::ReplySurbKeyDigestAlgorithm;
use nym_sphinx::receiver::{MessageReceiver, MessageRecoveryError, ReconstructedMessage}; use nym_sphinx::receiver::{MessageReceiver, MessageRecoveryError, ReconstructedMessage};
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender}; use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use nym_task::ShutdownToken; use nym_task::TaskClient;
use std::collections::HashSet; use std::collections::HashSet;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::{Duration, Instant};
use tracing::*;
// The interval at which we check for stale buffers // The interval at which we check for stale buffers
const STALE_BUFFER_CHECK_INTERVAL: Duration = Duration::from_secs(10); const STALE_BUFFER_CHECK_INTERVAL: Duration = Duration::from_secs(10);
@@ -54,7 +54,7 @@ struct ReceivedMessagesBufferInner<R: MessageReceiver> {
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
// Periodically check for stale buffers to clean up // Periodically check for stale buffers to clean up
last_stale_check: crate::client::helpers::Instant, last_stale_check: Instant,
} }
impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> { impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
@@ -154,7 +154,7 @@ impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
} }
fn cleanup_stale_buffers(&mut self) { fn cleanup_stale_buffers(&mut self) {
let now = get_time_now(); let now = Instant::now();
if now - self.last_stale_check > STALE_BUFFER_CHECK_INTERVAL { if now - self.last_stale_check > STALE_BUFFER_CHECK_INTERVAL {
self.last_stale_check = now; self.last_stale_check = now;
self.message_receiver self.message_receiver
@@ -171,7 +171,7 @@ struct ReceivedMessagesBuffer<R: MessageReceiver> {
inner: Arc<Mutex<ReceivedMessagesBufferInner<R>>>, inner: Arc<Mutex<ReceivedMessagesBufferInner<R>>>,
reply_key_storage: SentReplyKeys, reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
shutdown_token: ShutdownToken, task_client: TaskClient,
} }
impl<R: MessageReceiver> ReceivedMessagesBuffer<R> { impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
@@ -180,7 +180,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
reply_key_storage: SentReplyKeys, reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
stats_tx: ClientStatsSender, stats_tx: ClientStatsSender,
shutdown_token: ShutdownToken, task_client: TaskClient,
) -> Self { ) -> Self {
ReceivedMessagesBuffer { ReceivedMessagesBuffer {
inner: Arc::new(Mutex::new(ReceivedMessagesBufferInner { inner: Arc::new(Mutex::new(ReceivedMessagesBufferInner {
@@ -190,15 +190,14 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
message_sender: None, message_sender: None,
recently_reconstructed: HashSet::new(), recently_reconstructed: HashSet::new(),
stats_tx, stats_tx,
last_stale_check: get_time_now(), last_stale_check: Instant::now(),
})), })),
reply_key_storage, reply_key_storage,
reply_controller_sender, reply_controller_sender,
shutdown_token, task_client,
} }
} }
#[allow(clippy::panic)]
async fn disconnect_sender(&mut self) { async fn disconnect_sender(&mut self) {
let mut guard = self.inner.lock().await; let mut guard = self.inner.lock().await;
if guard.message_sender.is_none() { if guard.message_sender.is_none() {
@@ -209,7 +208,6 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
guard.message_sender = None; guard.message_sender = None;
} }
#[allow(clippy::panic)]
async fn connect_sender(&mut self, sender: ReconstructedMessagesSender) { async fn connect_sender(&mut self, sender: ReconstructedMessagesSender) {
let mut guard = self.inner.lock().await; let mut guard = self.inner.lock().await;
if guard.message_sender.is_some() { if guard.message_sender.is_some() {
@@ -223,7 +221,10 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
let stored_messages = std::mem::take(&mut guard.messages); let stored_messages = std::mem::take(&mut guard.messages);
if !stored_messages.is_empty() { if !stored_messages.is_empty() {
if let Err(err) = sender.unbounded_send(stored_messages) { if let Err(err) = sender.unbounded_send(stored_messages) {
error!("The sender channel we just received is already invalidated - {err:?}"); error!(
"The sender channel we just received is already invalidated - {:?}",
err
);
// put the values back to the buffer // put the values back to the buffer
// the returned error has two fields: err: SendError and val: T, // the returned error has two fields: err: SendError and val: T,
// where val is the value that was failed to get sent; // where val is the value that was failed to get sent;
@@ -309,15 +310,13 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
} }
}; };
if !reply_surbs.is_empty() { if let Err(err) = self.reply_controller_sender.send_additional_surbs(
if let Err(err) = self.reply_controller_sender.send_additional_surbs( msg.sender_tag,
msg.sender_tag, reply_surbs,
reply_surbs, from_surb_request,
from_surb_request, ) {
) { if !self.task_client.is_shutdown_poll() {
if !self.shutdown_token.is_cancelled() { error!("{err}");
error!("{err}");
}
} }
} }
} }
@@ -338,7 +337,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
.reply_controller_sender .reply_controller_sender
.send_additional_surbs_request(*recipient, amount) .send_additional_surbs_request(*recipient, amount)
{ {
if !self.shutdown_token.is_cancelled() { if !self.task_client.is_shutdown_poll() {
error!("{err}"); error!("{err}");
} }
} }
@@ -465,22 +464,22 @@ pub enum ReceivedBufferMessage {
ReceiverDisconnect, ReceiverDisconnect,
} }
pub(crate) struct RequestReceiver<R: MessageReceiver> { struct RequestReceiver<R: MessageReceiver> {
received_buffer: ReceivedMessagesBuffer<R>, received_buffer: ReceivedMessagesBuffer<R>,
query_receiver: ReceivedBufferRequestReceiver, query_receiver: ReceivedBufferRequestReceiver,
shutdown_token: ShutdownToken, task_client: TaskClient,
} }
impl<R: MessageReceiver> RequestReceiver<R> { impl<R: MessageReceiver> RequestReceiver<R> {
fn new( fn new(
received_buffer: ReceivedMessagesBuffer<R>, received_buffer: ReceivedMessagesBuffer<R>,
query_receiver: ReceivedBufferRequestReceiver, query_receiver: ReceivedBufferRequestReceiver,
shutdown_token: ShutdownToken, task_client: TaskClient,
) -> Self { ) -> Self {
RequestReceiver { RequestReceiver {
received_buffer, received_buffer,
query_receiver, query_receiver,
shutdown_token, task_client,
} }
} }
@@ -495,71 +494,67 @@ impl<R: MessageReceiver> RequestReceiver<R> {
} }
} }
pub(crate) async fn run(&mut self) { async fn run(&mut self) {
debug!("Started RequestReceiver with graceful shutdown support"); debug!("Started RequestReceiver with graceful shutdown support");
loop { while !self.task_client.is_shutdown() {
tokio::select! { tokio::select! {
biased; biased;
_ = self.shutdown_token.cancelled() => { _ = self.task_client.recv() => {
tracing::trace!("RequestReceiver: Received shutdown"); log::trace!("RequestReceiver: Received shutdown");
break;
} }
request = self.query_receiver.next() => { request = self.query_receiver.next() => {
if let Some(message) = request { if let Some(message) = request {
self.handle_message(message).await self.handle_message(message).await
} else { } else {
tracing::trace!("RequestReceiver: Stopping since channel closed"); log::trace!("RequestReceiver: Stopping since channel closed");
self.shutdown_token.cancelled().await;
break; break;
} }
}, },
} }
} }
tracing::debug!("RequestReceiver: Exiting"); self.task_client.recv().await;
log::debug!("RequestReceiver: Exiting");
} }
} }
pub(crate) struct FragmentedMessageReceiver<R: MessageReceiver> { struct FragmentedMessageReceiver<R: MessageReceiver> {
received_buffer: ReceivedMessagesBuffer<R>, received_buffer: ReceivedMessagesBuffer<R>,
mixnet_packet_receiver: MixnetMessageReceiver, mixnet_packet_receiver: MixnetMessageReceiver,
shutdown_token: ShutdownToken, task_client: TaskClient,
} }
impl<R: MessageReceiver> FragmentedMessageReceiver<R> { impl<R: MessageReceiver> FragmentedMessageReceiver<R> {
fn new( fn new(
received_buffer: ReceivedMessagesBuffer<R>, received_buffer: ReceivedMessagesBuffer<R>,
mixnet_packet_receiver: MixnetMessageReceiver, mixnet_packet_receiver: MixnetMessageReceiver,
shutdown_token: ShutdownToken, task_client: TaskClient,
) -> Self { ) -> Self {
FragmentedMessageReceiver { FragmentedMessageReceiver {
received_buffer, received_buffer,
mixnet_packet_receiver, mixnet_packet_receiver,
shutdown_token, task_client,
} }
} }
pub(crate) async fn run(&mut self) -> Result<(), MessageRecoveryError> { async fn run(&mut self) -> Result<(), MessageRecoveryError> {
debug!("Started FragmentedMessageReceiver with graceful shutdown support"); debug!("Started FragmentedMessageReceiver with graceful shutdown support");
loop { while !self.task_client.is_shutdown() {
tokio::select! { tokio::select! {
biased;
_ = self.shutdown_token.cancelled() => {
tracing::trace!("FragmentedMessageReceiver: Received shutdown");
break;
}
new_messages = self.mixnet_packet_receiver.next() => { new_messages = self.mixnet_packet_receiver.next() => {
if let Some(new_messages) = new_messages { if let Some(new_messages) = new_messages {
self.received_buffer.handle_new_received(new_messages).await?; self.received_buffer.handle_new_received(new_messages).await?;
} else { } else {
tracing::trace!("FragmentedMessageReceiver: Stopping since channel closed"); log::trace!("FragmentedMessageReceiver: Stopping since channel closed");
self.shutdown_token.cancelled().await;
break; break;
} }
}, },
_ = self.task_client.recv_with_delay() => {
log::trace!("FragmentedMessageReceiver: Received shutdown");
}
} }
} }
tracing::debug!("FragmentedMessageReceiver: Exiting"); self.task_client.recv_timeout().await;
log::debug!("FragmentedMessageReceiver: Exiting");
Ok(()) Ok(())
} }
} }
@@ -577,31 +572,42 @@ impl<R: MessageReceiver + Clone + Send + 'static> ReceivedMessagesBufferControll
reply_key_storage: SentReplyKeys, reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender, reply_controller_sender: ReplyControllerSender,
metrics_reporter: ClientStatsSender, metrics_reporter: ClientStatsSender,
shutdown_token: ShutdownToken, task_client: TaskClient,
) -> Self { ) -> Self {
let received_buffer = ReceivedMessagesBuffer::new( let received_buffer = ReceivedMessagesBuffer::new(
local_encryption_keypair, local_encryption_keypair,
reply_key_storage, reply_key_storage,
reply_controller_sender, reply_controller_sender,
metrics_reporter, metrics_reporter,
shutdown_token.clone(), task_client.fork("received_messages_buffer"),
); );
ReceivedMessagesBufferController { ReceivedMessagesBufferController {
fragmented_message_receiver: FragmentedMessageReceiver::new( fragmented_message_receiver: FragmentedMessageReceiver::new(
received_buffer.clone(), received_buffer.clone(),
mixnet_packet_receiver, mixnet_packet_receiver,
shutdown_token.clone(), task_client.fork("fragmented_message_receiver"),
), ),
request_receiver: RequestReceiver::new( request_receiver: RequestReceiver::new(
received_buffer, received_buffer,
query_receiver, query_receiver,
shutdown_token.clone(), task_client.with_suffix("request_receiver"),
), ),
} }
} }
pub(crate) fn into_tasks(self) -> (FragmentedMessageReceiver<R>, RequestReceiver<R>) { pub fn start(self) {
(self.fragmented_message_receiver, self.request_receiver) let mut fragmented_message_receiver = self.fragmented_message_receiver;
let mut request_receiver = self.request_receiver;
spawn_future(async move {
match fragmented_message_receiver.run().await {
Ok(_) => {}
Err(e) => error!("{e}"),
}
});
spawn_future(async move {
request_receiver.run().await;
});
} }
} }
@@ -1,169 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use nym_topology::NymTopologyMetadata;
use nym_validator_client::models::{
EpochId, KeyRotationId, KeyRotationInfoResponse, KeyRotationState,
};
use std::time::Duration;
use time::OffsetDateTime;
#[derive(Clone, Copy)]
pub(crate) enum SurbRefreshState {
WaitingForNextRotation { last_known: KeyRotationId },
ScheduledForNextInvocation,
}
#[derive(Clone, Copy)]
pub(crate) struct ReferenceEpoch {
pub(crate) absolute_epoch_id: EpochId,
pub(crate) start_time: OffsetDateTime,
}
#[derive(Clone, Copy)]
pub(crate) struct KeyRotationConfig {
pub(crate) epoch_duration: Duration,
pub(crate) rotation_state: KeyRotationState,
pub(crate) reference_epoch: ReferenceEpoch,
}
impl From<KeyRotationInfoResponse> for KeyRotationConfig {
fn from(value: KeyRotationInfoResponse) -> Self {
KeyRotationConfig {
epoch_duration: value.details.epoch_duration,
rotation_state: value.details.key_rotation_state,
reference_epoch: ReferenceEpoch {
absolute_epoch_id: value.details.current_absolute_epoch_id,
start_time: value.details.current_epoch_start,
},
}
}
}
impl KeyRotationConfig {
pub(crate) fn rotation_lifetime(&self) -> Duration {
(self.rotation_state.validity_epochs + 1) * self.epoch_duration
}
pub(crate) fn key_rotation_id(&self, current_absolute_epoch_id: EpochId) -> KeyRotationId {
self.rotation_state
.key_rotation_id(current_absolute_epoch_id)
}
// this is called with the assumption that now is always > reference epoch start
pub(crate) fn expected_current_epoch_id(&self, now: OffsetDateTime) -> EpochId {
let diff_secs = (now - self.reference_epoch.start_time).as_seconds_f64();
let epochs = (diff_secs / self.epoch_duration.as_secs_f64()).floor() as u32;
self.reference_epoch.absolute_epoch_id + epochs
}
fn initial_rotation_epoch_start(&self) -> OffsetDateTime {
let epochs_diff = self
.reference_epoch
.absolute_epoch_id
.saturating_sub(self.rotation_state.initial_epoch_id);
self.reference_epoch.start_time - epochs_diff * self.epoch_duration
}
pub(crate) fn key_rotation_start(&self, key_rotation_id: KeyRotationId) -> OffsetDateTime {
let rotation_duration = self.rotation_state.validity_epochs * self.epoch_duration;
let initial_start = self.initial_rotation_epoch_start();
// note: key rotation starts from 0
initial_start + rotation_duration * key_rotation_id
}
pub(crate) fn expected_current_key_rotation_id(&self, now: OffsetDateTime) -> KeyRotationId {
let expected_current_epoch = self.expected_current_epoch_id(now);
self.key_rotation_id(expected_current_epoch)
}
pub(crate) fn expected_current_key_rotation_start(
&self,
now: OffsetDateTime,
) -> OffsetDateTime {
let expected_current_key_rotation_id = self.expected_current_key_rotation_id(now);
self.key_rotation_start(expected_current_key_rotation_id)
}
pub(crate) fn epoch_stuck(&self, topology_metadata: NymTopologyMetadata) -> bool {
// add leeway of 2mins each direction since transition is not instantaneous
let lower_bound = topology_metadata.refreshed_at - Duration::from_secs(2);
let upper_bound = topology_metadata.refreshed_at + Duration::from_secs(2);
let expected_epoch_lower = self.expected_current_epoch_id(lower_bound);
let expected_epoch_upper = self.expected_current_epoch_id(upper_bound);
topology_metadata.absolute_epoch_id != expected_epoch_lower
&& topology_metadata.absolute_epoch_id != expected_epoch_upper
}
}
#[cfg(test)]
mod tests {
use super::*;
use time::macros::datetime;
fn mock_config() -> KeyRotationConfig {
KeyRotationConfig {
epoch_duration: Duration::from_secs(60 * 60),
rotation_state: KeyRotationState {
validity_epochs: 10,
initial_epoch_id: 80,
},
reference_epoch: ReferenceEpoch {
absolute_epoch_id: 100,
start_time: datetime!(2025-06-30 12:00:00+00:00),
},
}
}
#[test]
fn expected_current_key_rotation_start() {
// rot0: 80-89
// rot1: 90-99
// rot2: 100-109
// rot3: 110-119
// ... etc
let cfg = mock_config();
assert_eq!(
cfg.initial_rotation_epoch_start(),
datetime!(2025-06-29 16:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 12:00:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 100);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 2);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 12:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 12:30:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 100);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 2);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 12:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 13:01:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 101);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 2);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 12:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 22:02:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 110);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 3);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 22:00:00+00:00)
);
}
}
File diff suppressed because it is too large Load Diff
@@ -1,901 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::real_messages_control::acknowledgement_control::PendingAcknowledgement;
use crate::client::real_messages_control::message_handler::{
FragmentWithMaxRetransmissions, MessageHandler, PreparationError,
};
use crate::client::replies::reply_controller::key_rotation_helpers::SurbRefreshState;
use crate::client::replies::reply_controller::Config;
use crate::client::topology_control::TopologyAccessor;
use crate::client::transmission_buffer::TransmissionBuffer;
use futures::channel::oneshot;
use nym_client_core_surb_storage::{ReceivedReplySurb, ReceivedReplySurbsMap};
use nym_crypto::aes::cipher::crypto_common::rand_core::CryptoRng;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::anonymous_replies::ReplySurbWithKeyRotation;
use nym_sphinx::chunking::fragment::FragmentIdentifier;
use nym_task::connections::{ConnectionId, TransmissionLane};
use nym_topology::NymTopologyMetadata;
use rand::Rng;
use std::cmp::{max, min};
use std::collections::btree_map::Entry;
use std::collections::{BTreeMap, HashMap};
use std::mem;
use std::sync::{Arc, Weak};
use time::OffsetDateTime;
use tracing::{debug, error, info, trace, warn};
struct SenderData {
current_clear_rerequest_counter: usize,
pending_replies: TransmissionBuffer<FragmentWithMaxRetransmissions>,
pending_retransmissions: BTreeMap<FragmentIdentifier, Weak<PendingAcknowledgement>>,
last_request_failure: OffsetDateTime,
}
impl Default for SenderData {
fn default() -> Self {
SenderData {
current_clear_rerequest_counter: 0,
pending_replies: Default::default(),
pending_retransmissions: Default::default(),
last_request_failure: OffsetDateTime::UNIX_EPOCH,
}
}
}
impl SenderData {
fn total_pending(&self) -> usize {
let pending_replies = self.pending_replies.total_size();
let pending_retransmissions = self.pending_retransmissions.len();
let total_pending = pending_retransmissions + pending_replies;
debug!("total queue size: {total_pending} = pending data {pending_replies} + pending retransmission {pending_retransmissions}");
total_pending
}
pub(crate) fn increment_current_clear_rerequest_counter(&mut self) {
self.current_clear_rerequest_counter += 1;
}
pub(crate) fn reset_current_clear_rerequest_counter(&mut self) {
self.current_clear_rerequest_counter = 0;
}
pub(crate) fn reset_last_request_failure(&mut self, now: OffsetDateTime) -> OffsetDateTime {
mem::replace(&mut self.last_request_failure, now)
}
}
/// Reply controller responsible for controlling receiver-related part
/// of replies, such as requesting additional reply SURBs
pub struct ReceiverReplyController<R> {
config: Config,
surb_refresh_state: SurbRefreshState,
topology_access: TopologyAccessor,
surb_senders: HashMap<AnonymousSenderTag, SenderData>,
unavailable: HashMap<AnonymousSenderTag, OffsetDateTime>,
surbs_storage: ReceivedReplySurbsMap,
// TODO: incorporate that field at some point
// and use binomial distribution to determine the expected required number
// of surbs required to send the message through
// expected_reliability: f32,
message_handler: MessageHandler<R>,
}
impl<R> ReceiverReplyController<R>
where
R: CryptoRng + Rng,
{
pub(crate) fn new(
config: Config,
storage: ReceivedReplySurbsMap,
message_handler: MessageHandler<R>,
) -> Self {
let topology_access = message_handler.topology_access_handle().clone();
ReceiverReplyController {
config,
surb_refresh_state: SurbRefreshState::WaitingForNextRotation {
last_known: config
.key_rotation
.expected_current_key_rotation_id(OffsetDateTime::now_utc()),
},
topology_access,
surb_senders: Default::default(),
unavailable: Default::default(),
surbs_storage: storage,
message_handler,
}
}
fn get_or_create_surb_sender(&mut self, tag: &AnonymousSenderTag) -> &mut SenderData {
self.surb_senders.entry(*tag).or_default()
}
async fn current_topology_metadata(&self) -> Option<NymTopologyMetadata> {
self.topology_access.current_metadata().await
}
fn insert_pending_replies<I: IntoIterator<Item = FragmentWithMaxRetransmissions>>(
&mut self,
recipient: &AnonymousSenderTag,
fragments: I,
lane: TransmissionLane,
) {
trace!("buffering pending replies for {recipient}");
self.surb_senders
.entry(*recipient)
.or_default()
.pending_replies
.store(&lane, fragments)
}
fn re_insert_pending_replies(
&mut self,
recipient: &AnonymousSenderTag,
fragments: Vec<(TransmissionLane, FragmentWithMaxRetransmissions)>,
) {
trace!("re-inserting pending replies for {recipient}");
// the buffer should ALWAYS exist at this point, if it doesn't, it's a bug...
self.surb_senders
.entry(*recipient)
.or_default()
.pending_replies
.store_multiple(fragments)
}
fn re_insert_pending_retransmission(
&mut self,
recipient: &AnonymousSenderTag,
data: Vec<Arc<PendingAcknowledgement>>,
) {
trace!("re-inserting pending retransmissions for {recipient}");
// SAFETY: the underlying entry MUST exist as we've just got data from there
// and we hold a mut reference
#[allow(clippy::expect_used)]
let map_entry = &mut self
.surb_senders
.get_mut(recipient)
.expect("our pending retransmission entry is somehow gone!")
.pending_retransmissions;
for pending in data {
// if it's 0, we don't need to do anything - we just got that ack!
if Arc::strong_count(&pending) > 1 {
let id = pending.inner_fragment_identifier();
let downgraded = Arc::downgrade(&pending);
map_entry.insert(id, downgraded);
}
}
}
fn should_request_more_surbs(&self, target: &AnonymousSenderTag) -> bool {
trace!("checking if we should request more surbs from {target}");
let total_queue = self
.surb_senders
.get(target)
.map(|pending| pending.total_pending())
.unwrap_or_default();
// only consider 'fresh' surbs
let available_surbs = self.surbs_storage.available_fresh_surbs(target);
let pending_surbs = self.surbs_storage.pending_reception(target) as usize;
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_surbs_threshold = self.surbs_storage.max_surb_threshold();
let min_surbs_threshold_buffer =
self.config.reply_surbs.minimum_reply_surb_threshold_buffer;
// After clearing the queue, we want to have at least `min_surbs_threshold` surbs available
// and reserved for requesting additional surbs, and in addition to that we also want to
// have `min_surbs_threshold_buffer` surbs available proactively.
let target_surbs_after_clearing_queue = min_surbs_threshold + min_surbs_threshold_buffer;
// Check if we have enough surbs to handle the total queue and maintain minimum thresholds
let total_required_surbs = total_queue + target_surbs_after_clearing_queue;
let total_available_surbs = pending_surbs + available_surbs;
debug!("available surbs: {available_surbs} pending surbs: {pending_surbs} threshold range: {min_surbs_threshold}..+{min_surbs_threshold_buffer}..{max_surbs_threshold}");
// We should request more surbs if:
// 1. We haven't hit the maximum surb threshold, and
// 2. We don't have enough surbs to handle the queue plus minimum thresholds
let is_below_max_threshold = total_available_surbs < max_surbs_threshold;
let is_below_required_surbs = total_available_surbs < total_required_surbs;
is_below_max_threshold && is_below_required_surbs
}
pub(crate) async fn handle_send_reply(
&mut self,
recipient_tag: AnonymousSenderTag,
data: Vec<u8>,
lane: TransmissionLane,
max_retransmissions: Option<u32>,
) {
if !self.surbs_storage.contains_surbs_for(&recipient_tag) {
if self
.unavailable
.insert(recipient_tag, OffsetDateTime::now_utc())
.is_none()
{
// don't report it every single time
warn!("received reply request for {recipient_tag} but we don't have any surbs stored for that recipient!");
} else {
trace!("received reply request for {recipient_tag} but we don't have any surbs stored for that recipient!");
}
return;
}
trace!("handling reply to {recipient_tag}");
let mut fragments = self.message_handler.split_reply_message(data);
let total_size = fragments.len();
trace!("This reply requires {total_size} SURBs");
// for the purposes of sending reply, do allow using possibly stale entries
let available_surbs = self.surbs_storage.available_surbs(&recipient_tag);
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_to_send = if available_surbs > min_surbs_threshold {
min(fragments.len(), available_surbs - min_surbs_threshold)
} else {
0
};
if max_to_send > 0 {
let (surbs, surbs_left) = self
.surbs_storage
.get_reply_surbs(&recipient_tag, max_to_send);
debug!(
"retrieved {} reply surbs. {surbs_left} surbs remaining in storage",
surbs.as_ref().map(|s| s.len()).unwrap_or_default()
);
if let Some(reply_surbs) = surbs {
let to_send = fragments
.drain(..reply_surbs.len())
.map(|f| FragmentWithMaxRetransmissions {
fragment: f,
max_retransmissions,
})
.collect::<Vec<_>>();
if let Err(err) = self
.message_handler
.try_send_reply_chunks_on_lane(
recipient_tag,
to_send.clone(),
reply_surbs,
lane,
)
.await
{
let err = err.return_unused_surbs(&self.surbs_storage, &recipient_tag);
warn!("failed to send reply to {recipient_tag}: {err}");
info!(
"buffering {no_fragments} fragments for {recipient_tag}",
no_fragments = to_send.len()
);
self.insert_pending_replies(&recipient_tag, to_send, lane);
}
}
}
// if there's leftover data we didn't send because we didn't have enough (or any) surbs - buffer it
if !fragments.is_empty() {
// Ideally we should have enough surbs above the minimum threshold to handle sending
// new replies without having to first request more surbs. That's why I'd like to log
// these cases as they might indicate a problem with the surb management.
debug!(
"buffering {no_fragments} fragments for {recipient_tag}",
no_fragments = fragments.len()
);
let fragments: Vec<_> = fragments
.into_iter()
.map(|fragment| FragmentWithMaxRetransmissions {
fragment,
max_retransmissions,
})
.collect();
self.insert_pending_replies(&recipient_tag, fragments, lane);
}
if self.should_request_more_surbs(&recipient_tag) {
self.request_reply_surbs_for_queue_clearing(recipient_tag)
.await;
}
}
async fn request_additional_reply_surbs(
&mut self,
target: AnonymousSenderTag,
amount: u32,
) -> Result<(), PreparationError> {
debug!("requesting {amount} additional reply surbs for {target}");
let (reply_surb, _) = self
.surbs_storage
.get_reply_surb_ignoring_threshold(&target);
let reply_surb = reply_surb.ok_or(PreparationError::NotEnoughSurbs {
available: 0,
required: 1,
})?;
if let Err(err) = self
.message_handler
.try_request_additional_reply_surbs(target, reply_surb, amount)
.await
{
let err = err.return_unused_surbs(&self.surbs_storage, &target);
warn!("failed to request additional surbs from {target}: {err}",);
return Err(err);
} else {
self.surbs_storage
.increment_pending_reception(&target, amount);
}
Ok(())
}
async fn try_clear_pending_retransmission(&mut self, target: AnonymousSenderTag) {
trace!("trying to clear pending retransmission queue");
let available_surbs = self.surbs_storage.available_surbs(&target);
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_to_clear = if available_surbs > min_surbs_threshold {
available_surbs - min_surbs_threshold
} else {
trace!("we don't have enough surbs for retransmission queue clearing...");
return;
};
trace!("we can clear up to {max_to_clear} entries");
let Some(pending) = self.surb_senders.get_mut(&target) else {
trace!("no pending entry for {target}!");
return;
};
let mut to_take = Vec::new();
while to_take.len() < max_to_clear {
if let Some((_, data)) = pending.pending_retransmissions.pop_first() {
// no need to do anything if we failed to upgrade the reference,
// it means we got the ack while the data was waiting in the queue
if let Some(upgraded) = data.upgrade() {
to_take.push(upgraded)
}
} else {
// our map is empty!
break;
}
}
if to_take.is_empty() {
// no need to do anything
return;
}
let (surbs_for_reply, _) = self.surbs_storage.get_reply_surbs(&target, to_take.len());
let Some(surbs_for_reply) = surbs_for_reply else {
error!("somehow different task has stolen our reply surbs! - this should have been impossible");
self.re_insert_pending_retransmission(&target, to_take);
return;
};
let to_send_vec = to_take.iter().map(|ack| ack.fragment_data()).collect();
let prepared_fragments = match self
.message_handler
.prepare_reply_chunks_for_sending(to_send_vec, surbs_for_reply)
.await
{
Ok(prepared) => prepared,
Err(err) => {
let err = err.return_unused_surbs(&self.surbs_storage, &target);
self.re_insert_pending_retransmission(&target, to_take);
warn!("failed to clear pending retransmission queue for {target}: {err}",);
return;
}
};
// we can't fail at this point, so drop all references to acks so that timer updates wouldn't blow up
drop(to_take);
self.message_handler
.send_retransmission_reply_chunks(prepared_fragments, TransmissionLane::Retransmission)
.await;
}
fn pop_at_most_pending_replies(
&mut self,
from: &AnonymousSenderTag,
amount: usize,
) -> Option<Vec<(TransmissionLane, FragmentWithMaxRetransmissions)>> {
// if possible, pop all pending replies, if not, pop only entries for which we'd have a reply surb
let pending = self.surb_senders.get_mut(from)?;
let total = pending.pending_replies.total_size();
trace!("pending queue has {total} elements");
if total == 0 {
return None;
}
pending
.pending_replies
.pop_at_most_n_next_messages_at_random(amount)
}
#[allow(clippy::panic)]
async fn try_clear_pending_queue(&mut self, target: AnonymousSenderTag) {
trace!("trying to clear pending queue");
let available_surbs = self.surbs_storage.available_surbs(&target);
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_to_clear = if available_surbs > min_surbs_threshold {
available_surbs - min_surbs_threshold
} else {
trace!("we don't have enough surbs for queue clearing...");
return;
};
trace!("we can clear up to {max_to_clear} entries");
// we're guaranteed to not get more entries than we have reply surbs for
if let Some(to_send) = self.pop_at_most_pending_replies(&target, max_to_clear) {
let to_send_clone = to_send.clone();
if to_send_clone.is_empty() {
panic!(
"please let the devs know if you ever see this message (reply_controller.rs)"
);
}
let (surbs_for_reply, _) = self
.surbs_storage
.get_reply_surbs(&target, to_send_clone.len());
let Some(surbs_for_reply) = surbs_for_reply else {
error!("somehow different task has stolen our reply surbs! - this should have been impossible");
self.re_insert_pending_replies(&target, to_send);
return;
};
if let Err(err) = self
.message_handler
.try_send_reply_chunks(target, to_send_clone, surbs_for_reply)
.await
{
let err = err.return_unused_surbs(&self.surbs_storage, &target);
self.re_insert_pending_replies(&target, to_send);
warn!("failed to clear pending queue for {target}: {err}");
}
} else {
trace!("the pending queue is empty");
}
}
fn reset_rerequest_counter(&mut self, from: &AnonymousSenderTag) {
if let Some(pending) = self.surb_senders.get_mut(from) {
pending.reset_current_clear_rerequest_counter()
}
}
pub(crate) async fn handle_received_surbs(
&mut self,
from: AnonymousSenderTag,
reply_surbs: Vec<ReplySurbWithKeyRotation>,
from_surb_request: bool,
) {
trace!("handling received surbs");
// clear the requesting flag since we should have been asking for surbs
if from_surb_request {
self.surbs_storage
.decrement_pending_reception(&from, reply_surbs.len() as u32);
}
// store received surbs
self.surbs_storage.insert_fresh_surbs(&from, reply_surbs);
// reset, if applicable, request counter
self.reset_rerequest_counter(&from);
// use as many as we can for clearing pending retransmission queue
self.try_clear_pending_retransmission(from).await;
// use as many as we can for clearing pending 'normal' queue
self.try_clear_pending_queue(from).await;
// if we have to, request more
if self.should_request_more_surbs(&from) {
self.request_reply_surbs_for_queue_clearing(from).await;
}
}
fn buffer_pending_ack(
&mut self,
recipient: AnonymousSenderTag,
ack_ref: Arc<PendingAcknowledgement>,
weak_ack_ref: Weak<PendingAcknowledgement>,
) {
let frag_id = ack_ref.inner_fragment_identifier();
let pending = self.surb_senders.entry(recipient).or_default();
if let Entry::Vacant(e) = pending.pending_retransmissions.entry(frag_id) {
e.insert(weak_ack_ref);
} else {
warn!(
"we're already trying to retransmit {frag_id}. We must be really behind in surbs!"
);
}
}
pub(crate) async fn handle_reply_retransmission(
&mut self,
recipient_tag: AnonymousSenderTag,
timed_out_ack: Weak<PendingAcknowledgement>,
extra_surbs_request: bool,
) {
// seems we got the ack in the end
let ack_ref = match timed_out_ack.upgrade() {
Some(ack) => ack,
None => {
debug!("we received the ack for one of the reply packets as we were putting it in the retransmission queue");
return;
}
};
// if this is retransmission for obtaining additional reply surbs,
// we can dip below the storage threshold
let (maybe_reply_surb, _) = if extra_surbs_request {
self.surbs_storage
.get_reply_surb_ignoring_threshold(&recipient_tag)
} else {
self.surbs_storage.get_reply_surb(&recipient_tag)
};
if let Some(reply_surb) = maybe_reply_surb {
match self
.message_handler
.try_prepare_single_reply_chunk_for_sending(reply_surb, ack_ref.fragment_data())
.await
{
Ok(prepared) => {
// drop the ack ref so that controller would not panic on `UpdateTimer` if that task
// got to handle the action before this function terminated (which is very much
// possible if `forward_messages` takes a while)
drop(ack_ref);
self.message_handler
.update_ack_delay(prepared.fragment_identifier, prepared.total_delay);
self.message_handler
.forward_messages(vec![prepared.into()], TransmissionLane::Retransmission)
.await;
}
Err(err) => {
let err = err.return_unused_surbs(&self.surbs_storage, &recipient_tag);
warn!("failed to prepare message for retransmission - {err}");
// we buffer that packet and to try another day
self.buffer_pending_ack(recipient_tag, ack_ref, timed_out_ack);
if self.should_request_more_surbs(&recipient_tag) {
self.request_reply_surbs_for_queue_clearing(recipient_tag)
.await;
}
}
};
} else {
self.buffer_pending_ack(recipient_tag, ack_ref, timed_out_ack);
if self.should_request_more_surbs(&recipient_tag) {
self.request_reply_surbs_for_queue_clearing(recipient_tag)
.await;
}
}
}
// to be honest this doesn't make a lot of sense in the context of `connection_id`,
// it should really be asked per tag
pub(crate) fn handle_lane_queue_length(
&self,
connection_id: ConnectionId,
response_channel: oneshot::Sender<usize>,
) {
// TODO: if we ever have duplicate ids for different senders, it means our rng is super weak
// thus I don't think we have to worry about it?
let lane = TransmissionLane::ConnectionId(connection_id);
for buf in self.surb_senders.values().map(|p| &p.pending_replies) {
if let Some(length) = buf.lane_length(&lane) {
if response_channel.send(length).is_err() {
error!("the requester for lane queue length has dropped the response channel!")
}
return;
}
}
// make sure that if we didn't find that lane, we reply with 0
if response_channel.send(0).is_err() {
error!("the requester for lane queue length has dropped the response channel!")
}
}
// TODO: modify this method to more accurately determine the amount of surbs it needs to request
// it should take into consideration the average latency, sending rate and queue size.
// it should request as many surbs as it takes to saturate its sending rate before next batch arrives
async fn request_reply_surbs_for_queue_clearing(&mut self, target: AnonymousSenderTag) {
trace!("requesting surbs for queue clearing");
let total_queue = self
.surb_senders
.get(&target)
.map(|pending| pending.total_pending() as u32)
.unwrap_or_default();
let min_surbs_buffer = self.config.reply_surbs.minimum_reply_surb_threshold_buffer as u32;
// To proactively request additional surbs, we aim to have a buffer of extra surbs in our
// storage.
let total_queue_with_buffer = total_queue + min_surbs_buffer;
let request_size = min(
self.config.reply_surbs.maximum_reply_surb_request_size,
max(
total_queue_with_buffer,
self.config.reply_surbs.minimum_reply_surb_request_size,
),
);
if let Err(err) = self
.request_additional_reply_surbs(target, request_size)
.await
{
let now = OffsetDateTime::now_utc();
let sender_info = self.get_or_create_surb_sender(&target);
let last_failure = sender_info.reset_last_request_failure(now);
// only log at higher level if it's the first time this error has occurred in a while
if now - last_failure > time::Duration::seconds(30) {
warn!("failed to request more surbs to clear pending queue of size {total_queue} (attempted to request: {request_size}): {err}")
} else {
debug!("failed to request more surbs to clear pending queue of size {total_queue} (attempted to request: {request_size}): {err}")
}
}
}
pub(crate) async fn inspect_stale_pending_data(&mut self) {
let mut to_request = Vec::new();
let mut to_remove = Vec::new();
let now = OffsetDateTime::now_utc();
for (pending_reply_target, vals) in self.surb_senders.iter_mut() {
// for now recreate old behaviour
let retransmission_buf = &vals.pending_replies;
if retransmission_buf.is_empty() {
continue;
}
let Some(last_received_time) = self
.surbs_storage
.surbs_last_received_at(pending_reply_target)
else {
error!("we have {} pending replies for {pending_reply_target}, but we somehow never received any reply surbs from them!", retransmission_buf.total_size());
to_remove.push(*pending_reply_target);
continue;
};
let diff = now - last_received_time;
let max_rerequest_wait = self
.config
.reply_surbs
.maximum_reply_surb_rerequest_waiting_period;
let max_drop_wait = self
.config
.reply_surbs
.maximum_reply_surb_drop_waiting_period;
let max_rerequests = self.config.reply_surbs.maximum_reply_surbs_rerequests;
// if we have already requested extra surbs because of the stale entry,
// don't do it again (otherwise we'll get stuck in a constant cycle of requesting more surbs
// if client is offline)
if vals.current_clear_rerequest_counter > max_rerequests {
to_remove.push(*pending_reply_target);
debug!("we have reached the maximum threshold of attempting to request surbs from {pending_reply_target}. dropping the sender");
continue;
}
if diff > max_rerequest_wait {
if diff > max_drop_wait {
to_remove.push(*pending_reply_target)
} else {
debug!("We haven't received any surbs in {} from {pending_reply_target}. Going to explicitly ask for more", humantime::format_duration(diff.unsigned_abs()));
vals.increment_current_clear_rerequest_counter();
to_request.push(*pending_reply_target);
}
}
}
for pending_reply_target in to_request {
self.request_reply_surbs_for_queue_clearing(pending_reply_target)
.await;
self.surbs_storage
.reset_pending_reception(&pending_reply_target)
}
for to_remove in to_remove {
// TODO: in the 'old' version we just removed pending messages,
// not retransmissions, but I think those should follow the same logic.
// if something breaks because of that. I guess here is your explanation, future reader
self.surb_senders.remove(&to_remove);
}
}
pub(crate) async fn check_surb_refresh(&mut self) {
let Some(current_rotation_id) = self.topology_access.current_key_rotation_id().await else {
warn!("failed to retrieve current key rotation id from the network topology");
return;
};
if let SurbRefreshState::WaitingForNextRotation { last_known } = self.surb_refresh_state {
if last_known == current_rotation_id {
trace!("no changes in key rotation id");
} else {
// key rotation actually changed and given the polling rate (1/8th epoch) we should have plenty
// of time to perform the upgrade.
// but wait for one more call before doing this so that the clients could also resync
// their topologies and discover new rotation
self.surb_refresh_state = SurbRefreshState::ScheduledForNextInvocation;
}
return;
}
// here we are in `SurbRefreshState::ScheduledForNextInvocation` state
let mut marked_as_stale = HashMap::new();
// 1. mark all existing surbs we have as possibly stale
for mut map_entry in self.surbs_storage.as_raw_iter_mut() {
let (sender, received) = map_entry.pair_mut();
let num_downgraded = received.downgrade_freshness();
trace!("{sender}: {num_downgraded} downgraded");
if num_downgraded != 0 {
marked_as_stale.insert(*sender, num_downgraded);
}
}
// 2. attempt to re-request the equivalent number of fresh surbs
// TODO PROBLEM: if our request gets lost, we might be in trouble...
// we need some sort of retry mechanism
for (sender, num_to_request) in marked_as_stale {
if self
.request_additional_reply_surbs(sender, num_to_request as u32)
.await
.is_err()
{
warn!("surb refresh request failed")
}
}
self.surb_refresh_state = SurbRefreshState::WaitingForNextRotation {
last_known: current_rotation_id,
};
}
pub(crate) async fn inspect_and_clear_stale_data(&mut self, now: OffsetDateTime) {
// technically we don't know if epoch is stuck, but we're flying in blind here,
// so we have to assume the worst and not purge anything depending on proper epoch progression
let is_epoch_stuck = self
.current_topology_metadata()
.await
.map(|m| self.config.key_rotation.epoch_stuck(m))
.unwrap_or(false);
// expected time of when the CURRENT key rotation has begun
let expected_current_key_rotation_start = self
.config
.key_rotation
.expected_current_key_rotation_start(now);
// expected ID of the CURRENT key rotation
let expected_current_key_rotation = self
.config
.key_rotation
.expected_current_key_rotation_id(now);
// time of the start of one epoch BEFORE the CURRENT rotation has begun
// this indicates the starting time of when packets with the current keys might have been constructed
let prior_epoch_start =
expected_current_key_rotation_start - self.config.key_rotation.epoch_duration;
// time of the start of one epoch AFTER the current rotation has begun
// this indicates the end of transition period and any packets constructed with keys different
// from the current one are definitely invalid
let following_epoch_start =
expected_current_key_rotation_start + self.config.key_rotation.epoch_duration;
// define a closure for validating individual surbs
// (we have to run it twice for different piles)
let basic_surb_retention_logic = |received_surb: &ReceivedReplySurb| {
if is_epoch_stuck {
let diff = now - received_surb.received_at();
return diff < self.config.key_rotation.rotation_lifetime();
}
if received_surb.received_at() < prior_epoch_start {
// it's definitely from previous rotation
return false;
}
let surb_rotation = received_surb.key_rotation();
if surb_rotation.is_unknown() {
// can't do anything, so just retain it
return true;
}
// TODO: will this backfire during transition period where we need surbs to refresh surbs
// and we failed to send a request?
if surb_rotation.is_even() && expected_current_key_rotation % 2 == 1 {
return false;
}
if surb_rotation.is_odd() && expected_current_key_rotation % 2 == 0 {
return false;
}
true
};
// 1. purge full old clients data (this applies to RECEIVER)
self.surbs_storage.retain(|_, received| {
if is_epoch_stuck {
// if epoch is stuck, we can't do much (because we don't know for certain if rotation has advanced)
// apart from the basic check of surbs being received more than maximum lifetime of a rotation
// because at that point we know they must be invalid
let diff = now - received.surbs_last_received_at();
return diff < self.config.key_rotation.rotation_lifetime();
}
// if surbs were received more than 1h before the start of the current rotation,
// they're DEFINITELY invalid.
// if it was up until 1h AFTER the start of the current rotation they MIGHT be valid -
// we don't know for sure, unless the client explicitly attached rotation information
// (which only applies to more recent versions of clients so we can't 100% rely on that)
if received.surbs_last_received_at() < prior_epoch_start {
return false;
}
// 1.1. check individual surbs (same basic logic applies)
received.retain_fresh_surbs(&basic_surb_retention_logic);
// 1.2. check the possibly stale entries
// 1.2.1. check if we're beyond the key rotation transition period,
// if so those surbs are definitely unusable
if now > following_epoch_start {
received.drop_possibly_stale_surbs();
}
// 1.2.2. otherwise continue with the same logic as the fresh ones
received.retain_possibly_stale_surbs(&basic_surb_retention_logic);
// no surbs left, we're not expecting any AND we haven't received anything in a while
// (i.e. sender probably abandoned us)
let max_drop_wait = self
.config
.reply_surbs
.maximum_reply_surb_drop_waiting_period;
let last_received = received.surbs_last_received_at();
let possibly_abandoned = last_received + max_drop_wait < now;
if received.is_empty() && received.pending_reception() == 0 && possibly_abandoned {
return false;
}
true
});
// 1.3 inspect old unavailable receivers to clear any stale data
self.unavailable
.retain(|_, last_reported| now - *last_reported < time::Duration::seconds(30));
}
}
@@ -3,12 +3,12 @@
use crate::client::real_messages_control::acknowledgement_control::PendingAcknowledgement; use crate::client::real_messages_control::acknowledgement_control::PendingAcknowledgement;
use futures::channel::{mpsc, oneshot}; use futures::channel::{mpsc, oneshot};
use log::error;
use nym_sphinx::addressing::clients::Recipient; use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag; use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::anonymous_replies::ReplySurbWithKeyRotation; use nym_sphinx::anonymous_replies::ReplySurb;
use nym_task::connections::{ConnectionId, TransmissionLane}; use nym_task::connections::{ConnectionId, TransmissionLane};
use std::sync::Weak; use std::sync::Weak;
use tracing::error;
pub(crate) fn new_control_channels() -> (ReplyControllerSender, ReplyControllerReceiver) { pub(crate) fn new_control_channels() -> (ReplyControllerSender, ReplyControllerReceiver) {
let (tx, rx) = mpsc::unbounded(); let (tx, rx) = mpsc::unbounded();
@@ -81,7 +81,7 @@ impl ReplyControllerSender {
pub(crate) fn send_additional_surbs( pub(crate) fn send_additional_surbs(
&self, &self,
sender_tag: AnonymousSenderTag, sender_tag: AnonymousSenderTag,
reply_surbs: Vec<ReplySurbWithKeyRotation>, reply_surbs: Vec<ReplySurb>,
from_surb_request: bool, from_surb_request: bool,
) -> Result<(), ReplyControllerSenderError> { ) -> Result<(), ReplyControllerSenderError> {
self.0 self.0
@@ -167,7 +167,7 @@ pub enum ReplyControllerMessage {
AdditionalSurbs { AdditionalSurbs {
sender_tag: AnonymousSenderTag, sender_tag: AnonymousSenderTag,
reply_surbs: Vec<ReplySurbWithKeyRotation>, reply_surbs: Vec<ReplySurb>,
from_surb_request: bool, from_surb_request: bool,
}, },
@@ -1,101 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::replies::reply_controller::Config;
use nym_client_core_surb_storage::{CombinedReplyStorage, SentReplyKeys, UsedSenderTags};
use nym_crypto::aes::cipher::crypto_common::rand_core::CryptoRng;
use nym_sphinx::addressing::Recipient;
use rand::Rng;
use std::cmp::min;
use std::time::Duration;
use time::OffsetDateTime;
use tracing::{debug, trace, warn};
/// Reply controller responsible for controlling sender-related part
/// of replies, such as checking if any reply keys are stale
pub struct SenderReplyController<R> {
config: Config,
tags_storage: UsedSenderTags,
sent_reply_keys: SentReplyKeys,
message_handler: MessageHandler<R>,
}
impl<R> SenderReplyController<R>
where
R: CryptoRng + Rng,
{
pub(crate) fn new(
config: Config,
storage: &CombinedReplyStorage,
message_handler: MessageHandler<R>,
) -> Self {
SenderReplyController {
config,
tags_storage: storage.tags_storage(),
sent_reply_keys: storage.key_storage(),
message_handler,
}
}
pub(crate) async fn handle_surb_request(&mut self, recipient: Recipient, mut amount: u32) {
// 1. check whether we sent any surbs in the past to this recipient, otherwise
// they have no business in asking for more
if !self.tags_storage.exists(&recipient) {
warn!("{recipient} asked us for reply SURBs even though we never sent them any anonymous messages before!");
return;
}
// 2. check whether the requested amount is within sane range
if amount
> self
.config
.reply_surbs
.maximum_allowed_reply_surb_request_size
{
warn!("The requested reply surb amount is larger than our maximum allowed ({amount} > {}). Lowering it to a more sane value...", self.config.reply_surbs.maximum_allowed_reply_surb_request_size);
amount = self
.config
.reply_surbs
.maximum_allowed_reply_surb_request_size;
}
// 3. construct and send the surbs away
// (send them in smaller batches to make the experience a bit smoother
let mut remaining = amount;
while remaining > 0 {
let to_send = min(remaining, 100);
if let Err(err) = self
.message_handler
.try_send_additional_reply_surbs(
recipient,
to_send,
nym_sphinx::params::PacketType::Mix,
)
.await
{
warn!("failed to send additional surbs to {recipient} - {err}");
} else {
trace!("sent {to_send} reply SURBs to {recipient}");
}
remaining -= to_send;
}
}
pub(crate) fn inspect_and_clear_stale_data(&self, now: OffsetDateTime) {
// check reply keys (this applies to SENDER)
self.sent_reply_keys.retain(|_, reply_key| {
let diff = now - reply_key.sent_at;
if diff > self.config.reply_surbs.maximum_reply_key_age {
let std_diff = Duration::try_from(diff).unwrap_or_default();
let diff_formatted = humantime::format_duration(std_diff);
debug!("it's been {diff_formatted} since we created this reply key. it's probably never going to get used, so we're going to purge it...");
false
} else {
true
}
});
}
}
@@ -16,17 +16,21 @@
#![warn(clippy::todo)] #![warn(clippy::todo)]
#![warn(clippy::dbg_macro)] #![warn(clippy::dbg_macro)]
use crate::client::inbound_messages::{InputMessage, InputMessageSender};
use futures::StreamExt; use futures::StreamExt;
use nym_client_core_config_types::StatsReporting; use nym_client_core_config_types::StatsReporting;
use nym_sphinx::addressing::Recipient; use nym_sphinx::addressing::Recipient;
use nym_statistics_common::clients::{ use nym_statistics_common::clients::{
ClientStatsController, ClientStatsReceiver, ClientStatsSender, ClientStatsController, ClientStatsReceiver, ClientStatsSender,
}; };
use nym_task::{connections::TransmissionLane, ShutdownToken, ShutdownTracker}; use nym_task::{connections::TransmissionLane, TaskClient};
use std::time::Duration; use std::time::Duration;
/// Time interval between reporting statistics locally (logging/shutdown_token) use crate::{
client::inbound_messages::{InputMessage, InputMessageSender},
spawn_future,
};
/// Time interval between reporting statistics locally (logging/task_client)
const LOCAL_REPORT_INTERVAL: Duration = Duration::from_secs(2); const LOCAL_REPORT_INTERVAL: Duration = Duration::from_secs(2);
/// Interval for taking snapshots of the statistics /// Interval for taking snapshots of the statistics
const SNAPSHOT_INTERVAL: Duration = Duration::from_millis(500); const SNAPSHOT_INTERVAL: Duration = Duration::from_millis(500);
@@ -47,6 +51,9 @@ pub(crate) struct StatisticsControl {
/// Config for stats reporting (enabled, address, interval) /// Config for stats reporting (enabled, address, interval)
reporting_config: StatsReporting, reporting_config: StatsReporting,
/// Task client for listening for shutdown
task_client: TaskClient,
} }
impl StatisticsControl { impl StatisticsControl {
@@ -55,20 +62,24 @@ impl StatisticsControl {
client_type: String, client_type: String,
client_stats_id: String, client_stats_id: String,
report_tx: InputMessageSender, report_tx: InputMessageSender,
shutdown_token: ShutdownToken, task_client: TaskClient,
) -> (Self, ClientStatsSender) { ) -> (Self, ClientStatsSender) {
let (stats_tx, stats_rx) = tokio::sync::mpsc::unbounded_channel(); let (stats_tx, stats_rx) = tokio::sync::mpsc::unbounded_channel();
let stats = ClientStatsController::new(client_stats_id, client_type); let stats = ClientStatsController::new(client_stats_id, client_type);
let mut task_client_stats_sender = task_client.fork("stats_sender");
task_client_stats_sender.disarm();
( (
StatisticsControl { StatisticsControl {
stats, stats,
stats_rx, stats_rx,
report_tx, report_tx,
reporting_config, reporting_config,
task_client,
}, },
ClientStatsSender::new(Some(stats_tx), shutdown_token), ClientStatsSender::new(Some(stats_tx), task_client_stats_sender),
) )
} }
@@ -82,15 +93,14 @@ impl StatisticsControl {
None, None,
); );
if let Err(err) = self.report_tx.send(report_message).await { if let Err(err) = self.report_tx.send(report_message).await {
tracing::error!("Failed to report client stats: {err:?}"); log::error!("Failed to report client stats: {:?}", err);
} else { } else {
self.stats.reset(); self.stats.reset();
} }
} }
// manually control the shutdown mechanism as we don't want to get interrupted mid-snapshot async fn run(&mut self) {
pub async fn run(&mut self, shutdown_token: ShutdownToken) { log::debug!("Started StatisticsControl with graceful shutdown support");
tracing::debug!("Started StatisticsControl with graceful shutdown support");
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
let mut stats_report_interval = tokio_stream::wrappers::IntervalStream::new( let mut stats_report_interval = tokio_stream::wrappers::IntervalStream::new(
@@ -119,17 +129,17 @@ impl StatisticsControl {
let mut snapshot_interval = let mut snapshot_interval =
gloo_timers::future::IntervalStream::new(SNAPSHOT_INTERVAL.as_millis() as u32); gloo_timers::future::IntervalStream::new(SNAPSHOT_INTERVAL.as_millis() as u32);
loop { while !self.task_client.is_shutdown() {
tokio::select! { tokio::select! {
biased; biased;
_ = shutdown_token.cancelled() => { _ = self.task_client.recv() => {
tracing::trace!("StatisticsControl: Received shutdown"); log::trace!("StatisticsControl: Received shutdown");
break; break;
}, },
stats_event = self.stats_rx.recv() => match stats_event { stats_event = self.stats_rx.recv() => match stats_event {
Some(stats_event) => self.stats.handle_event(stats_event), Some(stats_event) => self.stats.handle_event(stats_event),
None => { None => {
tracing::trace!("StatisticsControl: shutting down due to closed stats channel"); log::trace!("StatisticsControl: shutting down due to closed stats channel");
break; break;
} }
}, },
@@ -147,11 +157,17 @@ impl StatisticsControl {
} }
_ = local_report_interval.next() => { _ = local_report_interval.next() => {
self.stats.local_report(); self.stats.local_report(&mut self.task_client);
} }
} }
} }
tracing::debug!("StatisticsControl: Exiting"); log::debug!("StatisticsControl: Exiting");
}
pub(crate) fn start(mut self) {
spawn_future(async move {
self.run().await;
})
} }
pub(crate) fn create_and_start( pub(crate) fn create_and_start(
@@ -159,22 +175,16 @@ impl StatisticsControl {
client_type: String, client_type: String,
client_stats_id: String, client_stats_id: String,
report_tx: InputMessageSender, report_tx: InputMessageSender,
shutdown_tracker: &ShutdownTracker, task_client: TaskClient,
) -> ClientStatsSender { ) -> ClientStatsSender {
let (mut controller, sender) = Self::create( let (controller, sender) = Self::create(
reporting_config, reporting_config,
client_type, client_type,
client_stats_id, client_stats_id,
report_tx, report_tx,
shutdown_tracker.child_shutdown_token(), task_client,
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move {
controller.run(shutdown_token).await;
},
"StatisticsControl",
); );
controller.start();
sender sender
} }
} }
@@ -2,8 +2,7 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use nym_sphinx::addressing::clients::Recipient; use nym_sphinx::addressing::clients::Recipient;
use nym_topology::{NymRouteProvider, NymTopology, NymTopologyError, NymTopologyMetadata}; use nym_topology::{NymRouteProvider, NymTopology, NymTopologyError};
use nym_validator_client::models::KeyRotationId;
use std::ops::Deref; use std::ops::Deref;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc; use std::sync::Arc;
@@ -126,7 +125,7 @@ impl TopologyAccessor {
.map(|p| p.topology.clone()) .map(|p| p.topology.clone())
} }
pub async fn current_route_provider(&self) -> Option<RwLockReadGuard<'_, NymRouteProvider>> { pub async fn current_route_provider(&self) -> Option<RwLockReadGuard<NymRouteProvider>> {
let provider = self.inner.topology.read().await; let provider = self.inner.topology.read().await;
if provider.topology.is_empty() { if provider.topology.is_empty() {
None None
@@ -135,21 +134,6 @@ impl TopologyAccessor {
} }
} }
pub async fn current_mixnet_epoch_id(&self) -> Option<u32> {
let route_provider = self.current_route_provider().await?;
Some(route_provider.absolute_epoch_id())
}
pub async fn current_key_rotation_id(&self) -> Option<KeyRotationId> {
let route_provider = self.current_route_provider().await?;
Some(route_provider.current_key_rotation())
}
pub async fn current_metadata(&self) -> Option<NymTopologyMetadata> {
let route_provider = self.current_route_provider().await?;
Some(route_provider.metadata())
}
pub async fn manually_change_topology(&self, new_topology: NymTopology) { pub async fn manually_change_topology(&self, new_topology: NymTopology) {
self.inner.controlled_manually.store(true, Ordering::SeqCst); self.inner.controlled_manually.store(true, Ordering::SeqCst);
self.inner.update(Some(new_topology)).await; self.inner.update(Some(new_topology)).await;
@@ -1,12 +1,14 @@
// Copyright 2021-2023 - Nym Technologies SA <contact@nymtech.net> // Copyright 2021-2023 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use crate::spawn_future;
pub(crate) use accessor::{TopologyAccessor, TopologyReadPermit}; pub(crate) use accessor::{TopologyAccessor, TopologyReadPermit};
use futures::StreamExt; use futures::StreamExt;
use log::*;
use nym_sphinx::addressing::nodes::NodeIdentity; use nym_sphinx::addressing::nodes::NodeIdentity;
use nym_task::TaskClient;
use nym_topology::NymTopologyError; use nym_topology::NymTopologyError;
use std::time::Duration; use std::time::Duration;
use tracing::*;
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
use tokio::time::sleep; use tokio::time::sleep;
@@ -18,7 +20,7 @@ mod accessor;
pub mod nym_api_provider; pub mod nym_api_provider;
pub use nym_api_provider::{Config as NymApiTopologyProviderConfig, NymApiTopologyProvider}; pub use nym_api_provider::{Config as NymApiTopologyProviderConfig, NymApiTopologyProvider};
pub use nym_topology::provider_trait::{ToTopologyMetadata, TopologyProvider}; pub use nym_topology::provider_trait::TopologyProvider;
// TODO: move it to config later // TODO: move it to config later
const MAX_FAILURE_COUNT: usize = 10; const MAX_FAILURE_COUNT: usize = 10;
@@ -39,6 +41,8 @@ pub struct TopologyRefresher {
refresh_rate: Duration, refresh_rate: Duration,
consecutive_failure_count: usize, consecutive_failure_count: usize,
task_client: TaskClient,
} }
impl TopologyRefresher { impl TopologyRefresher {
@@ -46,12 +50,14 @@ impl TopologyRefresher {
cfg: TopologyRefresherConfig, cfg: TopologyRefresherConfig,
topology_accessor: TopologyAccessor, topology_accessor: TopologyAccessor,
topology_provider: Box<dyn TopologyProvider + Send + Sync>, topology_provider: Box<dyn TopologyProvider + Send + Sync>,
task_client: TaskClient,
) -> Self { ) -> Self {
TopologyRefresher { TopologyRefresher {
topology_provider, topology_provider,
topology_accessor, topology_accessor,
refresh_rate: cfg.refresh_rate, refresh_rate: cfg.refresh_rate,
consecutive_failure_count: 0, consecutive_failure_count: 0,
task_client,
} }
} }
@@ -138,30 +144,37 @@ impl TopologyRefresher {
} }
} }
// it's perfectly fine if task is interrupted mid-refresh pub fn start(mut self) {
// there's no data to persist or send over spawn_future(async move {
pub async fn run(&mut self) { debug!("Started TopologyRefresher with graceful shutdown support");
debug!("Started TopologyRefresher with graceful shutdown support");
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
let mut interval = let mut interval = tokio_stream::wrappers::IntervalStream::new(tokio::time::interval(
tokio_stream::wrappers::IntervalStream::new(tokio::time::interval(self.refresh_rate)); self.refresh_rate,
));
#[cfg(target_arch = "wasm32")] #[cfg(target_arch = "wasm32")]
let mut interval = let mut interval =
gloo_timers::future::IntervalStream::new(self.refresh_rate.as_millis() as u32); gloo_timers::future::IntervalStream::new(self.refresh_rate.as_millis() as u32);
// We already have an initial topology, so no need to refresh it immediately. // We already have an initial topology, so no need to refresh it immediately.
// My understanding is that js setInterval does not fire immediately, so it's not // My understanding is that js setInterval does not fire immediately, so it's not
// needed there. // needed there.
#[cfg(not(target_arch = "wasm32"))] #[cfg(not(target_arch = "wasm32"))]
interval.next().await; interval.next().await;
while interval.next().await.is_some() { while !self.task_client.is_shutdown() {
self.try_refresh().await; tokio::select! {
} _ = interval.next() => {
self.try_refresh().await;
// this should never get triggered },
error!("topology refresher interval has been exhausted!") _ = self.task_client.recv() => {
log::trace!("TopologyRefresher: Received shutdown");
},
}
}
self.task_client.recv_timeout().await;
log::debug!("TopologyRefresher: Exiting");
})
} }
} }
@@ -2,14 +2,13 @@
// SPDX-License-Identifier: Apache-2.0 // SPDX-License-Identifier: Apache-2.0
use async_trait::async_trait; use async_trait::async_trait;
use nym_mixnet_contract_common::EpochRewardedSet; use log::{debug, error, warn};
use nym_topology::provider_trait::{ToTopologyMetadata, TopologyProvider}; use nym_topology::provider_trait::TopologyProvider;
use nym_topology::NymTopology; use nym_topology::NymTopology;
use nym_validator_client::nym_api::NymApiClientExt; use nym_validator_client::UserAgent;
use rand::prelude::SliceRandom; use rand::prelude::SliceRandom;
use rand::thread_rng; use rand::thread_rng;
use std::cmp::min; use std::cmp::min;
use tracing::{debug, error, warn};
use url::Url; use url::Url;
#[derive(Debug)] #[derive(Debug)]
@@ -41,43 +40,38 @@ impl Config {
pub struct NymApiTopologyProvider { pub struct NymApiTopologyProvider {
config: Config, config: Config,
validator_client: nym_http_api_client::Client, validator_client: nym_validator_client::client::NymApiClient,
nym_api_urls: Vec<Url>, nym_api_urls: Vec<Url>,
currently_used_api: usize, currently_used_api: usize,
use_bincode: bool,
} }
impl NymApiTopologyProvider { impl NymApiTopologyProvider {
pub fn new( pub fn new(
config: impl Into<Config>, config: impl Into<Config>,
mut nym_api_urls: Vec<Url>, mut nym_api_urls: Vec<Url>,
validator_client: nym_http_api_client::Client, user_agent: Option<UserAgent>,
) -> Self { ) -> Self {
nym_api_urls.shuffle(&mut thread_rng()); nym_api_urls.shuffle(&mut thread_rng());
let mut provider = NymApiTopologyProvider {
let validator_client = if let Some(user_agent) = user_agent {
nym_validator_client::client::NymApiClient::new_with_user_agent(
nym_api_urls[0].clone(),
user_agent,
)
} else {
nym_validator_client::client::NymApiClient::new(nym_api_urls[0].clone())
};
NymApiTopologyProvider {
config: config.into(), config: config.into(),
validator_client, validator_client,
nym_api_urls, nym_api_urls,
currently_used_api: 0, currently_used_api: 0,
use_bincode: true, }
};
// Set all API URLs - the client will try them in order with automatic failover
provider.validator_client.change_base_urls(
provider
.nym_api_urls
.iter()
.map(|u| u.clone().into())
.collect(),
);
provider
} }
pub fn disable_bincode(&mut self) { pub fn disable_bincode(&mut self) {
self.use_bincode = false; self.validator_client.use_bincode = false;
// Note: The unified client doesn't support toggling bincode after creation.
// This would require recreating the client without bincode.
// For now, we'll track the preference but it won't take effect.
warn!("Disabling bincode on existing client is not currently supported");
} }
fn use_next_nym_api(&mut self) { fn use_next_nym_api(&mut self) {
@@ -87,105 +81,63 @@ impl NymApiTopologyProvider {
} }
self.currently_used_api = (self.currently_used_api + 1) % self.nym_api_urls.len(); self.currently_used_api = (self.currently_used_api + 1) % self.nym_api_urls.len();
self.validator_client
// Provide all URLs starting from the next one in rotation order .change_nym_api(self.nym_api_urls[self.currently_used_api].clone())
// This enables automatic failover to other endpoints
let rotated_urls: Vec<_> = self
.nym_api_urls
.iter()
.cycle()
.skip(self.currently_used_api)
.take(self.nym_api_urls.len())
.map(|u| u.clone().into())
.collect();
self.validator_client.change_base_urls(rotated_urls)
} }
async fn get_current_compatible_topology(&mut self) -> Option<NymTopology> { async fn get_current_compatible_topology(&mut self) -> Option<NymTopology> {
let rewarded_set_fut = self.validator_client.get_current_rewarded_set(); let rewarded_set_fut = self.validator_client.get_current_rewarded_set();
let topology = if self.config.use_extended_topology { let topology = if self.config.use_extended_topology {
let all_nodes_fut = self.validator_client.get_all_basic_nodes_with_metadata(); let all_nodes_fut = self.validator_client.get_all_basic_nodes();
// Join rewarded_set_fut and all_nodes_fut concurrently // Join rewarded_set_fut and all_nodes_fut concurrently
let (rewarded_set, all_nodes_res) = futures::try_join!(rewarded_set_fut, all_nodes_fut) let (rewarded_set, all_nodes) = futures::try_join!(rewarded_set_fut, all_nodes_fut)
.inspect_err(|err| error!("failed to get network nodes: {err}")) .inspect_err(|err| error!("failed to get network nodes: {err}"))
.ok()?; .ok()?;
let metadata = all_nodes_res.metadata;
let all_nodes = all_nodes_res.nodes;
debug!( debug!(
"there are {} nodes on the network (before filtering)", "there are {} nodes on the network (before filtering)",
all_nodes.len() all_nodes.len()
); );
let nodes_filtered = all_nodes let mut topology = NymTopology::new_empty(rewarded_set);
.into_iter() topology.add_additional_nodes(all_nodes.iter().filter(|n| {
.filter(|n| n.performance.round_to_integer() >= self.config.min_node_performance()) n.performance.round_to_integer() >= self.config.min_node_performance()
.collect::<Vec<_>>(); }));
let epoch_rewarded_set: EpochRewardedSet = rewarded_set.into(); topology
NymTopology::new(
metadata.to_topology_metadata(),
epoch_rewarded_set,
Vec::new(),
)
.with_skimmed_nodes(&nodes_filtered)
} else { } else {
// if we're not using extended topology, we're only getting active set mixnodes and gateways // if we're not using extended topology, we're only getting active set mixnodes and gateways
let mixnodes_fut = self let mixnodes_fut = self
.validator_client .validator_client
.get_all_basic_active_mixing_assigned_nodes_with_metadata(); .get_all_basic_active_mixing_assigned_nodes();
// TODO: we really should be getting ACTIVE gateways only // TODO: we really should be getting ACTIVE gateways only
let gateways_fut = self let gateways_fut = self.validator_client.get_all_basic_entry_assigned_nodes();
.validator_client
.get_all_basic_entry_assigned_nodes_with_metadata();
let (rewarded_set, mixnodes_res, gateways_res) = let (rewarded_set, mixnodes, gateways) =
futures::try_join!(rewarded_set_fut, mixnodes_fut, gateways_fut) futures::try_join!(rewarded_set_fut, mixnodes_fut, gateways_fut)
.inspect_err(|err| { .inspect_err(|err| {
error!("failed to get network nodes: {err}"); error!("failed to get network nodes: {err}");
}) })
.ok()?; .ok()?;
let metadata = mixnodes_res.metadata;
let mixnodes = mixnodes_res.nodes;
if !gateways_res.metadata.consistency_check(&metadata) {
warn!("inconsistent nodes metadata between mixnodes and gateways calls! {metadata:?} and {:?}", gateways_res.metadata);
return None;
}
let gateways = gateways_res.nodes;
debug!( debug!(
"there are {} mixnodes and {} gateways in total (before performance filtering)", "there are {} mixnodes and {} gateways in total (before performance filtering)",
mixnodes.len(), mixnodes.len(),
gateways.len() gateways.len()
); );
let mut nodes = Vec::new(); let mut topology = NymTopology::new_empty(rewarded_set);
for mix in mixnodes { topology.add_additional_nodes(mixnodes.iter().filter(|m| {
if mix.performance.round_to_integer() >= self.config.min_mixnode_performance { m.performance.round_to_integer() >= self.config.min_mixnode_performance
nodes.push(mix) }));
} topology.add_additional_nodes(gateways.iter().filter(|m| {
} m.performance.round_to_integer() >= self.config.min_gateway_performance
for gateway in gateways { }));
if gateway.performance.round_to_integer() >= self.config.min_gateway_performance {
nodes.push(gateway)
}
}
let epoch_rewarded_set: EpochRewardedSet = rewarded_set.into(); topology
NymTopology::new(
metadata.to_topology_metadata(),
epoch_rewarded_set,
Vec::new(),
)
.with_skimmed_nodes(&nodes)
}; };
if !topology.is_minimally_routable() { if !topology.is_minimally_routable() {

Some files were not shown because too many files have changed in this diff Show More