Compare commits

..

35 Commits

Author SHA1 Message Date
durch f4d0ac855c Add periodic route data cleanup to epoch operations
Implements automatic cleanup of route monitoring results to prevent
unbounded storage growth while maintaining data for performance analysis.

- Add purge_old_routes() method to StorageManager following existing patterns
- Integrate route cleanup into purge_old_statuses() wrapper function
- Route data now purged every epoch with 48-hour retention, to facilitate comparisons with legacy data
- Update logging to reflect cleanup of both node statuses and routes
2025-06-06 09:12:21 +02:00
durch eb1c7d649e Client per request 2025-06-05 12:06:15 +02:00
durch 75f34ef51b Add timeout to locust 2025-06-05 11:43:04 +02:00
durch 4f7fa557d5 Optimize database queries by eliminating N+1 patterns in simulation system
This commit addresses critical N+1 query performance issues identified in
the reward simulation system. The optimizations significantly reduce database
round trips and improve performance when processing large datasets.

**Key Optimizations:**

1. **Batch Identity Key Lookups**
   - Added `get_mixnode_identity_keys_batch()` and `get_gateway_identity_keys_batch()`
   - Updated simulation performance conversion to use batch operations
   - Reduced from N individual queries to 2 batch queries

2. **Batch Node Classification**
   - Added `classify_nodes_batch()` method for mixnode/gateway determination
   - Updated reliability calculation methods to use batch classification
   - Reduced from N individual lookups to 2 batch queries

3. **Batch Epoch Metadata Enhancement**
   - Added `count_simulated_node_performance_for_epochs_batch()`
   - Added `get_available_calculation_methods_for_epochs_batch()`
   - Updated API handlers to use batch operations for metadata enhancement
   - Reduced from 2N queries to 2 batch queries for epoch data

4. **Bulk Insert Optimizations**
   - Converted individual INSERT operations to use `sqlx::QueryBuilder::push_values()`
   - Optimized simulation data insertion methods
   - Eliminated transaction overhead from individual inserts

**Performance Impact:**
- Before: N+2N database queries for N nodes/epochs
- After: 2+2 batch queries regardless of dataset size
- Significant performance improvement for large simulation datasets

All changes maintain backward compatibility while providing substantial
performance benefits for the reward simulation system.
2025-06-05 10:45:08 +02:00
durch a96fb098c2 Locust sleep if no clients are available 2025-06-05 09:54:39 +02:00
durch ad5c6ab829 Enhance simulation system with performance comparison framework
Refactors the simulation system to focus on performance methodology comparison
rather than reward amounts, enabling robust analysis of old vs new calculation
methods. Key improvements:

- Replace simulated_rewards table with performance_comparisons for better metrics
- Add performance_rankings table for ranking analysis across methodologies
- Enhance database schema with additional performance tracking fields
- Update simulation coordinator to use performance-focused data structures
- Add comprehensive performance ranking calculations
- Improve API models and handlers for performance comparison workflows
- Update SQLx query cache with new database schema changes

This provides a foundation for data-driven performance methodology evaluation
while maintaining separation from actual reward calculations.
2025-06-04 16:59:42 +02:00
durch b3d07e8832 Tests 2025-06-04 11:28:47 +02:00
durch e761255174 Add complete simulation API layer for reward method comparison
This completes Phase 3 of the simulation system implementation:

- Add comprehensive REST API endpoints for simulation data access
- Implement /v1/simulation/* routes with full CRUD operations
- Support JSON/CSV export for external analysis
- Add statistical comparison between old vs new methods
- Provide node performance history tracking
- Include proper error handling and response formatting
- Simplify simulation coordinator to remove unused complex return types
- Clean up dead code while maintaining all functionality
- Pass clippy with no warnings

The simulation API provides complete access to:
- Simulation epoch listing and details
- Method comparison analytics (old 24h vs new 1h)
- Node performance analysis across epochs
- Route reliability statistics
- Export capabilities for further analysis

All simulation data is persisted and accessible via REST endpoints.
2025-06-03 15:38:58 +02:00
durch e4a20f9cf5 Implement core simulation logic for dual reward calculations
Add complete simulation engine that compares old (24h cache-based) vs new (1h route-based)
reward calculation methodologies with full integration into epoch operations.

Core Simulation Engine:
- Add SimulationCoordinator with configurable time windows and comparison settings
- Implement dual calculation methods with proper Performance type conversions
- Add comprehensive error handling with DatabaseError variant in RewardingError
- Store simulation results in database with proper relationship constraints

Old Method Implementation (24h Cache-Based):
- Wrap existing reliability calculation using get_all_avg_mix_reliability_in_last_24hr()
- Convert reliability percentages to Performance types using from_percentage_value()
- Maintain exact same logic as production for accurate baseline comparison
- Generate simulation data structures with proper metadata

New Method Implementation (1h Route-Based):
- Leverage calculate_corrected_node_reliabilities_for_interval() for route analysis
- Support configurable time windows (default 1 hour vs 24 hours)
- Provide detailed route statistics including success rates and failure analysis
- Convert route reliability data to Performance types with naive_try_from_f64()

Epoch Operations Integration:
- Extend EpochAdvancer struct with optional SimulationConfig field
- Update constructor and start method to accept simulation configuration
- Add simulation trigger in perform_epoch_operations() before real rewarding
- Ensure simulation failures don't break epoch advancement process

CLI Integration:
- Update run.rs to handle both --enable-rewarding and --simulate-rewarding modes
- Create SimulationConfig from rewarding.debug configuration settings
- Implement mutual exclusivity between real rewarding and simulation mode
- Skip permission checks for simulation-only mode (no blockchain transactions)

The simulation system runs in parallel with epoch operations, storing comparative
data for analysis without affecting production reward distribution.
2025-06-03 14:52:13 +02:00
durch 1eefe8a579 Add simulated rewarding system foundation
Implement foundation for simulated reward calculations to compare old (24h cache-based)
vs new (1h route-based) methodologies without blockchain transactions.

Database Changes:
- Add migration with 4 new tables for simulation system
- simulated_reward_epochs: tracks each simulation run
- simulated_node_performance: stores performance calculations
- simulated_rewards: stores reward calculation results
- simulated_route_analysis: metadata for route analysis
- Add comprehensive indexes for efficient querying

Configuration Changes:
- Add simulation_mode flag to rewarding configuration
- Add CLI flag --simulate-rewarding with proper dependencies/conflicts
- Add validation for simulation-specific settings
- Add time window configuration for new method (default: 1 hour)

Storage Layer:
- Add model structs with SQLx FromRow derives for all simulation tables
- Add comprehensive CRUD methods for simulation data management
- Add proper type annotations to fix SQLx compile issues
- Maintain separation between simulation and real rewarding logic

The simulation mode is mutually exclusive with real rewarding and does not
require mnemonic since no blockchain transactions are performed.
2025-06-03 12:39:44 +02:00
durch e9dc848950 Bump nym-api 2025-06-02 12:49:13 +02:00
durch 81162fba7e Allow nym-api init fail 2025-06-02 12:45:21 +02:00
durch be36da68b1 Clap value_delimiter 2025-06-02 11:58:12 +02:00
durch 21a56e307f Bump NNM 2025-06-02 09:31:09 +02:00
durch bd966383be Towards untangling nym-api client 2025-06-02 09:30:36 +02:00
durch 7626785ce4 Bump NM version 2025-05-27 15:40:59 +02:00
durch 6f79d39d48 Filter out non mixnodes 2025-05-27 15:40:33 +02:00
durch 014b5f767a Log APIs used 2025-05-27 15:23:18 +02:00
durch e0966565e6 Mnemonic to run, bump 2025-05-27 13:38:13 +02:00
durch c6aec663b7 Bump nym-api version 2025-05-27 13:18:31 +02:00
durch 7d041ddd44 Explicit mnemonic to entrypoint 2025-05-27 13:02:38 +02:00
durch 5d8bdc6570 Bunch of new query files 2025-05-27 10:28:00 +02:00
durch 06c412b3ba Remove debug logging 2025-05-27 10:25:27 +02:00
durch 356cf00106 Put the monitoring back properly 2025-05-27 10:25:27 +02:00
durch 58493a69aa Fix submission URLs 2025-05-27 10:25:27 +02:00
durch e881da834b More NM logging 2025-05-27 10:25:27 +02:00
durch eee9d8ab0c DEBUG: disable epoch operations, less noisy logging 2025-05-27 10:25:27 +02:00
durch 09026307f4 Debug logging for nym-api 2025-05-27 10:25:27 +02:00
durch 507ddf246c Stagger out route sending 2025-05-27 10:25:26 +02:00
durch 8d8ce29113 Update NM readme, fmt 2025-05-27 10:25:26 +02:00
durch 3be9e06bef sqlx prepare, bunch of nits 2025-05-27 10:25:26 +02:00
durch 770078a9ed Delete test script 2025-05-27 10:25:26 +02:00
durch fcffebfe45 Raw route handling and reliability corrections 2025-05-27 10:25:19 +02:00
durch 9c7d79683b Force routing through all nodes 2025-05-27 10:21:02 +02:00
durch c7f34d04c0 Support submitting to multiple APIs 2025-05-27 10:21:02 +02:00
2209 changed files with 152425 additions and 114818 deletions
+3 -3
View File
@@ -415,9 +415,9 @@
}
},
"node_modules/undici": {
"version": "5.29.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-5.29.0.tgz",
"integrity": "sha512-raqeBD6NQK4SkWhQzeYKd1KmIG6dllBOTt55Rmkt4HtI9mwdWtJljnrXjAFUBLTSN67HWrOIZ3EPF4kjUw80Bg==",
"version": "5.28.5",
"resolved": "https://registry.npmjs.org/undici/-/undici-5.28.5.tgz",
"integrity": "sha512-zICwjrDrcrUE0pyyJc1I2QzBkLM8FINsgOrt6WjA+BgajVq9Nxu2PbFFXUrAggLfDXlZGZBVZYw7WNV5KiBiBA==",
"license": "MIT",
"dependencies": {
"@fastify/busboy": "^2.0.0"
@@ -21,7 +21,7 @@ jobs:
strategy:
fail-fast: false
matrix:
platform: [ arc-linux-latest ]
platform: [ arc-ubuntu-22.04 ]
runs-on: ${{ matrix.platform }}
env:
@@ -38,14 +38,15 @@ jobs:
rm -rf ci-builds || true
mkdir -p $OUTPUT_DIR
echo $OUTPUT_DIR
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install libudev-dev
- name: Sets env vars for tokio if set in manual dispatch inputs
if: github.event_name == 'workflow_dispatch' && inputs.add_tokio_unstable == true
run: |
echo "RUSTFLAGS=--cfg tokio_unstable" >> $GITHUB_ENV
echo "CARGO_FEATURES=--features tokio-console" >> $GITHUB_ENV
echo 'RUSTFLAGS="--cfg tokio_unstable"' >> $GITHUB_ENV
if: github.event_name == 'workflow_dispatch' && inputs.add_tokio_unstable == true
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
@@ -102,6 +103,7 @@ jobs:
if [ ${{ github.event_name == 'workflow_dispatch' && inputs.enable_deb == true }} = true ]; then
cp target/debian/*.deb $OUTPUT_DIR
fi
- name: Deploy branch to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
+1 -1
View File
@@ -9,7 +9,7 @@ on:
jobs:
wasm:
runs-on: arc-linux-latest
runs-on: arc-ubuntu-22.04
env:
CARGO_TERM_COLOR: always
RUSTUP_PERMIT_COPY_RENAME: 1
+9 -18
View File
@@ -5,6 +5,7 @@ on:
paths:
- 'clients/**'
- 'common/**'
- 'explorer-api/**'
- 'gateway/**'
- 'integrations/**'
- 'nym-api/**'
@@ -12,7 +13,6 @@ on:
- 'nym-network-monitor/**'
- 'nym-node/**'
- 'nym-node-status-api/**'
- 'nym-statistics-api/**'
- 'nym-outfox/**'
- 'nym-validator-rewarder/**'
- 'nyx-chain-watcher/**'
@@ -38,7 +38,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ arc-linux-latest, custom-windows-11, custom-macos-15 ]
os: [ arc-ubuntu-22.04, custom-windows-11, custom-runner-mac-m1 ]
runs-on: ${{ matrix.os }}
env:
CARGO_TERM_COLOR: always
@@ -46,9 +46,9 @@ jobs:
RUSTUP_PERMIT_COPY_RENAME: 1
steps:
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools protobuf-compiler cmake
run: sudo apt-get update && sudo apt-get -y install libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools protobuf-compiler
continue-on-error: true
if: contains(matrix.os, 'linux')
if: contains(matrix.os, 'ubuntu')
- name: Check out repository code
uses: actions/checkout@v4
@@ -63,7 +63,7 @@ jobs:
# To avoid running out of disk space, skip generating debug symbols
- name: Set debug to false (unix)
if: contains(matrix.os, 'linux') || contains(matrix.os, 'mac')
if: contains(matrix.os, 'ubuntu') || contains(matrix.os, 'mac')
run: |
sed -i.bak 's/\[profile.dev\]/\[profile.dev\]\ndebug = false/' Cargo.toml
git diff
@@ -81,35 +81,26 @@ jobs:
command: fmt
args: --all -- --check
- name: Clippy (macos)
if: contains(matrix.os, 'mac')
uses: actions-rs/cargo@v1
with:
command: clippy
args: --workspace --all-targets --exclude nym-gateway-probe -- -D warnings
- name: Clippy (non-macos)
if: contains(matrix.os, 'linux') || contains(matrix.os, 'windows')
- name: Clippy
uses: actions-rs/cargo@v1
with:
command: clippy
args: --workspace --all-targets -- -D warnings
- name: Build all binaries
uses: actions-rs/cargo@v1
with:
command: build
- name: Build all examples
if: contains(matrix.os, 'linux')
if: contains(matrix.os, 'ubuntu')
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --examples
- name: Run all tests
if: contains(matrix.os, 'linux')
if: contains(matrix.os, 'ubuntu')
uses: actions-rs/cargo@v1
env:
NYM_API: https://sandbox-nym-api1.nymtech.net/api
@@ -118,7 +109,7 @@ jobs:
args: --workspace
- name: Run expensive tests
if: (github.ref == 'refs/heads/develop' || github.event.pull_request.base.ref == 'develop' || github.event.pull_request.base.ref == 'master') && contains(matrix.os, 'linux')
if: (github.ref == 'refs/heads/develop' || github.event.pull_request.base.ref == 'develop' || github.event.pull_request.base.ref == 'master') && contains(matrix.os, 'ubuntu')
uses: actions-rs/cargo@v1
with:
command: test
@@ -10,13 +10,13 @@ env:
jobs:
check-if-tag-exists:
runs-on: arc-linux-latest-dind
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Checkout repo
uses: actions/checkout@v4
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
@@ -44,10 +44,8 @@ jobs:
echo "Tag is empty"
exit 1
fi
# first, list all tags for logging purposes
curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq
# check if there's a matching tag
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq -r --arg tag "$TAG" 'any(.tags[]; . == $tag)' )
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq --arg tag $TAG '.tags | contains([$tag])' )
if [[ $exists = "true" ]]; then
echo "Version '$TAG' defined in Cargo.toml ALREADY EXISTS as tag in harbor repo"
exit 1
@@ -55,5 +53,5 @@ jobs:
echo "Version '$TAG' doesn't exist on the remote"
else
echo "Unknown output '$exists'"
exit 2
exit 1
fi
@@ -1,59 +0,0 @@
name: ci-check-nym-stats-api-version
on:
pull_request:
paths:
- "nym-statistics-api/**"
env:
WORKING_DIRECTORY: "nym-statistics-api"
jobs:
check-if-tag-exists:
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Checkout repo
uses: actions/checkout@v4
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Check if git tag exists
run: |
TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
if [[ -z "$TAG" ]]; then
echo "Tag is empty"
exit 1
fi
git ls-remote --tags origin | awk '{print $2}'
if git ls-remote --tags origin | awk '{print $2}' | grep -q "refs/tags/$TAG$" ; then
echo "Tag '$TAG' ALREADY EXISTS on the remote"
exit 1
else
echo "Tag '$TAG' does not exist on the remote"
fi
- name: Check if harbor tag exists
run: |
TAG=${{ steps.get_version.outputs.result }}
registry=https://harbor.nymte.ch
repo_name=nym/nym-statistics-api
if [[ -z $TAG ]]; then
echo "Tag is empty"
exit 1
fi
# first, list all tags for logging purposes
curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq
# check if there's a matching tag
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq -r --arg tag "$TAG" 'any(.tags[]; . == $tag)' )
if [[ $exists = "true" ]]; then
echo "Version '$TAG' defined in Cargo.toml ALREADY EXISTS as tag in harbor repo"
exit 1
elif [[ $exists = "false" ]]; then
echo "Version '$TAG' doesn't exist on the remote"
else
echo "Unknown output '$exists'"
exit 2
fi
@@ -11,7 +11,7 @@ jobs:
strategy:
fail-fast: false
matrix:
platform: [ arc-linux-latest-dind ]
platform: [ arc-ubuntu-22.04 ]
runs-on: ${{ matrix.platform }}
env:
@@ -28,22 +28,34 @@ jobs:
mkdir -p $OUTPUT_DIR
echo $OUTPUT_DIR
- name: Build contracts
run: make optimize-contracts
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
target: wasm32-unknown-unknown
override: true
- name: Check optimized contracts
run: make docker-check-contracts
- name: Install wasm-opt
uses: ./.github/actions/install-wasm-opt
with:
version: '114'
- name: Install cosmwasm-check
run: cargo install cosmwasm-check
- name: Build release contracts
run: make contracts
- name: Prepare build output
shell: bash
env:
OUTPUT_DIR: ci-contract-builds/${{ github.ref_name }}
run: |
find contracts/artifacts -maxdepth 1 -type f -name '*.wasm' -exec cp {} $OUTPUT_DIR \;
# Also include the optimizer-generated checksums if present
if [ -f contracts/artifacts/checksums.txt ]; then
cp contracts/artifacts/checksums.txt $OUTPUT_DIR
fi
cp contracts/target/wasm32-unknown-unknown/release/mixnet_contract.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/vesting_contract.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_coconut_dkg.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw3_flex_multisig.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw4_group.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_ecash.wasm $OUTPUT_DIR
- name: Deploy branch to CI www
continue-on-error: true
+3 -5
View File
@@ -17,10 +17,9 @@ jobs:
build:
# since it's going to be compiled into wasm, there's absolutely
# no point in running CI on different OS-es
runs-on: arc-linux-latest
runs-on: ubuntu-22.04
env:
CARGO_TERM_COLOR: always
RUSTUP_PERMIT_COPY_RENAME: 1
steps:
- uses: actions/checkout@v4
@@ -28,8 +27,7 @@ jobs:
uses: actions-rs/toolchain@v1
with:
profile: minimal
# pinned due to issues building contracts
toolchain: 1.86.0
toolchain: stable
target: wasm32-unknown-unknown
override: true
components: rustfmt, clippy
@@ -54,7 +52,7 @@ jobs:
uses: actions-rs/cargo@v1
with:
command: test
args: --lib --manifest-path contracts/Cargo.toml --all-features
args: --lib --manifest-path contracts/Cargo.toml
- name: Check formatting
uses: actions-rs/cargo@v1
+1 -1
View File
@@ -10,7 +10,7 @@ on:
jobs:
build:
runs-on: arc-linux-latest
runs-on: arc-ubuntu-22.04
env:
RUSTUP_PERMIT_COPY_RENAME: 1
defaults:
+1 -1
View File
@@ -11,7 +11,7 @@ on:
jobs:
build:
runs-on: arc-linux-latest
runs-on: arc-ubuntu-22.04
env:
CARGO_TERM_COLOR: always
RUSTUP_PERMIT_COPY_RENAME: 1
+7 -7
View File
@@ -4,14 +4,14 @@ on:
workflow_dispatch:
pull_request:
paths:
- "wasm/**"
- "clients/client-core/**"
- "common/**"
- ".github/workflows/ci-sdk-wasm.yml"
- 'wasm/**'
- 'clients/client-core/**'
- 'common/**'
- '.github/workflows/ci-sdk-wasm.yml'
jobs:
wasm:
runs-on: arc-linux-latest
runs-on: arc-ubuntu-22.04
env:
CARGO_TERM_COLOR: always
RUSTUP_PERMIT_COPY_RENAME: 1
@@ -33,7 +33,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: "1.24.6"
go-version: "1.23.7"
- name: Install wasm-pack
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
@@ -41,7 +41,7 @@ jobs:
- name: Install wasm-opt
uses: ./.github/actions/install-wasm-opt
with:
version: "116"
version: '116'
- name: Install wasm-bindgen-cli
run: cargo install wasm-bindgen-cli
-19
View File
@@ -1,19 +0,0 @@
name: Run SonarQube Scan
on:
push:
branches:
- develop
# pull_request:
# types: [opened, synchronize, reopened]
jobs:
sonarqube:
name: SonarQube
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: SonarQube Scan
uses: SonarSource/sonarqube-scan-action@v5
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
+1 -1
View File
@@ -6,7 +6,7 @@ jobs:
greeting:
runs-on: ubuntu-latest
steps:
- uses: actions/first-interaction@v3
- uses: actions/first-interaction@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
issue-message: 'Thank you for raising this issue'
+1 -1
View File
@@ -31,7 +31,7 @@ jobs:
- name: Check out repository code
uses: actions/checkout@v4
- name: Download report from previous job
uses: actions/download-artifact@v5
uses: actions/download-artifact@v4
with:
name: report
path: .github/workflows/support-files/notifications
+6 -8
View File
@@ -19,11 +19,7 @@ jobs:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-binaries-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
strategy:
fail-fast: false
matrix:
include:
- os: arc-linux-latest
target: x86_64-unknown-linux-gnu
runs-on: ${{ matrix.os }}
runs-on: arc-ubuntu-22.04
outputs:
release_id: ${{ steps.create-release.outputs.id }}
@@ -56,7 +52,7 @@ jobs:
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: 1.88.0
toolchain: stable
override: true
- name: Build all binaries
@@ -70,6 +66,7 @@ jobs:
with:
name: my-artifact
path: |
target/release/explorer-api
target/release/nym-client
target/release/nym-socks5-client
target/release/nym-api
@@ -78,13 +75,14 @@ jobs:
target/release/nymvisor
target/release/nym-node
retention-days: 30
- id: create-release
name: Upload to release based on tag name
uses: softprops/action-gh-release@da05d552573ad5aba039eaac05058a918a7bf631
uses: softprops/action-gh-release@v2
if: github.event_name == 'release'
with:
files: |
target/release/explorer-api
target/release/nym-client
target/release/nym-socks5-client
target/release/nym-api
@@ -25,7 +25,7 @@ jobs:
uses: actions/checkout@v4
- name: Install Java
uses: actions/setup-java@v5
uses: actions/setup-java@v4
with:
distribution: "temurin"
java-version: "17"
@@ -91,7 +91,7 @@ jobs:
- name: Checkout
uses: actions/checkout@v4
- name: Download binary artifact
uses: actions/download-artifact@v5
uses: actions/download-artifact@v4
with:
name: nyms5-apk-arch64
path: apk
+2 -2
View File
@@ -8,7 +8,7 @@ env:
jobs:
build-container:
runs-on: arc-linux-latest-dind
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-credential-proxy/Cargo.toml
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-network-monitor/Cargo.toml
+16 -39
View File
@@ -5,22 +5,15 @@ on:
inputs:
gateway_probe_git_ref:
type: string
default: nym-vpn-core-v1.4.0
required: true
description: Which gateway probe git ref to build the image with
release_image:
description: 'Tag image as a release'
required: true
default: false
type: boolean
env:
WORKING_DIRECTORY: "nym-node-status-api/nym-node-status-agent"
CONTAINER_NAME: "node-status-agent"
jobs:
build-container:
runs-on: arc-linux-latest-dind
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
@@ -38,10 +31,10 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version
run: |
VERSION=$(yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml)
echo "result=$VERSION" >> $GITHUB_OUTPUT
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: cleanup-gateway-probe-ref
id: cleanup_gateway_probe_ref
@@ -50,35 +43,19 @@ jobs:
GIT_REF_SLUG="${GATEWAY_PROBE_GIT_REF//\//-}"
echo "git_ref=${GIT_REF_SLUG}" >> $GITHUB_OUTPUT
- name: Set GIT_TAG variable
run: echo "GIT_TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}" >> $GITHUB_ENV
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
fi
- name: Initialize RELEASE_TAG
run: echo "RELEASE_TAG=" >> $GITHUB_ENV
- name: Set RELEASE_TAG for release
if: github.event.inputs.release_image == 'true'
run: echo "RELEASE_TAG=golden-" >> $GITHUB_ENV
- name: Set IMAGE_NAME_AND_TAGS variable
run: echo "IMAGE_NAME_AND_TAGS=${{ env.CONTAINER_NAME }}:${{ env.RELEASE_TAG }}${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}" >> $GITHUB_ENV
- name: New env vars
run: echo "RELEASE_TAG='$RELEASE_TAG' GIT_TAG='$GIT_TAG' IMAGE_NAME_AND_TAGS='$IMAGE_NAME_AND_TAGS'"
# - name: Remove existing tag if exists
# run: |
# if git rev-parse $${{ env.GIT_TAG }} >/dev/null 2>&1; then
# git push --delete origin $${{ env.GIT_TAG }}
# git tag -d $${{ env.GIT_TAG }}
# fi
# - name: Create tag
# run: |
# git tag -a $${{ env.GIT_TAG }} -m "Version ${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}"
# git push origin $${{ env.GIT_TAG }}
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }} -m "Version ${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
- name: BuildAndPushImageOnHarbor
run: |
docker build --build-arg GIT_REF=${{ github.event.inputs.gateway_probe_git_ref }} -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.IMAGE_NAME_AND_TAGS }}
docker build --build-arg GIT_REF=${{ github.event.inputs.gateway_probe_git_ref }} -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+21 -40
View File
@@ -1,20 +1,14 @@
name: Build and upload Node Status API container to harbor.nymte.ch
on:
workflow_dispatch:
inputs:
release_image:
description: 'Tag image as a release'
required: true
default: false
type: boolean
env:
WORKING_DIRECTORY: "nym-node-status-api/nym-node-status-api"
CONTAINER_NAME: "node-status-api"
jobs:
build-container:
runs-on: arc-linux-latest-dind
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
@@ -32,43 +26,30 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Check if tag exists
run: |
VERSION=$(yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml)
echo "result=$VERSION" >> $GITHUB_OUTPUT
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
echo "Tag ${{ steps.get_version.outputs.result }} already exists"
fi
- name: Set GIT_TAG variable
run: echo "GIT_TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}" >> $GITHUB_ENV
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
fi
- name: Initialise RELEASE_TAG
run: echo "RELEASE_TAG=" >> $GITHUB_ENV
- name: Set RELEASE_TAG for release
if: github.event.inputs.release_image == 'true'
run: echo "RELEASE_TAG=golden-" >> $GITHUB_ENV
- name: Set IMAGE_NAME_AND_TAGS variable
run: echo "IMAGE_NAME_AND_TAGS=${{ env.CONTAINER_NAME }}:${{ env.RELEASE_TAG }}${{ steps.get_version.outputs.result }}" >> $GITHUB_ENV
- name: New env vars
run: echo "RELEASE_TAG='$RELEASE_TAG' GIT_TAG='$GIT_TAG' IMAGE_NAME_AND_TAGS='$IMAGE_NAME_AND_TAGS'"
# - name: Remove existing tag if exists, then create
# run: |
# if git rev-parse "$GIT_TAG" >/dev/null 2>&1; then
# echo "Tag '$GIT_TAG' already exists, deleting"
# git push --delete origin "$GIT_TAG"
# git tag -d "$GIT_TAG"
# echo "Tag '$GIT_TAG' deleted"
# else
# echo "Tag '$GIT_TAG' does not exist, creating it"
# git tag -a $GIT_TAG -m "Version ${{ steps.get_version.outputs.result }}"
# git push origin $GIT_TAG
# echo "Tag '$GIT_TAG' created"
# fi
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.IMAGE_NAME_AND_TAGS }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-api/Cargo.toml
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
@@ -1,42 +0,0 @@
name: Build and upload Nym Statistics API container to harbor.nymte.ch
on:
workflow_dispatch:
env:
WORKING_DIRECTORY: "nym-statistics-api"
CONTAINER_NAME: "nym-statistics-api"
jobs:
build-container:
runs-on: arc-linux-latest-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: harbor.nymte.ch
username: ${{ secrets.HARBOR_ROBOT_USERNAME }}
password: ${{ secrets.HARBOR_ROBOT_SECRET }}
- name: Checkout repo
uses: actions/checkout@v4
- name: Configure git identity
run: |
git config --global user.email "lawrence@nymtech.net"
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.47.1
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
+2 -2
View File
@@ -35,13 +35,12 @@ validator-api/keypair
contracts/mixnet/code_id
contracts/mixnet/Justfile
contracts/mixnet/Makefile
artifacts
contracts/artifacts
validator-config
*.patch
validator-api-config.toml
dist
storybook-static
envs/qwerty.env
.parcel-cache
**/.DS_Store
cpu-cycles/libcpucycles/build
@@ -63,3 +62,4 @@ nym-api/redocly/formatted-openapi.json
**/settings.sql
**/enter_db.sh
CLAUDE.md
-392
View File
@@ -4,398 +4,6 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
## [Unreleased]
## [2025.18-jarlsberg] (2025-10-14)
- ns-api: add descriptions to dVPN gateway responses ([#6102])
- NS API: use new probe download filesize and milliseconds field ([#6097])
- ns-api: use download files size from probes instead of parsing filenames ([#6095])
- ns-api: add new fields for probe output for query_metadata and download file size and duration in ms ([#6091])
- Bugfix/bloomfilters purge ([#6089])
- Hotfix: Update API source in node ping tester script ([#6082])
- Get wireguard keypair as arg instead of reading it from disk ([#6078])
- Feature: Ping probe all nodes /described nodes from a server ([#6074])
- Node Status API: add bridge information to dVPN endpoint ([#6069])
- frontdoor typo fix ([#6067])
- Feature: Node rewards tracker ([#6064])
- [chore] Clippy fix ([#6060])
- Registration Client ([#6059])
- Bugfix: Nym node CLI download nym-node exception ([#6058])
- Feature: Nym node html landing page ([#6053])
- feat: DKG contract method for updating announce address ([#6050])
- feat: NS ticket faucet ([#6047])
- Bridge proto client params in Self-Described ([#6035])
- Node Status API: remove sqlite support ([#6004])
- Benny/ci contract fix ([#5962])
[#6102]: https://github.com/nymtech/nym/pull/6102
[#6097]: https://github.com/nymtech/nym/pull/6097
[#6095]: https://github.com/nymtech/nym/pull/6095
[#6091]: https://github.com/nymtech/nym/pull/6091
[#6089]: https://github.com/nymtech/nym/pull/6089
[#6082]: https://github.com/nymtech/nym/pull/6082
[#6078]: https://github.com/nymtech/nym/pull/6078
[#6074]: https://github.com/nymtech/nym/pull/6074
[#6069]: https://github.com/nymtech/nym/pull/6069
[#6067]: https://github.com/nymtech/nym/pull/6067
[#6064]: https://github.com/nymtech/nym/pull/6064
[#6060]: https://github.com/nymtech/nym/pull/6060
[#6059]: https://github.com/nymtech/nym/pull/6059
[#6058]: https://github.com/nymtech/nym/pull/6058
[#6053]: https://github.com/nymtech/nym/pull/6053
[#6050]: https://github.com/nymtech/nym/pull/6050
[#6047]: https://github.com/nymtech/nym/pull/6047
[#6035]: https://github.com/nymtech/nym/pull/6035
[#6004]: https://github.com/nymtech/nym/pull/6004
[#5962]: https://github.com/nymtech/nym/pull/5962
## [2025.17-isabirra] (2025-09-29)
- Bugfix | Fix the registration handshake ([#6062])
- Convenience for ShutdownTracker ([#6038])
- chore: made http-api-client-macro doctest compile ([#6037])
- feat: refresh mixnet contract on epoch progression ([#6023])
- chore: remove legacy nodes from nym api [and kinda-ish from node status api] ([#6021])
- Feature/credential proxy crate ([#6018])
- Moving clients crate from vpn-client repo to here ([#6015])
- Feature/cancellation migration ([#6014])
- Use default value for the ports until api is deployed ([#6007])
- bugfix: return from MixTrafficController if client request channel has closed ([#6002])
- Revert "Create an axum_test client for more integrated unit testing (… ([#5999])
- chore: upgraded syn to 2.0 and removed nym-execute ([#5998])
- feat: use `ShutdownToken` (`CancellationToken` inside) for nym-api ([#5997])
- bugfix: Recipient deserialisation for deserialisers missing bytes specialisation ([#5991])
- chore: use updated version of simulate endpoint ([#5988])
- chore: purge temp databases on build ([#5984])
- Bump sha.js from 2.4.11 to 2.4.12 ([#5983])
- Feature: Delegation program stake checker and adjuster ([#5980])
- build(deps): bump actions/setup-java from 4 to 5 ([#5975])
- Domain fronting integration ([#5974])
- chore: internal hidden command to force advance nyx epoch ([#5964])
- Create an axum_test client for more integrated unit testing ([#5956])
- feat: shared library for attempting to retrieve update mode attestation ([#5954])
- Bump slab from 0.4.10 to 0.4.11 ([#5952])
- build(deps): bump actions/first-interaction from 1 to 3 ([#5950])
- fix: use WASM compatible time API in client ([#5948])
- feat: credential proxy deposit pool ([#5945])
- build(deps): bump actions/download-artifact from 4 to 5 ([#5939])
- feat: nym signers monitor ([#5933])
- Bump console from 0.15.11 to 0.16.0 ([#5931])
- Bump mock_instant from 0.5.3 to 0.6.0 ([#5930])
- Bump tokio from 1.46.1 to 1.47.1 ([#5929])
- Bump defguard_wireguard_rs from v0.4.7 to v0.7.5 ([#5928])
- Bump indicatif from 0.17.11 to 0.18.0 ([#5924])
- Feature: Nym node autorun CLI ([#5916])
- build(deps): bump mikefarah/yq from 4.45.4 to 4.47.1 ([#5911])
- build(deps): bump pbkdf2 from 3.1.2 to 3.1.3 ([#5869])
[#6062]: https://github.com/nymtech/nym/pull/6062
[#6038]: https://github.com/nymtech/nym/pull/6038
[#6037]: https://github.com/nymtech/nym/pull/6037
[#6023]: https://github.com/nymtech/nym/pull/6023
[#6021]: https://github.com/nymtech/nym/pull/6021
[#6018]: https://github.com/nymtech/nym/pull/6018
[#6015]: https://github.com/nymtech/nym/pull/6015
[#6014]: https://github.com/nymtech/nym/pull/6014
[#6007]: https://github.com/nymtech/nym/pull/6007
[#6002]: https://github.com/nymtech/nym/pull/6002
[#5999]: https://github.com/nymtech/nym/pull/5999
[#5998]: https://github.com/nymtech/nym/pull/5998
[#5997]: https://github.com/nymtech/nym/pull/5997
[#5991]: https://github.com/nymtech/nym/pull/5991
[#5988]: https://github.com/nymtech/nym/pull/5988
[#5984]: https://github.com/nymtech/nym/pull/5984
[#5983]: https://github.com/nymtech/nym/pull/5983
[#5980]: https://github.com/nymtech/nym/pull/5980
[#5975]: https://github.com/nymtech/nym/pull/5975
[#5974]: https://github.com/nymtech/nym/pull/5974
[#5964]: https://github.com/nymtech/nym/pull/5964
[#5956]: https://github.com/nymtech/nym/pull/5956
[#5954]: https://github.com/nymtech/nym/pull/5954
[#5952]: https://github.com/nymtech/nym/pull/5952
[#5950]: https://github.com/nymtech/nym/pull/5950
[#5948]: https://github.com/nymtech/nym/pull/5948
[#5945]: https://github.com/nymtech/nym/pull/5945
[#5939]: https://github.com/nymtech/nym/pull/5939
[#5933]: https://github.com/nymtech/nym/pull/5933
[#5931]: https://github.com/nymtech/nym/pull/5931
[#5930]: https://github.com/nymtech/nym/pull/5930
[#5929]: https://github.com/nymtech/nym/pull/5929
[#5928]: https://github.com/nymtech/nym/pull/5928
[#5924]: https://github.com/nymtech/nym/pull/5924
[#5916]: https://github.com/nymtech/nym/pull/5916
[#5911]: https://github.com/nymtech/nym/pull/5911
[#5869]: https://github.com/nymtech/nym/pull/5869
## [2025.16-halloumi] (2025-09-16)
- Backport metadata endpoint ([#6010])
- bugfix: make sure tables are removed in correct order to not trigger FK constraint issue ([#5987])
- chore: move authenticator into gateway crate ([#5982])
- Fix the ns api ci workflow ([#5981])
- Remove freshness check on testrun submit ([#5977])
- Update sysinfo to the latest ([#5976])
- bugfix: manually calculate per node work on rewarded set changes ([#5972])
- fixing the ci for ns agent ([#5965])
- Feature/testing utils ([#5963])
- bugfix: fix ci-build for linux (and use updated runner) ([#5958])
- chore: updated refs to cheddar rev of nym repo ([#5955])
- http api client adjustment ([#5953])
- chore: fix rust 1.89 clippy issues ([#5944])
- Wireguard metadata client library ([#5943])
- chore: remove unused import ([#5942])
- feat: introduce additional checks when attempting to send to bounded channels ([#5941])
- Move credential verifier in peer controller ([#5938])
- change PK/FK on expiration date signatures tables ([#5934])
- Wireguard private metadata ([#5915])
[#6010]: https://github.com/nymtech/nym/pull/6010
[#5987]: https://github.com/nymtech/nym/pull/5987
[#5982]: https://github.com/nymtech/nym/pull/5982
[#5981]: https://github.com/nymtech/nym/pull/5981
[#5977]: https://github.com/nymtech/nym/pull/5977
[#5976]: https://github.com/nymtech/nym/pull/5976
[#5972]: https://github.com/nymtech/nym/pull/5972
[#5965]: https://github.com/nymtech/nym/pull/5965
[#5963]: https://github.com/nymtech/nym/pull/5963
[#5958]: https://github.com/nymtech/nym/pull/5958
[#5955]: https://github.com/nymtech/nym/pull/5955
[#5953]: https://github.com/nymtech/nym/pull/5953
[#5944]: https://github.com/nymtech/nym/pull/5944
[#5943]: https://github.com/nymtech/nym/pull/5943
[#5942]: https://github.com/nymtech/nym/pull/5942
[#5941]: https://github.com/nymtech/nym/pull/5941
[#5938]: https://github.com/nymtech/nym/pull/5938
[#5934]: https://github.com/nymtech/nym/pull/5934
[#5915]: https://github.com/nymtech/nym/pull/5915
## [2025.15-gruyere] (2025-08-20)
- Migrate strum to 0.27.2 ([#5960])
- WG exit policy scripts update ([#5921])
- Make DNS Resolver fallback optional ([#5920])
- nym-node debug command to reset providers db ([#5914])
- basic zulip client for sending messages ([#5913])
- chore: allow compatibility with 'CDLA-Permissive-2.0' ([#5910])
- feat: ecash liveness check ([#5890])
- Remove old free credential handle ([#5864])
[#5960]: https://github.com/nymtech/nym/pull/5960
[#5921]: https://github.com/nymtech/nym/pull/5921
[#5920]: https://github.com/nymtech/nym/pull/5920
[#5914]: https://github.com/nymtech/nym/pull/5914
[#5913]: https://github.com/nymtech/nym/pull/5913
[#5910]: https://github.com/nymtech/nym/pull/5910
[#5890]: https://github.com/nymtech/nym/pull/5890
[#5864]: https://github.com/nymtech/nym/pull/5864
## [2025.14-feta] (2025-08-05)
- chore: nym node tokio console ([#5909])
- Feature/dkg snapshot epoch ([#5900])
- Feature/dkg epoch dealers query ([#5899])
- sqlx-pool-guard: allocate more memory on windows ([#5896])
- Support mnemonic in the NS agent ([#5883])
- Allow PG database backend ([#5880])
[#5909]: https://github.com/nymtech/nym/pull/5909
[#5900]: https://github.com/nymtech/nym/pull/5900
[#5899]: https://github.com/nymtech/nym/pull/5899
[#5896]: https://github.com/nymtech/nym/pull/5896
[#5883]: https://github.com/nymtech/nym/pull/5883
[#5880]: https://github.com/nymtech/nym/pull/5880
## [2025.13-emmental] (2025-07-22)
- fix: don't allow mixnode running in exit mode ([#5898])
- fix contract build process in Makefile ([#5892])
- bugfix: ignore 'Send' responses when claiming bandwidth ([#5884])
- Update push-node-status-agent.yaml ([#5882])
- listen for shutdown signals during nym-node startup ([#5879])
- feat: forbid running mixnode + entry on the same node ([#5878])
- chore: 1.88 clippy ([#5877])
- Batch SQL writes for packet stats ([#5874])
- fix the broken link ([#5873])
- Set busy_timeout in sqlx ([#5872])
- feat: basic performance contract integration [within Nym API] ([#5871])
- scraper bugfix: ignore precommits from missing validators ([#5867])
- Return true remaining ([#5866])
- Make Mix hops optional for Mixnet Client SURBs ([#5861])
- Check gateway supported versions ([#5860])
- Add build info endpoints ([#5857])
- Clear out screaming logs ([#5856])
- fix removal of qa env ([#5855])
- Use display when printing paths ([#5853])
- feat: initial performance contract ([#5833])
- Security patches for the `dkg` crate ([#5828])
- HTTP Discovery objects & network defaults ([#5814])
[#5898]: https://github.com/nymtech/nym/pull/5898
[#5892]: https://github.com/nymtech/nym/pull/5892
[#5884]: https://github.com/nymtech/nym/pull/5884
[#5882]: https://github.com/nymtech/nym/pull/5882
[#5879]: https://github.com/nymtech/nym/pull/5879
[#5878]: https://github.com/nymtech/nym/pull/5878
[#5877]: https://github.com/nymtech/nym/pull/5877
[#5874]: https://github.com/nymtech/nym/pull/5874
[#5873]: https://github.com/nymtech/nym/pull/5873
[#5872]: https://github.com/nymtech/nym/pull/5872
[#5871]: https://github.com/nymtech/nym/pull/5871
[#5867]: https://github.com/nymtech/nym/pull/5867
[#5866]: https://github.com/nymtech/nym/pull/5866
[#5861]: https://github.com/nymtech/nym/pull/5861
[#5860]: https://github.com/nymtech/nym/pull/5860
[#5857]: https://github.com/nymtech/nym/pull/5857
[#5856]: https://github.com/nymtech/nym/pull/5856
[#5855]: https://github.com/nymtech/nym/pull/5855
[#5853]: https://github.com/nymtech/nym/pull/5853
[#5833]: https://github.com/nymtech/nym/pull/5833
[#5828]: https://github.com/nymtech/nym/pull/5828
[#5814]: https://github.com/nymtech/nym/pull/5814
## [2025.12-dolcelatte] (2025-07-07)
- bugfix: key-rotation + reply SURBs ([#5876])
- Bugfix/backwards compat ([#5865])
- bugfix: allow gateways to permit authentication from v4 clients ([#5862])
- fixed client route for obtaining v2 list of gateways ([#5859])
- Updated browser extension piece removal ([#5849])
- Remove/old env references ([#5848])
- Remove qa env ([#5847])
- remove not used old mock-api ([#5845])
- remove bity dir ([#5844])
- build(deps-dev): bump webpack-dev-server from 4.13.2 to 5.2.1 in /wasm/mix-fetch/internal-dev ([#5843])
- Amended the buy section ([#5841])
- Removing test-net faucet ([#5840])
- Feature/node status dvpn directory ([#5829])
- build(deps-dev): bump webpack-dev-server from 4.15.2 to 5.2.1 in /nym-credential-proxy/vpn-api-lib-wasm/internal-dev ([#5826])
- bugfix: fix swapped total and circulating supplies ([#5822])
- build(deps): bump tar-fs from 3.0.8 to 3.0.9 in /sdk/typescript/tests/integration-tests/mix-fetch ([#5821])
- Url scheme warning log ([#5819])
- chore: adjust heuristic for wireguard peer activity ([#5818])
- Use the same client bandwidth for top up ([#5813])
- Replace chrono with time in NS API ([#5811])
- build(deps-dev): bump http-proxy-middleware from 2.0.4 to 2.0.9 in /clients/native/examples/js-examples/websocket ([#5810])
- build(deps): bump tokio from 1.44.2 to 1.45.1 ([#5798])
- Close sqlite pool before moving or reopening databases ([#5796])
- HTTP Client Retries, Fallbacks, and Redirects ([#5789])
- feat: key rotation ([#5777])
- build(deps): bump next from 14.2.15 to 14.2.26 in /documentation/docs ([#5772])
- build(deps): bump undici from 5.28.5 to 5.29.0 in /.github/actions/nym-hash-releases/src ([#5771])
- build(deps): bump cargo_metadata from 0.18.1 to 0.19.2 ([#5765])
- build(deps): bump tempfile from 3.19.1 to 3.20.0 ([#5764])
- [Feature] Noise XKpsk3 integration (2025 version) ([#5692])
- feature: nympool contract ([#5464])
- chore: fixed typo in API endpoint parameter ([#5449])
[#5876]: https://github.com/nymtech/nym/pull/5876
[#5865]: https://github.com/nymtech/nym/pull/5865
[#5862]: https://github.com/nymtech/nym/pull/5862
[#5859]: https://github.com/nymtech/nym/pull/5859
[#5849]: https://github.com/nymtech/nym/pull/5849
[#5848]: https://github.com/nymtech/nym/pull/5848
[#5847]: https://github.com/nymtech/nym/pull/5847
[#5845]: https://github.com/nymtech/nym/pull/5845
[#5844]: https://github.com/nymtech/nym/pull/5844
[#5843]: https://github.com/nymtech/nym/pull/5843
[#5841]: https://github.com/nymtech/nym/pull/5841
[#5840]: https://github.com/nymtech/nym/pull/5840
[#5829]: https://github.com/nymtech/nym/pull/5829
[#5826]: https://github.com/nymtech/nym/pull/5826
[#5822]: https://github.com/nymtech/nym/pull/5822
[#5821]: https://github.com/nymtech/nym/pull/5821
[#5819]: https://github.com/nymtech/nym/pull/5819
[#5818]: https://github.com/nymtech/nym/pull/5818
[#5813]: https://github.com/nymtech/nym/pull/5813
[#5811]: https://github.com/nymtech/nym/pull/5811
[#5810]: https://github.com/nymtech/nym/pull/5810
[#5798]: https://github.com/nymtech/nym/pull/5798
[#5796]: https://github.com/nymtech/nym/pull/5796
[#5789]: https://github.com/nymtech/nym/pull/5789
[#5777]: https://github.com/nymtech/nym/pull/5777
[#5772]: https://github.com/nymtech/nym/pull/5772
[#5771]: https://github.com/nymtech/nym/pull/5771
[#5765]: https://github.com/nymtech/nym/pull/5765
[#5764]: https://github.com/nymtech/nym/pull/5764
[#5692]: https://github.com/nymtech/nym/pull/5692
[#5464]: https://github.com/nymtech/nym/pull/5464
[#5449]: https://github.com/nymtech/nym/pull/5449
## [2025.11-cheddar] (2025-06-10)
- No autoremoval of peers ([#5831])
- Set cached storage counters to 0 ([#5812])
- hack: temporarily use next.config.js instead of next.config.ts ([#5805])
- chore: resolve 1.87 clippy warnings ([#5802])
- Nym Statistics API ([#5800])
- QoL: RequestPath trait for http-api-client ([#5788])
- Fix contains ticketbook function that always returned true ([#5787])
- swap a decode into a fromrow to please future postgres feature ([#5785])
- Make address cache configurable ([#5784])
- Track wireguard credential retries ([#5783])
[#5831]: https://github.com/nymtech/nym/pull/5831
[#5812]: https://github.com/nymtech/nym/pull/5812
[#5805]: https://github.com/nymtech/nym/pull/5805
[#5802]: https://github.com/nymtech/nym/pull/5802
[#5800]: https://github.com/nymtech/nym/pull/5800
[#5788]: https://github.com/nymtech/nym/pull/5788
[#5787]: https://github.com/nymtech/nym/pull/5787
[#5785]: https://github.com/nymtech/nym/pull/5785
[#5784]: https://github.com/nymtech/nym/pull/5784
[#5783]: https://github.com/nymtech/nym/pull/5783
## [2025.10-brie] (2025-05-27)
- Backport PR 5779 ([#5801])
- Expanded Accept Encoding for `reqwest` ([#5779])
- Teach HttpClientError how to report its status code and timeout ([#5770])
- Skip refreshing the topology on startup as we already have an initial set ([#5768])
- Fetch the topology from the nym-api concurrently ([#5767])
- feat: use bincode by default in NymApiClient + remove feature-lock ([#5761])
- Instrument create_request ([#5760])
- Add node_bonded field to delegations ([#5759])
- build(deps): bump mikefarah/yq from 4.45.1 to 4.45.4 ([#5758])
- Raw route submissions ([#5756])
- feat: expires header for `/active` nym-api responses ([#5755])
- Decrease default average packet delay to 15 ms ([#5754])
- build(deps): bump the patch-updates group across 1 directory with 12 updates ([#5753])
- Remove pretty_env_logger and switch remaining crates to use tracing ([#5749])
- Update pretty_env_logger to latest to not depend on unmaintained crate atty ([#5748])
- Upgrade prometheus crate to fix security warning ([#5747])
- Downgrade deranged crate to 0.4.0 ([#5746])
- feat: nym-api bincode + yaml support ([#5745])
- fix parallel feature in ecash crate with send + sync ([#5744])
- Remove old test directory - Update validator docker ([#5743])
- [Feature] `RememberMe` is the new don't `ForgetMe` ([#5742])
- build(deps): bump ammonia from 4.0.0 to 4.1.0 ([#5739])
- build(deps): bump base-x from 3.0.9 to 3.0.11 in /testnet-faucet ([#5737])
- build(deps): bump http-proxy-middleware from 2.0.8 to 2.0.9 ([#5730])
[#5801]: https://github.com/nymtech/nym/pull/5801
[#5779]: https://github.com/nymtech/nym/pull/5779
[#5770]: https://github.com/nymtech/nym/pull/5770
[#5768]: https://github.com/nymtech/nym/pull/5768
[#5767]: https://github.com/nymtech/nym/pull/5767
[#5761]: https://github.com/nymtech/nym/pull/5761
[#5760]: https://github.com/nymtech/nym/pull/5760
[#5759]: https://github.com/nymtech/nym/pull/5759
[#5758]: https://github.com/nymtech/nym/pull/5758
[#5756]: https://github.com/nymtech/nym/pull/5756
[#5755]: https://github.com/nymtech/nym/pull/5755
[#5754]: https://github.com/nymtech/nym/pull/5754
[#5753]: https://github.com/nymtech/nym/pull/5753
[#5749]: https://github.com/nymtech/nym/pull/5749
[#5748]: https://github.com/nymtech/nym/pull/5748
[#5747]: https://github.com/nymtech/nym/pull/5747
[#5746]: https://github.com/nymtech/nym/pull/5746
[#5745]: https://github.com/nymtech/nym/pull/5745
[#5744]: https://github.com/nymtech/nym/pull/5744
[#5743]: https://github.com/nymtech/nym/pull/5743
[#5742]: https://github.com/nymtech/nym/pull/5742
[#5739]: https://github.com/nymtech/nym/pull/5739
[#5737]: https://github.com/nymtech/nym/pull/5737
[#5730]: https://github.com/nymtech/nym/pull/5730
## [2025.9-appenzeller] (2025-05-13)
- build(deps): bump clap from 4.5.36 to 4.5.37 in the patch-updates group ([#5722])
-686
View File
@@ -1,686 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Nym is a privacy platform that uses mixnet technology to protect against metadata surveillance. The platform consists of several key components:
- Mixnet nodes (mixnodes) for packet mixing
- Gateways (entry/exit points for the network)
- Clients for interacting with the network
- Network monitoring tools
- Validators for network consensus
- Various service providers and integrations
## Build Commands
### Rust Components
```bash
# Default build (debug)
cargo build
# Release build
cargo build --release
# Build a specific package
cargo build -p <package-name>
# Build main components
make build
# Build release versions of main binaries and contracts
make build-release
# Build specific binaries
make build-nym-cli
cargo build -p nym-node --release
cargo build -p nym-api --release
```
### Testing
```bash
# Run clippy, unit tests, and formatting
make test
# Run all tests including slow tests
make test-all
# Run clippy on all workspaces
make clippy
# Run unit tests for a specific package
cargo test -p <package-name>
# Run only expensive/ignored tests
cargo test --workspace -- --ignored
# Run API tests
dotenv -f envs/sandbox.env -- cargo test --test public-api-tests
# Run tests with specific log level
RUST_LOG=debug cargo test -p <package-name>
# Run specific test scripts
./nym-node/tests/test_apis.sh
./scripts/wireguard-exit-policy/exit-policy-tests.sh
```
### Linting and Formatting
```bash
# Run rustfmt on all code
make fmt
# Check formatting without modifying
cargo fmt --all -- --check
# Run clippy with all targets
cargo clippy --workspace --all-targets -- -D warnings
# TypeScript linting
yarn lint
yarn lint:fix
yarn types:lint:fix
# Check dependencies for security/licensing issues
cargo deny check
```
### WASM Components
```bash
# Build all WASM components
make sdk-wasm-build
# Build TypeScript SDK
yarn build:sdk
npx lerna run --scope @nymproject/sdk build --stream
# Build and test WASM components
make sdk-wasm
# Build specific WASM packages
cd wasm/client && make
cd wasm/mix-fetch && make
cd wasm/node-tester && make
```
### Contract Development
```bash
# Build all contracts
make contracts
# Build contracts in release mode
make build-release-contracts
# Generate contract schemas
make contract-schema
# Run wasm-opt on contracts
make wasm-opt-contracts
# Check contracts with cosmwasm-check
make cosmwasm-check-contracts
```
### Running Components
```bash
# Run nym-node as a mixnode
cargo run -p nym-node -- run --mode mixnode
# Run nym-node as a gateway
cargo run -p nym-node -- run --mode gateway
# Run the network monitor
cargo run -p nym-network-monitor
# Run the API server
cargo run -p nym-api
# Run with specific environment
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Start a local network
./scripts/localnet_start.sh
```
## Architecture
The Nym platform consists of various components organized as a monorepo:
1. **Core Mixnet Infrastructure**:
- `nym-node`: Core binary supporting mixnode and gateway modes
- `common/nymsphinx`: Implementation of the Sphinx packet format
- `common/topology`: Network topology management
- `common/types`: Shared data types across components
2. **Network Monitoring**:
- `nym-network-monitor`: Monitors the network's reliability and performance
- `nym-api`: API server for network stats and monitoring data
- Metrics tracking for nodes, routes, and overall network health
3. **Client Implementations**:
- `clients/native`: Native Rust client implementation
- `clients/socks5`: SOCKS5 proxy client for standard applications
- `wasm`: WebAssembly client implementations (for browsers)
- `nym-connect`: Desktop and mobile clients
4. **Blockchain & Smart Contracts**:
- `common/cosmwasm-smart-contracts`: Smart contract implementations
- `contracts`: CosmWasm contracts for the Nym network
- `common/ledger`: Blockchain integration
5. **Utilities & Tools**:
- `tools`: Various CLI tools and utilities
- `sdk`: SDKs for different languages and platforms
- `documentation`: Documentation generation and management
## Packet System
Nym uses a modified Sphinx packet format for its mixnet:
1. **Message Chunking**:
- Messages are divided into "sets" and "fragments"
- Each fragment fits in a single Sphinx packet
- The `common/nymsphinx/chunking` module handles message fragmentation
2. **Routing**:
- Packets traverse through 3 layers of mixnodes
- Routing information is encrypted in layers (onion routing)
- The final gateway receives and processes the messages
3. **Monitoring**:
- Monitoring system tracks packet delivery through the network
- Routes are analyzed for reliability statistics
- Node performance metrics are collected
## Network Protocol
Nym implements the Loopix mixnet design with several key privacy features:
1. **Continuous-time Mixing**:
- Each mixnode delays messages independently with an exponential distribution
- This creates random reordering of packets, destroying timing correlations
- Offers better anonymity properties than batch mixing approaches
2. **Cover Traffic**:
- Clients and nodes generate dummy "loop" packets that circulate through the network
- These packets are indistinguishable from real traffic
- Creates a baseline level of traffic that hides actual communication patterns
- Provides unobservability (hiding when and how much real traffic is being sent)
3. **Stratified Network Architecture**:
- Traffic flows through Entry Gateway → 3 Mixnode Layers → Exit Gateway
- Path selection is independent per-message (unlike Tor)
- Each node connects only to adjacent layers
4. **Anonymous Replies**:
- Single-Use Reply Blocks (SURBs) allow receiving messages without revealing identity
- Enables bidirectional communication while maintaining privacy
## Network Monitoring Architecture
The network monitoring system is a core component that measures mixnet reliability:
1. The `nym-network-monitor` sends test packets through the network
2. These packets follow predefined routes through multiple mixnodes
3. Metrics are collected about:
- Successful and failed packet deliveries
- Node reliability (percentage of successful packet handling)
- Route reliability (which specific route combinations work best)
4. Results are stored in the database and used by `nym-api` to:
- Present node performance statistics
- Determine network rewards
- Provide route selection guidance to clients
In the current branch, metrics collection is being enhanced with a fanout approach to submit to multiple API endpoints.
## Development Environment
### Required Dependencies
- Rust toolchain (stable, 1.80+)
- Node.js (v20+) and yarn for TypeScript components
- SQLite for local database development
- PostgreSQL for API database (optional, for full API functionality)
- CosmWasm tools for contract development
- For building contracts: `wasm-opt` tool from `binaryen`
- Python 3.8+ for some scripts
- Docker (optional, for containerized development)
- protoc (Protocol Buffers compiler) for some components
### Environment Configurations
The `envs/` directory contains pre-configured environments:
#### Available Environments
- **`local.env`**: Local development environment
- Points to local services (localhost)
- Uses test mnemonics and keys
- Ideal for testing without external dependencies
- **`sandbox.env`**: Sandbox test network
- Public test network with real nodes
- Test tokens available from faucet
- Contract addresses for sandbox deployment
- API: https://sandbox-nym-api1.nymtech.net
- **`mainnet.env`**: Production mainnet
- Real network with real tokens
- Production contract addresses
- API: https://validator.nymtech.net
- Use with caution!
- **`canary.env`**: Canary deployment
- Pre-release testing environment
- Tests new features before mainnet
- **`mainnet-local-api.env`**: Hybrid environment
- Uses mainnet contracts but local API
- Useful for API development against mainnet data
#### Key Environment Variables
```bash
# Network configuration
NETWORK_NAME=sandbox # Network identifier
BECH32_PREFIX=n # Address prefix (n for sandbox, n for mainnet)
NYM_API=https://sandbox-nym-api1.nymtech.net/api
NYXD=https://rpc.sandbox.nymtech.net
NYM_API_NETWORK=sandbox
# Contract addresses (network-specific)
MIXNET_CONTRACT_ADDRESS=n1xr3rq8yvd7qplsw5yx90ftsr2zdhg4e9z60h5duusgxpv72hud3sjkxkav
VESTING_CONTRACT_ADDRESS=n1unyuj8qnmygvzuex3dwmg9yzt9alhvyeat0uu0jedg2wj33efl5qackslz
# ... other contract addresses
# Mnemonic for testing (NEVER use in production)
MNEMONIC="clutch captain shoe salt awake harvest setup primary inmate ugly among become"
# API Keys and tokens
IPINFO_API_TOKEN=your_token_here
AUTHENTICATOR_PASSWORD=password_here
# Logging
RUST_LOG=info # Options: error, warn, info, debug, trace
RUST_BACKTRACE=1 # Enable backtraces
# Database
DATABASE_URL=postgresql://user:pass@localhost/nym_api
```
#### Using Environment Files
```bash
# Load environment and run command
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Export to shell
source envs/sandbox.env
# Use with make targets
dotenv -f envs/sandbox.env -- make run-api-tests
```
## Initial Setup
### First Time Setup
1. **Install Prerequisites**
```bash
# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install Node.js and yarn
# Via nvm (recommended):
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
nvm install 20
npm install -g yarn
# Install build tools
# Ubuntu/Debian:
sudo apt-get install build-essential pkg-config libssl-dev protobuf-compiler libpq-dev
# macOS:
brew install protobuf postgresql
# Install wasm-opt for contract builds
npm install -g wasm-opt
# Add wasm target for Rust
rustup target add wasm32-unknown-unknown
```
2. **Clone and Setup Repository**
```bash
git clone https://github.com/nymtech/nym.git
cd nym/nym
# Install JavaScript dependencies
yarn install
# Build the project
make build
```
3. **Database Setup (Optional, for API development)**
```bash
# Install PostgreSQL
# Create database
createdb nym_api
# Run migrations (from nym-api directory)
cd nym-api
sqlx migrate run
```
### Quick Start
```bash
# Run a mixnode locally
dotenv -f envs/sandbox.env -- cargo run -p nym-node -- run --mode mixnode --id my-mixnode
# Run a gateway locally
dotenv -f envs/sandbox.env -- cargo run -p nym-node -- run --mode gateway --id my-gateway
# Run the API server
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Run a client
cargo run -p nym-client -- init --id my-client
cargo run -p nym-client -- run --id my-client
```
## CI/CD Pipeline
The project uses GitHub Actions for CI/CD with several key workflows:
1. **Build and Test**:
- `ci-build.yml`: Main build workflow for Rust components
- Tests are run on multiple platforms (Linux, Windows, macOS)
- Includes formatting check (rustfmt) and linting (clippy)
2. **Release Process**:
- Binary artifacts are published on release tags
- Multiple platform builds are created
3. **Documentation**:
- Documentation is automatically built and deployed
## Database Structure
The system uses SQLite databases with tables like:
- `mixnode_status`: Status information about mixnodes
- `gateway_status`: Status information about gateways
- `routes`: Route performance information (success/failure of specific paths)
- `monitor_run`: Information about monitoring test runs
## Development Workflows
### Running a Node
To run the mixnode or gateway:
```bash
# Run nym-node as a mixnode with specified identity
cargo run -p nym-node -- run --mode mixnode --id my-mixnode
# Run nym-node as a gateway
cargo run -p nym-node -- run --mode gateway --id my-gateway
```
### Configuration
Nodes can be configured with files in various locations:
- Command-line arguments
- Environment variables
- `.env` files specified with `--config-env-file`
### Monitoring
To monitor the health of your node:
- View logs for real-time information
- Use the node's HTTP API for status information
- Check the explorer for public node statistics
## Common Libraries
- `common/types`: Shared data types across all components
- `common/crypto`: Cryptographic primitives and wrappers
- `common/client-core`: Core client functionality
- `common/gateway-client`: Client-gateway communication
- `common/task`: Task management and concurrency utilities
- `common/nymsphinx`: Sphinx packet implementation for mixnet
- `common/topology`: Network topology management
- `common/credentials`: Credential system for privacy-preserving authentication
- `common/bandwidth-controller`: Bandwidth management and accounting
## Code Conventions
- Error handling: Use anyhow/thiserror for structured error handling
- Logging: Use the tracing framework for logging and diagnostics
- State management: Generally use Tokio/futures for async code
- Configuration: Use the config crate and env vars with defaults
- Database: Use sqlx for type-safe database queries
- Follow clippy recommendations and rustfmt formatting
- Use semantic commit messages: feat, fix, docs, refactor, test, chore
## When Making Changes
- Run `make test` before submitting PRs
- Follow Rust naming conventions
- Use `clippy` to check for common issues
- Update SQLx query caches when modifying DB queries: `cargo sqlx prepare`
- Consider backward compatibility for protocol changes
- Use lefthook pre-commit hooks for TypeScript formatting
- Run `cargo deny check` to verify dependency compliance
- Test against both sandbox and local environments when possible
- Update relevant documentation and CHANGELOG.md
## Development Tools
### Useful Cargo Commands
```bash
# Check for outdated dependencies
cargo outdated
# Analyze binary size
cargo bloat --release -p nym-node
# Generate dependency graph
cargo tree -p nym-api
# Run with instrumentation
cargo run --features profiling -p nym-node
# Check for security advisories
cargo audit
```
### Database Tools
```bash
# SQLx CLI for migrations
cargo install sqlx-cli
# Create new migration
cd nym-api && sqlx migrate add <migration_name>
# Prepare query metadata for offline compilation
cargo sqlx prepare --workspace
# View database schema
./nym-api/enter_db.sh
```
### Development Scripts
- `scripts/build_topology.py`: Generate network topology files
- `scripts/node_api_check.py`: Verify node API endpoints
- `scripts/network_tunnel_manager.sh`: Manage network tunnels
- `scripts/localnet_start.sh`: Start a local test network
- Various deployment scripts in `deployment/` for different environments
## Debugging
- Enable more verbose logging with the RUST_LOG environment variable:
```
RUST_LOG=debug,nym_node=trace cargo run -p nym-node -- run --mode mixnode
```
- Use the HTTP API endpoints for status information
- Check monitoring data in the database for network performance metrics
- For complex issues, use tracing tools to follow packet flow
- Enable backtraces: `RUST_BACKTRACE=full`
- For WASM debugging: Use browser developer tools with source maps
## Deployment and Advanced Configurations
### Deployment Structure
The `deployment/` directory contains Ansible playbooks and configurations for various deployment scenarios:
- **`aws/`**: AWS-specific deployment configurations
- **`mixnode/`**: Mixnode deployment playbooks
- **`gateway/`**: Gateway deployment playbooks
- **`validator/`**: Validator node deployment
- **`sandbox-v2/`**: Complete sandbox environment setup
- **`big-dipper-2/`**: Block explorer deployment
### Sandbox V2 Deployment
The sandbox-v2 deployment (`deployment/sandbox-v2/`) provides a complete test environment:
```bash
# Key playbooks:
- deploy.yaml # Main deployment orchestrator
- deploy-mixnodes.yaml # Deploy mixnodes
- deploy-gateways.yaml # Deploy gateways
- deploy-validators.yaml # Deploy validator nodes
- deploy-nym-api.yaml # Deploy API services
```
### Custom Environment Setup
To create a custom environment:
1. Copy an existing env file: `cp envs/sandbox.env envs/custom.env`
2. Modify the network endpoints and contract addresses
3. Update the `NETWORK_NAME` to your identifier
4. Set appropriate mnemonics and keys (use fresh ones for production!)
### Contract Addresses
Contract addresses are network-specific and defined in environment files:
- Mixnet contract: Manages mixnode/gateway registry
- Vesting contract: Handles token vesting schedules
- Coconut contracts: Privacy-preserving credentials
- Name service: Human-readable address mapping
- Ecash contract: Electronic cash functionality
### Local Network Setup
For a completely local network:
```bash
# Start local chain
./scripts/localnet_start.sh
# Deploy contracts
cd contracts
make deploy-local
# Start nodes with local config
dotenv -f envs/local.env -- cargo run -p nym-node -- run --mode mixnode
```
## Common Issues and Troubleshooting
### Database Issues
- When modifying database queries, you must update SQLx query caches:
```bash
cargo sqlx prepare
```
- If you see SQLx errors about missing query files, this is likely the cause
- For "database is locked" errors with SQLite, ensure only one process accesses the DB
- For PostgreSQL connection issues, verify DATABASE_URL and that the server is running
### API Connection Issues
- Check the environment variables pointing to the APIs (NYM_API, NYXD)
- Verify network connectivity and API health endpoints
- For authentication issues, check node keys and credentials
- Common endpoints to verify:
- API health: `$NYM_API/health`
- Chain status: `$NYXD/status`
- Contract info: `$NYXD/cosmwasm/wasm/v1/contract/$CONTRACT_ADDRESS`
### Build Problems
- Clean dependencies with `cargo clean` for a fresh build
- Check for compatible Rust version (1.80+ recommended)
- For smart contract builds, ensure wasm-opt is installed: `npm install -g wasm-opt`
- For cross-compilation issues, check target-specific dependencies
- WASM build issues: Ensure wasm32-unknown-unknown target is installed:
```bash
rustup target add wasm32-unknown-unknown
```
- For "cannot find -lpq" errors, install PostgreSQL development files:
```bash
# Ubuntu/Debian
sudo apt-get install libpq-dev
# macOS
brew install postgresql
```
### Environment Issues
- Contract address mismatches: Ensure you're using the correct environment file
- "Account sequence mismatch": The account nonce is out of sync, wait and retry
- Token decimal issues: Sandbox uses different decimal places than mainnet
- API version mismatches: Ensure your local API version matches the network
- "Insufficient funds": Get test tokens from faucet (sandbox) or check balance
- Gateway/mixnode bonding issues: Verify minimum stake requirements
## Working with Routes and Monitoring
1. Route monitoring metrics are stored in a `routes` table with:
- Layer node IDs (layer1, layer2, layer3, gw)
- Success flag (boolean)
- Timestamp
2. To analyze routes:
- Check `NetworkAccount` and `AccountingRoute` in `nym-network-monitor/src/accounting.rs`
- View monitoring logic in `common/nymsphinx/chunking/monitoring.rs`
- Observe how routes are submitted to the database in the `submit_accounting_routes_to_db` function
## Performance Optimization
### Profiling and Benchmarking
```bash
# Run benchmarks
cargo bench -p nym-node
# Profile with perf (Linux)
cargo build --release --features profiling
perf record --call-graph=dwarf ./target/release/nym-node run --mode mixnode
perf report
# Generate flamegraph
cargo install flamegraph
cargo flamegraph --bin nym-node -- run --mode mixnode
```
### Common Performance Considerations
- Use bounded channels for backpressure
- Batch database operations where possible
- Monitor memory usage with `RUST_LOG=nym_node::metrics=debug`
- Use connection pooling for database connections
- Consider using `jemalloc` for better memory allocation performance
Generated
+900 -1664
View File
File diff suppressed because it is too large Load Diff
+39 -64
View File
@@ -31,20 +31,14 @@ members = [
"common/client-libs/mixnet-client",
"common/client-libs/validator-client",
"common/commands",
"common/nym-common",
"common/config",
"common/cosmwasm-smart-contracts/coconut-dkg",
"common/cosmwasm-smart-contracts/contracts-common",
"common/cosmwasm-smart-contracts/contracts-common-testing",
"common/cosmwasm-smart-contracts/easy_addr",
"common/cosmwasm-smart-contracts/contracts-common", "common/cosmwasm-smart-contracts/easy_addr",
"common/cosmwasm-smart-contracts/ecash-contract",
"common/cosmwasm-smart-contracts/group-contract",
"common/cosmwasm-smart-contracts/mixnet-contract",
"common/cosmwasm-smart-contracts/multisig-contract",
"common/cosmwasm-smart-contracts/nym-performance-contract",
"common/cosmwasm-smart-contracts/nym-pool-contract",
"common/cosmwasm-smart-contracts/vesting-contract",
"common/credential-proxy",
"common/credential-storage",
"common/credential-utils",
"common/credential-verification",
@@ -52,15 +46,13 @@ members = [
"common/credentials-interface",
"common/crypto",
"common/dkg",
"common/ecash-signer-check",
"common/ecash-signer-check-types",
"common/ecash-time",
"common/execute",
"common/exit-policy",
"common/gateway-requests",
"common/gateway-stats-storage",
"common/gateway-storage",
"common/http-api-client",
"common/http-api-client-macro",
"common/http-api-common",
"common/inclusion-probability",
"common/ip-packet-requests",
@@ -69,13 +61,9 @@ members = [
"common/network-defaults",
"common/node-tester-utils",
"common/nonexhaustive-delayqueue",
"common/nym-cache",
"common/nym-connection-monitor",
"common/nym-id",
"common/nym-metrics",
"common/nym_offline_compact_ecash",
"common/nymnoise",
"common/nymnoise/keys",
"common/nymsphinx",
"common/nymsphinx/acknowledgements",
"common/nymsphinx/addressing",
@@ -89,7 +77,6 @@ members = [
"common/nymsphinx/types",
"common/nyxd-scraper",
"common/pemstore",
"common/registration",
"common/serde-helpers",
"common/service-provider-requests-common",
"common/socks5-client-core",
@@ -98,33 +85,25 @@ members = [
"common/statistics",
"common/store-cipher",
"common/task",
"common/test-utils",
"common/ticketbooks-merkle",
"common/topology",
"common/tun",
"common/types",
"common/upgrade-mode-check",
"common/verloc",
"common/wasm/client-core",
"common/wasm/storage",
"common/wasm/utils",
"common/wireguard",
"common/wireguard-private-metadata/client",
"common/wireguard-private-metadata/server",
"common/wireguard-private-metadata/shared",
"common/wireguard-private-metadata/tests",
"common/wireguard-types",
"common/zulip-client",
"documentation/autodoc",
"gateway",
"integrations/bity",
"nym-api",
"nym-api/nym-api-requests",
"nym-authenticator-client",
"nym-browser-extension/storage",
"nym-credential-proxy/nym-credential-proxy",
"nym-credential-proxy/nym-credential-proxy-requests",
"nym-credential-proxy/vpn-api-lib-wasm",
"nym-ip-packet-client",
"nym-network-monitor",
"nym-node",
"nym-node-status-api/nym-node-status-agent",
@@ -133,19 +112,16 @@ members = [
"nym-node/nym-node-metrics",
"nym-node/nym-node-requests",
"nym-outfox",
"nym-registration-client",
"nym-signers-monitor",
"nym-statistics-api",
"nym-validator-rewarder",
"nyx-chain-watcher",
"sdk/ffi/cpp",
"sdk/ffi/go",
"sdk/ffi/shared",
"sdk/rust/nym-sdk",
"service-providers/authenticator",
"service-providers/common",
"service-providers/ip-packet-router",
"service-providers/network-requester",
"sqlx-pool-guard",
"tools/echo-server",
"tools/internal/contract-state-importer/importer-cli",
"tools/internal/contract-state-importer/importer-contract",
@@ -153,8 +129,9 @@ members = [
# "tools/internal/sdk-version-bump",
"tools/internal/ssl-inject",
"tools/internal/testnet-manager",
"tools/internal/testnet-manager",
"tools/internal/testnet-manager/dkg-bypass-contract",
"tools/internal/validator-status-check",
"tools/internal/testnet-manager/dkg-bypass-contract", "tools/internal/validator-status-check",
"tools/nym-cli",
"tools/nym-id-cli",
"tools/nym-nr-query",
@@ -165,7 +142,6 @@ members = [
"wasm/mix-fetch",
"wasm/node-tester",
"wasm/zknym-lib",
"nym-gateway-probe"
]
default-members = [
@@ -176,24 +152,24 @@ default-members = [
"nym-node",
"nym-node-status-api/nym-node-status-agent",
"nym-node-status-api/nym-node-status-api",
"nym-statistics-api",
"nym-validator-rewarder",
"nyx-chain-watcher",
"service-providers/authenticator",
"service-providers/ip-packet-router",
"service-providers/network-requester",
"tools/nymvisor",
]
exclude = ["contracts", "nym-wallet", "cpu-cycles"]
exclude = ["explorer", "contracts", "nym-wallet", "cpu-cycles"]
[workspace.package]
authors = ["Nym Technologies SA"]
repository = "https://github.com/nymtech/nym"
homepage = "https://nymtech.net"
documentation = "https://nymtech.net"
edition = "2024"
edition = "2021"
license = "Apache-2.0"
rust-version = "1.85"
rust-version = "1.80"
readme = "README.md"
[workspace.dependencies]
@@ -215,13 +191,14 @@ base64 = "0.22.1"
base85rs = "0.1.3"
bincode = "1.3.3"
bip39 = { version = "2.0.0", features = ["zeroize"] }
bit-vec = "0.7.0" # can we unify those?
bitvec = "1.0.0"
blake3 = "1.7.0"
bloomfilter = "3.0.1"
bs58 = "0.5.1"
bytecodec = "0.4.15"
bytes = "1.10.1"
cargo_metadata = "0.19.2"
cargo_metadata = "0.18.1"
celes = "2.6.0"
cfg-if = "1.0.0"
chacha20 = "0.9.0"
@@ -233,8 +210,8 @@ clap_complete = "4.5"
clap_complete_fig = "4.5"
colored = "2.2"
comfy-table = "7.1.4"
console = "0.16.0"
console-subscriber = "0.4.1"
console = "0.15.11"
console-subscriber = "0.1.1"
console_error_panic_hook = "0.1"
const-str = "0.5.6"
const_format = "0.2.34"
@@ -242,13 +219,14 @@ criterion = "0.5"
csv = "1.3.1"
ctr = "0.9.1"
cupid = "0.6.1"
curve25519-dalek = "4.1"
dashmap = "5.5.3"
# We want https://github.com/DefGuard/wireguard-rs/pull/64 , but there's no crates.io release being pushed out anymore
defguard_wireguard_rs = { git = "https://github.com/DefGuard/wireguard-rs.git", rev = "v0.4.7" }
digest = "0.10.7"
dirs = "6.0"
dirs = "5.0"
doc-comment = "0.3"
dotenvy = "0.15.6"
dyn-clone = "1.0.19"
ecdsa = "0.16"
ed25519-dalek = "2.1"
encoding_rs = "0.8.35"
@@ -262,8 +240,11 @@ futures = "0.3.31"
futures-util = "0.3"
generic-array = "0.14.7"
getrandom = "0.2.10"
getset = "0.1.5"
handlebars = "3.5.5"
headers = "0.4.0"
hex = "0.4.3"
hex-literal = "0.3.3"
hickory-resolver = "0.25"
hkdf = "0.12.3"
hmac = "0.12.1"
@@ -275,22 +256,22 @@ humantime = "2.2.0"
humantime-serde = "1.1.1"
hyper = "1.6.0"
hyper-util = "0.1"
indicatif = "0.18.0"
indicatif = "0.17.11"
inquire = "0.6.2"
inventory = "0.3.21"
ip_network = "0.4.1"
ipnetwork = "0.20"
itertools = "0.14.0"
jwt-simple = { version = "0.12.12", default-features = false, features = ["pure-rust"] }
k256 = "0.13"
lazy_static = "1.5.0"
ledger-transport = "0.10.0"
ledger-transport-hid = "0.10.0"
log = "0.4"
maxminddb = "0.23.0"
mime = "0.3.17"
moka = { version = "0.12", features = ["future"] }
nix = "0.27.1"
notify = "5.1.0"
okapi = "0.7.0"
once_cell = "1.21.3"
opentelemetry = "0.19.0"
opentelemetry-jaeger = "0.18.0"
@@ -298,18 +279,20 @@ parking_lot = "0.12.3"
pem = "0.8"
petgraph = "0.6.5"
pin-project = "1.1"
pnet_packet = "0.35.0"
pin-project-lite = "0.2.16"
publicsuffix = "2.3.0"
proc_pidinfo = "0.1.3"
quote = "1"
rand = "0.8.5"
rand_chacha = "0.3"
rand_core = "0.6.3"
rand_distr = "0.4"
rand_pcg = "0.3.1"
rand_seeder = "0.2.3"
rayon = "1.5.1"
regex = "1.10.6"
reqwest = { version = "0.12.15", default-features = false }
rs_merkle = "1.5.0"
safer-ffi = "0.1.13"
schemars = "0.8.22"
semver = "1.0.26"
serde = "1.0.219"
@@ -320,24 +303,21 @@ serde_json_path = "0.7.2"
serde_repr = "0.1"
serde_with = "3.9.0"
serde_yaml = "0.9.25"
serde_plain = "1.0.2"
sha2 = "0.10.9"
si-scale = "0.2.3"
snow = "0.9.6"
sphinx-packet = "=0.6.0"
sqlx = "0.8.6"
strum = "0.27.2"
strum_macros = "0.27.2"
sqlx = "0.7.4"
strum = "0.26"
strum_macros = "0.26"
subtle-encoding = "0.5"
syn = "2"
sysinfo = "0.37.0"
syn = "1"
sysinfo = "0.33.0"
tap = "1.0.1"
tar = "0.4.44"
test-with = { version = "0.15.4", default-features = false }
tempfile = "3.20"
tempfile = "3.19"
thiserror = "2.0"
time = "0.3.41"
tokio = "1.47"
tokio = "1.44"
tokio-postgres = "0.7"
tokio-stream = "0.1.17"
tokio-test = "0.4.4"
@@ -353,10 +333,8 @@ tracing-opentelemetry = "0.19.0"
tracing-subscriber = "0.3.19"
tracing-tree = "0.2.2"
tracing-indicatif = "0.3.9"
tracing-test = "0.2.5"
ts-rs = "10.1.0"
tungstenite = { version = "0.20.1", default-features = false }
typed-builder = "0.23.0"
uniffi = "0.29.2"
uniffi_build = "0.29.0"
url = "2.5"
@@ -365,8 +343,8 @@ utoipa-swagger-ui = "8.1"
utoipauto = "0.2"
uuid = "*"
vergen = { version = "=8.3.1", default-features = false }
vergen-gitcl = { version = "1.0.8", default-features = false }
walkdir = "2"
wasm-bindgen-test = "0.3.49"
x25519-dalek = "2.0.0"
zeroize = "1.7.0"
@@ -384,6 +362,9 @@ subtle = "2.5.0"
# cosmwasm-related
cosmwasm-schema = "=2.2.2"
cosmwasm-std = "=2.2.2"
# use 1.0.1 as that's the version used by cosmwasm-std 2.2.1
# (and ideally we don't want to pull the same dependency twice)
serde-json-wasm = "=1.0.1"
# same version as used by cosmwasm
cw-utils = "=2.0.0"
cw-storage-plus = "=2.0.0"
@@ -391,7 +372,6 @@ cw2 = { version = "=2.0.0" }
cw3 = { version = "=2.0.0" }
cw4 = { version = "=2.0.0" }
cw-controllers = { version = "=2.0.0" }
cw-multi-test = "=2.3.2"
# cosmrs-related
bip32 = { version = "0.5.3", default-features = false }
@@ -405,19 +385,17 @@ prost = { version = "0.13", default-features = false }
# wasm-related dependencies
gloo-utils = "0.2.0"
gloo-net = "0.6.0"
gloo-timers = "0.3.0"
indexed_db_futures = "0.6.4"
js-sys = "0.3.76"
serde-wasm-bindgen = "0.6.5"
tsify = "0.4.5"
tokio_with_wasm = { version = "0.8.7" }
wasm-bindgen = "0.2.99"
wasm-bindgen-futures = "0.4.49"
wasm-bindgen-test = "0.3.49"
wasmtimer = "0.4.1"
web-sys = "0.3.76"
# for local development:
#[patch.crates-io]
#sphinx-packet = { path = "../sphinx" }
@@ -449,9 +427,6 @@ opt-level = 'z'
# lto = true
opt-level = 'z'
[workspace.lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tokio_unstable)'] }
[workspace.lints.clippy]
unwrap_used = "deny"
expect_used = "deny"
+15 -71
View File
@@ -12,11 +12,7 @@ help:
@echo " clippy: run clippy for all workspaces"
@echo " test: run clippy, unit tests, and formatting."
@echo " test-all: like test, but also includes the expensive tests"
@echo " deb: build debian packages"
@echo ""
@echo "Contract building targets:"
@echo " contracts: build contracts for development (includes wasm-opt)"
@echo " publish-contracts: build contracts using Docker optimizer (deterministic)"
@echo " deb: build debian packages
# -----------------------------------------------------------------------------
# Meta targets
@@ -107,16 +103,16 @@ sdk-wasm-build:
$(MAKE) -C nym-browser-extension/storage wasm-pack
$(MAKE) -C wasm/client
$(MAKE) -C wasm/node-tester
# $(MAKE) -C wasm/mix-fetch
$(MAKE) -C wasm/mix-fetch
$(MAKE) -C wasm/zknym-lib
# $(MAKE) -C wasm/full-nym-wasm
#$(MAKE) -C wasm/full-nym-wasm
# run this from npm/yarn to ensure tools are in the path, e.g. yarn build:sdk from root of repo
sdk-typescript-build:
npx lerna run --scope @nymproject/sdk build --stream
# npx lerna run --scope @nymproject/mix-fetch build --stream
# npx lerna run --scope @nymproject/node-tester build --stream
# yarn --cwd sdk/typescript/codegen/contract-clients build
npx lerna run --scope @nymproject/mix-fetch build --stream
npx lerna run --scope @nymproject/node-tester build --stream
yarn --cwd sdk/typescript/codegen/contract-clients build
# NOTE: These targets are part of the main workspace (but not as wasm32-unknown-unknown)
WASM_CRATES = extension-storage nym-client-wasm nym-node-tester-wasm zknym-lib
@@ -134,77 +130,25 @@ cargo-test: sdk-wasm-test
clippy: sdk-wasm-lint
# -----------------------------------------------------------------------------
# Build CosmWasm contracts (deterministic docker build)
# Build contracts ready for deploy
# -----------------------------------------------------------------------------
CONTRACTS=vesting_contract mixnet_contract nym_ecash cw3_flex_multisig cw4_group nym_coconut_dkg
CONTRACTS_WASM=$(addsuffix .wasm, $(CONTRACTS))
CONTRACTS_OUT_DIR=contracts/target/wasm32-unknown-unknown/release
WASM_CONTRACT_DIR := contracts/target/wasm32-unknown-unknown/release
# Find every direct contract folder that contains a Cargo.toml
CONTRACT_DIRS := $(shell find contracts -type f -name Cargo.toml \( ! -path "contracts/Cargo.toml" \) | grep -v integration-tests | xargs -n1 dirname | sort -u)
CONTRACTS_OUT_DIR = contracts/artifacts
# Build all contracts via the official CosmWasm optimizer image (one invocation per contract)
# See : https://github.com/CosmWasm/optimizer?tab=readme-ov-file#contracts-excluded-from-workspace
# The optimizer ships separate multi-arch images. ARM builds are *not* bit-for-bit identical to the
# canonical x86_64 build (see README notice in CosmWasm/optimizer). For reproducible artefacts we
# therefore always run the amd64 variant by default.
# Override with :
# $ COSMWASM_OPTIMIZER_IMAGE=cosmwasm/optimizer-arm64:0.17.0 make contracts-publish
#
COSMWASM_OPTIMIZER_IMAGE ?= cosmwasm/optimizer:0.17.0
COSMWASM_OPTIMIZER_PLATFORM ?= linux/amd64
COSMWASM_CHECK_IMAGE ?= rust:1.88
# Ensure clean build environment and run the optimizer
optimize-contracts:
@rm -rf artifacts 2>/dev/null || true
@echo "=== Ensuring clean build environment"
docker volume rm nym_contracts_cache 2>/dev/null || true
docker volume rm registry_cache 2>/dev/null || true
@for DIR in $(CONTRACT_DIRS); do \
echo "=== Optimizing $${DIR}"; \
docker run --rm --platform $(COSMWASM_OPTIMIZER_PLATFORM) \
-v $(CURDIR):/code \
--mount type=volume,source=nym_contracts_cache,target=/target \
--mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
-e CARGO_BUILD_INCREMENTAL=false \
-e RUSTFLAGS="-C target-cpu=generic -C debuginfo=0" \
-e SOURCE_DATE_EPOCH=1 \
$(COSMWASM_OPTIMIZER_IMAGE) $${DIR}; \
done
@mkdir -p $(CONTRACTS_OUT_DIR)
@cp artifacts/*.wasm $(CONTRACTS_OUT_DIR)/ 2>/dev/null || true
@cd $(CONTRACTS_OUT_DIR) && sha256sum *.wasm > checksums.txt
# Cleanup temporary artefacts directory
@rm -rf artifacts 2>/dev/null || true
# Check artifacts with cosmwasm-check inside the optimizer image
docker-check-contracts:
@docker run --rm --platform $(COSMWASM_OPTIMIZER_PLATFORM) \
-v $(CURDIR):/code --workdir /code \
--entrypoint /bin/sh \
$(COSMWASM_CHECK_IMAGE) -lc 'apt-get update && apt-get install -y --no-install-recommends llvm-dev libclang-dev pkg-config && export PATH="/usr/local/cargo/bin:/usr/local/rustup/bin:$$PATH" && cargo install cosmwasm-check --locked && WASMER_ENGINE=universal WASMER_COMPILER=singlepass cosmwasm-check contracts/artifacts/*.wasm'
contracts: build-release-contracts wasm-opt-contracts cosmwasm-check-contracts
wasm-opt-contracts:
@for WASM in $(WASM_CONTRACT_DIR)/*.wasm; do \
echo "Running wasm-opt on $$WASM"; \
wasm-opt --signext-lowering -Os $$WASM -o $$WASM ; \
for contract in $(CONTRACTS_WASM); do \
wasm-opt --signext-lowering -Os $(CONTRACTS_OUT_DIR)/$$contract -o $(CONTRACTS_OUT_DIR)/$$contract; \
done
cosmwasm-check-contracts:
@for WASM in $(WASM_CONTRACT_DIR)/*.wasm; do \
echo "Checking $$WASM"; \
cosmwasm-check $$WASM ; \
for contract in $(CONTRACTS_WASM); do \
cosmwasm-check $(CONTRACTS_OUT_DIR)/$$contract; \
done
# Default development build
contracts: build-release-contracts wasm-opt-contracts cosmwasm-check-contracts
# Publishing build used by CI deterministic Docker optimiser
publish-contracts: optimize-contracts cosmwasm-check-contracts
# Consider adding 's' to make plural consistent (beware: used in github workflow)
contract-schema:
$(MAKE) -C contracts schema
+2 -2
View File
@@ -1,10 +1,10 @@
[package]
name = "nym-client"
version = "1.1.64"
version = "1.1.55"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
description = "Implementation of the Nym Client"
edition = "2021"
rust-version = "1.85"
rust-version = "1.70"
license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -2048,11 +2048,10 @@
}
},
"node_modules/http-proxy-middleware": {
"version": "2.0.9",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.9.tgz",
"integrity": "sha512-c1IyJYLYppU574+YI7R4QyX2ystMtVXZwIdzazUIPIJsHuWNd+mho2j+bKoHftndicGj9yh+xjd+l0yj7VeT1Q==",
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.4.tgz",
"integrity": "sha512-m/4FxX17SUvz4lJ5WPXOHDUuCwIqXLfLHs1s0uZ3oYjhoXlx9csYxaOa0ElDEJ+h8Q4iJ1s+lTMbiCa4EXIJqg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/http-proxy": "^1.17.8",
"http-proxy": "^1.18.1",
@@ -6096,9 +6095,9 @@
}
},
"http-proxy-middleware": {
"version": "2.0.9",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.9.tgz",
"integrity": "sha512-c1IyJYLYppU574+YI7R4QyX2ystMtVXZwIdzazUIPIJsHuWNd+mho2j+bKoHftndicGj9yh+xjd+l0yj7VeT1Q==",
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.4.tgz",
"integrity": "sha512-m/4FxX17SUvz4lJ5WPXOHDUuCwIqXLfLHs1s0uZ3oYjhoXlx9csYxaOa0ElDEJ+h8Q4iJ1s+lTMbiCa4EXIJqg==",
"dev": true,
"requires": {
"@types/http-proxy": "^1.17.8",
+11 -16
View File
@@ -11,7 +11,7 @@ use nym_client_core::client::base_client::{
BaseClientBuilder, ClientInput, ClientOutput, ClientState,
};
use nym_sphinx::params::PacketType;
use nym_task::ShutdownManager;
use nym_task::TaskHandle;
use nym_validator_client::QueryHttpRpcNyxdClient;
use std::error::Error;
use std::path::PathBuf;
@@ -29,8 +29,6 @@ pub struct SocketClient {
/// Optional path to a .json file containing standalone network details.
custom_mixnet: Option<PathBuf>,
shutdown_manager: ShutdownManager,
}
impl SocketClient {
@@ -42,7 +40,6 @@ impl SocketClient {
SocketClient {
config,
custom_mixnet,
shutdown_manager: Default::default(),
}
}
@@ -52,7 +49,7 @@ impl SocketClient {
client_output: ClientOutput,
client_state: ClientState,
self_address: &Recipient,
shutdown_token: nym_task::ShutdownToken,
task_client: nym_task::TaskClient,
packet_type: PacketType,
) {
info!("Starting websocket listener...");
@@ -60,7 +57,6 @@ impl SocketClient {
let ClientInput {
connection_command_sender,
input_sender,
..
} = client_input;
let ClientOutput {
@@ -81,24 +77,24 @@ impl SocketClient {
shared_lane_queue_lengths,
reply_controller_sender,
Some(packet_type),
shutdown_token.clone(),
task_client.fork("websocket_handler"),
);
websocket::Listener::new(
config.socket.host,
config.socket.listening_port,
shutdown_token.child_token(),
task_client.with_suffix("websocket_listener"),
)
.start(websocket_handler);
}
/// blocking version of `start_socket` method. Will run forever (or until SIGINT is sent)
pub async fn run_socket_forever(self) -> Result<(), Box<dyn Error + Send + Sync>> {
let mut shutdown = self.start_socket().await?;
let shutdown = self.start_socket().await?;
shutdown.run_until_shutdown().await;
let res = shutdown.wait_for_shutdown().await;
log::info!("Stopping nym-client");
Ok(())
res
}
async fn initialise_storage(&self) -> Result<OnDiskPersistent, ClientError> {
@@ -115,7 +111,7 @@ impl SocketClient {
let dkg_query_client = if self.config.base.client.disabled_credentials_mode {
None
} else {
Some(default_query_dkg_client_from_config(&self.config.base)?)
Some(default_query_dkg_client_from_config(&self.config.base))
};
let storage = self.initialise_storage().await?;
@@ -123,7 +119,6 @@ impl SocketClient {
let mut base_client =
BaseClientBuilder::new(self.config().base(), storage, dkg_query_client)
.with_shutdown(self.shutdown_manager.shutdown_tracker_owned())
.with_user_agent(user_agent);
if let Some(custom_mixnet) = &self.custom_mixnet {
@@ -133,7 +128,7 @@ impl SocketClient {
Ok(base_client)
}
pub async fn start_socket(self) -> Result<ShutdownManager, ClientError> {
pub async fn start_socket(self) -> Result<TaskHandle, ClientError> {
if !self.config.socket.socket_type.is_websocket() {
return Err(ClientError::InvalidSocketMode);
}
@@ -152,13 +147,13 @@ impl SocketClient {
client_output,
client_state,
&self_address,
self.shutdown_manager.child_shutdown_token(),
started_client.task_handle.get_handle(),
packet_type,
);
info!("Client startup finished!");
info!("The address of this client is: {self_address}");
Ok(self.shutdown_manager)
Ok(started_client.task_handle)
}
}
+28 -22
View File
@@ -19,7 +19,7 @@ use nym_sphinx::receiver::ReconstructedMessage;
use nym_task::connections::{
ConnectionCommand, ConnectionCommandSender, ConnectionId, LaneQueueLengths, TransmissionLane,
};
use nym_task::ShutdownToken;
use nym_task::TaskClient;
use std::time::Duration;
use tokio::net::TcpStream;
use tokio::time::Instant;
@@ -44,7 +44,7 @@ pub(crate) struct HandlerBuilder {
lane_queue_lengths: LaneQueueLengths,
reply_controller_sender: ReplyControllerSender,
packet_type: Option<PacketType>,
shutdown_token: ShutdownToken,
task_client: TaskClient,
}
impl HandlerBuilder {
@@ -57,7 +57,7 @@ impl HandlerBuilder {
lane_queue_lengths: LaneQueueLengths,
reply_controller_sender: ReplyControllerSender,
packet_type: Option<PacketType>,
shutdown_token: ShutdownToken,
task_client: TaskClient,
) -> Self {
Self {
msg_input,
@@ -67,13 +67,14 @@ impl HandlerBuilder {
lane_queue_lengths,
reply_controller_sender,
packet_type,
shutdown_token,
task_client,
}
}
// TODO: make sure we only ever have one active handler
pub fn create_active_handler(&self) -> Handler {
let shutdown_token = self.shutdown_token.clone();
let mut task_client = self.task_client.fork("active_handler");
task_client.disarm();
Handler {
msg_input: self.msg_input.clone(),
client_connection_tx: self.client_connection_tx.clone(),
@@ -84,7 +85,7 @@ impl HandlerBuilder {
lane_queue_lengths: self.lane_queue_lengths.clone(),
reply_controller_sender: self.reply_controller_sender.clone(),
packet_type: self.packet_type,
shutdown_token,
task_client,
}
}
}
@@ -99,14 +100,19 @@ pub(crate) struct Handler {
lane_queue_lengths: LaneQueueLengths,
reply_controller_sender: ReplyControllerSender,
packet_type: Option<PacketType>,
shutdown_token: ShutdownToken,
task_client: TaskClient,
}
impl Drop for Handler {
fn drop(&mut self) {
let _ = self
if let Err(err) = self
.buffer_requester
.unbounded_send(ReceivedBufferMessage::ReceiverDisconnect);
.unbounded_send(ReceivedBufferMessage::ReceiverDisconnect)
{
if !self.task_client.is_shutdown_poll() {
error!("failed to disconnect the receiver from the buffer: {err}");
}
}
}
}
@@ -136,7 +142,7 @@ impl Handler {
{
Ok(length) => length,
Err(err) => {
if !self.shutdown_token.is_cancelled() {
if !self.task_client.is_shutdown_poll() {
error!(
"Failed to get reply queue length for connection {connection_id}: {err}"
);
@@ -186,7 +192,7 @@ impl Handler {
// the ack control is now responsible for chunking, etc.
let input_msg = InputMessage::new_regular(recipient, message, lane, self.packet_type);
if let Err(err) = self.msg_input.send(input_msg).await {
if !self.shutdown_token.is_cancelled() {
if !self.task_client.is_shutdown_poll() {
error!("Failed to send message to the input buffer: {err}");
}
}
@@ -219,7 +225,7 @@ impl Handler {
let input_msg =
InputMessage::new_anonymous(recipient, message, reply_surbs, lane, self.packet_type);
if let Err(err) = self.msg_input.send(input_msg).await {
if !self.shutdown_token.is_cancelled() {
if !self.task_client.is_shutdown_poll() {
error!("Failed to send anonymous message to the input buffer: {err}");
}
}
@@ -247,7 +253,7 @@ impl Handler {
let input_msg = InputMessage::new_reply(recipient_tag, message, lane, self.packet_type);
if let Err(err) = self.msg_input.send(input_msg).await {
if !self.shutdown_token.is_cancelled() {
if !self.task_client.is_shutdown_poll() {
error!("Failed to send reply message to the input buffer: {err}");
}
}
@@ -269,7 +275,7 @@ impl Handler {
.client_connection_tx
.unbounded_send(ConnectionCommand::Close(connection_id))
{
if !self.shutdown_token.is_cancelled() {
if !self.task_client.is_shutdown_poll() {
error!("Failed to send close connection command: {err}");
}
}
@@ -312,7 +318,7 @@ impl Handler {
async fn handle_text_message(&mut self, msg: String) -> Option<WsMessage> {
debug!("Handling text message request");
trace!("Content: {msg:?}");
trace!("Content: {:?}", msg);
self.received_response_type = ReceivedResponseType::Text;
let client_request = ClientRequest::try_from_text(msg);
@@ -388,14 +394,11 @@ impl Handler {
}
async fn listen_for_requests(&mut self, mut msg_receiver: ReconstructedMessagesReceiver) {
let shutdown_token = self.shutdown_token.clone();
let mut task_client = self.task_client.fork("select");
task_client.disarm();
loop {
while !task_client.is_shutdown() {
tokio::select! {
_ = shutdown_token.cancelled() => {
log::trace!("Websocket handler: Received shutdown");
break;
}
// we can either get a client request from the websocket
socket_msg = self.next_websocket_request() => {
if socket_msg.is_none() {
@@ -433,6 +436,9 @@ impl Handler {
break;
}
}
_ = task_client.recv() => {
log::trace!("Websocket handler: Received shutdown");
}
}
}
log::debug!("Websocket handler: Exiting");
@@ -458,7 +464,7 @@ impl Handler {
reconstructed_sender,
))
{
if !self.shutdown_token.is_cancelled() {
if !self.task_client.is_shutdown_poll() {
error!("failed to announce the receiver to the buffer: {err}");
}
}
+9 -9
View File
@@ -3,7 +3,7 @@
use super::handler::HandlerBuilder;
use log::*;
use nym_task::ShutdownToken;
use nym_task::TaskClient;
use std::net::IpAddr;
use std::{net::SocketAddr, process, sync::Arc};
use tokio::io::AsyncWriteExt;
@@ -23,15 +23,15 @@ impl State {
pub(crate) struct Listener {
address: SocketAddr,
state: State,
shutdown_token: ShutdownToken,
task_client: TaskClient,
}
impl Listener {
pub(crate) fn new(host: IpAddr, port: u16, shutdown_token: ShutdownToken) -> Self {
pub(crate) fn new(host: IpAddr, port: u16, task_client: TaskClient) -> Self {
Listener {
address: SocketAddr::new(host, port),
state: State::AwaitingConnection,
shutdown_token,
task_client,
}
}
@@ -46,11 +46,11 @@ impl Listener {
let notify = Arc::new(Notify::new());
while !self.shutdown_token.is_cancelled() {
while !self.task_client.is_shutdown() {
tokio::select! {
// When the handler finishes we check if shutdown is signalled
_ = notify.notified() => {
if self.shutdown_token.is_cancelled() {
if self.task_client.is_shutdown() {
log::trace!("Websocket listener: detected shutdown after connection closed");
break;
}
@@ -59,7 +59,7 @@ impl Listener {
}
// ... but when there is no connected client at the time of shutdown being
// signalled, we handle it here.
_ = self.shutdown_token.cancelled() => {
_ = self.task_client.recv() => {
if !self.state.is_connected() {
log::trace!("Not connected: shutting down");
break;
@@ -68,9 +68,9 @@ impl Listener {
new_conn = tcp_listener.accept() => {
match new_conn {
Ok((mut socket, remote_addr)) => {
debug!("Received connection from {remote_addr:?}");
debug!("Received connection from {:?}", remote_addr);
if self.state.is_connected() {
warn!("Tried to open a duplicate websocket connection. The request came from {remote_addr}");
warn!("Tried to open a duplicate websocket connection. The request came from {}", remote_addr);
// if we've already got a connection, don't allow another one
// while we only ever want to accept a single connection, we don't want
// to leave clients hanging (and also allow for reconnection if it somehow
+2 -2
View File
@@ -1,10 +1,10 @@
[package]
name = "nym-socks5-client"
version = "1.1.64"
version = "1.1.55"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
description = "A SOCKS5 localhost proxy that converts incoming messages to Sphinx and sends them to a Nym address"
edition = "2021"
rust-version = "1.85"
rust-version = "1.70"
license.workspace = true
[dependencies]
+6 -6
View File
@@ -1,8 +1,8 @@
// Copyright 2023 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use futures::StreamExt;
use futures::channel::mpsc;
use futures::StreamExt;
use notify::event::{DataChange, MetadataKind, ModifyKind};
use notify::{Config, Event, EventKind, RecommendedWatcher, RecursiveMode, Watcher};
use std::collections::HashMap;
@@ -96,10 +96,10 @@ impl AsyncFileWatcher {
// when testing I was consistently getting two `Modify(Data(Any))` events in quick succession
// (probably to modify content and metadata).
// we really only want to propagate one of them
if let Some(previous) = self.last_received.get(&event.kind)
&& now.duration_since(*previous) < self.tick_duration
{
return false;
if let Some(previous) = self.last_received.get(&event.kind) {
if now.duration_since(*previous) < self.tick_duration {
return false;
}
}
let Some(filters) = &self.filters else {
@@ -137,7 +137,7 @@ impl AsyncFileWatcher {
log::error!("the file watcher receiver has been dropped!");
}
} else {
log::debug!("will not propagate information about {event:?}");
log::debug!("will not propagate information about {:?}", event);
}
}
Err(err) => {
-2
View File
@@ -13,8 +13,6 @@ base64 = { workspace = true }
bincode = { workspace = true }
rand = { workspace = true }
serde = { workspace = true, features = ["derive"] }
semver = { workspace = true }
strum_macros = { workspace = true }
thiserror = { workspace = true }
nym-credentials-interface = { path = "../credentials-interface" }
@@ -1,273 +0,0 @@
// Copyright 2025 Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use nym_sphinx::addressing::Recipient;
use nym_wireguard_types::PeerPublicKey;
use crate::{
AuthenticatorVersion, Error,
latest::registration::IpPair,
traits::{FinalMessage, InitMessage, QueryBandwidthMessage, TopUpMessage, Versionable},
v2, v3, v4, v5,
};
// This is very redundant with AuthenticatorRequest and I reckon they could be smooshed.
// It is a bit out of scope for me at the moment though
#[derive(Debug)]
pub enum ClientMessage {
Initial(Box<dyn InitMessage + Send + Sync + 'static>),
Final(Box<dyn FinalMessage + Send + Sync + 'static>),
Query(Box<dyn QueryBandwidthMessage + Send + Sync + 'static>),
TopUp(Box<dyn TopUpMessage + Send + Sync + 'static>),
}
impl ClientMessage {
// check if message is wasteful e.g. contains a credential
pub fn is_wasteful(&self) -> bool {
match self {
Self::Final(msg) => msg.credential().is_some(),
Self::TopUp(_) => true,
Self::Initial(_) | Self::Query(_) => false,
}
}
fn version(&self) -> AuthenticatorVersion {
match self {
ClientMessage::Initial(msg) => msg.version(),
ClientMessage::Final(msg) => msg.version(),
ClientMessage::Query(msg) => msg.version(),
ClientMessage::TopUp(msg) => msg.version(),
}
}
pub fn bytes(&self, reply_to: Recipient) -> Result<(Vec<u8>, u64), Error> {
match self.version() {
AuthenticatorVersion::V1 => Err(Error::UnsupportedVersion),
AuthenticatorVersion::V2 => {
use v2::{
registration::{ClientMac, FinalMessage, GatewayClient, InitMessage},
request::AuthenticatorRequest,
};
match self {
ClientMessage::Initial(init_message) => {
let (req, id) = AuthenticatorRequest::new_initial_request(
InitMessage {
pub_key: init_message.pub_key(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Final(final_message) => {
let (req, id) = AuthenticatorRequest::new_final_request(
FinalMessage {
gateway_client: GatewayClient {
pub_key: final_message.gateway_client_pub_key(),
private_ip: final_message
.gateway_client_ipv4()
.ok_or(Error::UnsupportedMessage)?
.into(),
mac: ClientMac::new(final_message.gateway_client_mac()),
},
credential: final_message.credential(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Query(query_message) => {
let (req, id) = AuthenticatorRequest::new_query_request(
query_message.pub_key(),
reply_to,
);
Ok((req.to_bytes()?, id))
}
_ => Err(Error::UnsupportedMessage),
}
}
AuthenticatorVersion::V3 => {
use v3::{
registration::{ClientMac, FinalMessage, GatewayClient, InitMessage},
request::AuthenticatorRequest,
topup::TopUpMessage,
};
match self {
ClientMessage::Initial(init_message) => {
let (req, id) = AuthenticatorRequest::new_initial_request(
InitMessage {
pub_key: init_message.pub_key(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Final(final_message) => {
let (req, id) = AuthenticatorRequest::new_final_request(
FinalMessage {
gateway_client: GatewayClient {
pub_key: final_message.gateway_client_pub_key(),
private_ip: final_message
.gateway_client_ipv4()
.ok_or(Error::UnsupportedMessage)?
.into(),
mac: ClientMac::new(final_message.gateway_client_mac()),
},
credential: final_message.credential(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Query(query_message) => {
let (req, id) = AuthenticatorRequest::new_query_request(
query_message.pub_key(),
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::TopUp(top_up_message) => {
let (req, id) = AuthenticatorRequest::new_topup_request(
TopUpMessage {
pub_key: top_up_message.pub_key(),
credential: top_up_message.credential(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
}
}
AuthenticatorVersion::V4 => {
use v4::{
registration::{ClientMac, FinalMessage, GatewayClient, InitMessage},
request::AuthenticatorRequest,
topup::TopUpMessage,
};
match self {
ClientMessage::Initial(init_message) => {
let (req, id) = AuthenticatorRequest::new_initial_request(
InitMessage {
pub_key: init_message.pub_key(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Final(final_message) => {
let (req, id) = AuthenticatorRequest::new_final_request(
FinalMessage {
gateway_client: GatewayClient {
pub_key: final_message.gateway_client_pub_key(),
private_ips: IpPair {
ipv4: final_message
.gateway_client_ipv4()
.ok_or(Error::UnsupportedMessage)?,
ipv6: final_message
.gateway_client_ipv6()
.ok_or(Error::UnsupportedMessage)?,
}
.into(),
mac: ClientMac::new(final_message.gateway_client_mac()),
},
credential: final_message.credential(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::Query(query_message) => {
let (req, id) = AuthenticatorRequest::new_query_request(
query_message.pub_key(),
reply_to,
);
Ok((req.to_bytes()?, id))
}
ClientMessage::TopUp(top_up_message) => {
let (req, id) = AuthenticatorRequest::new_topup_request(
TopUpMessage {
pub_key: top_up_message.pub_key(),
credential: top_up_message.credential(),
},
reply_to,
);
Ok((req.to_bytes()?, id))
}
}
}
AuthenticatorVersion::V5 => {
use v5::{
registration::{ClientMac, FinalMessage, GatewayClient, InitMessage},
request::AuthenticatorRequest,
topup::TopUpMessage,
};
match self {
ClientMessage::Initial(init_message) => {
let (req, id) = AuthenticatorRequest::new_initial_request(InitMessage {
pub_key: init_message.pub_key(),
});
Ok((req.to_bytes()?, id))
}
ClientMessage::Final(final_message) => {
let (req, id) = AuthenticatorRequest::new_final_request(FinalMessage {
gateway_client: GatewayClient {
pub_key: final_message.gateway_client_pub_key(),
private_ips: IpPair {
ipv4: final_message
.gateway_client_ipv4()
.ok_or(Error::UnsupportedMessage)?,
ipv6: final_message
.gateway_client_ipv6()
.ok_or(Error::UnsupportedMessage)?,
},
mac: ClientMac::new(final_message.gateway_client_mac()),
},
credential: final_message.credential(),
});
Ok((req.to_bytes()?, id))
}
ClientMessage::Query(query_message) => {
let (req, id) =
AuthenticatorRequest::new_query_request(query_message.pub_key());
Ok((req.to_bytes()?, id))
}
ClientMessage::TopUp(top_up_message) => {
let (req, id) = AuthenticatorRequest::new_topup_request(TopUpMessage {
pub_key: top_up_message.pub_key(),
credential: top_up_message.credential(),
});
Ok((req.to_bytes()?, id))
}
}
}
AuthenticatorVersion::UNKNOWN => Err(Error::UnknownVersion),
}
}
pub fn use_surbs(&self) -> bool {
use AuthenticatorVersion::*;
match self.version() {
V1 | V2 | V3 | V4 => false,
V5 => true,
UNKNOWN => true,
}
}
}
// Same comment as above struct
#[derive(Debug)]
pub struct QueryMessageImpl {
pub pub_key: PeerPublicKey,
pub version: AuthenticatorVersion,
}
impl Versionable for QueryMessageImpl {
fn version(&self) -> AuthenticatorVersion {
self.version
}
}
impl QueryBandwidthMessage for QueryMessageImpl {
fn pub_key(&self) -> PeerPublicKey {
self.pub_key
}
}
+2 -13
View File
@@ -23,17 +23,6 @@ pub enum Error {
#[error("conversion: {0}")]
Conversion(String),
// TODO add version number for debugging
#[error("unknown version number")]
UnknownVersion,
// TODO add version number for debugging
#[error("unsupported request version")]
UnsupportedVersion,
#[error("gateway doesn't support this type of message")]
UnsupportedMessage,
#[error(transparent)]
Bincode(#[from] bincode::Error),
#[error("failed to serialize response packet: {source}")]
FailedToSerializeResponsePacket { source: Box<bincode::ErrorKind> },
}
+1 -6
View File
@@ -1,9 +1,6 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
pub mod client_message;
pub mod request;
pub mod response;
pub mod traits;
pub mod v1;
pub mod v2;
@@ -13,13 +10,11 @@ pub mod v5;
mod error;
mod util;
mod version;
pub use error::Error;
pub use v5 as latest;
pub use version::AuthenticatorVersion;
pub const CURRENT_VERSION: u8 = latest::VERSION;
pub const CURRENT_VERSION: u8 = 5;
fn make_bincode_serializer() -> impl bincode::Options {
use bincode::Options;
@@ -1,204 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use nym_service_provider_requests_common::{Protocol, ServiceProviderType};
use nym_sphinx::addressing::Recipient;
use crate::traits::{FinalMessage, InitMessage, QueryBandwidthMessage, TopUpMessage};
use crate::{v1, v2, v3, v4, v5};
#[derive(Debug)]
pub enum AuthenticatorRequest {
Initial {
msg: Box<dyn InitMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
Final {
msg: Box<dyn FinalMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
QueryBandwidth {
msg: Box<dyn QueryBandwidthMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
TopUpBandwidth {
msg: Box<dyn TopUpMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
}
impl From<v1::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v1::request::AuthenticatorRequest) -> Self {
match value.data {
v1::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v1::request::AuthenticatorRequestData::Final(gateway_client) => Self::Final {
msg: Box::new(gateway_client),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v1::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl From<v2::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v2::request::AuthenticatorRequest) -> Self {
match value.data {
v2::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v2::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v2::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl From<v3::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v3::request::AuthenticatorRequest) -> Self {
match value.data {
v3::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v3::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v3::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
v3::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl From<v4::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v4::request::AuthenticatorRequest) -> Self {
match value.data {
v4::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v4::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v4::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
v4::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl From<v5::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v5::request::AuthenticatorRequest) -> Self {
match value.data {
v5::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
},
v5::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
},
v5::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
}
}
v5::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
}
}
}
}
}
@@ -1,106 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::traits::{
Id, PendingRegistrationResponse, RegisteredResponse, RemainingBandwidthResponse,
TopUpBandwidthResponse,
};
use crate::{v2, v3, v4, v5};
#[derive(Debug)]
pub enum AuthenticatorResponse {
PendingRegistration(Box<dyn PendingRegistrationResponse + Send + Sync + 'static>),
Registered(Box<dyn RegisteredResponse + Send + Sync + 'static>),
RemainingBandwidth(Box<dyn RemainingBandwidthResponse + Send + Sync + 'static>),
TopUpBandwidth(Box<dyn TopUpBandwidthResponse + Send + Sync + 'static>),
}
impl Id for AuthenticatorResponse {
fn id(&self) -> u64 {
match self {
AuthenticatorResponse::PendingRegistration(pending_registration_response) => {
pending_registration_response.id()
}
AuthenticatorResponse::Registered(registered_response) => registered_response.id(),
AuthenticatorResponse::RemainingBandwidth(remaining_bandwidth_response) => {
remaining_bandwidth_response.id()
}
AuthenticatorResponse::TopUpBandwidth(top_up_bandwidth_response) => {
top_up_bandwidth_response.id()
}
}
}
}
impl From<v2::response::AuthenticatorResponse> for AuthenticatorResponse {
fn from(value: v2::response::AuthenticatorResponse) -> Self {
match value.data {
v2::response::AuthenticatorResponseData::PendingRegistration(
pending_registration_response,
) => Self::PendingRegistration(Box::new(pending_registration_response)),
v2::response::AuthenticatorResponseData::Registered(registered_response) => {
Self::Registered(Box::new(registered_response))
}
v2::response::AuthenticatorResponseData::RemainingBandwidth(
remaining_bandwidth_response,
) => Self::RemainingBandwidth(Box::new(remaining_bandwidth_response)),
}
}
}
impl From<v3::response::AuthenticatorResponse> for AuthenticatorResponse {
fn from(value: v3::response::AuthenticatorResponse) -> Self {
match value.data {
v3::response::AuthenticatorResponseData::PendingRegistration(
pending_registration_response,
) => Self::PendingRegistration(Box::new(pending_registration_response)),
v3::response::AuthenticatorResponseData::Registered(registered_response) => {
Self::Registered(Box::new(registered_response))
}
v3::response::AuthenticatorResponseData::RemainingBandwidth(
remaining_bandwidth_response,
) => Self::RemainingBandwidth(Box::new(remaining_bandwidth_response)),
v3::response::AuthenticatorResponseData::TopUpBandwidth(top_up_bandwidth_response) => {
Self::TopUpBandwidth(Box::new(top_up_bandwidth_response))
}
}
}
}
impl From<v4::response::AuthenticatorResponse> for AuthenticatorResponse {
fn from(value: v4::response::AuthenticatorResponse) -> Self {
match value.data {
v4::response::AuthenticatorResponseData::PendingRegistration(
pending_registration_response,
) => Self::PendingRegistration(Box::new(pending_registration_response)),
v4::response::AuthenticatorResponseData::Registered(registered_response) => {
Self::Registered(Box::new(registered_response))
}
v4::response::AuthenticatorResponseData::RemainingBandwidth(
remaining_bandwidth_response,
) => Self::RemainingBandwidth(Box::new(remaining_bandwidth_response)),
v4::response::AuthenticatorResponseData::TopUpBandwidth(top_up_bandwidth_response) => {
Self::TopUpBandwidth(Box::new(top_up_bandwidth_response))
}
}
}
}
impl From<v5::response::AuthenticatorResponse> for AuthenticatorResponse {
fn from(value: v5::response::AuthenticatorResponse) -> Self {
match value.data {
v5::response::AuthenticatorResponseData::PendingRegistration(
pending_registration_response,
) => Self::PendingRegistration(Box::new(pending_registration_response)),
v5::response::AuthenticatorResponseData::Registered(registered_response) => {
Self::Registered(Box::new(registered_response))
}
v5::response::AuthenticatorResponseData::RemainingBandwidth(
remaining_bandwidth_response,
) => Self::RemainingBandwidth(Box::new(remaining_bandwidth_response)),
v5::response::AuthenticatorResponseData::TopUpBandwidth(top_up_bandwidth_response) => {
Self::TopUpBandwidth(Box::new(top_up_bandwidth_response))
}
}
}
}
+220 -437
View File
@@ -1,105 +1,49 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use std::fmt;
use std::net::{Ipv4Addr, Ipv6Addr};
use nym_credentials_interface::CredentialSpendingData;
use nym_crypto::asymmetric::x25519::PrivateKey;
use nym_service_provider_requests_common::{Protocol, ServiceProviderType};
use nym_sphinx::addressing::clients::Recipient;
use nym_wireguard_types::PeerPublicKey;
use crate::latest::registration::IpPair;
use crate::{AuthenticatorVersion, Error, v1, v2, v3, v4, v5};
use crate::{
v1, v2, v3, v4,
v5::{self, registration::IpPair},
Error,
};
pub trait Versionable {
fn version(&self) -> AuthenticatorVersion;
#[derive(Copy, Clone, Debug)]
pub enum AuthenticatorVersion {
V1,
V2,
V3,
V4,
V5,
UNKNOWN,
}
impl Versionable for v1::GatewayClient {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V1
impl From<Protocol> for AuthenticatorVersion {
fn from(value: Protocol) -> Self {
if value.service_provider_type != ServiceProviderType::Authenticator {
AuthenticatorVersion::UNKNOWN
} else if value.version == v1::VERSION {
AuthenticatorVersion::V1
} else if value.version == v2::VERSION {
AuthenticatorVersion::V2
} else if value.version == v3::VERSION {
AuthenticatorVersion::V3
} else if value.version == v4::VERSION {
AuthenticatorVersion::V4
} else if value.version == v5::VERSION {
AuthenticatorVersion::V5
} else {
AuthenticatorVersion::UNKNOWN
}
}
}
impl Versionable for v1::registration::InitMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V1
}
}
impl Versionable for v2::registration::InitMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V2
}
}
impl Versionable for v3::registration::InitMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V3
}
}
impl Versionable for v4::registration::InitMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V4
}
}
impl Versionable for v5::registration::InitMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V5
}
}
impl Versionable for v2::registration::FinalMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V2
}
}
impl Versionable for v3::registration::FinalMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V3
}
}
impl Versionable for v4::registration::FinalMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V4
}
}
impl Versionable for v5::registration::FinalMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V5
}
}
impl Versionable for PeerPublicKey {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V3
}
}
impl Versionable for v3::topup::TopUpMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V3
}
}
impl Versionable for v4::topup::TopUpMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V4
}
}
impl Versionable for v5::topup::TopUpMessage {
fn version(&self) -> AuthenticatorVersion {
AuthenticatorVersion::V5
}
}
pub trait InitMessage: Versionable + fmt::Debug {
pub trait InitMessage {
fn pub_key(&self) -> PeerPublicKey;
}
@@ -133,18 +77,15 @@ impl InitMessage for v5::registration::InitMessage {
}
}
pub trait FinalMessage: Versionable + fmt::Debug {
fn gateway_client_pub_key(&self) -> PeerPublicKey;
pub trait FinalMessage {
fn pub_key(&self) -> PeerPublicKey;
fn verify(&self, private_key: &PrivateKey, nonce: u64) -> Result<(), Error>;
fn private_ips(&self) -> IpPair;
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr>;
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr>;
fn gateway_client_mac(&self) -> Vec<u8>;
fn credential(&self) -> Option<CredentialSpendingData>;
}
impl FinalMessage for v1::GatewayClient {
fn gateway_client_pub_key(&self) -> PeerPublicKey {
fn pub_key(&self) -> PeerPublicKey {
self.pub_key
}
@@ -156,28 +97,13 @@ impl FinalMessage for v1::GatewayClient {
self.private_ip.into()
}
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr> {
match self.private_ip {
std::net::IpAddr::V4(ipv4_addr) => Some(ipv4_addr),
std::net::IpAddr::V6(_) => None,
}
}
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr> {
None
}
fn gateway_client_mac(&self) -> Vec<u8> {
self.mac.to_vec()
}
fn credential(&self) -> Option<CredentialSpendingData> {
None
}
}
impl FinalMessage for v2::registration::FinalMessage {
fn gateway_client_pub_key(&self) -> PeerPublicKey {
fn pub_key(&self) -> PeerPublicKey {
self.gateway_client.pub_key
}
@@ -189,28 +115,13 @@ impl FinalMessage for v2::registration::FinalMessage {
self.gateway_client.private_ip.into()
}
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr> {
match self.gateway_client.private_ip {
std::net::IpAddr::V4(ipv4_addr) => Some(ipv4_addr),
std::net::IpAddr::V6(_) => None,
}
}
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr> {
None
}
fn gateway_client_mac(&self) -> Vec<u8> {
self.gateway_client.mac.to_vec()
}
fn credential(&self) -> Option<CredentialSpendingData> {
self.credential.clone()
}
}
impl FinalMessage for v3::registration::FinalMessage {
fn gateway_client_pub_key(&self) -> PeerPublicKey {
fn pub_key(&self) -> PeerPublicKey {
self.gateway_client.pub_key
}
@@ -222,28 +133,13 @@ impl FinalMessage for v3::registration::FinalMessage {
self.gateway_client.private_ip.into()
}
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr> {
match self.gateway_client.private_ip {
std::net::IpAddr::V4(ipv4_addr) => Some(ipv4_addr),
std::net::IpAddr::V6(_) => None,
}
}
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr> {
None
}
fn gateway_client_mac(&self) -> Vec<u8> {
self.gateway_client.mac.to_vec()
}
fn credential(&self) -> Option<CredentialSpendingData> {
self.credential.clone()
}
}
impl FinalMessage for v4::registration::FinalMessage {
fn gateway_client_pub_key(&self) -> PeerPublicKey {
fn pub_key(&self) -> PeerPublicKey {
self.gateway_client.pub_key
}
@@ -255,25 +151,13 @@ impl FinalMessage for v4::registration::FinalMessage {
self.gateway_client.private_ips.into()
}
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr> {
Some(self.gateway_client.private_ips.ipv4)
}
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr> {
Some(self.gateway_client.private_ips.ipv6)
}
fn gateway_client_mac(&self) -> Vec<u8> {
self.gateway_client.mac.to_vec()
}
fn credential(&self) -> Option<CredentialSpendingData> {
self.credential.clone()
}
}
impl FinalMessage for v5::registration::FinalMessage {
fn gateway_client_pub_key(&self) -> PeerPublicKey {
fn pub_key(&self) -> PeerPublicKey {
self.gateway_client.pub_key
}
@@ -285,24 +169,12 @@ impl FinalMessage for v5::registration::FinalMessage {
self.gateway_client.private_ips
}
fn gateway_client_ipv4(&self) -> Option<Ipv4Addr> {
Some(self.gateway_client.private_ips.ipv4)
}
fn gateway_client_ipv6(&self) -> Option<Ipv6Addr> {
Some(self.gateway_client.private_ips.ipv6)
}
fn gateway_client_mac(&self) -> Vec<u8> {
self.gateway_client.mac.to_vec()
}
fn credential(&self) -> Option<CredentialSpendingData> {
self.credential.clone()
}
}
pub trait QueryBandwidthMessage: Versionable + fmt::Debug {
pub trait QueryBandwidthMessage {
fn pub_key(&self) -> PeerPublicKey;
}
@@ -312,7 +184,7 @@ impl QueryBandwidthMessage for PeerPublicKey {
}
}
pub trait TopUpMessage: Versionable + fmt::Debug {
pub trait TopUpMessage {
fn pub_key(&self) -> PeerPublicKey;
fn credential(&self) -> CredentialSpendingData;
}
@@ -347,286 +219,197 @@ impl TopUpMessage for v5::topup::TopUpMessage {
}
}
pub trait Id {
fn id(&self) -> u64;
pub enum AuthenticatorRequest {
Initial {
msg: Box<dyn InitMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
Final {
msg: Box<dyn FinalMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
QueryBandwidth {
msg: Box<dyn QueryBandwidthMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
TopUpBandwidth {
msg: Box<dyn TopUpMessage + Send + Sync + 'static>,
protocol: Protocol,
reply_to: Option<Recipient>,
request_id: u64,
},
}
impl Id for v2::response::PendingRegistrationResponse {
fn id(&self) -> u64 {
self.request_id
impl From<v1::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v1::request::AuthenticatorRequest) -> Self {
match value.data {
v1::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v1::request::AuthenticatorRequestData::Final(gateway_client) => Self::Final {
msg: Box::new(gateway_client),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v1::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: Protocol {
version: value.version,
service_provider_type: ServiceProviderType::Authenticator,
},
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl Id for v3::response::PendingRegistrationResponse {
fn id(&self) -> u64 {
self.request_id
impl From<v2::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v2::request::AuthenticatorRequest) -> Self {
match value.data {
v2::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v2::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v2::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl Id for v4::response::PendingRegistrationResponse {
fn id(&self) -> u64 {
self.request_id
impl From<v3::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v3::request::AuthenticatorRequest) -> Self {
match value.data {
v3::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v3::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v3::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
v3::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl Id for v5::response::PendingRegistrationResponse {
fn id(&self) -> u64 {
self.request_id
impl From<v4::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v4::request::AuthenticatorRequest) -> Self {
match value.data {
v4::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v4::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
},
v4::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
v4::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: Some(value.reply_to),
request_id: value.request_id,
}
}
}
}
}
impl Id for v2::response::RegisteredResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v3::response::RegisteredResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v4::response::RegisteredResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v5::response::RegisteredResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v2::response::RemainingBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v3::response::RemainingBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v4::response::RemainingBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v5::response::RemainingBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v3::response::TopUpBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v4::response::TopUpBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
impl Id for v5::response::TopUpBandwidthResponse {
fn id(&self) -> u64 {
self.request_id
}
}
pub trait PendingRegistrationResponse: Id + fmt::Debug {
fn nonce(&self) -> u64;
fn verify(&self, gateway_key: &PrivateKey) -> std::result::Result<(), Error>;
fn pub_key(&self) -> PeerPublicKey;
fn private_ips(&self) -> IpPair;
}
impl PendingRegistrationResponse for v2::response::PendingRegistrationResponse {
fn nonce(&self) -> u64 {
self.reply.nonce
}
fn verify(&self, gateway_key: &PrivateKey) -> std::result::Result<(), Error> {
self.reply.gateway_data.verify(gateway_key, self.nonce())
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.gateway_data.pub_key
}
fn private_ips(&self) -> IpPair {
self.reply.gateway_data.private_ip.into()
}
}
impl PendingRegistrationResponse for v3::response::PendingRegistrationResponse {
fn nonce(&self) -> u64 {
self.reply.nonce
}
fn verify(&self, gateway_key: &PrivateKey) -> std::result::Result<(), Error> {
self.reply.gateway_data.verify(gateway_key, self.nonce())
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.gateway_data.pub_key
}
fn private_ips(&self) -> IpPair {
self.reply.gateway_data.private_ip.into()
}
}
impl PendingRegistrationResponse for v4::response::PendingRegistrationResponse {
fn nonce(&self) -> u64 {
self.reply.nonce
}
fn verify(&self, gateway_key: &PrivateKey) -> std::result::Result<(), Error> {
self.reply.gateway_data.verify(gateway_key, self.nonce())
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.gateway_data.pub_key
}
fn private_ips(&self) -> IpPair {
self.reply.gateway_data.private_ips.into()
}
}
impl PendingRegistrationResponse for v5::response::PendingRegistrationResponse {
fn nonce(&self) -> u64 {
self.reply.nonce
}
fn verify(&self, gateway_key: &PrivateKey) -> std::result::Result<(), Error> {
self.reply.gateway_data.verify(gateway_key, self.nonce())
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.gateway_data.pub_key
}
fn private_ips(&self) -> IpPair {
self.reply.gateway_data.private_ips
}
}
pub trait RegisteredResponse: Id + fmt::Debug {
fn private_ips(&self) -> IpPair;
fn pub_key(&self) -> PeerPublicKey;
fn wg_port(&self) -> u16;
}
impl RegisteredResponse for v2::response::RegisteredResponse {
fn private_ips(&self) -> IpPair {
self.reply.private_ip.into()
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.pub_key
}
fn wg_port(&self) -> u16 {
self.reply.wg_port
}
}
impl RegisteredResponse for v3::response::RegisteredResponse {
fn private_ips(&self) -> IpPair {
self.reply.private_ip.into()
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.pub_key
}
fn wg_port(&self) -> u16 {
self.reply.wg_port
}
}
impl RegisteredResponse for v4::response::RegisteredResponse {
fn private_ips(&self) -> IpPair {
self.reply.private_ips.into()
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.pub_key
}
fn wg_port(&self) -> u16 {
self.reply.wg_port
}
}
impl RegisteredResponse for v5::response::RegisteredResponse {
fn private_ips(&self) -> IpPair {
self.reply.private_ips
}
fn pub_key(&self) -> PeerPublicKey {
self.reply.pub_key
}
fn wg_port(&self) -> u16 {
self.reply.wg_port
}
}
pub trait RemainingBandwidthResponse: Id + fmt::Debug {
fn available_bandwidth(&self) -> Option<i64>;
}
impl RemainingBandwidthResponse for v2::response::RemainingBandwidthResponse {
fn available_bandwidth(&self) -> Option<i64> {
self.reply.as_ref().map(|r| r.available_bandwidth)
}
}
impl RemainingBandwidthResponse for v3::response::RemainingBandwidthResponse {
fn available_bandwidth(&self) -> Option<i64> {
self.reply.as_ref().map(|r| r.available_bandwidth)
}
}
impl RemainingBandwidthResponse for v4::response::RemainingBandwidthResponse {
fn available_bandwidth(&self) -> Option<i64> {
self.reply.as_ref().map(|r| r.available_bandwidth)
}
}
impl RemainingBandwidthResponse for v5::response::RemainingBandwidthResponse {
fn available_bandwidth(&self) -> Option<i64> {
self.reply.as_ref().map(|r| r.available_bandwidth)
}
}
pub trait TopUpBandwidthResponse: Id + fmt::Debug {
fn available_bandwidth(&self) -> i64;
}
impl TopUpBandwidthResponse for v3::response::TopUpBandwidthResponse {
fn available_bandwidth(&self) -> i64 {
self.reply.available_bandwidth
}
}
impl TopUpBandwidthResponse for v4::response::TopUpBandwidthResponse {
fn available_bandwidth(&self) -> i64 {
self.reply.available_bandwidth
}
}
impl TopUpBandwidthResponse for v5::response::TopUpBandwidthResponse {
fn available_bandwidth(&self) -> i64 {
self.reply.available_bandwidth
impl From<v5::request::AuthenticatorRequest> for AuthenticatorRequest {
fn from(value: v5::request::AuthenticatorRequest) -> Self {
match value.data {
v5::request::AuthenticatorRequestData::Initial(init_message) => Self::Initial {
msg: Box::new(init_message),
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
},
v5::request::AuthenticatorRequestData::Final(final_message) => Self::Final {
msg: final_message,
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
},
v5::request::AuthenticatorRequestData::QueryBandwidth(peer_public_key) => {
Self::QueryBandwidth {
msg: Box::new(peer_public_key),
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
}
}
v5::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
Self::TopUpBandwidth {
msg: top_up_message,
protocol: value.protocol,
reply_to: None,
request_id: value.request_id,
}
}
}
}
}
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use crate::error::Error;
use base64::{Engine, engine::general_purpose};
use base64::{engine::general_purpose, Engine};
use nym_wireguard_types::PeerPublicKey;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
@@ -108,7 +108,7 @@ impl GatewayClient {
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key
let dh = gateway_key.inner().diffie_hellman(&self.pub_key);
let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use crate::error::Error;
use base64::{Engine, engine::general_purpose};
use base64::{engine::general_purpose, Engine};
use nym_credentials_interface::CredentialSpendingData;
use nym_wireguard_types::PeerPublicKey;
use serde::{Deserialize, Serialize};
@@ -117,7 +117,7 @@ impl GatewayClient {
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key
let dh = gateway_key.inner().diffie_hellman(&self.pub_key);
let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use crate::error::Error;
use base64::{Engine, engine::general_purpose};
use base64::{engine::general_purpose, Engine};
use nym_credentials_interface::CredentialSpendingData;
use nym_wireguard_types::PeerPublicKey;
use serde::{Deserialize, Serialize};
@@ -117,7 +117,7 @@ impl GatewayClient {
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key
let dh = gateway_key.inner().diffie_hellman(&self.pub_key);
let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use crate::error::Error;
use base64::{Engine, engine::general_purpose};
use base64::{engine::general_purpose, Engine};
use nym_credentials_interface::CredentialSpendingData;
use nym_network_defaults::constants::{WG_TUN_DEVICE_IP_ADDRESS_V4, WG_TUN_DEVICE_IP_ADDRESS_V6};
use nym_wireguard_types::PeerPublicKey;
@@ -169,7 +169,7 @@ impl GatewayClient {
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key
let dh = gateway_key.inner().diffie_hellman(&self.pub_key);
let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use crate::error::Error;
use base64::{Engine, engine::general_purpose};
use base64::{engine::general_purpose, Engine};
use nym_credentials_interface::CredentialSpendingData;
use nym_network_defaults::constants::{WG_TUN_DEVICE_IP_ADDRESS_V4, WG_TUN_DEVICE_IP_ADDRESS_V6};
use nym_wireguard_types::PeerPublicKey;
@@ -28,6 +28,8 @@ pub type HmacSha256 = Hmac<Sha256>;
pub type Nonce = u64;
pub type Taken = Option<SystemTime>;
pub const BANDWIDTH_CAP_PER_DAY: u64 = 250 * 1024 * 1024 * 1024; // 250 GB
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct IpPair {
pub ipv4: Ipv4Addr,
@@ -167,7 +169,7 @@ impl GatewayClient {
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key
let dh = gateway_key.inner().diffie_hellman(&self.pub_key);
let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
@@ -1,195 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use super::{v1, v2, v3, v4, v5};
use nym_service_provider_requests_common::{Protocol, ServiceProviderType};
#[derive(Copy, Clone, Debug, PartialEq, strum_macros::Display)]
#[strum(serialize_all = "snake_case")]
pub enum AuthenticatorVersion {
/// introduced in wispa release (1.1.5)
V1,
/// introduced in aero release (1.1.9)
V2,
/// introduced in magura release (1.1.10)
V3,
/// introduced in crunch release (1.2.0)
V4,
/// introduced in dorina-patched release (1.6.1)
V5,
UNKNOWN,
}
impl AuthenticatorVersion {
pub const LATEST: Self = Self::V5;
pub const fn release_version(&self) -> semver::Version {
match self {
AuthenticatorVersion::V1 => semver::Version::new(1, 1, 5),
AuthenticatorVersion::V2 => semver::Version::new(1, 1, 9),
AuthenticatorVersion::V3 => semver::Version::new(1, 1, 10),
AuthenticatorVersion::V4 => semver::Version::new(1, 2, 0),
AuthenticatorVersion::V5 => semver::Version::new(1, 6, 1),
AuthenticatorVersion::UNKNOWN => semver::Version::new(0, 0, 0),
}
}
}
impl From<Protocol> for AuthenticatorVersion {
fn from(value: Protocol) -> Self {
if value.service_provider_type != ServiceProviderType::Authenticator {
AuthenticatorVersion::UNKNOWN
} else if value.version == v1::VERSION {
AuthenticatorVersion::V1
} else if value.version == v2::VERSION {
AuthenticatorVersion::V2
} else if value.version == v3::VERSION {
AuthenticatorVersion::V3
} else if value.version == v4::VERSION {
AuthenticatorVersion::V4
} else if value.version == v5::VERSION {
AuthenticatorVersion::V5
} else {
AuthenticatorVersion::UNKNOWN
}
}
}
impl From<u8> for AuthenticatorVersion {
fn from(value: u8) -> Self {
if value == v1::VERSION {
AuthenticatorVersion::V1
} else if value == v2::VERSION {
AuthenticatorVersion::V2
} else if value == v3::VERSION {
AuthenticatorVersion::V3
} else if value == v4::VERSION {
AuthenticatorVersion::V4
} else if value == v5::VERSION {
AuthenticatorVersion::V5
} else {
AuthenticatorVersion::UNKNOWN
}
}
}
impl From<&str> for AuthenticatorVersion {
fn from(value: &str) -> Self {
let Ok(semver) = semver::Version::parse(value) else {
return Self::UNKNOWN;
};
semver.into()
}
}
impl From<Option<&String>> for AuthenticatorVersion {
fn from(value: Option<&String>) -> Self {
match value {
None => Self::UNKNOWN,
Some(value) => value.as_str().into(),
}
}
}
impl From<String> for AuthenticatorVersion {
fn from(value: String) -> Self {
Self::from(value.as_str())
}
}
impl From<Option<String>> for AuthenticatorVersion {
fn from(value: Option<String>) -> Self {
value.as_ref().into()
}
}
impl From<semver::Version> for AuthenticatorVersion {
fn from(semver: semver::Version) -> Self {
if semver < AuthenticatorVersion::V1.release_version() {
return Self::UNKNOWN;
}
if semver < AuthenticatorVersion::V2.release_version() {
return Self::V1;
}
if semver < AuthenticatorVersion::V3.release_version() {
return Self::V2;
}
if semver < AuthenticatorVersion::V4.release_version() {
return Self::V3;
}
if semver < AuthenticatorVersion::V5.release_version() {
return Self::V4;
}
// if provided version is higher (or equal) to release version of V5,
// we return the latest (i.e. v5)
debug_assert_eq!(
Self::V5,
Self::LATEST,
"a new AuthenticatorVersion variant has been introduced without adjusting the `From<semver::Version>` trait"
);
Self::LATEST
}
}
#[cfg(test)]
mod tests {
use super::super::latest;
use super::*;
#[test]
fn strum_display() {
// sanity check on formatting and casing
assert_eq!("v1", AuthenticatorVersion::V1.to_string());
assert_eq!("v2", AuthenticatorVersion::V2.to_string());
assert_eq!("unknown", AuthenticatorVersion::UNKNOWN.to_string());
}
#[test]
fn u8_conversion() {
assert_eq!(AuthenticatorVersion::V1, AuthenticatorVersion::from(1u8));
assert_eq!(AuthenticatorVersion::V2, AuthenticatorVersion::from(2u8));
assert_eq!(
AuthenticatorVersion::UNKNOWN,
AuthenticatorVersion::from(latest::VERSION + 1)
);
assert_eq!(
AuthenticatorVersion::UNKNOWN,
AuthenticatorVersion::from(0u8)
);
assert_eq!(
AuthenticatorVersion::UNKNOWN,
AuthenticatorVersion::from(255u8)
);
}
#[test]
fn semver_checks() {
assert_eq!(AuthenticatorVersion::UNKNOWN, "1.1.4".into());
assert_eq!(AuthenticatorVersion::UNKNOWN, "0.1.0".into());
assert_eq!(AuthenticatorVersion::UNKNOWN, "1.0.4".into());
assert_eq!(AuthenticatorVersion::V1, "1.1.5".into());
assert_eq!(AuthenticatorVersion::V1, "1.1.6".into());
assert_eq!(AuthenticatorVersion::V1, "1.1.8".into());
assert_eq!(AuthenticatorVersion::V2, "1.1.9".into());
assert_eq!(AuthenticatorVersion::V3, "1.1.10".into());
assert_eq!(AuthenticatorVersion::V3, "1.1.11".into());
assert_eq!(AuthenticatorVersion::V3, "1.1.60".into());
assert_eq!(AuthenticatorVersion::V4, "1.2.0".into());
assert_eq!(AuthenticatorVersion::V4, "1.2.1".into());
assert_eq!(AuthenticatorVersion::V4, "1.5.1".into());
assert_eq!(AuthenticatorVersion::V4, "1.6.0".into());
assert_eq!(AuthenticatorVersion::V5, "1.6.1".into());
assert_eq!(AuthenticatorVersion::V5, "1.6.11".into());
assert_eq!(AuthenticatorVersion::V5, "1.7.0".into());
assert_eq!(AuthenticatorVersion::V5, "1.16.11".into());
assert_eq!(AuthenticatorVersion::V5, "1.17.0".into());
}
}
-1
View File
@@ -7,7 +7,6 @@ license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
async-trait = { workspace = true }
bip39 = { workspace = true }
log = { workspace = true }
rand = { workspace = true }
+1 -1
View File
@@ -11,7 +11,7 @@ impl std::fmt::Display for BandwidthStatusMessage {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
BandwidthStatusMessage::RemainingBandwidth(b) => {
write!(f, "remaining bandwidth: {b}")
write!(f, "remaining bandwidth: {}", b)
}
BandwidthStatusMessage::NoBandwidth => write!(f, "no bandwidth left"),
}
-2
View File
@@ -23,12 +23,10 @@ use nym_validator_client::nym_api::EpochId;
use nym_validator_client::nyxd::contract_traits::DkgQueryClient;
pub use event::BandwidthStatusMessage;
pub use traits::{BandwidthTicketProvider, DEFAULT_TICKETS_TO_SPEND};
pub mod acquire;
pub mod error;
mod event;
mod traits;
mod utils;
#[derive(Debug)]
-42
View File
@@ -1,42 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use async_trait::async_trait;
use nym_credential_storage::storage::Storage;
use nym_credentials_interface::TicketType;
use nym_crypto::asymmetric::ed25519;
use nym_validator_client::nyxd::contract_traits::DkgQueryClient;
use crate::{error::BandwidthControllerError, BandwidthController, PreparedCredential};
pub const DEFAULT_TICKETS_TO_SPEND: u32 = 1;
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait BandwidthTicketProvider: Send + Sync {
async fn get_ecash_ticket(
&self,
ticket_type: TicketType,
gateway_id: ed25519::PublicKey,
tickets_to_spend: u32,
) -> Result<PreparedCredential, BandwidthControllerError>;
}
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
impl<C, St> BandwidthTicketProvider for BandwidthController<C, St>
where
C: DkgQueryClient + Sync + Send,
St: nym_credential_storage::storage::Storage,
<St as Storage>::StorageError: Send + Sync + 'static,
{
async fn get_ecash_ticket(
&self,
ticket_type: TicketType,
gateway_id: ed25519::PublicKey,
tickets_to_spend: u32,
) -> Result<PreparedCredential, BandwidthControllerError> {
self.prepare_ecash_ticket(ticket_type, gateway_id.to_bytes(), tickets_to_spend)
.await
}
}
+3 -3
View File
@@ -13,7 +13,7 @@ use nym_credentials_interface::{
};
use nym_ecash_time::Date;
use nym_validator_client::coconut::all_ecash_api_clients;
use nym_validator_client::nym_api::{EpochId, NymApiClientExt};
use nym_validator_client::nym_api::EpochId;
use nym_validator_client::nyxd::contract_traits::DkgQueryClient;
use nym_validator_client::EcashApiClient;
use rand::prelude::SliceRandom;
@@ -207,7 +207,7 @@ where
<St as Storage>::StorageError: Send + Sync + 'static,
{
if let Some(stored) = storage
.get_expiration_date_signatures(expiration_date, epoch_id)
.get_expiration_date_signatures(expiration_date)
.await
.map_err(BandwidthControllerError::credential_storage_error)?
{
@@ -220,7 +220,7 @@ where
ecash_apis,
|api| async move {
api.api_client
.global_expiration_date_signatures(Some(expiration_date), Some(epoch_id))
.global_expiration_date_signatures(Some(expiration_date))
.await
},
format!("aggregated coin index signatures for date {expiration_date}"),
+2 -2
View File
@@ -1,8 +1,8 @@
use clap::Args;
use clap::builder::Command;
use clap::clap_derive::ValueEnum;
use clap_complete::Shell as ClapShell;
use clap::Args;
use clap_complete::generator::generate;
use clap_complete::Shell as ClapShell;
use std::io;
pub fn fig_generate(command: &mut Command, name: &str) {
+10 -17
View File
@@ -3,7 +3,7 @@ name = "nym-client-core"
version = "1.1.15"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
edition = "2021"
rust-version = "1.85"
rust-version = "1.76"
license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -13,10 +13,10 @@ async-trait = { workspace = true }
base64 = { workspace = true }
bs58 = { workspace = true }
clap = { workspace = true, optional = true }
cfg-if = { workspace = true }
comfy-table = { workspace = true, optional = true }
futures = { workspace = true }
humantime = { workspace = true }
humantime-serde = { workspace = true }
log = { workspace = true }
rand = { workspace = true }
rand_chacha = { workspace = true }
serde = { workspace = true, features = ["derive"] }
@@ -25,23 +25,26 @@ sha2 = { workspace = true }
si-scale = { workspace = true }
thiserror = { workspace = true }
url = { workspace = true, features = ["serde"] }
tokio = { workspace = true, features = ["macros"] }
time = { workspace = true }
tokio = { workspace = true, features = ["sync", "macros"] }
tracing = { workspace = true }
zeroize = { workspace = true }
# internal
nym-id = { path = "../nym-id" }
nym-bandwidth-controller = { path = "../bandwidth-controller" }
nym-config = { path = "../config" }
nym-crypto = { path = "../crypto" }
nym-gateway-client = { path = "../client-libs/gateway-client" }
nym-gateway-requests = { path = "../gateway-requests" }
nym-http-api-client = { path = "../http-api-client", features = ["network-defaults"] }
nym-http-api-client = { path = "../http-api-client" }
nym-metrics = { path = "../nym-metrics" }
nym-nonexhaustive-delayqueue = { path = "../nonexhaustive-delayqueue" }
nym-sphinx = { path = "../nymsphinx" }
nym-statistics-common = { path = "../statistics" }
nym-pemstore = { path = "../pemstore" }
nym-topology = { path = "../topology", features = ["persistence"] }
nym-mixnet-client = { path = "../client-libs/mixnet-client", default-features = false }
nym-validator-client = { path = "../client-libs/validator-client", default-features = false }
nym-task = { path = "../task" }
nym-credentials-interface = { path = "../credentials-interface" }
@@ -53,10 +56,6 @@ nym-client-core-config-types = { path = "./config-types", features = [
nym-client-core-surb-storage = { path = "./surb-storage" }
nym-client-core-gateways-storage = { path = "./gateways-storage" }
nym-ecash-time = { path = "../ecash-time" }
nym-mixnet-contract-common = { path = "../cosmwasm-smart-contracts/mixnet-contract" }
[target."cfg(not(target_arch = \"wasm32\"))".dependencies]
nym-mixnet-client = { path = "../client-libs/mixnet-client", default-features = false }
### For serving prometheus metrics
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.hyper]
@@ -69,6 +68,7 @@ workspace = true
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.hyper-util]
workspace = true
features = ["tokio"]
###
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio-stream]
workspace = true
@@ -102,7 +102,7 @@ workspace = true
features = ["tokio"]
[target."cfg(target_arch = \"wasm32\")".dependencies.gloo-timers]
workspace = true
version = "0.3.0"
features = ["futures"]
[target."cfg(target_arch = \"wasm32\")".dependencies.wasm-utils]
@@ -113,10 +113,6 @@ features = ["websocket"]
workspace = true
features = ["wasm-bindgen"]
[target."cfg(target_arch = \"wasm32\")".dependencies.tokio_with_wasm]
workspace = true
features = ["full"]
[dev-dependencies]
tempfile = { workspace = true }
@@ -128,6 +124,3 @@ fs-surb-storage = ["nym-client-core-surb-storage/fs-surb-storage"]
fs-gateways-storage = ["nym-client-core-gateways-storage/fs-gateways-storage"]
wasm = ["nym-gateway-client/wasm"]
metrics-server = []
[lints]
workspace = true
+15 -8
View File
@@ -57,7 +57,9 @@ const DEFAULT_MAXIMUM_ALLOWED_SURB_REQUEST_SIZE: u32 = 500;
const DEFAULT_MAXIMUM_REPLY_SURB_REREQUEST_WAITING_PERIOD: Duration = Duration::from_secs(10);
const DEFAULT_MAXIMUM_REPLY_SURB_DROP_WAITING_PERIOD: Duration = Duration::from_secs(5 * 60);
const DEFAULT_MAXIMUM_REPLY_SURB_REREQUESTS: usize = 5;
// 12 hours
const DEFAULT_MAXIMUM_REPLY_SURB_AGE: Duration = Duration::from_secs(12 * 60 * 60);
// 24 hours
const DEFAULT_MAXIMUM_REPLY_KEY_AGE: Duration = Duration::from_secs(24 * 60 * 60);
@@ -416,9 +418,6 @@ pub struct Traffic {
/// will be routed as usual, to the entry gateway, through three mix nodes, egressing
/// through the exit gateway. If mix hops are disabled, traffic will be routed directly
/// from the entry gateway to the exit gateway, bypassing the mix nodes.
///
/// This overrides the `use_legacy_sphinx_format` setting as reduced mix hops
/// requires use of the updated SURB packet format.
pub disable_mix_hops: bool,
}
@@ -626,9 +625,10 @@ pub struct ReplySurbs {
#[serde(with = "humantime_serde")]
pub maximum_reply_surb_drop_waiting_period: Duration,
/// Defines maximum number of times the client is going to re-request reply surbs
/// for clearing pending messages before giving up after making no progress.
pub maximum_reply_surbs_rerequests: usize,
/// Defines maximum amount of time given reply surb is going to be valid for.
/// This is going to be superseded by key rotation once implemented.
#[serde(with = "humantime_serde")]
pub maximum_reply_surb_age: Duration,
/// Defines maximum amount of time given reply key is going to be valid for.
/// This is going to be superseded by key rotation once implemented.
@@ -638,6 +638,9 @@ pub struct ReplySurbs {
/// Specifies the number of mixnet hops the packet should go through. If not specified, then
/// the default value is used.
pub surb_mix_hops: Option<u8>,
/// Specifies if we should reset all the sender tags on startup
pub fresh_sender_tags: bool,
}
impl Default for ReplySurbs {
@@ -652,9 +655,10 @@ impl Default for ReplySurbs {
maximum_reply_surb_rerequest_waiting_period:
DEFAULT_MAXIMUM_REPLY_SURB_REREQUEST_WAITING_PERIOD,
maximum_reply_surb_drop_waiting_period: DEFAULT_MAXIMUM_REPLY_SURB_DROP_WAITING_PERIOD,
maximum_reply_surbs_rerequests: DEFAULT_MAXIMUM_REPLY_SURB_REREQUESTS,
maximum_reply_surb_age: DEFAULT_MAXIMUM_REPLY_SURB_AGE,
maximum_reply_key_age: DEFAULT_MAXIMUM_REPLY_KEY_AGE,
surb_mix_hops: None,
fresh_sender_tags: false,
}
}
}
@@ -707,10 +711,13 @@ pub struct DebugConfig {
/// Defines all configuration options related to reply SURBs.
pub reply_surbs: ReplySurbs,
/// Defines all configuration options related to stats reporting.
pub stats_reporting: StatsReporting,
/// Defines all configuration options related to the forget me flag.
pub forget_me: ForgetMe,
/// Defines all configuration options related to the remember me flag.
pub remember_me: RememberMe,
}
@@ -189,13 +189,14 @@ impl From<ConfigV6> for Config {
.debug
.reply_surbs
.maximum_reply_surb_drop_waiting_period,
maximum_reply_surb_age: value.debug.reply_surbs.maximum_reply_surb_age,
maximum_reply_key_age: value.debug.reply_surbs.maximum_reply_key_age,
surb_mix_hops: value.debug.reply_surbs.surb_mix_hops,
minimum_reply_surb_threshold_buffer: value
.debug
.reply_surbs
.minimum_reply_surb_threshold_buffer,
..Default::default()
fresh_sender_tags: value.debug.reply_surbs.fresh_sender_tags,
},
stats_reporting: StatsReporting {
enabled: value.debug.stats_reporting.enabled,
@@ -543,8 +544,10 @@ pub struct DebugConfigV6 {
/// Defines all configuration options related to reply SURBs.
pub reply_surbs: ReplySurbsV6,
/// Defines all configuration options related to stats reporting.
pub stats_reporting: StatsReportingV6,
/// Defines all configuration options related to the forget me flag.
pub forget_me: ForgetMeV6,
@@ -3,18 +3,17 @@ name = "nym-client-core-gateways-storage"
version = "0.1.0"
edition = "2021"
license.workspace = true
rust-version.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
async-trait.workspace = true
cosmrs.workspace = true
log.workspace = true
serde = { workspace = true, features = ["derive"] }
thiserror.workspace = true
time.workspace = true
tokio = { workspace = true, features = ["sync"] }
tracing.workspace = true
url.workspace = true
zeroize = { workspace = true, features = ["zeroize_derive"] }
@@ -27,7 +26,6 @@ features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate", "time"]
optional = true
[build-dependencies]
anyhow = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
sqlx = { workspace = true, features = [
"runtime-tokio-rustls",
+4 -13
View File
@@ -2,30 +2,23 @@
// SPDX-License-Identifier: Apache-2.0
#[tokio::main]
async fn main() -> anyhow::Result<()> {
async fn main() {
#[cfg(feature = "fs-gateways-storage")]
{
use anyhow::Context;
use sqlx::{Connection, SqliteConnection};
use std::env;
let out_dir = env::var("OUT_DIR")?;
let out_dir = env::var("OUT_DIR").unwrap();
let database_path = format!("{out_dir}/gateways-storage-example.sqlite");
// remove the db file if it already existed from previous build
// in case it was from a different branch
if std::fs::exists(&database_path)? {
std::fs::remove_file(&database_path)?;
}
let mut conn = SqliteConnection::connect(&format!("sqlite://{database_path}?mode=rwc"))
.await
.context("Failed to create SQLx database connection")?;
.expect("Failed to create SQLx database connection");
sqlx::migrate!("./fs_gateways_migrations")
.run(&mut conn)
.await
.context("Failed to perform SQLx migrations")?;
.expect("Failed to perform SQLx migrations");
#[cfg(target_family = "unix")]
println!("cargo:rustc-env=DATABASE_URL=sqlite://{}", &database_path);
@@ -35,6 +28,4 @@ async fn main() -> anyhow::Result<()> {
// not a valid windows path... but hey, it works...
println!("cargo:rustc-env=DATABASE_URL=sqlite:///{}", &database_path);
}
Ok(())
}
@@ -2,7 +2,8 @@
// SPDX-License-Identifier: Apache-2.0
use crate::BadGateway;
use std::{io, path::PathBuf};
use std::io;
use std::path::PathBuf;
use thiserror::Error;
#[derive(Debug, Error)]
@@ -18,6 +19,7 @@ pub enum StorageError {
#[error("failed to perform sqlx migration: {source}")]
MigrationError {
#[source]
#[from]
source: sqlx::migrate::MigrateError,
},
@@ -30,6 +32,7 @@ pub enum StorageError {
#[error("failed to run the SQL query: {source}")]
QueryError {
#[source]
#[from]
source: sqlx::error::Error,
},
@@ -7,12 +7,12 @@ use crate::{
RawActiveGateway, RawCustomGatewayDetails, RawRegisteredGateway, RawRemoteGatewayDetails,
},
};
use log::{debug, error};
use sqlx::{
sqlite::{SqliteAutoVacuum, SqliteSynchronous},
ConnectOptions,
};
use std::path::Path;
use tracing::{debug, error};
#[derive(Debug, Clone)]
pub struct StorageManager {
@@ -31,6 +31,7 @@ impl StorageManager {
}
})?;
}
let opts = sqlx::sqlite::SqliteConnectOptions::new()
.journal_mode(sqlx::sqlite::SqliteJournalMode::Wal)
.synchronous(SqliteSynchronous::Normal)
@@ -86,7 +87,7 @@ impl StorageManager {
sqlx::query!("SELECT EXISTS (SELECT 1 FROM registered_gateway WHERE gateway_id_bs58 = ?) AS 'exists'", gateway_id)
.fetch_one(&self.connection_pool)
.await
.map(|result| result.exists == 1)
.map(|result| result.exists == Some(1))
}
pub(crate) async fn maybe_get_registered_gateway(
@@ -12,12 +12,12 @@ use crate::{
error::ClientCoreError,
init::types::{GatewaySelectionSpecification, GatewaySetup},
};
use log::info;
use nym_client_core_gateways_storage::GatewayDetails;
use nym_crypto::asymmetric::ed25519;
use nym_topology::NymTopology;
use nym_validator_client::UserAgent;
use std::path::PathBuf;
use tracing::info;
#[cfg_attr(feature = "cli", derive(clap::Args))]
#[derive(Debug, Clone)]
@@ -81,14 +81,14 @@ where
// Attempt to use a user-provided gateway, if possible
let user_chosen_gateway_id = common_args.gateway_id;
tracing::debug!("User chosen gateway id: {user_chosen_gateway_id:?}");
log::debug!("User chosen gateway id: {user_chosen_gateway_id:?}");
let selection_spec = GatewaySelectionSpecification::new(
user_chosen_gateway_id.map(|id| id.to_base58_string()),
Some(common_args.latency_based_selection),
common_args.force_tls_gateway,
);
tracing::debug!("Gateway selection specification: {selection_spec:?}");
log::debug!("Gateway selection specification: {selection_spec:?}");
let registered_gateways = get_all_registered_identities(&details_store).await?;
@@ -114,12 +114,13 @@ where
})?;
hardcoded_topology.entry_capable_nodes().cloned().collect()
} else {
let mut rng = rand::thread_rng();
crate::init::helpers::gateways_for_init(
&mut rng,
&core.client.nym_api_urls,
user_agent,
core.debug.topology.minimum_gateway_performance,
core.debug.topology.ignore_ingress_epoch_role,
None,
)
.await?
};
@@ -58,7 +58,6 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.signatures_path.unwrap())?
}
};
@@ -64,7 +64,6 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.credential_path.unwrap())?
}
};
@@ -58,7 +58,6 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.signatures_path.unwrap())?
}
};
@@ -58,7 +58,6 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.key_path.unwrap())?
}
};
@@ -12,6 +12,7 @@ use crate::{
},
init::types::{GatewaySelectionSpecification, GatewaySetup, InitResults},
};
use log::info;
use nym_client_core_gateways_storage::GatewayDetails;
use nym_crypto::asymmetric::ed25519;
use nym_sphinx::addressing::Recipient;
@@ -19,7 +20,6 @@ use nym_topology::NymTopology;
use nym_validator_client::UserAgent;
use rand::rngs::OsRng;
use std::path::PathBuf;
use tracing::info;
// we can suppress this warning (as suggested by linter itself) since we're only using it in our own code
#[allow(async_fn_in_trait)]
@@ -130,23 +130,23 @@ where
// Attempt to use a user-provided gateway, if possible
let user_chosen_gateway_id = common_args.gateway;
tracing::debug!("User chosen gateway id: {user_chosen_gateway_id:?}");
log::debug!("User chosen gateway id: {user_chosen_gateway_id:?}");
let selection_spec = GatewaySelectionSpecification::new(
user_chosen_gateway_id.map(|id| id.to_base58_string()),
Some(common_args.latency_based_selection),
common_args.force_tls_gateway,
);
tracing::debug!("Gateway selection specification: {selection_spec:?}");
log::debug!("Gateway selection specification: {selection_spec:?}");
// Load and potentially override config
tracing::debug!("Init arguments: {init_args:#?}");
log::debug!("Init arguments: {init_args:#?}");
let config = C::construct_config(&init_args);
tracing::debug!("Constructed config: {config:#?}");
log::debug!("Constructed config: {config:#?}");
let paths = config.common_paths();
let core = config.core_config();
tracing::info!(
log::info!(
"Using nym-api: {}",
core.client
.nym_api_urls
@@ -173,12 +173,13 @@ where
})?;
hardcoded_topology.entry_capable_nodes().cloned().collect()
} else {
let mut rng = rand::thread_rng();
crate::init::helpers::gateways_for_init(
&mut rng,
&core.client.nym_api_urls,
user_agent,
core.debug.topology.minimum_gateway_performance,
core.debug.topology.ignore_ingress_epoch_role,
None,
)
.await?
};
+97 -398
View File
@@ -7,19 +7,17 @@ use super::statistics_control::StatisticsControl;
use crate::client::base_client::storage::helpers::store_client_keys;
use crate::client::base_client::storage::MixnetClientStorage;
use crate::client::cover_traffic_stream::LoopCoverTrafficStream;
use crate::client::event_control::EventControl;
use crate::client::inbound_messages::{InputMessage, InputMessageReceiver, InputMessageSender};
use crate::client::key_manager::persistence::KeyStore;
use crate::client::key_manager::ClientKeys;
use crate::client::mix_traffic::transceiver::{GatewayReceiver, GatewayTransceiver, RemoteGateway};
use crate::client::mix_traffic::{BatchMixMessageSender, MixTrafficController, MixTrafficEvent};
use crate::client::mix_traffic::{BatchMixMessageSender, MixTrafficController};
use crate::client::real_messages_control;
use crate::client::real_messages_control::RealMessagesController;
use crate::client::received_buffer::{
ReceivedBufferRequestReceiver, ReceivedBufferRequestSender, ReceivedMessagesBufferController,
};
use crate::client::replies::reply_controller;
use crate::client::replies::reply_controller::key_rotation_helpers::KeyRotationConfig;
use crate::client::replies::reply_controller::{ReplyControllerReceiver, ReplyControllerSender};
use crate::client::replies::reply_storage::{
CombinedReplyStorage, PersistentReplyStorage, ReplyStorageBackend, SentReplyKeys,
@@ -28,14 +26,15 @@ use crate::client::topology_control::nym_api_provider::NymApiTopologyProvider;
use crate::client::topology_control::{
TopologyAccessor, TopologyRefresher, TopologyRefresherConfig,
};
use crate::config;
use crate::config::{Config, DebugConfig};
use crate::error::ClientCoreError;
use crate::init::{
setup_gateway,
types::{GatewaySetup, InitialisationResult},
};
use crate::{config, spawn_future};
use futures::channel::mpsc;
use log::*;
use nym_bandwidth_controller::BandwidthController;
use nym_client_core_config_types::{ForgetMe, RememberMe};
use nym_client_core_gateways_storage::{GatewayDetails, GatewaysDetailsStore};
@@ -49,34 +48,23 @@ use nym_gateway_client::{
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::addressing::nodes::NodeIdentity;
use nym_sphinx::params::PacketType;
use nym_sphinx::receiver::{ReconstructedMessage, SphinxMessageReceiver};
use nym_statistics_common::clients::ClientStatsSender;
use nym_statistics_common::generate_client_stats_id;
use nym_task::connections::{ConnectionCommandReceiver, ConnectionCommandSender, LaneQueueLengths};
use nym_task::ShutdownTracker;
use nym_task::{TaskClient, TaskHandle};
use nym_topology::provider_trait::TopologyProvider;
use nym_topology::HardcodedTopologyProvider;
use nym_validator_client::nym_api::NymApiClientExt;
use nym_validator_client::{nyxd::contract_traits::DkgQueryClient, UserAgent};
use rand::prelude::SliceRandom;
use rand::rngs::OsRng;
use rand::thread_rng;
use std::fmt::Debug;
use std::os::raw::c_int as RawFd;
use std::path::Path;
use std::sync::Arc;
use time::OffsetDateTime;
use tokio::sync::mpsc::Sender;
use url::Url;
#[cfg(target_arch = "wasm32")]
#[cfg(debug_assertions)]
use wasm_utils::console_log;
/// Default number of retries for Nym API requests when using network details with domain fronting.
/// This allows the client to try alternative URLs if the primary endpoint is unavailable.
const DEFAULT_NYM_API_RETRIES: usize = 3;
#[cfg(all(
not(target_arch = "wasm32"),
feature = "fs-surb-storage",
@@ -87,28 +75,10 @@ pub mod non_wasm_helpers;
pub mod helpers;
pub mod storage;
#[derive(Clone, Copy, Debug)]
pub enum MixnetClientEvent {
Traffic(MixTrafficEvent),
}
pub type EventReceiver = mpsc::UnboundedReceiver<MixnetClientEvent>;
#[derive(Clone)]
pub struct EventSender(pub mpsc::UnboundedSender<MixnetClientEvent>);
impl EventSender {
pub fn send(&self, event: MixnetClientEvent) {
if let Err(err) = self.0.unbounded_send(event) {
tracing::warn!("Failed to send error event. The caller event reader was closed: {err}");
}
}
}
#[derive(Clone)]
pub struct ClientInput {
pub connection_command_sender: ConnectionCommandSender,
pub input_sender: InputMessageSender,
pub client_request_sender: ClientRequestSender,
}
impl ClientInput {
@@ -120,6 +90,7 @@ impl ClientInput {
}
}
#[derive(Clone)]
pub struct ClientOutput {
pub received_buffer_request_sender: ReceivedBufferRequestSender,
}
@@ -159,11 +130,9 @@ pub enum ClientInputStatus {
}
impl ClientInputStatus {
#[allow(clippy::panic)]
pub fn register_producer(&mut self) -> ClientInput {
match std::mem::replace(self, ClientInputStatus::Connected) {
ClientInputStatus::AwaitingProducer { client_input } => client_input,
// critical failure implying misuse of software
ClientInputStatus::Connected => panic!("producer was already registered before"),
}
}
@@ -175,11 +144,9 @@ pub enum ClientOutputStatus {
}
impl ClientOutputStatus {
#[allow(clippy::panic)]
pub fn register_consumer(&mut self) -> ClientOutput {
match std::mem::replace(self, ClientOutputStatus::Connected) {
ClientOutputStatus::AwaitingConsumer { client_output } => client_output,
// critical failure implying misuse of software
ClientOutputStatus::Connected => panic!("consumer was already registered before"),
}
}
@@ -216,14 +183,10 @@ pub struct BaseClientBuilder<C, S: MixnetClientStorage> {
client_store: S,
dkg_query_client: Option<C>,
// Optional API URLs for domain fronting support
nym_api_urls: Option<Vec<nym_network_defaults::ApiUrl>>,
wait_for_gateway: bool,
custom_topology_provider: Option<Box<dyn TopologyProvider + Send + Sync>>,
custom_gateway_transceiver: Option<Box<dyn GatewayTransceiver + Send>>,
shutdown: Option<ShutdownTracker>,
event_tx: Option<EventSender>,
shutdown: Option<TaskClient>,
user_agent: Option<UserAgent>,
setup_method: GatewaySetup,
@@ -248,12 +211,10 @@ where
config: base_config,
client_store,
dkg_query_client,
nym_api_urls: None,
wait_for_gateway: false,
custom_topology_provider: None,
custom_gateway_transceiver: None,
shutdown: None,
event_tx: None,
user_agent: None,
setup_method: GatewaySetup::MustLoad { gateway_id: None },
#[cfg(unix)]
@@ -271,16 +232,6 @@ where
self
}
/// Set Nym API URLs for domain fronting support.
///
/// When provided, the client will use these API URLs (which include front_hosts)
/// to construct HTTP clients with domain fronting enabled.
#[must_use]
pub fn with_nym_api_urls(mut self, nym_api_urls: Vec<nym_network_defaults::ApiUrl>) -> Self {
self.nym_api_urls = Some(nym_api_urls);
self
}
#[must_use]
pub fn with_forget_me(mut self, forget_me: &ForgetMe) -> Self {
self.config.debug.forget_me = *forget_me;
@@ -321,17 +272,11 @@ where
}
#[must_use]
pub fn with_shutdown(mut self, shutdown: ShutdownTracker) -> Self {
pub fn with_shutdown(mut self, shutdown: TaskClient) -> Self {
self.shutdown = Some(shutdown);
self
}
#[must_use]
pub fn with_event_tx(mut self, event_tx: EventSender) -> Self {
self.event_tx = Some(event_tx);
self
}
#[must_use]
pub fn with_user_agent(mut self, user_agent: UserAgent) -> Self {
self.user_agent = Some(user_agent);
@@ -362,18 +307,6 @@ where
details.client_address()
}
fn start_event_control(
parent_event_tx: Option<EventSender>,
children_event_rx: EventReceiver,
shutdown_tracker: &ShutdownTracker,
) {
let event_control = EventControl::new(parent_event_tx, children_event_rx);
shutdown_tracker.try_spawn_named_with_shutdown(
async move { event_control.run().await },
"EventControl",
);
}
// future constantly pumping loop cover traffic at some specified average rate
// the pumped traffic goes to the MixTrafficController
fn start_cover_traffic_stream(
@@ -383,11 +316,11 @@ where
topology_accessor: TopologyAccessor,
mix_tx: BatchMixMessageSender,
stats_tx: ClientStatsSender,
shutdown_tracker: &ShutdownTracker,
task_client: TaskClient,
) {
tracing::info!("Starting loop cover traffic stream...");
info!("Starting loop cover traffic stream...");
let mut stream = LoopCoverTrafficStream::new(
let stream = LoopCoverTrafficStream::new(
ack_key,
debug_config.acknowledgements.average_ack_delay,
mix_tx,
@@ -396,15 +329,15 @@ where
debug_config.traffic,
debug_config.cover_traffic,
stats_tx,
task_client,
);
shutdown_tracker
.try_spawn_named_with_shutdown(async move { stream.run().await }, "CoverTrafficStream");
stream.start();
}
#[allow(clippy::too_many_arguments)]
fn start_real_traffic_controller(
controller_config: real_messages_control::Config,
key_rotation_config: KeyRotationConfig,
topology_accessor: TopologyAccessor,
ack_receiver: AcknowledgementReceiver,
input_receiver: InputMessageReceiver,
@@ -414,14 +347,14 @@ where
reply_controller_receiver: ReplyControllerReceiver,
lane_queue_lengths: LaneQueueLengths,
client_connection_rx: ConnectionCommandReceiver,
task_client: TaskClient,
packet_type: PacketType,
stats_tx: ClientStatsSender,
shutdown_tracker: &ShutdownTracker,
) {
tracing::info!("Starting real traffic stream...");
info!("Starting real traffic stream...");
let real_messages_controller = RealMessagesController::new(
RealMessagesController::new(
controller_config,
key_rotation_config,
ack_receiver,
input_receiver,
mix_sender,
@@ -432,63 +365,9 @@ where
lane_queue_lengths,
client_connection_rx,
stats_tx,
shutdown_tracker.clone_shutdown_token(),
);
// break out all the subtasks
let (mut out_queue_control, mut reply_controller, ack_controller) =
real_messages_controller.into_tasks();
let (
mut ack_listener,
mut input_listener,
mut retransmission_listener,
mut sent_notification_listener,
mut ack_action_controller,
) = ack_controller.into_tasks();
shutdown_tracker.try_spawn_named(
async move { out_queue_control.run().await },
"RealMessagesController::OutQueueControl",
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move { reply_controller.run(shutdown_token).await },
"RealMessagesController::ReplyController",
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move { ack_listener.run(shutdown_token).await },
"AcknowledgementController::AcknowledgementListener",
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move { input_listener.run(shutdown_token).await },
"AcknowledgementController::InputMessageListener",
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move { retransmission_listener.run(shutdown_token).await },
"AcknowledgementController::RetransmissionRequestListener",
);
shutdown_tracker.try_spawn_named_with_shutdown(
async move {
sent_notification_listener.run().await;
},
"AcknowledgementController::SentNotificationListener",
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move { ack_action_controller.run(shutdown_token).await },
"AcknowledgementController::ActionController",
);
// .start(packet_type);
task_client,
)
.start(packet_type);
}
// buffer controlling all messages fetched from provider
@@ -499,29 +378,21 @@ where
mixnet_receiver: MixnetMessageReceiver,
reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender,
shutdown: TaskClient,
metrics_reporter: ClientStatsSender,
shutdown_tracker: &ShutdownTracker,
) {
tracing::info!("Starting received messages buffer controller...");
let controller = ReceivedMessagesBufferController::<SphinxMessageReceiver>::new(
local_encryption_keypair,
query_receiver,
mixnet_receiver,
reply_key_storage,
reply_controller_sender,
metrics_reporter,
shutdown_tracker.clone_shutdown_token(),
);
let (mut msg_receiver, mut req_receiver) = controller.into_tasks();
shutdown_tracker.try_spawn_named(
async move { msg_receiver.run().await },
"ReceivedMessagesBufferController::FragmentedMessageReceiver",
);
shutdown_tracker.try_spawn_named(
async move { req_receiver.run().await },
"ReceivedMessagesBufferController::RequestReceiver",
);
info!("Starting received messages buffer controller...");
let controller: ReceivedMessagesBufferController<SphinxMessageReceiver> =
ReceivedMessagesBufferController::new(
local_encryption_keypair,
query_receiver,
mixnet_receiver,
reply_key_storage,
reply_controller_sender,
metrics_reporter,
shutdown,
);
controller.start()
}
#[allow(clippy::too_many_arguments)]
@@ -533,7 +404,7 @@ where
packet_router: PacketRouter,
stats_reporter: ClientStatsSender,
#[cfg(unix)] connection_fd_callback: Option<Arc<dyn Fn(RawFd) + Send + Sync>>,
shutdown_tracker: &ShutdownTracker,
shutdown: TaskClient,
) -> Result<GatewayClient<C, S::CredentialStore>, ClientCoreError>
where
<S::KeyStore as KeyStore>::StorageError: Send + Sync + 'static,
@@ -552,7 +423,7 @@ where
packet_router,
bandwidth_controller,
stats_reporter,
shutdown_tracker.clone_shutdown_token(),
shutdown,
)
} else {
let cfg = GatewayConfig::new(
@@ -577,15 +448,15 @@ where
stats_reporter,
#[cfg(unix)]
connection_fd_callback,
shutdown_tracker.clone_shutdown_token(),
shutdown,
)
};
let gateway_failure = |err| {
tracing::error!("Could not authenticate and start up the gateway connection - {err}");
log::error!("Could not authenticate and start up the gateway connection - {err}");
ClientCoreError::GatewayClientError {
gateway_id: details.gateway_id.to_base58_string(),
source: Box::new(err),
source: err,
}
};
@@ -613,7 +484,7 @@ where
details_store
.upgrade_stored_remote_gateway_key(gateway_client.gateway_identity(), &updated_key)
.await.map_err(|err| {
tracing::error!("failed to store upgraded gateway key! this connection might be forever broken now: {err}");
error!("failed to store upgraded gateway key! this connection might be forever broken now: {err}");
ClientCoreError::GatewaysDetailsStoreError { source: Box::new(err) }
})?
}
@@ -640,7 +511,7 @@ where
packet_router: PacketRouter,
stats_reporter: ClientStatsSender,
#[cfg(unix)] connection_fd_callback: Option<Arc<dyn Fn(RawFd) + Send + Sync>>,
shutdown_tracker: &ShutdownTracker,
mut shutdown: TaskClient,
) -> Result<Box<dyn GatewayTransceiver + Send>, ClientCoreError>
where
<S::KeyStore as KeyStore>::StorageError: Send + Sync + 'static,
@@ -657,6 +528,7 @@ where
Err(ClientCoreError::CustomGatewaySelectionExpected)
} else {
// and make sure to invalidate the task client, so we wouldn't cause premature shutdown
shutdown.disarm();
custom_gateway_transceiver.set_packet_router(packet_router)?;
Ok(custom_gateway_transceiver)
};
@@ -672,7 +544,7 @@ where
stats_reporter,
#[cfg(unix)]
connection_fd_callback,
shutdown_tracker,
shutdown,
)
.await?;
@@ -683,14 +555,14 @@ where
custom_provider: Option<Box<dyn TopologyProvider + Send + Sync>>,
config_topology: config::Topology,
nym_api_urls: Vec<Url>,
nym_api_client: nym_http_api_client::Client,
user_agent: Option<UserAgent>,
) -> Box<dyn TopologyProvider + Send + Sync> {
// if no custom provider was ... provided ..., create one using nym-api
custom_provider.unwrap_or_else(|| {
Box::new(NymApiTopologyProvider::new(
config_topology,
nym_api_urls,
nym_api_client,
user_agent,
))
})
}
@@ -703,28 +575,30 @@ where
topology_accessor: TopologyAccessor,
local_gateway: NodeIdentity,
wait_for_gateway: bool,
shutdown_tracker: &ShutdownTracker,
mut task_client: TaskClient,
) -> Result<(), ClientCoreError> {
let topology_refresher_config =
TopologyRefresherConfig::new(topology_config.topology_refresh_rate);
if topology_config.disable_refreshing {
// if we're not spawning the refresher, don't cause shutdown immediately
tracing::info!("The background topology refresher is not going to be started");
info!("The background topology refesher is not going to be started");
task_client.disarm();
}
let mut topology_refresher = TopologyRefresher::new(
topology_refresher_config,
topology_accessor,
topology_provider,
task_client,
);
// before returning, block entire runtime to refresh the current network view so that any
// components depending on topology would see a non-empty view
tracing::info!("Obtaining initial network topology");
info!("Obtaining initial network topology");
topology_refresher.try_refresh().await;
if let Err(err) = topology_refresher.ensure_topology_is_routable().await {
tracing::error!(
log::error!(
"The current network topology seem to be insufficient to route any packets through \
- check if enough nodes and a gateway are online - source: {err}"
);
@@ -746,13 +620,13 @@ where
.wait_for_gateway(local_gateway, waiting_timeout)
.await
{
tracing::error!(
error!(
"the gateway did not come back online within the specified timeout: {err}"
);
return Err(err.into());
}
} else {
tracing::error!("the gateway we're supposedly connected to does not exist. We'll not be able to send any packets to ourselves: {err}");
error!("the gateway we're supposedly connected to does not exist. We'll not be able to send any packets to ourselves: {err}");
return Err(err.into());
}
}
@@ -760,11 +634,8 @@ where
if !topology_config.disable_refreshing {
// don't spawn the refresher if we don't want to be refreshing the topology.
// only use the initial values obtained
tracing::info!("Starting topology refresher...");
shutdown_tracker.try_spawn_named_with_shutdown(
async move { topology_refresher.run().await },
"TopologyRefresher",
);
info!("Starting topology refresher...");
topology_refresher.start();
}
Ok(())
@@ -775,9 +646,9 @@ where
user_agent: Option<UserAgent>,
client_stats_id: String,
input_sender: Sender<InputMessage>,
shutdown_tracker: &ShutdownTracker,
task_client: TaskClient,
) -> ClientStatsSender {
tracing::info!("Starting statistics control...");
info!("Starting statistics control...");
StatisticsControl::create_and_start(
config.debug.stats_reporting,
user_agent
@@ -785,71 +656,45 @@ where
.unwrap_or("unknown".to_string()),
client_stats_id,
input_sender.clone(),
shutdown_tracker,
task_client,
)
}
fn start_mix_traffic_controller(
gateway_transceiver: Box<dyn GatewayTransceiver + Send>,
shutdown_tracker: &ShutdownTracker,
event_tx: EventSender,
shutdown: TaskClient,
) -> (BatchMixMessageSender, ClientRequestSender) {
tracing::info!("Starting mix traffic controller...");
let mut mix_traffic_controller = MixTrafficController::new(
gateway_transceiver,
shutdown_tracker.clone_shutdown_token(),
event_tx,
);
let mix_tx = mix_traffic_controller.mix_tx();
let client_tx = mix_traffic_controller.client_tx();
shutdown_tracker.try_spawn_named(
async move { mix_traffic_controller.run().await },
"MixTrafficController",
);
info!("Starting mix traffic controller...");
let (mix_traffic_controller, mix_tx, client_tx) =
MixTrafficController::new(gateway_transceiver, shutdown);
mix_traffic_controller.start();
(mix_tx, client_tx)
}
// TODO: rename it as it implies the data is persistent whilst one can use InMemBackend
async fn setup_persistent_reply_storage(
backend: S::ReplyStore,
key_rotation_config: KeyRotationConfig,
shutdown_tracker: &ShutdownTracker,
shutdown: TaskClient,
) -> Result<CombinedReplyStorage, ClientCoreError>
where
<S::ReplyStore as ReplyStorageBackend>::StorageError: Sync + Send,
S::ReplyStore: Send + Sync,
{
tracing::trace!("Setup persistent reply storage");
let now = OffsetDateTime::now_utc();
let expected_current_key_rotation_start =
key_rotation_config.expected_current_key_rotation_start(now);
// time of the start of one epoch BEFORE the CURRENT rotation has begun
// this indicates the starting time of when packets with the current keys might have been constructed
// (i.e. any surbs OLDER than that MUST BE invalid)
let prior_epoch_start =
expected_current_key_rotation_start - key_rotation_config.epoch_duration;
log::trace!("Setup persistent reply storage");
let persistent_storage = PersistentReplyStorage::new(backend);
let mem_store = persistent_storage
.load_state_from_backend(prior_epoch_start)
.load_state_from_backend()
.await
.map_err(|err| ClientCoreError::SurbStorageError {
source: Box::new(err),
})?;
let store_clone = mem_store.clone();
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move {
persistent_storage
.flush_on_shutdown(store_clone, shutdown_token)
.await
},
"PersistentReplyStorage::flush_on_shutdown",
);
spawn_future(async move {
persistent_storage
.flush_on_shutdown(store_clone, shutdown)
.await
});
Ok(mem_store)
}
@@ -866,11 +711,11 @@ where
{
// if client keys do not exist already, create and persist them
if key_store.load_keys().await.is_err() {
tracing::info!("could not find valid client keys - a new set will be generated");
info!("could not find valid client keys - a new set will be generated");
let mut rng = OsRng;
let keys = if let Some(derivation_material) = derivation_material {
ClientKeys::from_master_key(&mut rng, &derivation_material)
.map_err(|_| ClientCoreError::HkdfDerivationError)?
.map_err(|_| ClientCoreError::HkdfDerivationError {})?
} else {
ClientKeys::generate_new(&mut rng)
};
@@ -880,77 +725,6 @@ where
setup_gateway(setup_method, key_store, details_store).await
}
fn construct_nym_api_client(
nym_api_urls: Option<&Vec<nym_network_defaults::ApiUrl>>,
config: &Config,
user_agent: Option<UserAgent>,
) -> Result<nym_http_api_client::Client, ClientCoreError> {
tracing::debug!(
"construct_nym_api_client called with nym_api_urls: {}",
nym_api_urls.is_some()
);
// If API URLs are provided, use new_with_fronted_urls() which handles domain fronting
if let Some(nym_api_urls) = nym_api_urls {
if nym_api_urls.is_empty() {
tracing::warn!("Provided nym_api_urls is empty, falling back to config endpoints");
} else {
tracing::info!(
"Building nym-api client from provided URLs (with domain fronting support): {} URLs",
nym_api_urls.len()
);
let mut builder =
nym_http_api_client::ClientBuilder::new_with_fronted_urls(nym_api_urls.clone())
.map_err(ClientCoreError::from)?
.with_retries(DEFAULT_NYM_API_RETRIES);
if let Some(user_agent) = user_agent {
builder = builder.with_user_agent(user_agent);
}
return builder.build().map_err(ClientCoreError::from);
}
}
// Fallback to basic client for backwards compatibility
tracing::debug!("Building basic nym-api HTTP client from config endpoints");
let mut nym_api_urls = config.get_nym_api_endpoints();
if nym_api_urls.is_empty() {
tracing::warn!("No API endpoints configured in config, this may cause issues");
}
nym_api_urls.shuffle(&mut thread_rng());
// Convert config URLs to ApiUrl format for consistency
let api_urls: Vec<nym_network_defaults::ApiUrl> = nym_api_urls
.into_iter()
.map(|url| nym_network_defaults::ApiUrl {
url: url.to_string(),
front_hosts: None,
})
.collect();
tracing::debug!("Using {} config API endpoints", api_urls.len());
let mut builder = nym_http_api_client::ClientBuilder::new_with_fronted_urls(api_urls)
.map_err(ClientCoreError::from)?
.with_retries(DEFAULT_NYM_API_RETRIES)
.with_bincode();
if let Some(user_agent) = user_agent {
builder = builder.with_user_agent(user_agent);
}
builder.build().map_err(ClientCoreError::from)
}
async fn determine_key_rotation_state(
client: &nym_http_api_client::Client,
) -> Result<KeyRotationConfig, ClientCoreError> {
Ok(client.get_key_rotation_info().await?.into())
}
pub async fn start_base(mut self) -> Result<BaseClient, ClientCoreError>
where
S::ReplyStore: Send + Sync,
@@ -959,12 +733,7 @@ where
<S::CredentialStore as CredentialStorage>::StorageError: Send + Sync + 'static,
<S::GatewaysDetailsStore as GatewaysDetailsStore>::StorageError: Sync + Send,
{
tracing::info!("Starting nym client");
#[cfg(debug_assertions)]
#[cfg(target_arch = "wasm32")]
{
console_log!("Starting base Nym Client");
}
info!("Starting nym client");
// derive (or load) client keys and gateway configuration
let init_res = Self::initialise_keys_and_gateway(
@@ -993,22 +762,17 @@ where
// channels responsible for controlling real messages
let (input_sender, input_receiver) = tokio::sync::mpsc::channel::<InputMessage>(1);
// channels responsible for event management
let (event_sender, event_receiver) = mpsc::unbounded();
// channels responsible for controlling ack messages
let (ack_sender, ack_receiver) = mpsc::unbounded();
let shared_topology_accessor =
TopologyAccessor::new(self.config.debug.topology.ignore_egress_epoch_role);
// Create a shutdown tracker for this client - either as a child of provided tracker
// or get one from the registry
let shutdown_tracker = match self.shutdown {
Some(parent_tracker) => parent_tracker.clone(),
None => nym_task::create_sdk_shutdown_tracker()?,
};
Self::start_event_control(self.event_tx, event_receiver, &shutdown_tracker);
// Shutdown notifier for signalling tasks to stop
let shutdown = self
.shutdown
.map(Into::<TaskHandle>::into)
.unwrap_or_default()
.name_if_unnamed("BaseNymClient");
// channels responsible for dealing with reply-related fun
let (reply_controller_sender, reply_controller_receiver) =
@@ -1025,18 +789,11 @@ where
.dkg_query_client
.map(|client| BandwidthController::new(credential_store, client));
let nym_api_client = Self::construct_nym_api_client(
self.nym_api_urls.as_ref(),
&self.config,
self.user_agent.clone(),
)?;
let key_rotation_config = Self::determine_key_rotation_state(&nym_api_client).await?;
let topology_provider = Self::setup_topology_provider(
self.custom_topology_provider.take(),
self.config.debug.topology,
self.config.get_nym_api_endpoints(),
nym_api_client,
self.user_agent.clone(),
);
let stats_reporter = Self::start_statistics_control(
@@ -1044,7 +801,7 @@ where
self.user_agent.clone(),
generate_client_stats_id(*self_address.identity()),
input_sender.clone(),
&shutdown_tracker.clone(),
shutdown.fork("statistics_control"),
);
// needs to be started as the first thing to block if required waiting for the gateway
@@ -1054,14 +811,14 @@ where
shared_topology_accessor.clone(),
self_address.gateway(),
self.wait_for_gateway,
&shutdown_tracker.clone(),
shutdown.fork("topology_refresher"),
)
.await?;
let gateway_packet_router = PacketRouter::new(
ack_sender,
mixnet_messages_sender,
shutdown_tracker.clone_shutdown_token(),
shutdown.get_handle().named("gateway-packet-router"),
);
let gateway_transceiver = Self::setup_gateway_transceiver(
@@ -1074,15 +831,14 @@ where
stats_reporter.clone(),
#[cfg(unix)]
self.connection_fd_callback,
&shutdown_tracker.clone(),
shutdown.fork("gateway_transceiver"),
)
.await?;
let gateway_ws_fd = gateway_transceiver.ws_fd();
let reply_storage = Self::setup_persistent_reply_storage(
reply_storage_backend,
key_rotation_config,
&shutdown_tracker.clone(),
shutdown.fork("persistent_reply_storage"),
)
.await?;
@@ -1092,8 +848,8 @@ where
mixnet_messages_receiver,
reply_storage.key_storage(),
reply_controller_sender.clone(),
shutdown.fork("received_messages_buffer"),
stats_reporter.clone(),
&shutdown_tracker.clone(),
);
// The message_sender is the transmitter for any component generating sphinx packets
@@ -1103,8 +859,7 @@ where
let (message_sender, client_request_sender) = Self::start_mix_traffic_controller(
gateway_transceiver,
&shutdown_tracker.clone(),
EventSender(event_sender),
shutdown.fork("mix_traffic_controller"),
);
// Channels that the websocket listener can use to signal downstream to the real traffic
@@ -1123,7 +878,6 @@ where
Self::start_real_traffic_controller(
controller_config,
key_rotation_config,
shared_topology_accessor.clone(),
ack_receiver,
input_receiver,
@@ -1133,8 +887,9 @@ where
reply_controller_receiver,
shared_lane_queue_lengths.clone(),
client_connection_rx,
shutdown.fork("real_traffic_controller"),
self.config.debug.traffic.packet_type,
stats_reporter.clone(),
&shutdown_tracker.clone(),
);
if !self
@@ -1150,19 +905,12 @@ where
shared_topology_accessor.clone(),
message_sender,
stats_reporter.clone(),
&shutdown_tracker.clone(),
shutdown.fork("cover_traffic_stream"),
);
}
tracing::debug!("Core client startup finished!");
tracing::debug!("The address of this client is: {self_address}");
#[cfg(debug_assertions)]
#[cfg(target_arch = "wasm32")]
{
console_log!("Core client startup finished!");
console_log!("Rust::start_base: the address of this client is: {self_address}");
}
debug!("Core client startup finished!");
debug!("The address of this client is: {self_address}");
Ok(BaseClient {
address: self_address,
@@ -1171,7 +919,6 @@ where
client_input: ClientInput {
connection_command_sender: client_connection_tx,
input_sender,
client_request_sender,
},
},
client_output: ClientOutputStatus::AwaitingConsumer {
@@ -1186,7 +933,8 @@ where
gateway_connection: GatewayConnection { gateway_ws_fd },
},
stats_reporter,
shutdown_handle: shutdown_tracker, // The primary tracker for this client
task_handle: shutdown,
client_request_sender,
forget_me: self.config.debug.forget_me,
remember_me: self.config.debug.remember_me,
})
@@ -1200,57 +948,8 @@ pub struct BaseClient {
pub client_output: ClientOutputStatus,
pub client_state: ClientState,
pub stats_reporter: ClientStatsSender,
pub shutdown_handle: ShutdownTracker,
pub client_request_sender: ClientRequestSender,
pub task_handle: TaskHandle,
pub forget_me: ForgetMe,
pub remember_me: RememberMe,
}
#[cfg(test)]
mod tests {
use super::*;
use nym_network_defaults::{ApiUrl, NymNetworkDetails};
#[test]
fn test_network_details_with_multiple_urls() {
// Verify that network details can be configured with multiple API URLs
let mut network_details = NymNetworkDetails::new_empty();
network_details.nym_api_urls = Some(vec![
ApiUrl {
url: "https://validator.nymtech.net/api/".to_string(),
front_hosts: None,
},
ApiUrl {
url: "https://nym-frontdoor.vercel.app/api/".to_string(),
front_hosts: Some(vec!["vercel.app".to_string(), "vercel.com".to_string()]),
},
]);
assert_eq!(network_details.nym_api_urls.as_ref().unwrap().len(), 2);
assert!(network_details.nym_api_urls.as_ref().unwrap()[1]
.front_hosts
.is_some());
}
#[test]
fn test_network_details_with_front_hosts() {
// Verify that ApiUrl can store domain fronting configuration
let api_url = ApiUrl {
url: "https://nym-frontdoor.vercel.app/api/".to_string(),
front_hosts: Some(vec!["vercel.app".to_string(), "vercel.com".to_string()]),
};
assert_eq!(api_url.url, "https://nym-frontdoor.vercel.app/api/");
assert_eq!(api_url.front_hosts.as_ref().unwrap().len(), 2);
assert!(api_url
.front_hosts
.as_ref()
.unwrap()
.contains(&"vercel.app".to_string()));
}
#[test]
fn test_default_nym_api_retries_constant() {
// Verify the retry constant is set correctly
assert_eq!(DEFAULT_NYM_API_RETRIES, 3);
}
}
@@ -1,30 +1,32 @@
// Copyright 2022-2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::{
client::replies::reply_storage::{fs_backend, CombinedReplyStorage, ReplyStorageBackend},
config,
config::Config,
error::ClientCoreError,
use crate::client::replies::reply_storage::{
fs_backend, CombinedReplyStorage, ReplyStorageBackend,
};
use crate::config;
use crate::config::Config;
use crate::error::ClientCoreError;
use log::{error, info, trace};
use nym_bandwidth_controller::BandwidthController;
use nym_client_core_gateways_storage::OnDiskGatewaysDetails;
use nym_credential_storage::storage::Storage as CredentialStorage;
use nym_validator_client::{nyxd, QueryHttpRpcNyxdClient};
use std::{io, path::Path};
use nym_validator_client::nyxd;
use nym_validator_client::QueryHttpRpcNyxdClient;
use std::path::Path;
use std::{fs, io};
use time::OffsetDateTime;
use tracing::{error, info, trace};
use url::Url;
async fn setup_fresh_backend<P: AsRef<Path>>(
db_path: P,
surb_config: &config::ReplySurbs,
) -> Result<fs_backend::Backend, ClientCoreError> {
info!("Creating fresh surb database");
info!("creating fresh surb database");
let mut storage_backend = match fs_backend::Backend::init(db_path).await {
Ok(backend) => backend,
Err(err) => {
error!("setup_fresh_backend: Failed to setup persistent storage backend for our reply needs: {err}");
error!("failed to setup persistent storage backend for our reply needs: {err}");
return Err(ClientCoreError::SurbStorageError {
source: Box::new(err),
});
@@ -38,15 +40,14 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
surb_config.minimum_reply_surb_storage_threshold,
surb_config.maximum_reply_surb_storage_threshold,
);
match storage_backend.init_fresh(&mem_store).await {
Ok(()) => Ok(storage_backend),
Err(err) => {
storage_backend.shutdown().await;
Err(ClientCoreError::SurbStorageError {
source: Box::new(err),
})
}
}
storage_backend
.init_fresh(&mem_store)
.await
.map_err(|err| ClientCoreError::SurbStorageError {
source: Box::new(err),
})?;
Ok(storage_backend)
}
// fn setup_inactive_backend(surb_config: &config::ReplySurbs) -> fs_backend::Backend {
@@ -57,11 +58,12 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
// )
// }
async fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> {
fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> {
let db_path = db_path.as_ref();
debug_assert!(db_path.exists());
let now = OffsetDateTime::now_utc().unix_timestamp();
let suffix = format!("_{now}.corrupted");
let new_extension =
@@ -70,15 +72,11 @@ async fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()
} else {
suffix
};
let renamed = db_path.with_extension(new_extension);
tokio::fs::rename(db_path, &renamed).await.inspect_err(|_| {
error!(
"Failed to rename corrupt database file: {} to {}",
db_path.display(),
renamed.display()
);
})
let mut renamed = db_path.to_owned();
renamed.set_extension(new_extension);
fs::rename(db_path, renamed)
}
pub async fn setup_fs_reply_surb_backend<P: AsRef<Path>>(
@@ -89,12 +87,13 @@ pub async fn setup_fs_reply_surb_backend<P: AsRef<Path>>(
// the existing one
let db_path = db_path.as_ref();
if db_path.exists() {
info!("Loading existing surb database");
match fs_backend::Backend::try_load(db_path).await {
info!("loading existing surb database");
match fs_backend::Backend::try_load(db_path, surb_config.fresh_sender_tags).await {
Ok(backend) => Ok(backend),
Err(err) => {
error!("setup_fs_reply_surb_backend: Failed to setup persistent storage backend for our reply needs: {err}. We're going to create a fresh database instead. This behaviour might change in the future");
archive_corrupted_database(db_path).await?;
error!("failed to setup persistent storage backend for our reply needs: {err}. We're going to create a fresh database instead. This behaviour might change in the future");
archive_corrupted_database(db_path)?;
setup_fresh_backend(db_path, surb_config).await
}
}
@@ -114,32 +113,41 @@ pub async fn setup_fs_gateways_storage<P: AsRef<Path>>(
})
}
pub fn create_bandwidth_controller_with_urls<St: CredentialStorage>(
nyxd_url: Url,
storage: St,
) -> Result<BandwidthController<QueryHttpRpcNyxdClient, St>, ClientCoreError> {
let client = default_query_dkg_client(nyxd_url)?;
Ok(BandwidthController::new(storage, client))
}
pub fn default_query_dkg_client_from_config(
pub fn create_bandwidth_controller<St: CredentialStorage>(
config: &Config,
) -> Result<QueryHttpRpcNyxdClient, ClientCoreError> {
storage: St,
) -> BandwidthController<QueryHttpRpcNyxdClient, St> {
let nyxd_url = config
.get_validator_endpoints()
.pop()
.ok_or(ClientCoreError::RpcClientMissingUrl)?;
.expect("No nyxd validator endpoint provided");
create_bandwidth_controller_with_urls(nyxd_url, storage)
}
pub fn create_bandwidth_controller_with_urls<St: CredentialStorage>(
nyxd_url: Url,
storage: St,
) -> BandwidthController<QueryHttpRpcNyxdClient, St> {
let client = default_query_dkg_client(nyxd_url);
BandwidthController::new(storage, client)
}
pub fn default_query_dkg_client_from_config(config: &Config) -> QueryHttpRpcNyxdClient {
let nyxd_url = config
.get_validator_endpoints()
.pop()
.expect("No nyxd validator endpoint provided");
default_query_dkg_client(nyxd_url)
}
pub fn default_query_dkg_client(nyxd_url: Url) -> Result<QueryHttpRpcNyxdClient, ClientCoreError> {
pub fn default_query_dkg_client(nyxd_url: Url) -> QueryHttpRpcNyxdClient {
let details = nym_network_defaults::NymNetworkDetails::new_from_env();
let client_config = nyxd::Config::try_from_nym_network_details(&details)
.map_err(|source| ClientCoreError::InvalidNetworkDetails { source })?;
.expect("failed to construct validator client config");
// overwrite env configuration with config URLs
QueryHttpRpcNyxdClient::connect(client_config, nyxd_url.as_str())
.map_err(|source| ClientCoreError::RpcClientCreationFailure { source })
.expect("Could not construct query client")
}
@@ -3,21 +3,22 @@
use crate::client::mix_traffic::BatchMixMessageSender;
use crate::client::topology_control::TopologyAccessor;
use crate::config;
use crate::{config, spawn_future};
use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt};
use log::*;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::cover::generate_loop_cover_packet;
use nym_sphinx::params::{PacketSize, PacketType};
use nym_sphinx::utils::sample_poisson_duration;
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use nym_task::TaskClient;
use rand::{rngs::OsRng, CryptoRng, Rng};
use std::pin::Pin;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::mpsc::error::TrySendError;
use tracing::*;
#[cfg(not(target_arch = "wasm32"))]
use tokio::time::{sleep, Sleep};
@@ -68,6 +69,8 @@ where
packet_type: PacketType,
stats_tx: ClientStatsSender,
task_client: TaskClient,
}
impl<R> Stream for LoopCoverTrafficStream<R>
@@ -114,6 +117,7 @@ impl LoopCoverTrafficStream<OsRng> {
traffic_config: config::Traffic,
cover_config: config::CoverTraffic,
stats_tx: ClientStatsSender,
task_client: TaskClient,
) -> Self {
let rng = OsRng;
@@ -133,6 +137,7 @@ impl LoopCoverTrafficStream<OsRng> {
use_legacy_sphinx_format: traffic_config.use_legacy_sphinx_format,
packet_type: traffic_config.packet_type,
stats_tx,
task_client,
}
}
@@ -205,10 +210,10 @@ impl LoopCoverTrafficStream<OsRng> {
TrySendError::Full(_) => {
// This isn't a problem, if the channel is full means we're already sending the
// max amount of messages downstream can handle.
tracing::trace!("Failed to send cover message - channel full");
log::debug!("Failed to send cover message - channel full");
}
TrySendError::Closed(_) => {
tracing::warn!("Failed to send cover message - channel closed");
log::warn!("Failed to send cover message - channel closed");
}
}
} else {
@@ -225,24 +230,16 @@ impl LoopCoverTrafficStream<OsRng> {
// JS: due to identical logical structure to OutQueueControl::on_message(), this is also
// presumably required to prevent bugs in the future. Exact reason is still unknown to me.
// TODO: temporary and BAD workaround for wasm (we should find a way to yield here in wasm)
#[cfg(not(target_arch = "wasm32"))]
{
tokio::task::yield_now().await;
}
#[cfg(target_arch = "wasm32")]
{
tokio_with_wasm::task::yield_now().await;
}
tokio::task::yield_now().await;
}
// it's fine if cover traffic stream task gets killed whilst processing next message
#[allow(clippy::panic)]
pub async fn run(&mut self) {
pub fn start(mut self) {
if self.cover_traffic.disable_loop_cover_traffic_stream {
// we should have never got here in the first place - the task should have never been created to begin with
// so panic and review the code that lead to this branch
panic!("attempted to run LoopCoverTrafficStream while config explicitly disabled it.")
panic!("attempted to start LoopCoverTrafficStream while config explicitly disabled it.")
}
// we should set initial delay only when we actually start the stream
@@ -252,11 +249,29 @@ impl LoopCoverTrafficStream<OsRng> {
);
self.set_next_delay(sampled);
while self.next().await.is_some() {
self.on_new_message().await;
}
let mut shutdown = self.task_client.fork("select");
// this should never get triggered
error!("cover traffic stream has been exhausted!")
spawn_future(async move {
debug!("Started LoopCoverTrafficStream with graceful shutdown support");
while !shutdown.is_shutdown() {
tokio::select! {
biased;
_ = shutdown.recv() => {
log::trace!("LoopCoverTrafficStream: Received shutdown");
}
next = self.next() => {
if next.is_some() {
self.on_new_message().await;
} else {
log::trace!("LoopCoverTrafficStream: Stopping since channel closed");
break;
}
}
}
}
shutdown.recv_timeout().await;
log::debug!("LoopCoverTrafficStream: Exiting");
})
}
}
@@ -1,40 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use futures::StreamExt;
use crate::client::base_client::{EventReceiver, EventSender, MixnetClientEvent};
/// Launches and manages task events, propagating upwards what is not strictly internal.
pub(crate) struct EventControl {
parent_event_tx: Option<EventSender>,
children_event_rx: EventReceiver,
}
impl EventControl {
pub(crate) fn new(
parent_event_tx: Option<EventSender>,
children_event_rx: EventReceiver,
) -> Self {
EventControl {
parent_event_tx,
children_event_rx,
}
}
fn is_internal(event: MixnetClientEvent) -> bool {
match event {
MixnetClientEvent::Traffic(_) => false,
}
}
pub(crate) async fn run(mut self) {
while let Some(event) = self.children_event_rx.next().await {
if let Some(parent_event_tx) = &self.parent_event_tx {
if !Self::is_internal(event) {
parent_event_tx.send(event);
}
}
}
}
}
@@ -2,9 +2,9 @@
// SPDX-License-Identifier: Apache-2.0
#![allow(unused_imports)]
use std::time::Duration;
#[cfg(target_arch = "wasm32")]
pub use wasmtimer::{std::Instant, tokio::*};
pub type IntervalStream = gloo_timers::future::IntervalStream;
@@ -135,9 +135,7 @@ impl InputMessage {
recipient_tag,
data,
lane,
// \/ set it to SOME sane default so that if we run out of surbs and constantly
// fail to request more, we wouldn't be stuck in limbo
max_retransmissions: Some(10),
max_retransmissions: None,
};
if let Some(packet_type) = packet_type {
InputMessage::new_wrapper(message, packet_type)
+98 -110
View File
@@ -1,14 +1,13 @@
// Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::{
base_client::{EventSender, MixnetClientEvent},
mix_traffic::transceiver::GatewayTransceiver,
};
use crate::client::mix_traffic::transceiver::GatewayTransceiver;
use crate::error::ClientCoreError;
use crate::spawn_future;
use log::*;
use nym_gateway_requests::ClientRequest;
use nym_sphinx::forwarding::packet::MixPacket;
use nym_task::ShutdownToken;
use tracing::*;
use nym_task::TaskClient;
use transceiver::ErasedGatewayError;
pub type BatchMixMessageSender = tokio::sync::mpsc::Sender<Vec<MixPacket>>;
@@ -20,41 +19,33 @@ pub mod transceiver;
// We remind ourselves that 32 x 32kb = 1024kb, a reasonable size for a network buffer.
pub const MIX_MESSAGE_RECEIVER_BUFFER_SIZE: usize = 32;
/// Reduced from 100 to 20 to fail fast (~1-2 seconds instead of ~6 seconds).
/// If we can't send 20 packets in a row, the gateway is unreachable.
const MAX_FAILURE_COUNT: usize = 20;
const MAX_FAILURE_COUNT: usize = 100;
// that's also disgusting.
pub struct Empty;
#[derive(Clone, Copy, Debug)]
pub enum MixTrafficEvent {
FailedSendingSphinx,
}
pub struct MixTrafficController {
gateway_transceiver: Box<dyn GatewayTransceiver + Send>,
mix_tx: BatchMixMessageSender,
mix_rx: BatchMixMessageReceiver,
client_rx: ClientRequestReceiver,
client_tx: ClientRequestSender,
// TODO: this is temporary work-around.
// in long run `gateway_client` will be moved away from `MixTrafficController` anyway.
consecutive_gateway_failure_count: usize,
shutdown_token: ShutdownToken,
event_tx: EventSender,
task_client: TaskClient,
}
impl MixTrafficController {
pub fn new<T>(
gateway_transceiver: T,
shutdown_token: ShutdownToken,
event_tx: EventSender,
) -> MixTrafficController
task_client: TaskClient,
) -> (
MixTrafficController,
BatchMixMessageSender,
ClientRequestSender,
)
where
T: GatewayTransceiver + Send + 'static,
{
@@ -63,32 +54,41 @@ impl MixTrafficController {
let (client_sender, client_receiver) = tokio::sync::mpsc::channel(8);
MixTrafficController {
gateway_transceiver: Box::new(gateway_transceiver),
mix_tx: message_sender,
mix_rx: message_receiver,
client_rx: client_receiver,
client_tx: client_sender,
consecutive_gateway_failure_count: 0,
shutdown_token,
event_tx,
}
(
MixTrafficController {
gateway_transceiver: Box::new(gateway_transceiver),
mix_rx: message_receiver,
client_rx: client_receiver,
consecutive_gateway_failure_count: 0,
task_client,
},
message_sender,
client_sender,
)
}
pub fn new_dynamic(
gateway_transceiver: Box<dyn GatewayTransceiver + Send>,
shutdown_token: ShutdownToken,
event_tx: EventSender,
) -> MixTrafficController {
Self::new(gateway_transceiver, shutdown_token, event_tx)
}
pub fn client_tx(&self) -> ClientRequestSender {
self.client_tx.clone()
}
pub fn mix_tx(&self) -> BatchMixMessageSender {
self.mix_tx.clone()
task_client: TaskClient,
) -> (
MixTrafficController,
BatchMixMessageSender,
ClientRequestSender,
) {
let (message_sender, message_receiver) =
tokio::sync::mpsc::channel(MIX_MESSAGE_RECEIVER_BUFFER_SIZE);
let (client_sender, client_receiver) = tokio::sync::mpsc::channel(8);
(
MixTrafficController {
gateway_transceiver,
mix_rx: message_receiver,
client_rx: client_receiver,
consecutive_gateway_failure_count: 0,
task_client,
},
message_sender,
client_sender,
)
}
async fn on_messages(
@@ -96,84 +96,72 @@ impl MixTrafficController {
mut mix_packets: Vec<MixPacket>,
) -> Result<(), ErasedGatewayError> {
debug_assert!(!mix_packets.is_empty());
let send_future = if mix_packets.len() == 1 {
// SAFETY: we just checked we have one packet
#[allow(clippy::unwrap_used)]
let result = if mix_packets.len() == 1 {
let mix_packet = mix_packets.pop().unwrap();
self.gateway_transceiver.send_mix_packet(mix_packet)
self.gateway_transceiver.send_mix_packet(mix_packet).await
} else {
self.gateway_transceiver.batch_send_mix_packets(mix_packets)
self.gateway_transceiver
.batch_send_mix_packets(mix_packets)
.await
};
tokio::select! {
biased;
_ = self.shutdown_token.cancelled() => {
trace!("received shutdown while handling messages");
Ok(())
}
result = send_future => {
if result.is_err() {
self.consecutive_gateway_failure_count += 1;
} else {
trace!("We *might* have managed to forward sphinx packet(s) to the gateway!");
self.consecutive_gateway_failure_count = 0;
}
result
}
if result.is_err() {
self.consecutive_gateway_failure_count += 1;
} else {
trace!("We *might* have managed to forward sphinx packet(s) to the gateway!");
self.consecutive_gateway_failure_count = 0;
}
result
}
async fn on_client_request(&mut self, client_request: ClientRequest) {
tokio::select! {
biased;
_ = self.shutdown_token.cancelled() => {
trace!("received shutdown while handling client request");
}
result = self.gateway_transceiver.send_client_request(client_request) => {
if let Err(err) = result {
error!("Failed to send client request: {err}")
}
}
}
}
pub fn start(mut self) {
spawn_future(async move {
debug!("Started MixTrafficController with graceful shutdown support");
pub async fn run(&mut self) {
debug!("Started MixTrafficController with graceful shutdown support");
loop {
tokio::select! {
biased;
_ = self.shutdown_token.cancelled() => {
trace!("MixTrafficController: Received shutdown");
break;
}
// mix_rx should never error out as we're holding one instance of the sender
Some(mix_packets) = self.mix_rx.recv() => {
if let Err(err) = self.on_messages(mix_packets).await {
error!("Failed to send sphinx packet(s) to the gateway: {err}");
if self.consecutive_gateway_failure_count == MAX_FAILURE_COUNT {
// Disconnect from the gateway. If we should try to re-connect
// is handled at a higher layer.
error!("Failed to send sphinx packet to the gateway {MAX_FAILURE_COUNT} times in a row - assuming the gateway is dead");
// Do we need to handle the embedded mixnet client case
// separately?
self.event_tx.send(MixnetClientEvent::Traffic(MixTrafficEvent::FailedSendingSphinx));
// IMO it shouldn't be signalled from there but it is how it is
// TODO : report the failure upwards and shutdown from upwards
// Gateway is dead, we have to shut down currently
error!("Signalling shutdown from the MixTrafficController");
self.shutdown_token.cancel();
while !self.task_client.is_shutdown() {
tokio::select! {
mix_packets = self.mix_rx.recv() => match mix_packets {
Some(mix_packets) => {
if let Err(err) = self.on_messages(mix_packets).await {
error!("Failed to send sphinx packet(s) to the gateway: {err}");
if self.consecutive_gateway_failure_count == MAX_FAILURE_COUNT {
// Disconnect from the gateway. If we should try to re-connect
// is handled at a higher layer.
error!("Failed to send sphinx packet to the gateway {MAX_FAILURE_COUNT} times in a row - assuming the gateway is dead");
// Do we need to handle the embedded mixnet client case
// separately?
self.task_client.send_we_stopped(Box::new(ClientCoreError::GatewayFailedToForwardMessages));
break;
}
}
},
None => {
log::trace!("MixTrafficController: Stopping since channel closed");
break;
}
},
client_request = self.client_rx.recv() => match client_request {
Some(client_request) => {
match self.gateway_transceiver.send_client_request(client_request).await {
Ok(_) => (),
Err(e) => error!("Failed to send client request: {}", e),
};
},
None => {
log::trace!("MixTrafficController, client request channel closed");
}
},
_ = self.task_client.recv() => {
log::trace!("MixTrafficController: Received shutdown");
break;
}
},
// client_rx should never error out as we're holding one instance of the sender
Some(client_request) = self.client_rx.recv() => {
self.on_client_request(client_request).await;
}
}
}
debug!("MixTrafficController: Exiting");
self.task_client.recv_timeout().await;
log::debug!("MixTrafficController: Exiting");
});
}
}
@@ -2,6 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use async_trait::async_trait;
use log::{debug, error};
use nym_credential_storage::storage::Storage as CredentialStorage;
use nym_crypto::asymmetric::ed25519;
use nym_gateway_client::error::GatewayClientError;
@@ -13,7 +14,6 @@ use nym_validator_client::nyxd::contract_traits::DkgQueryClient;
use std::fmt::Debug;
use std::os::raw::c_int as RawFd;
use thiserror::Error;
use tracing::{debug, error};
#[cfg(not(target_arch = "wasm32"))]
use futures::channel::oneshot;
@@ -27,7 +27,7 @@ fn erase_err<E: std::error::Error + Send + Sync + 'static>(err: E) -> ErasedGate
ErasedGatewayError(Box::new(err))
}
/// This combines the functionalities of being able to send and receive mix packets.
/// This combines combines the functionalities of being able to send and receive mix packets.
#[async_trait]
pub trait GatewayTransceiver: GatewaySender + GatewayReceiver {
fn gateway_identity(&self) -> ed25519::PublicKey;
@@ -87,7 +87,7 @@ impl<G: GatewayTransceiver + ?Sized + Send> GatewayTransceiver for Box<G> {
message: ClientRequest,
) -> Result<(), GatewayClientError> {
let _ = (**self).send_client_request(message.clone()).await?;
tracing::debug!("Sent client request: {:?}", message);
log::debug!("Sent client request: {:?}", message);
Ok(())
}
}
@@ -269,8 +269,6 @@ pub struct MockGateway {
}
impl Default for MockGateway {
// test code
#[allow(clippy::unwrap_used)]
fn default() -> Self {
MockGateway {
dummy_identity: "3ebjp1Fb9hdcS1AR6AZihgeJiMHkB5jjJUsvqNnfQwU7"
-1
View File
@@ -3,7 +3,6 @@
pub mod base_client;
pub mod cover_traffic_stream;
pub(crate) mod event_control;
pub(crate) mod helpers;
pub mod inbound_messages;
pub mod key_manager;
@@ -5,22 +5,23 @@ use super::action_controller::{AckActionSender, Action};
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use futures::StreamExt;
use log::*;
use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::{
acknowledgements::{identifier::recover_identifier, AckKey},
chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID},
};
use nym_task::ShutdownToken;
use nym_task::TaskClient;
use std::sync::Arc;
use tracing::*;
/// Module responsible for listening for any data resembling acknowledgements from the network
/// and firing actions to remove them from the 'Pending' state.
pub(crate) struct AcknowledgementListener {
pub(super) struct AcknowledgementListener {
ack_key: Arc<AckKey>,
ack_receiver: AcknowledgementReceiver,
action_sender: AckActionSender,
stats_tx: ClientStatsSender,
task_client: TaskClient,
}
impl AcknowledgementListener {
@@ -29,12 +30,14 @@ impl AcknowledgementListener {
ack_receiver: AcknowledgementReceiver,
action_sender: AckActionSender,
stats_tx: ClientStatsSender,
task_client: TaskClient,
) -> Self {
AcknowledgementListener {
ack_key,
ack_receiver,
action_sender,
stats_tx,
task_client,
}
}
@@ -62,12 +65,17 @@ impl AcknowledgementListener {
return;
}
trace!("Received {frag_id} from the mix network");
trace!("Received {} from the mix network", frag_id);
self.stats_tx
.report(PacketStatisticsEvent::RealAckReceived(ack_content.len()).into());
let _ = self
if let Err(err) = self
.action_sender
.unbounded_send(Action::new_remove(frag_id));
.unbounded_send(Action::new_remove(frag_id))
{
if !self.task_client.is_shutdown_poll() {
error!("Failed to send remove action to action controller: {err}");
}
}
}
async fn handle_ack_receiver_item(&mut self, item: Vec<Vec<u8>>) {
@@ -77,26 +85,24 @@ impl AcknowledgementListener {
}
}
pub(crate) async fn run(&mut self, shutdown_token: ShutdownToken) {
pub(super) async fn run(&mut self) {
debug!("Started AcknowledgementListener with graceful shutdown support");
loop {
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("AcknowledgementListener: Received shutdown");
break;
}
acks = self.ack_receiver.next() => match acks {
Some(acks) => self.handle_ack_receiver_item(acks).await,
None => {
tracing::trace!("AcknowledgementListener: Stopping since channel closed");
log::trace!("AcknowledgementListener: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("AcknowledgementListener: Received shutdown");
}
}
}
tracing::debug!("AcknowledgementListener: Exiting");
self.task_client.recv_timeout().await;
log::debug!("AcknowledgementListener: Exiting");
}
}
@@ -5,14 +5,14 @@ use super::PendingAcknowledgement;
use crate::client::real_messages_control::acknowledgement_control::RetransmissionRequestSender;
use futures::channel::mpsc;
use futures::StreamExt;
use log::*;
use nym_nonexhaustive_delayqueue::{Expired, NonExhaustiveDelayQueue, QueueKey};
use nym_sphinx::chunking::fragment::FragmentIdentifier;
use nym_sphinx::Delay as SphinxDelay;
use nym_task::ShutdownToken;
use nym_task::TaskClient;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use tracing::*;
pub(crate) type AckActionSender = mpsc::UnboundedSender<Action>;
pub(crate) type AckActionReceiver = mpsc::UnboundedReceiver<Action>;
@@ -82,7 +82,7 @@ impl Config {
}
}
pub(crate) struct ActionController {
pub(super) struct ActionController {
/// Configurable parameters of the `ActionController`
config: Config,
@@ -102,6 +102,8 @@ pub(crate) struct ActionController {
/// Channel for notifying `RetransmissionRequestListener` about expired acknowledgements.
retransmission_sender: RetransmissionRequestSender,
task_client: TaskClient,
}
impl ActionController {
@@ -109,6 +111,7 @@ impl ActionController {
config: Config,
retransmission_sender: RetransmissionRequestSender,
incoming_actions: AckActionReceiver,
task_client: TaskClient,
) -> Self {
ActionController {
config,
@@ -116,13 +119,14 @@ impl ActionController {
pending_acks_timers: NonExhaustiveDelayQueue::new(),
incoming_actions,
retransmission_sender,
task_client,
}
}
fn handle_insert(&mut self, pending_acks: Vec<PendingAcknowledgement>) {
for pending_ack in pending_acks {
let frag_id = pending_ack.message_chunk.fragment_identifier();
trace!("{frag_id} is inserted");
trace!("{} is inserted", frag_id);
if self
.pending_acks_data
@@ -157,16 +161,22 @@ impl ActionController {
let new_queue_key = self.pending_acks_timers.insert(frag_id, timeout);
*queue_key = Some(new_queue_key)
} else {
debug!("Tried to START TIMER on pending ack that is already gone! - {frag_id}");
debug!(
"Tried to START TIMER on pending ack that is already gone! - {}",
frag_id
);
}
}
fn handle_remove(&mut self, frag_id: FragmentIdentifier) {
trace!("{frag_id} is getting removed");
trace!("{} is getting removed", frag_id);
match self.pending_acks_data.remove(&frag_id) {
None => {
debug!("Tried to REMOVE pending ack that is already gone! - {frag_id}");
debug!(
"Tried to REMOVE pending ack that is already gone! - {}",
frag_id
);
}
Some((_, queue_key)) => {
if let Some(queue_key) = queue_key {
@@ -178,7 +188,10 @@ impl ActionController {
} else {
// I'm not 100% sure if having a `None` key is even possible here
// (REMOVE would have to be called before START TIMER),
debug!("Tried to REMOVE pending ack without TIMER active - {frag_id}");
debug!(
"Tried to REMOVE pending ack without TIMER active - {}",
frag_id
);
}
}
}
@@ -187,26 +200,27 @@ impl ActionController {
// initiated basically as a first step of retransmission. At first data has its delay updated
// (as new sphinx packet was created with new expected delivery time)
fn handle_update_pending_ack(&mut self, frag_id: FragmentIdentifier, delay: SphinxDelay) {
trace!("{frag_id} is updating its delay");
trace!("{} is updating its delay", frag_id);
// TODO: is it possible to solve this without either locking or temporarily removing the value?
if let Some((pending_ack_data, queue_key)) = self.pending_acks_data.remove(&frag_id) {
// SAFETY: this Action is triggered by `RetransmissionRequestListener` (for 'normal' packets)
// this Action is triggered by `RetransmissionRequestListener` (for 'normal' packets)
// or `ReplyController` (for 'reply' packets) which held the other potential
// reference to this Arc. HOWEVER, before the Action was pushed onto the queue, the reference
// was dropped hence this unwrap is safe.
#[allow(clippy::unwrap_used)]
let mut inner_data = Arc::try_unwrap(pending_ack_data).unwrap();
inner_data.update_retransmitted(delay);
self.pending_acks_data
.insert(frag_id, (Arc::new(inner_data), queue_key));
} else {
debug!("Tried to UPDATE TIMER on pending ack that is already gone! - {frag_id}");
debug!(
"Tried to UPDATE TIMER on pending ack that is already gone! - {}",
frag_id
);
}
}
// note: when the entry expires it's automatically removed from pending_acks_timers
#[allow(clippy::panic)]
fn handle_expired_ack_timer(&mut self, expired_ack: Expired<FragmentIdentifier>) {
let frag_id = expired_ack.into_inner();
@@ -222,9 +236,14 @@ impl ActionController {
// downgrading an arc and then upgrading vs cloning is difference of 30ns vs 15ns
// so it's literally a NO difference while it might prevent us from unnecessarily
// resending data (in maybe 1 in 1 million cases, but it's something)
let _ = self
if let Err(err) = self
.retransmission_sender
.unbounded_send(Arc::downgrade(pending_ack_data));
.unbounded_send(Arc::downgrade(pending_ack_data))
{
if !self.task_client.is_shutdown_poll() {
log::error!("Failed to send pending ack for retransmission: {err}");
}
}
} else {
// this shouldn't cause any issues but shouldn't have happened to begin with!
error!("An already removed pending ack has expired")
@@ -242,20 +261,15 @@ impl ActionController {
}
}
pub(crate) async fn run(&mut self, shutdown_token: ShutdownToken) {
pub(super) async fn run(&mut self) {
debug!("Started ActionController with graceful shutdown support");
loop {
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("ActionController: Received shutdown");
break;
}
action = self.incoming_actions.next() => match action {
Some(action) => self.process_action(action),
None => {
tracing::trace!(
log::trace!(
"ActionController: Stopping since incoming actions channel closed"
);
break;
@@ -264,12 +278,17 @@ impl ActionController {
expired_ack = self.pending_acks_timers.next() => match expired_ack {
Some(expired_ack) => self.handle_expired_ack_timer(expired_ack),
None => {
tracing::trace!("ActionController: Stopping since ack channel closed");
log::trace!("ActionController: Stopping since ack channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("ActionController: Received shutdown");
break;
}
}
}
tracing::debug!("ActionController: Exiting");
self.task_client.recv_timeout().await;
log::debug!("ActionController: Exiting");
}
}
@@ -5,25 +5,26 @@ use crate::client::inbound_messages::{InputMessage, InputMessageReceiver};
use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::real_messages_control::real_traffic_stream::RealMessage;
use crate::client::replies::reply_controller::ReplyControllerSender;
use log::*;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::forwarding::packet::MixPacket;
use nym_sphinx::params::PacketType;
use nym_task::connections::TransmissionLane;
use nym_task::ShutdownToken;
use nym_task::TaskClient;
use rand::{CryptoRng, Rng};
use tracing::*;
/// Module responsible for dealing with the received messages: splitting them, creating acknowledgements,
/// putting everything into sphinx packets, etc.
/// It also makes an initial sending attempt for said messages.
pub(crate) struct InputMessageListener<R>
pub(super) struct InputMessageListener<R>
where
R: CryptoRng + Rng,
{
input_receiver: InputMessageReceiver,
message_handler: MessageHandler<R>,
reply_controller_sender: ReplyControllerSender,
task_client: TaskClient,
}
impl<R> InputMessageListener<R>
@@ -37,11 +38,13 @@ where
input_receiver: InputMessageReceiver,
message_handler: MessageHandler<R>,
reply_controller_sender: ReplyControllerSender,
task_client: TaskClient,
) -> Self {
InputMessageListener {
input_receiver,
message_handler,
reply_controller_sender,
task_client,
}
}
@@ -65,9 +68,14 @@ where
max_retransmissions: Option<u32>,
) {
// offload reply handling to the dedicated task
let _ =
if let Err(err) =
self.reply_controller_sender
.send_reply(recipient_tag, data, lane, max_retransmissions);
.send_reply(recipient_tag, data, lane, max_retransmissions)
{
if !self.task_client.is_shutdown_poll() {
error!("failed to send a reply - {err}");
}
}
}
async fn handle_plain_message(
@@ -112,7 +120,6 @@ where
}
}
#[allow(clippy::panic)]
async fn on_input_message(&mut self, msg: InputMessage) {
match msg {
InputMessage::Regular {
@@ -206,35 +213,31 @@ where
self.handle_premade_packets(msgs, lane).await
}
// MessageWrappers can't be nested
InputMessage::MessageWrapper { .. } => {
panic!("attempted to use nested MessageWrapper")
}
InputMessage::MessageWrapper { .. } => unimplemented!(),
},
};
}
pub(crate) async fn run(&mut self, shutdown_token: ShutdownToken) {
pub(super) async fn run(&mut self) {
debug!("Started InputMessageListener with graceful shutdown support");
loop {
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("InputMessageListener: Received shutdown");
break;
}
input_msg = self.input_receiver.recv() => match input_msg {
Some(input_msg) => {
self.on_input_message(input_msg).await;
},
None => {
tracing::trace!("InputMessageListener: Stopping since channel closed");
log::trace!("InputMessageListener: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("InputMessageListener: Received shutdown");
}
}
}
tracing::debug!("InputMessageListener: Exiting");
self.task_client.recv_timeout().await;
log::debug!("InputMessageListener: Exiting");
}
}
@@ -10,8 +10,10 @@ use self::{
use crate::client::inbound_messages::InputMessageReceiver;
use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::replies::reply_controller::ReplyControllerSender;
use crate::spawn_future;
use action_controller::AckActionReceiver;
use futures::channel::mpsc;
use log::*;
use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::params::{PacketSize, PacketType};
@@ -22,6 +24,7 @@ use nym_sphinx::{
Delay as SphinxDelay,
};
use nym_statistics_common::clients::ClientStatsSender;
use nym_task::TaskClient;
use rand::{CryptoRng, Rng};
use std::{
sync::{Arc, Weak},
@@ -187,9 +190,6 @@ pub(super) struct Config {
/// Predefined packet size used for the encapsulated messages.
packet_size: PacketSize,
/// Type of packets used for retransmissions
packet_type: PacketType,
}
impl Config {
@@ -197,14 +197,12 @@ impl Config {
maximum_retransmissions: Option<u32>,
ack_wait_addition: Duration,
ack_wait_multiplier: f64,
packet_type: PacketType,
) -> Self {
Config {
maximum_retransmissions,
ack_wait_addition,
ack_wait_multiplier,
packet_size: Default::default(),
packet_type,
}
}
@@ -214,7 +212,7 @@ impl Config {
}
}
pub(crate) struct AcknowledgementController<R>
pub(super) struct AcknowledgementController<R>
where
R: CryptoRng + Rng,
{
@@ -236,6 +234,7 @@ where
message_handler: MessageHandler<R>,
reply_controller_sender: ReplyControllerSender,
stats_tx: ClientStatsSender,
task_client: TaskClient,
) -> Self {
let (retransmission_tx, retransmission_rx) = mpsc::unbounded();
@@ -245,6 +244,7 @@ where
action_config,
retransmission_tx,
connectors.ack_action_receiver,
task_client.fork("action_controller"),
);
// will listen for any acks coming from the network
@@ -253,6 +253,7 @@ where
connectors.ack_receiver,
connectors.ack_action_sender.clone(),
stats_tx,
task_client.fork("acknowledgement_listener"),
);
// will listen for any new messages from the client
@@ -260,6 +261,7 @@ where
connectors.input_receiver,
message_handler.clone(),
reply_controller_sender.clone(),
task_client.fork("input_message_listener"),
);
// will listen for any ack timeouts and trigger retransmission
@@ -269,13 +271,16 @@ where
message_handler,
retransmission_rx,
reply_controller_sender,
config.packet_type,
task_client.fork("retransmission_request_listener"),
);
// will listen for events indicating the packet was sent through the network so that
// the retransmission timer should be started.
let sent_notification_listener =
SentNotificationListener::new(connectors.sent_notifier, connectors.ack_action_sender);
let sent_notification_listener = SentNotificationListener::new(
connectors.sent_notifier,
connectors.ack_action_sender,
task_client.with_suffix("sent_notification_listener"),
);
AcknowledgementController {
acknowledgement_listener,
@@ -286,21 +291,36 @@ where
}
}
pub(crate) fn into_tasks(
self,
) -> (
AcknowledgementListener,
InputMessageListener<R>,
RetransmissionRequestListener<R>,
SentNotificationListener,
ActionController,
) {
(
self.acknowledgement_listener,
self.input_message_listener,
self.retransmission_request_listener,
self.sent_notification_listener,
self.action_controller,
)
pub(super) fn start(self, packet_type: PacketType) {
let mut acknowledgement_listener = self.acknowledgement_listener;
let mut input_message_listener = self.input_message_listener;
let mut retransmission_request_listener = self.retransmission_request_listener;
let mut sent_notification_listener = self.sent_notification_listener;
let mut action_controller = self.action_controller;
spawn_future(async move {
acknowledgement_listener.run().await;
debug!("The acknowledgement listener has finished execution!");
});
spawn_future(async move {
input_message_listener.run().await;
debug!("The input listener has finished execution!");
});
spawn_future(async move {
retransmission_request_listener.run(packet_type).await;
debug!("The retransmission request listener has finished execution!");
});
spawn_future(async move {
sent_notification_listener.run().await;
debug!("The sent notification listener has finished execution!");
});
spawn_future(async move {
action_controller.run().await;
debug!("The controller has finished execution!");
});
}
}
@@ -10,22 +10,22 @@ use crate::client::real_messages_control::message_handler::{MessageHandler, Prep
use crate::client::real_messages_control::real_traffic_stream::RealMessage;
use crate::client::replies::reply_controller::ReplyControllerSender;
use futures::StreamExt;
use log::*;
use nym_sphinx::chunking::fragment::Fragment;
use nym_sphinx::preparer::PreparedFragment;
use nym_sphinx::{addressing::clients::Recipient, params::PacketType};
use nym_task::{connections::TransmissionLane, ShutdownToken};
use nym_task::{connections::TransmissionLane, TaskClient};
use rand::{CryptoRng, Rng};
use std::sync::{Arc, Weak};
use tracing::*;
// responsible for packet retransmission upon fired timer
pub(crate) struct RetransmissionRequestListener<R> {
pub(super) struct RetransmissionRequestListener<R> {
maximum_retransmissions: Option<u32>,
action_sender: AckActionSender,
message_handler: MessageHandler<R>,
request_receiver: RetransmissionRequestReceiver,
reply_controller_sender: ReplyControllerSender,
packet_type: PacketType,
task_client: TaskClient,
}
impl<R> RetransmissionRequestListener<R>
@@ -38,7 +38,7 @@ where
message_handler: MessageHandler<R>,
request_receiver: RetransmissionRequestReceiver,
reply_controller_sender: ReplyControllerSender,
packet_type: PacketType,
task_client: TaskClient,
) -> Self {
RetransmissionRequestListener {
maximum_retransmissions,
@@ -46,7 +46,7 @@ where
message_handler,
request_receiver,
reply_controller_sender,
packet_type,
task_client,
}
}
@@ -67,6 +67,7 @@ where
async fn on_retransmission_request(
&mut self,
weak_timed_out_ack: Weak<PendingAcknowledgement>,
packet_type: PacketType,
) {
let timed_out_ack = match weak_timed_out_ack.upgrade() {
Some(timed_out_ack) => timed_out_ack,
@@ -96,18 +97,22 @@ where
} => {
// if this is retransmission for reply, offload it to the dedicated task
// that deals with all the surbs
let _ = self.reply_controller_sender.send_retransmission_data(
if let Err(err) = self.reply_controller_sender.send_retransmission_data(
*recipient_tag,
weak_timed_out_ack,
*extra_surb_request,
);
) {
if !self.task_client.is_shutdown_poll() {
error!("Failed to send retransmission data to the reply controller: {err}");
}
}
return;
}
PacketDestination::KnownRecipient(recipient) => {
self.prepare_normal_retransmission_chunk(
**recipient,
timed_out_ack.message_chunk.clone(),
self.packet_type,
packet_type,
)
.await
}
@@ -148,9 +153,14 @@ where
// is sent to the `OutQueueControl` and has gone through its internal queue
// with the additional poisson delay.
// And since Actions are executed in order `UpdateTimer` will HAVE TO be executed before `StartTimer`
let _ = self
if let Err(err) = self
.action_sender
.unbounded_send(Action::new_update_pending_ack(frag_id, new_delay));
.unbounded_send(Action::new_update_pending_ack(frag_id, new_delay))
{
if !self.task_client.is_shutdown_poll() {
error!("Failed to send update pending ack action to the controller: {err}");
}
}
// send to `OutQueueControl` to eventually send to the mix network
self.message_handler
@@ -164,26 +174,24 @@ where
.await
}
pub(crate) async fn run(&mut self, shutdown_token: ShutdownToken) {
pub(super) async fn run(&mut self, packet_type: PacketType) {
debug!("Started RetransmissionRequestListener with graceful shutdown support");
loop {
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("RetransmissionRequestListener: Received shutdown");
break;
}
timed_out_ack = self.request_receiver.next() => match timed_out_ack {
Some(timed_out_ack) => self.on_retransmission_request(timed_out_ack).await,
Some(timed_out_ack) => self.on_retransmission_request(timed_out_ack, packet_type).await,
None => {
tracing::trace!("RetransmissionRequestListener: Stopping since channel closed");
log::trace!("RetransmissionRequestListener: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("RetransmissionRequestListener: Received shutdown");
}
}
}
tracing::debug!("RetransmissionRequestListener: Exiting");
self.task_client.recv_timeout().await;
log::debug!("RetransmissionRequestListener: Exiting");
}
}
@@ -4,26 +4,30 @@
use super::action_controller::{AckActionSender, Action};
use super::SentPacketNotificationReceiver;
use futures::StreamExt;
use log::*;
use nym_sphinx::chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID};
use tracing::*;
use nym_task::TaskClient;
/// Module responsible for starting up retransmission timers.
/// It is required because when we send our packet to the `real traffic stream` controlled
/// by a poisson timer, there's no guarantee the message will be sent immediately, so we might
/// accidentally fire retransmission way quicker than we should have.
pub(crate) struct SentNotificationListener {
pub(super) struct SentNotificationListener {
sent_notifier: SentPacketNotificationReceiver,
action_sender: AckActionSender,
task_client: TaskClient,
}
impl SentNotificationListener {
pub(super) fn new(
sent_notifier: SentPacketNotificationReceiver,
action_sender: AckActionSender,
task_client: TaskClient,
) -> Self {
SentNotificationListener {
sent_notifier,
action_sender,
task_client,
}
}
@@ -32,18 +36,37 @@ impl SentNotificationListener {
trace!("sent off a cover message - no need to start retransmission timer!");
return;
}
let _ = self
if let Err(err) = self
.action_sender
.unbounded_send(Action::new_start_timer(frag_id));
.unbounded_send(Action::new_start_timer(frag_id))
{
if !self.task_client.is_shutdown_poll() {
error!("Failed to send start timer action to action controller: {err}");
}
}
}
pub(crate) async fn run(&mut self) {
pub(super) async fn run(&mut self) {
debug!("Started SentNotificationListener with graceful shutdown support");
while let Some(frag_id) = self.sent_notifier.next().await {
self.on_sent_message(frag_id).await;
while !self.task_client.is_shutdown() {
tokio::select! {
frag_id = self.sent_notifier.next() => match frag_id {
Some(frag_id) => {
self.on_sent_message(frag_id).await;
}
None => {
log::trace!("SentNotificationListener: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("SentNotificationListener: Received shutdown");
break;
}
}
}
tracing::debug!("SentNotificationListener: Exiting");
assert!(self.task_client.is_shutdown_poll());
log::debug!("SentNotificationListener: Exiting");
}
}
@@ -9,18 +9,17 @@ use crate::client::real_messages_control::{AckActionSender, Action};
use crate::client::replies::reply_controller::MaxRetransmissions;
use crate::client::replies::reply_storage::{ReceivedReplySurbsMap, SentReplyKeys, UsedSenderTags};
use crate::client::topology_control::{TopologyAccessor, TopologyReadPermit};
use nym_client_core_surb_storage::RetrievedReplySurb;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::{AnonymousSenderTag, RepliableMessage, ReplyMessage};
use nym_sphinx::anonymous_replies::ReplySurbWithKeyRotation;
use nym_sphinx::anonymous_replies::{ReplySurb, SurbEncryptionKey};
use nym_sphinx::chunking::fragment::{Fragment, FragmentIdentifier};
use nym_sphinx::message::NymMessage;
use nym_sphinx::params::{PacketSize, PacketType};
use nym_sphinx::preparer::{MessagePreparer, PreparedFragment};
use nym_sphinx::Delay;
use nym_task::connections::TransmissionLane;
use nym_task::ShutdownToken;
use nym_task::TaskClient;
use nym_topology::{NymRouteProvider, NymTopologyError};
use rand::{CryptoRng, Rng};
use std::collections::HashMap;
@@ -35,9 +34,6 @@ pub enum PreparationError {
#[error(transparent)]
NymTopologyError(#[from] NymTopologyError),
#[error("message wasn't split into any fragments!")]
EmptyFragments,
#[error("message too long for a single SURB, splitting into {fragments} fragments.")]
MessageTooLongForSingleSurb { fragments: usize },
@@ -48,7 +44,7 @@ pub enum PreparationError {
}
impl PreparationError {
fn return_surbs(self, returned_surbs: Vec<RetrievedReplySurb>) -> SurbWrappedPreparationError {
fn return_surbs(self, returned_surbs: Vec<ReplySurb>) -> SurbWrappedPreparationError {
SurbWrappedPreparationError {
source: self,
returned_surbs: Some(returned_surbs),
@@ -62,7 +58,7 @@ pub struct SurbWrappedPreparationError {
#[source]
source: PreparationError,
returned_surbs: Option<Vec<RetrievedReplySurb>>,
returned_surbs: Option<Vec<ReplySurb>>,
}
impl<T> From<T> for SurbWrappedPreparationError
@@ -84,7 +80,7 @@ impl SurbWrappedPreparationError {
target: &AnonymousSenderTag,
) -> PreparationError {
if let Some(reply_surbs) = self.returned_surbs {
surb_storage.re_insert_reply_surbs(target, reply_surbs)
surb_storage.insert_surbs(target, reply_surbs)
}
self.source
}
@@ -106,9 +102,6 @@ pub(crate) struct Config {
/// will be routed as usual, to the entry gateway, through three mix nodes, egressing
/// through the exit gateway. If mix hops are disabled, traffic will be routed directly
/// from the entry gateway to the exit gateway, bypassing the mix nodes.
///
/// This overrides the `use_legacy_sphinx_format` setting as reduced mix hops
/// requires use of the updated SURB packet format.
disable_mix_hops: bool,
/// Average delay a data packet is going to get delay at a single mixnode.
@@ -163,12 +156,8 @@ impl Config {
}
/// Configure whether messages senders using this config should use mix hops or not when sending messages.
///
/// This overrides the `use_legacy_sphinx_format` setting as disabled mix hops
/// requires use of the updated SURB packet format.
pub fn disable_mix_hops(mut self, disable_mix_hops: bool) -> Self {
self.disable_mix_hops = disable_mix_hops;
self.use_legacy_sphinx_format = false;
self
}
}
@@ -189,7 +178,7 @@ pub(crate) struct MessageHandler<R> {
topology_access: TopologyAccessor,
reply_key_storage: SentReplyKeys,
tag_storage: UsedSenderTags,
shutdown_token: ShutdownToken,
task_client: TaskClient,
}
impl<R> MessageHandler<R>
@@ -205,7 +194,7 @@ where
topology_access: TopologyAccessor,
reply_key_storage: SentReplyKeys,
tag_storage: UsedSenderTags,
shutdown_token: ShutdownToken,
task_client: TaskClient,
) -> Self
where
R: Copy,
@@ -228,14 +217,10 @@ where
topology_access,
reply_key_storage,
tag_storage,
shutdown_token,
task_client,
}
}
pub(crate) fn topology_access_handle(&self) -> &TopologyAccessor {
&self.topology_access
}
fn get_or_create_sender_tag(&mut self, recipient: &Recipient) -> AnonymousSenderTag {
if let Some(existing) = self.tag_storage.try_get_existing(recipient) {
trace!("we already had sender tag for {recipient}");
@@ -283,10 +268,10 @@ where
}
}
async fn generate_reply_surbs(
async fn generate_reply_surbs_with_keys(
&mut self,
amount: usize,
) -> Result<Vec<ReplySurbWithKeyRotation>, PreparationError> {
) -> Result<(Vec<ReplySurb>, Vec<SurbEncryptionKey>), PreparationError> {
let topology_permit = self.topology_access.get_read_permit().await;
let topology = self.get_topology(&topology_permit)?;
@@ -296,14 +281,19 @@ where
topology,
)?;
Ok(reply_surbs)
let reply_keys = reply_surbs
.iter()
.map(|s| *s.encryption_key())
.collect::<Vec<_>>();
Ok((reply_surbs, reply_keys))
}
pub(crate) async fn try_send_single_surb_message(
&mut self,
target: AnonymousSenderTag,
message: ReplyMessage,
reply_surb: RetrievedReplySurb,
reply_surb: ReplySurb,
is_extra_surb_request: bool,
) -> Result<(), SurbWrappedPreparationError> {
let msg = NymMessage::new_reply(message);
@@ -323,16 +313,6 @@ where
});
}
if fragment.is_empty() {
error!("CRITICAL FAILURE: our split message didn't result in any sendable fragments");
return Err(SurbWrappedPreparationError {
source: PreparationError::EmptyFragments,
returned_surbs: Some(vec![reply_surb]),
});
}
// SAFETY: we just checked we have one fragment
#[allow(clippy::unwrap_used)]
let chunk = fragment.pop().unwrap();
let chunk_clone = chunk.clone();
let prepared_fragment = self
@@ -344,10 +324,7 @@ where
Some(chunk.fragment_identifier()),
);
let delay = prepared_fragment.total_delay;
// we have to set a maximum number of retransmissions in case we fail to retrieve
// surbs for a long period of time; we don't want to be stuck constantly resending the data
let max_retransmissions = Some(10);
let max_retransmissions = None;
let pending_ack = PendingAcknowledgement::new_anonymous(
chunk,
delay,
@@ -370,7 +347,7 @@ where
pub(crate) async fn try_request_additional_reply_surbs(
&mut self,
from: AnonymousSenderTag,
reply_surb: RetrievedReplySurb,
reply_surb: ReplySurb,
amount: u32,
) -> Result<(), SurbWrappedPreparationError> {
debug!("requesting {amount} reply SURBs from {from}");
@@ -410,9 +387,11 @@ where
&mut self,
target: AnonymousSenderTag,
fragments: Vec<FragmentWithMaxRetransmissions>,
reply_surbs: impl IntoIterator<Item = RetrievedReplySurb>,
reply_surbs: Vec<ReplySurb>,
lane: TransmissionLane,
) -> Result<(), SurbWrappedPreparationError> {
// TODO: technically this is performing an unnecessary cloning, but in the grand scheme of things
// is it really that bad?
self.try_send_reply_chunks(
target,
fragments.into_iter().map(|f| (lane, f)).collect(),
@@ -425,7 +404,7 @@ where
&mut self,
target: AnonymousSenderTag,
fragments: Vec<(TransmissionLane, FragmentWithMaxRetransmissions)>,
reply_surbs: impl IntoIterator<Item = RetrievedReplySurb>,
reply_surbs: Vec<ReplySurb>,
) -> Result<(), SurbWrappedPreparationError> {
let prepared_fragments = self
.prepare_reply_chunks_for_sending(
@@ -548,7 +527,6 @@ where
pending_acks.push(pending_ack);
}
drop(topology_permit);
self.insert_pending_acks(pending_acks);
self.forward_messages(real_messages, lane).await;
@@ -563,12 +541,8 @@ where
) -> Result<(), PreparationError> {
debug!("Sending additional reply SURBs with packet type {packet_type}");
let sender_tag = self.get_or_create_sender_tag(&recipient);
let reply_surbs = self.generate_reply_surbs(amount as usize).await?;
let reply_keys = reply_surbs
.iter()
.map(|s| *s.encryption_key())
.collect::<Vec<_>>();
let (reply_surbs, reply_keys) =
self.generate_reply_surbs_with_keys(amount as usize).await?;
let message = NymMessage::new_repliable(RepliableMessage::new_additional_surbs(
self.config.use_legacy_sphinx_format,
@@ -588,7 +562,7 @@ where
)
.await?;
tracing::trace!("storing {} reply keys", reply_keys.len());
log::trace!("storing {} reply keys", reply_keys.len());
self.reply_key_storage.insert_multiple(reply_keys);
Ok(())
@@ -605,12 +579,9 @@ where
) -> Result<(), SurbWrappedPreparationError> {
debug!("Sending message with reply SURBs with packet type {packet_type}");
let sender_tag = self.get_or_create_sender_tag(&recipient);
let reply_surbs = self.generate_reply_surbs(num_reply_surbs as usize).await?;
let reply_keys = reply_surbs
.iter()
.map(|s| *s.encryption_key())
.collect::<Vec<_>>();
let (reply_surbs, reply_keys) = self
.generate_reply_surbs_with_keys(num_reply_surbs as usize)
.await?;
let message = NymMessage::new_repliable(RepliableMessage::new_data(
self.config.use_legacy_sphinx_format,
@@ -628,7 +599,7 @@ where
)
.await?;
tracing::trace!("storing {} reply keys", reply_keys.len());
log::trace!("storing {} reply keys", reply_keys.len());
self.reply_key_storage.insert_multiple(reply_keys);
Ok(())
@@ -658,12 +629,20 @@ where
pub(crate) async fn prepare_reply_chunks_for_sending(
&mut self,
fragments: Vec<Fragment>,
reply_surbs: impl IntoIterator<Item = RetrievedReplySurb>,
reply_surbs: Vec<ReplySurb>,
) -> Result<Vec<PreparedFragment>, SurbWrappedPreparationError> {
debug_assert_eq!(
fragments.len(),
reply_surbs.len(),
"attempted to send {} fragments with {} reply surbs",
fragments.len(),
reply_surbs.len()
);
let topology_permit = self.topology_access.get_read_permit().await;
let topology = match self.get_topology(&topology_permit) {
Ok(topology) => topology,
Err(err) => return Err(err.return_surbs(reply_surbs.into_iter().collect())),
Err(err) => return Err(err.return_surbs(reply_surbs)),
};
Ok(fragments
@@ -671,13 +650,12 @@ where
.zip(reply_surbs.into_iter())
.map(|(fragment, reply_surb)| {
// unwrap here is fine as we know we have a valid topology
#[allow(clippy::unwrap_used)]
self.message_preparer
.prepare_reply_chunk_for_sending(
fragment,
topology,
&self.config.ack_key,
reply_surb.into(),
reply_surb,
PacketType::Mix,
)
.unwrap()
@@ -687,7 +665,7 @@ where
pub(crate) async fn try_prepare_single_reply_chunk_for_sending(
&mut self,
reply_surb: RetrievedReplySurb,
reply_surb: ReplySurb,
chunk: Fragment,
) -> Result<PreparedFragment, SurbWrappedPreparationError> {
let topology_permit = self.topology_access.get_read_permit().await;
@@ -700,7 +678,7 @@ where
chunk,
topology,
&self.config.ack_key,
reply_surb.into(),
reply_surb,
PacketType::Mix,
)?;
@@ -712,7 +690,7 @@ where
.action_sender
.unbounded_send(Action::UpdatePendingAck(id, new_delay))
{
if !self.shutdown_token.is_cancelled() {
if !self.task_client.is_shutdown_poll() {
error!("Failed to send update action to the controller: {err}");
}
}
@@ -723,7 +701,7 @@ where
.action_sender
.unbounded_send(Action::new_insert(pending_acks))
{
if !self.shutdown_token.is_cancelled() {
if !self.task_client.is_shutdown_poll() {
error!("Failed to send insert action to the controller: {err}");
}
}
@@ -731,21 +709,17 @@ where
// tells real message sender (with the poisson timer) to send this to the mix network
pub(crate) async fn forward_messages(
&mut self,
&self,
messages: Vec<RealMessage>,
transmission_lane: TransmissionLane,
) {
tokio::select! {
biased;
_ = self.shutdown_token.cancelled() => {
trace!("received shutdown while attempting to forward mixnet messages");
}
sending_res = self.real_message_sender.send((messages, transmission_lane)) => {
if sending_res.is_err() {
error!(
"failed to forward mixnet messages due to closed channel (outside of shutdown!)"
);
}
if let Err(err) = self
.real_message_sender
.send((messages, transmission_lane))
.await
{
if !self.task_client.is_shutdown_poll() {
error!("Failed to forward messages to the real message sender: {err}");
}
}
}
@@ -14,23 +14,27 @@ use crate::client::replies::reply_controller::{
ReplyController, ReplyControllerReceiver, ReplyControllerSender,
};
use crate::client::replies::reply_storage::CombinedReplyStorage;
use crate::client::{
inbound_messages::InputMessageReceiver, mix_traffic::BatchMixMessageSender,
real_messages_control::acknowledgement_control::AcknowledgementControllerConnectors,
topology_control::TopologyAccessor,
};
use crate::config;
use crate::{
client::{
inbound_messages::InputMessageReceiver, mix_traffic::BatchMixMessageSender,
real_messages_control::acknowledgement_control::AcknowledgementControllerConnectors,
topology_control::TopologyAccessor,
},
spawn_future,
};
use futures::channel::mpsc;
use log::*;
use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::params::PacketType;
use nym_statistics_common::clients::ClientStatsSender;
use nym_task::connections::{ConnectionCommandReceiver, LaneQueueLengths};
use nym_task::ShutdownToken;
use nym_task::TaskClient;
use rand::{rngs::OsRng, CryptoRng, Rng};
use std::sync::Arc;
use crate::client::replies::reply_controller::key_rotation_helpers::KeyRotationConfig;
pub(crate) use acknowledgement_control::{AckActionSender, Action};
pub(crate) mod acknowledgement_control;
@@ -64,7 +68,6 @@ impl<'a> From<&'a Config> for acknowledgement_control::Config {
cfg.traffic.maximum_number_of_retransmissions,
cfg.acks.ack_wait_addition,
cfg.acks.ack_wait_multiplier,
cfg.traffic.packet_type,
)
.with_custom_packet_size(cfg.traffic.primary_packet_size)
}
@@ -82,6 +85,12 @@ impl<'a> From<&'a Config> for real_traffic_stream::Config {
}
}
impl<'a> From<&'a Config> for reply_controller::Config {
fn from(cfg: &'a Config) -> Self {
reply_controller::Config::new(cfg.reply_surbs)
}
}
impl<'a> From<&'a Config> for message_handler::Config {
fn from(cfg: &'a Config) -> Self {
message_handler::Config::new(
@@ -130,7 +139,6 @@ impl RealMessagesController<OsRng> {
#[allow(clippy::too_many_arguments)]
pub(crate) fn new(
config: Config,
key_rotation_config: KeyRotationConfig,
ack_receiver: AcknowledgementReceiver,
input_receiver: InputMessageReceiver,
mix_sender: BatchMixMessageSender,
@@ -142,7 +150,7 @@ impl RealMessagesController<OsRng> {
lane_queue_lengths: LaneQueueLengths,
client_connection_rx: ConnectionCommandReceiver,
stats_tx: ClientStatsSender,
shutdown_token: ShutdownToken,
task_client: TaskClient,
) -> Self {
let rng = OsRng;
@@ -161,8 +169,7 @@ impl RealMessagesController<OsRng> {
// create all configs for the components
let ack_control_config = (&config).into();
let out_queue_config = (&config).into();
let reply_controller_config =
reply_controller::Config::new(config.reply_surbs, key_rotation_config);
let reply_controller_config = (&config).into();
let message_handler_config = (&config).into();
// create the actual components
@@ -174,7 +181,7 @@ impl RealMessagesController<OsRng> {
topology_access.clone(),
reply_storage.key_storage(),
reply_storage.tags_storage(),
shutdown_token.clone(),
task_client.fork("message_handler"),
);
let ack_control = AcknowledgementController::new(
@@ -184,6 +191,7 @@ impl RealMessagesController<OsRng> {
message_handler.clone(),
reply_controller_sender,
stats_tx.clone(),
task_client.fork("ack_control"),
);
let reply_control = ReplyController::new(
@@ -191,6 +199,7 @@ impl RealMessagesController<OsRng> {
message_handler,
reply_storage,
reply_controller_receiver,
task_client.fork("reply_controller"),
);
let out_queue_control = OutQueueControl::new(
@@ -203,7 +212,7 @@ impl RealMessagesController<OsRng> {
lane_queue_lengths,
client_connection_rx,
stats_tx,
shutdown_token.clone(),
task_client.with_suffix("out_queue_control"),
);
RealMessagesController {
@@ -213,13 +222,20 @@ impl RealMessagesController<OsRng> {
}
}
pub fn into_tasks(
self,
) -> (
OutQueueControl<OsRng>,
ReplyController<OsRng>,
AcknowledgementController<OsRng>,
) {
(self.out_queue_control, self.reply_control, self.ack_control)
pub fn start(self, packet_type: PacketType) {
let mut out_queue_control = self.out_queue_control;
let ack_control = self.ack_control;
let mut reply_control = self.reply_control;
spawn_future(async move {
out_queue_control.run().await;
debug!("The out queue controller has finished execution!");
});
spawn_future(async move {
reply_control.run().await;
debug!("The reply controller has finished execution!");
});
ack_control.start(packet_type);
}
}
@@ -9,6 +9,7 @@ use crate::client::transmission_buffer::TransmissionBuffer;
use crate::config;
use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt};
use log::*;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::chunking::fragment::FragmentIdentifier;
@@ -21,19 +22,18 @@ use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, C
use nym_task::connections::{
ConnectionCommand, ConnectionCommandReceiver, ConnectionId, LaneQueueLengths, TransmissionLane,
};
use nym_task::ShutdownToken;
use nym_task::TaskClient;
use rand::{CryptoRng, Rng};
use std::pin::Pin;
use std::sync::Arc;
use std::time::Duration;
use tracing::*;
#[cfg(not(target_arch = "wasm32"))]
use tokio::time::{sleep, Sleep};
// use wasm_utils::console_log;
#[cfg(target_arch = "wasm32")]
use wasmtimer::tokio::{sleep, Sleep};
mod sending_delay_controller;
/// Configurable parameters of the `OutQueueControl`
@@ -119,7 +119,7 @@ where
/// Channel used for sending metrics events (specifically `PacketStatistics` events) to the metrics tracker.
stats_tx: ClientStatsSender,
shutdown_token: ShutdownToken,
task_client: TaskClient,
}
#[derive(Debug)]
@@ -179,7 +179,7 @@ where
lane_queue_lengths: LaneQueueLengths,
client_connection_rx: ConnectionCommandReceiver,
stats_tx: ClientStatsSender,
shutdown_token: ShutdownToken,
task_client: TaskClient,
) -> Self {
OutQueueControl {
config,
@@ -194,7 +194,7 @@ where
client_connection_rx,
lane_queue_lengths,
stats_tx,
shutdown_token,
task_client,
}
}
@@ -202,7 +202,7 @@ where
// well technically the message was not sent just yet, but now it's up to internal
// queues and client load rather than the required delay. So realistically we can treat
// whatever is about to happen as negligible additional delay.
trace!("{frag_id} is about to get sent to the mixnet");
trace!("{} is about to get sent to the mixnet", frag_id);
if let Err(err) = self.sent_notifier.unbounded_send(frag_id) {
error!("Failed to notify about sent message: {err}");
}
@@ -249,8 +249,6 @@ where
}
};
// SAFETY: our topology must be valid at this point
#[allow(clippy::expect_used)]
(
generate_loop_cover_packet(
&mut self.rng,
@@ -280,35 +278,17 @@ where
}
};
let sending_res = tokio::select! {
biased;
_ = self.shutdown_token.cancelled() => {
trace!("received shutdown signal while attempting to send mix message");
return
}
sending_res = self.mix_tx.send(vec![next_message]) => {
sending_res
}
};
match sending_res {
Err(_) => {
if !self.shutdown_token.is_cancelled() {
tracing::error!(
"failed to send mixnet packet due to closed channel (outside of shutdown!)"
);
}
// Early return to avoid further processing when channel is closed
return;
}
Ok(_) => {
let event = if fragment_id.is_some() {
PacketStatisticsEvent::RealPacketSent(packet_size)
} else {
PacketStatisticsEvent::CoverPacketSent(packet_size)
};
self.stats_tx.report(event.into());
if let Err(err) = self.mix_tx.send(vec![next_message]).await {
if !self.task_client.is_shutdown_poll() {
log::error!("Failed to send: {err}");
}
} else {
let event = if fragment_id.is_some() {
PacketStatisticsEvent::RealPacketSent(packet_size)
} else {
PacketStatisticsEvent::CoverPacketSent(packet_size)
};
self.stats_tx.report(event.into());
}
// notify ack controller about sending our message only after we actually managed to push it
@@ -327,19 +307,13 @@ where
// ready and hence was immediately re-scheduled causing other tasks to be starved;
// yield makes it go back the scheduling queue regardless of its value availability
// TODO: temporary and BAD workaround for wasm (we should find a way to yield here in wasm)
#[cfg(not(target_arch = "wasm32"))]
{
tokio::task::yield_now().await;
}
#[cfg(target_arch = "wasm32")]
{
tokio_with_wasm::task::yield_now().await;
}
tokio::task::yield_now().await;
}
fn on_close_connection(&mut self, connection_id: ConnectionId) {
tracing::debug!("Removing lane for connection: {connection_id}");
log::debug!("Removing lane for connection: {connection_id}");
self.transmission_buffer
.remove(&TransmissionLane::ConnectionId(connection_id));
}
@@ -351,7 +325,7 @@ where
fn adjust_current_average_message_sending_delay(&mut self) {
let used_slots = self.mix_tx.max_capacity() - self.mix_tx.capacity();
tracing::trace!(
log::trace!(
"used_slots: {used_slots}, current_multiplier: {}",
self.sending_delay_controller.current_multiplier()
);
@@ -360,7 +334,7 @@ where
.sending_delay_controller
.is_backpressure_currently_detected(used_slots)
{
tracing::trace!("Backpressure detected");
log::trace!("Backpressure detected");
self.sending_delay_controller.record_backpressure_detected();
}
@@ -462,11 +436,9 @@ where
Poll::Ready(None) => Poll::Ready(None),
Poll::Ready(Some((real_messages, conn_id))) => {
tracing::trace!("handling real_messages: size: {}", real_messages.len());
log::trace!("handling real_messages: size: {}", real_messages.len());
self.transmission_buffer.store(&conn_id, real_messages);
// SAFETY: we just stored the message
#[allow(clippy::expect_used)]
let real_next = self.pop_next_message().expect("Just stored one");
Poll::Ready(Some(StreamMessage::Real(Box::new(real_next))))
@@ -511,12 +483,10 @@ where
Poll::Ready(None) => Poll::Ready(None),
Poll::Ready(Some((real_messages, conn_id))) => {
tracing::trace!("handling real_messages: size: {}", real_messages.len());
log::trace!("handling real_messages: size: {}", real_messages.len());
// First store what we got for the given connection id
self.transmission_buffer.store(&conn_id, real_messages);
// SAFETY: we just stored the message
#[allow(clippy::expect_used)]
let real_next = self.pop_next_message().expect("we just added one");
Poll::Ready(Some(StreamMessage::Real(Box::new(real_next))))
@@ -544,7 +514,9 @@ where
}
#[cfg(not(target_arch = "wasm32"))]
fn log_status(&self) {
fn log_status(&self, shutdown: &mut TaskClient) {
use crate::error::ClientCoreStatusMessage;
let packets = self.transmission_buffer.total_size();
let lanes = self.transmission_buffer.lanes();
let mult = self.sending_delay_controller.current_multiplier();
@@ -566,70 +538,69 @@ where
};
if packets > 1000 {
tracing::warn!("{status_str}");
log::warn!("{status_str}");
} else if packets > 0 {
tracing::info!("{status_str}");
log::info!("{status_str}");
} else {
tracing::debug!("{status_str}");
log::debug!("{status_str}");
}
// leave the code commented in case somebody wanted to restore this logic with a different channel
// // Send status message to whoever is listening (possibly UI)
// if mult == self.sending_delay_controller.max_multiplier() {
// shutdown.send_status_msg(Box::new(ClientCoreStatusMessage::GatewayIsVerySlow));
// } else if mult > self.sending_delay_controller.min_multiplier() {
// shutdown.send_status_msg(Box::new(ClientCoreStatusMessage::GatewayIsSlow));
// }
// Send status message to whoever is listening (possibly UI)
if mult == self.sending_delay_controller.max_multiplier() {
shutdown.send_status_msg(Box::new(ClientCoreStatusMessage::GatewayIsVerySlow));
} else if mult > self.sending_delay_controller.min_multiplier() {
shutdown.send_status_msg(Box::new(ClientCoreStatusMessage::GatewayIsSlow));
}
}
pub(crate) async fn run(&mut self) {
pub(super) async fn run(&mut self) {
debug!("Started OutQueueControl with graceful shutdown support");
// avoid borrow on self
let shutdown_token = self.shutdown_token.clone();
let mut shutdown = self.task_client.fork("select");
#[cfg(not(target_arch = "wasm32"))]
{
let mut status_timer = tokio::time::interval(Duration::from_secs(5));
loop {
while !shutdown.is_shutdown() {
tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("OutQueueControl: Received shutdown");
_ = shutdown.recv() => {
log::trace!("OutQueueControl: Received shutdown");
break;
}
_ = status_timer.tick() => {
self.log_status();
self.log_status(&mut shutdown);
}
next_message = self.next() => if let Some(next_message) = next_message {
self.on_message(next_message).await;
} else {
tracing::trace!("OutQueueControl: Stopping since channel closed");
log::trace!("OutQueueControl: Stopping since channel closed");
break;
}
}
}
shutdown.recv_timeout().await;
}
#[cfg(target_arch = "wasm32")]
{
loop {
while !shutdown.is_shutdown() {
tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("OutQueueControl: Received shutdown");
break;
_ = shutdown.recv() => {
log::trace!("OutQueueControl: Received shutdown");
}
next_message = self.next() => if let Some(next_message) = next_message {
self.on_message(next_message).await;
} else {
tracing::trace!("OutQueueControl: Stopping since channel closed");
log::trace!("OutQueueControl: Stopping since channel closed");
break;
}
}
}
}
tracing::debug!("OutQueueControl: Exiting");
log::debug!("OutQueueControl: Exiting");
}
}
@@ -83,13 +83,11 @@ impl SendingDelayController {
self.current_multiplier
}
#[allow(dead_code)]
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn min_multiplier(&self) -> u32 {
self.lower_bound
}
#[allow(dead_code)]
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn max_multiplier(&self) -> u32 {
self.upper_bound
@@ -100,12 +98,12 @@ impl SendingDelayController {
self.current_multiplier =
(self.current_multiplier + 1).clamp(self.lower_bound, self.upper_bound);
self.time_when_changed = get_time_now();
tracing::debug!(
log::debug!(
"Increasing sending delay multiplier to: {}",
self.current_multiplier
);
} else {
tracing::warn!("Trying to increase delay multipler higher than allowed");
log::warn!("Trying to increase delay multipler higher than allowed");
}
}
@@ -114,7 +112,7 @@ impl SendingDelayController {
self.current_multiplier =
(self.current_multiplier - 1).clamp(self.lower_bound, self.upper_bound);
self.time_when_changed = get_time_now();
tracing::debug!(
log::debug!(
"Decreasing sending delay multiplier to: {}",
self.current_multiplier
);
@@ -166,11 +164,11 @@ impl SendingDelayController {
self.current_multiplier()
);
if self.current_multiplier() > 0 {
tracing::debug!("{status_str}");
log::debug!("{}", status_str);
} else if self.current_multiplier() > 1 {
tracing::info!("{status_str}");
log::info!("{}", status_str);
} else if self.current_multiplier() > 2 {
tracing::warn!("{status_str}");
log::warn!("{}", status_str);
}
self.time_when_logged_about_elevated_multiplier = now;
}
@@ -1,13 +1,14 @@
// Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::helpers::get_time_now;
use crate::client::replies::{
reply_controller::ReplyControllerSender, reply_storage::SentReplyKeys,
};
use crate::spawn_future;
use futures::channel::mpsc;
use futures::lock::Mutex;
use futures::StreamExt;
use log::*;
use nym_crypto::asymmetric::x25519;
use nym_crypto::Digest;
use nym_gateway_client::MixnetMessageReceiver;
@@ -19,11 +20,10 @@ use nym_sphinx::message::{NymMessage, PlainMessage};
use nym_sphinx::params::ReplySurbKeyDigestAlgorithm;
use nym_sphinx::receiver::{MessageReceiver, MessageRecoveryError, ReconstructedMessage};
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use nym_task::ShutdownToken;
use nym_task::TaskClient;
use std::collections::HashSet;
use std::sync::Arc;
use std::time::Duration;
use tracing::*;
use std::time::{Duration, Instant};
// The interval at which we check for stale buffers
const STALE_BUFFER_CHECK_INTERVAL: Duration = Duration::from_secs(10);
@@ -54,7 +54,7 @@ struct ReceivedMessagesBufferInner<R: MessageReceiver> {
stats_tx: ClientStatsSender,
// Periodically check for stale buffers to clean up
last_stale_check: crate::client::helpers::Instant,
last_stale_check: Instant,
}
impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
@@ -154,7 +154,7 @@ impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
}
fn cleanup_stale_buffers(&mut self) {
let now = get_time_now();
let now = Instant::now();
if now - self.last_stale_check > STALE_BUFFER_CHECK_INTERVAL {
self.last_stale_check = now;
self.message_receiver
@@ -171,7 +171,7 @@ struct ReceivedMessagesBuffer<R: MessageReceiver> {
inner: Arc<Mutex<ReceivedMessagesBufferInner<R>>>,
reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender,
shutdown_token: ShutdownToken,
task_client: TaskClient,
}
impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
@@ -180,7 +180,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender,
stats_tx: ClientStatsSender,
shutdown_token: ShutdownToken,
task_client: TaskClient,
) -> Self {
ReceivedMessagesBuffer {
inner: Arc::new(Mutex::new(ReceivedMessagesBufferInner {
@@ -190,15 +190,14 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
message_sender: None,
recently_reconstructed: HashSet::new(),
stats_tx,
last_stale_check: get_time_now(),
last_stale_check: Instant::now(),
})),
reply_key_storage,
reply_controller_sender,
shutdown_token,
task_client,
}
}
#[allow(clippy::panic)]
async fn disconnect_sender(&mut self) {
let mut guard = self.inner.lock().await;
if guard.message_sender.is_none() {
@@ -209,7 +208,6 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
guard.message_sender = None;
}
#[allow(clippy::panic)]
async fn connect_sender(&mut self, sender: ReconstructedMessagesSender) {
let mut guard = self.inner.lock().await;
if guard.message_sender.is_some() {
@@ -223,7 +221,10 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
let stored_messages = std::mem::take(&mut guard.messages);
if !stored_messages.is_empty() {
if let Err(err) = sender.unbounded_send(stored_messages) {
error!("The sender channel we just received is already invalidated - {err:?}");
error!(
"The sender channel we just received is already invalidated - {:?}",
err
);
// put the values back to the buffer
// the returned error has two fields: err: SendError and val: T,
// where val is the value that was failed to get sent;
@@ -309,15 +310,13 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
}
};
if !reply_surbs.is_empty() {
if let Err(err) = self.reply_controller_sender.send_additional_surbs(
msg.sender_tag,
reply_surbs,
from_surb_request,
) {
if !self.shutdown_token.is_cancelled() {
error!("{err}");
}
if let Err(err) = self.reply_controller_sender.send_additional_surbs(
msg.sender_tag,
reply_surbs,
from_surb_request,
) {
if !self.task_client.is_shutdown_poll() {
error!("{err}");
}
}
}
@@ -338,7 +337,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
.reply_controller_sender
.send_additional_surbs_request(*recipient, amount)
{
if !self.shutdown_token.is_cancelled() {
if !self.task_client.is_shutdown_poll() {
error!("{err}");
}
}
@@ -465,22 +464,22 @@ pub enum ReceivedBufferMessage {
ReceiverDisconnect,
}
pub(crate) struct RequestReceiver<R: MessageReceiver> {
struct RequestReceiver<R: MessageReceiver> {
received_buffer: ReceivedMessagesBuffer<R>,
query_receiver: ReceivedBufferRequestReceiver,
shutdown_token: ShutdownToken,
task_client: TaskClient,
}
impl<R: MessageReceiver> RequestReceiver<R> {
fn new(
received_buffer: ReceivedMessagesBuffer<R>,
query_receiver: ReceivedBufferRequestReceiver,
shutdown_token: ShutdownToken,
task_client: TaskClient,
) -> Self {
RequestReceiver {
received_buffer,
query_receiver,
shutdown_token,
task_client,
}
}
@@ -495,71 +494,67 @@ impl<R: MessageReceiver> RequestReceiver<R> {
}
}
pub(crate) async fn run(&mut self) {
async fn run(&mut self) {
debug!("Started RequestReceiver with graceful shutdown support");
loop {
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = self.shutdown_token.cancelled() => {
tracing::trace!("RequestReceiver: Received shutdown");
break;
_ = self.task_client.recv() => {
log::trace!("RequestReceiver: Received shutdown");
}
request = self.query_receiver.next() => {
if let Some(message) = request {
self.handle_message(message).await
} else {
tracing::trace!("RequestReceiver: Stopping since channel closed");
self.shutdown_token.cancelled().await;
log::trace!("RequestReceiver: Stopping since channel closed");
break;
}
},
}
}
tracing::debug!("RequestReceiver: Exiting");
self.task_client.recv().await;
log::debug!("RequestReceiver: Exiting");
}
}
pub(crate) struct FragmentedMessageReceiver<R: MessageReceiver> {
struct FragmentedMessageReceiver<R: MessageReceiver> {
received_buffer: ReceivedMessagesBuffer<R>,
mixnet_packet_receiver: MixnetMessageReceiver,
shutdown_token: ShutdownToken,
task_client: TaskClient,
}
impl<R: MessageReceiver> FragmentedMessageReceiver<R> {
fn new(
received_buffer: ReceivedMessagesBuffer<R>,
mixnet_packet_receiver: MixnetMessageReceiver,
shutdown_token: ShutdownToken,
task_client: TaskClient,
) -> Self {
FragmentedMessageReceiver {
received_buffer,
mixnet_packet_receiver,
shutdown_token,
task_client,
}
}
pub(crate) async fn run(&mut self) -> Result<(), MessageRecoveryError> {
async fn run(&mut self) -> Result<(), MessageRecoveryError> {
debug!("Started FragmentedMessageReceiver with graceful shutdown support");
loop {
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = self.shutdown_token.cancelled() => {
tracing::trace!("FragmentedMessageReceiver: Received shutdown");
break;
}
new_messages = self.mixnet_packet_receiver.next() => {
if let Some(new_messages) = new_messages {
self.received_buffer.handle_new_received(new_messages).await?;
} else {
tracing::trace!("FragmentedMessageReceiver: Stopping since channel closed");
self.shutdown_token.cancelled().await;
log::trace!("FragmentedMessageReceiver: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv_with_delay() => {
log::trace!("FragmentedMessageReceiver: Received shutdown");
}
}
}
tracing::debug!("FragmentedMessageReceiver: Exiting");
self.task_client.recv_timeout().await;
log::debug!("FragmentedMessageReceiver: Exiting");
Ok(())
}
}
@@ -577,31 +572,42 @@ impl<R: MessageReceiver + Clone + Send + 'static> ReceivedMessagesBufferControll
reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender,
metrics_reporter: ClientStatsSender,
shutdown_token: ShutdownToken,
task_client: TaskClient,
) -> Self {
let received_buffer = ReceivedMessagesBuffer::new(
local_encryption_keypair,
reply_key_storage,
reply_controller_sender,
metrics_reporter,
shutdown_token.clone(),
task_client.fork("received_messages_buffer"),
);
ReceivedMessagesBufferController {
fragmented_message_receiver: FragmentedMessageReceiver::new(
received_buffer.clone(),
mixnet_packet_receiver,
shutdown_token.clone(),
task_client.fork("fragmented_message_receiver"),
),
request_receiver: RequestReceiver::new(
received_buffer,
query_receiver,
shutdown_token.clone(),
task_client.with_suffix("request_receiver"),
),
}
}
pub(crate) fn into_tasks(self) -> (FragmentedMessageReceiver<R>, RequestReceiver<R>) {
(self.fragmented_message_receiver, self.request_receiver)
pub fn start(self) {
let mut fragmented_message_receiver = self.fragmented_message_receiver;
let mut request_receiver = self.request_receiver;
spawn_future(async move {
match fragmented_message_receiver.run().await {
Ok(_) => {}
Err(e) => error!("{e}"),
}
});
spawn_future(async move {
request_receiver.run().await;
});
}
}
@@ -1,169 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use nym_topology::NymTopologyMetadata;
use nym_validator_client::models::{
EpochId, KeyRotationId, KeyRotationInfoResponse, KeyRotationState,
};
use std::time::Duration;
use time::OffsetDateTime;
#[derive(Clone, Copy)]
pub(crate) enum SurbRefreshState {
WaitingForNextRotation { last_known: KeyRotationId },
ScheduledForNextInvocation,
}
#[derive(Clone, Copy)]
pub(crate) struct ReferenceEpoch {
pub(crate) absolute_epoch_id: EpochId,
pub(crate) start_time: OffsetDateTime,
}
#[derive(Clone, Copy)]
pub(crate) struct KeyRotationConfig {
pub(crate) epoch_duration: Duration,
pub(crate) rotation_state: KeyRotationState,
pub(crate) reference_epoch: ReferenceEpoch,
}
impl From<KeyRotationInfoResponse> for KeyRotationConfig {
fn from(value: KeyRotationInfoResponse) -> Self {
KeyRotationConfig {
epoch_duration: value.details.epoch_duration,
rotation_state: value.details.key_rotation_state,
reference_epoch: ReferenceEpoch {
absolute_epoch_id: value.details.current_absolute_epoch_id,
start_time: value.details.current_epoch_start,
},
}
}
}
impl KeyRotationConfig {
pub(crate) fn rotation_lifetime(&self) -> Duration {
(self.rotation_state.validity_epochs + 1) * self.epoch_duration
}
pub(crate) fn key_rotation_id(&self, current_absolute_epoch_id: EpochId) -> KeyRotationId {
self.rotation_state
.key_rotation_id(current_absolute_epoch_id)
}
// this is called with the assumption that now is always > reference epoch start
pub(crate) fn expected_current_epoch_id(&self, now: OffsetDateTime) -> EpochId {
let diff_secs = (now - self.reference_epoch.start_time).as_seconds_f64();
let epochs = (diff_secs / self.epoch_duration.as_secs_f64()).floor() as u32;
self.reference_epoch.absolute_epoch_id + epochs
}
fn initial_rotation_epoch_start(&self) -> OffsetDateTime {
let epochs_diff = self
.reference_epoch
.absolute_epoch_id
.saturating_sub(self.rotation_state.initial_epoch_id);
self.reference_epoch.start_time - epochs_diff * self.epoch_duration
}
pub(crate) fn key_rotation_start(&self, key_rotation_id: KeyRotationId) -> OffsetDateTime {
let rotation_duration = self.rotation_state.validity_epochs * self.epoch_duration;
let initial_start = self.initial_rotation_epoch_start();
// note: key rotation starts from 0
initial_start + rotation_duration * key_rotation_id
}
pub(crate) fn expected_current_key_rotation_id(&self, now: OffsetDateTime) -> KeyRotationId {
let expected_current_epoch = self.expected_current_epoch_id(now);
self.key_rotation_id(expected_current_epoch)
}
pub(crate) fn expected_current_key_rotation_start(
&self,
now: OffsetDateTime,
) -> OffsetDateTime {
let expected_current_key_rotation_id = self.expected_current_key_rotation_id(now);
self.key_rotation_start(expected_current_key_rotation_id)
}
pub(crate) fn epoch_stuck(&self, topology_metadata: NymTopologyMetadata) -> bool {
// add leeway of 2mins each direction since transition is not instantaneous
let lower_bound = topology_metadata.refreshed_at - Duration::from_secs(2);
let upper_bound = topology_metadata.refreshed_at + Duration::from_secs(2);
let expected_epoch_lower = self.expected_current_epoch_id(lower_bound);
let expected_epoch_upper = self.expected_current_epoch_id(upper_bound);
topology_metadata.absolute_epoch_id != expected_epoch_lower
&& topology_metadata.absolute_epoch_id != expected_epoch_upper
}
}
#[cfg(test)]
mod tests {
use super::*;
use time::macros::datetime;
fn mock_config() -> KeyRotationConfig {
KeyRotationConfig {
epoch_duration: Duration::from_secs(60 * 60),
rotation_state: KeyRotationState {
validity_epochs: 10,
initial_epoch_id: 80,
},
reference_epoch: ReferenceEpoch {
absolute_epoch_id: 100,
start_time: datetime!(2025-06-30 12:00:00+00:00),
},
}
}
#[test]
fn expected_current_key_rotation_start() {
// rot0: 80-89
// rot1: 90-99
// rot2: 100-109
// rot3: 110-119
// ... etc
let cfg = mock_config();
assert_eq!(
cfg.initial_rotation_epoch_start(),
datetime!(2025-06-29 16:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 12:00:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 100);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 2);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 12:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 12:30:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 100);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 2);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 12:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 13:01:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 101);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 2);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 12:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 22:02:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 110);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 3);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 22:00:00+00:00)
);
}
}
File diff suppressed because it is too large Load Diff
@@ -1,901 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::real_messages_control::acknowledgement_control::PendingAcknowledgement;
use crate::client::real_messages_control::message_handler::{
FragmentWithMaxRetransmissions, MessageHandler, PreparationError,
};
use crate::client::replies::reply_controller::key_rotation_helpers::SurbRefreshState;
use crate::client::replies::reply_controller::Config;
use crate::client::topology_control::TopologyAccessor;
use crate::client::transmission_buffer::TransmissionBuffer;
use futures::channel::oneshot;
use nym_client_core_surb_storage::{ReceivedReplySurb, ReceivedReplySurbsMap};
use nym_crypto::aes::cipher::crypto_common::rand_core::CryptoRng;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::anonymous_replies::ReplySurbWithKeyRotation;
use nym_sphinx::chunking::fragment::FragmentIdentifier;
use nym_task::connections::{ConnectionId, TransmissionLane};
use nym_topology::NymTopologyMetadata;
use rand::Rng;
use std::cmp::{max, min};
use std::collections::btree_map::Entry;
use std::collections::{BTreeMap, HashMap};
use std::mem;
use std::sync::{Arc, Weak};
use time::OffsetDateTime;
use tracing::{debug, error, info, trace, warn};
struct SenderData {
current_clear_rerequest_counter: usize,
pending_replies: TransmissionBuffer<FragmentWithMaxRetransmissions>,
pending_retransmissions: BTreeMap<FragmentIdentifier, Weak<PendingAcknowledgement>>,
last_request_failure: OffsetDateTime,
}
impl Default for SenderData {
fn default() -> Self {
SenderData {
current_clear_rerequest_counter: 0,
pending_replies: Default::default(),
pending_retransmissions: Default::default(),
last_request_failure: OffsetDateTime::UNIX_EPOCH,
}
}
}
impl SenderData {
fn total_pending(&self) -> usize {
let pending_replies = self.pending_replies.total_size();
let pending_retransmissions = self.pending_retransmissions.len();
let total_pending = pending_retransmissions + pending_replies;
debug!("total queue size: {total_pending} = pending data {pending_replies} + pending retransmission {pending_retransmissions}");
total_pending
}
pub(crate) fn increment_current_clear_rerequest_counter(&mut self) {
self.current_clear_rerequest_counter += 1;
}
pub(crate) fn reset_current_clear_rerequest_counter(&mut self) {
self.current_clear_rerequest_counter = 0;
}
pub(crate) fn reset_last_request_failure(&mut self, now: OffsetDateTime) -> OffsetDateTime {
mem::replace(&mut self.last_request_failure, now)
}
}
/// Reply controller responsible for controlling receiver-related part
/// of replies, such as requesting additional reply SURBs
pub struct ReceiverReplyController<R> {
config: Config,
surb_refresh_state: SurbRefreshState,
topology_access: TopologyAccessor,
surb_senders: HashMap<AnonymousSenderTag, SenderData>,
unavailable: HashMap<AnonymousSenderTag, OffsetDateTime>,
surbs_storage: ReceivedReplySurbsMap,
// TODO: incorporate that field at some point
// and use binomial distribution to determine the expected required number
// of surbs required to send the message through
// expected_reliability: f32,
message_handler: MessageHandler<R>,
}
impl<R> ReceiverReplyController<R>
where
R: CryptoRng + Rng,
{
pub(crate) fn new(
config: Config,
storage: ReceivedReplySurbsMap,
message_handler: MessageHandler<R>,
) -> Self {
let topology_access = message_handler.topology_access_handle().clone();
ReceiverReplyController {
config,
surb_refresh_state: SurbRefreshState::WaitingForNextRotation {
last_known: config
.key_rotation
.expected_current_key_rotation_id(OffsetDateTime::now_utc()),
},
topology_access,
surb_senders: Default::default(),
unavailable: Default::default(),
surbs_storage: storage,
message_handler,
}
}
fn get_or_create_surb_sender(&mut self, tag: &AnonymousSenderTag) -> &mut SenderData {
self.surb_senders.entry(*tag).or_default()
}
async fn current_topology_metadata(&self) -> Option<NymTopologyMetadata> {
self.topology_access.current_metadata().await
}
fn insert_pending_replies<I: IntoIterator<Item = FragmentWithMaxRetransmissions>>(
&mut self,
recipient: &AnonymousSenderTag,
fragments: I,
lane: TransmissionLane,
) {
trace!("buffering pending replies for {recipient}");
self.surb_senders
.entry(*recipient)
.or_default()
.pending_replies
.store(&lane, fragments)
}
fn re_insert_pending_replies(
&mut self,
recipient: &AnonymousSenderTag,
fragments: Vec<(TransmissionLane, FragmentWithMaxRetransmissions)>,
) {
trace!("re-inserting pending replies for {recipient}");
// the buffer should ALWAYS exist at this point, if it doesn't, it's a bug...
self.surb_senders
.entry(*recipient)
.or_default()
.pending_replies
.store_multiple(fragments)
}
fn re_insert_pending_retransmission(
&mut self,
recipient: &AnonymousSenderTag,
data: Vec<Arc<PendingAcknowledgement>>,
) {
trace!("re-inserting pending retransmissions for {recipient}");
// SAFETY: the underlying entry MUST exist as we've just got data from there
// and we hold a mut reference
#[allow(clippy::expect_used)]
let map_entry = &mut self
.surb_senders
.get_mut(recipient)
.expect("our pending retransmission entry is somehow gone!")
.pending_retransmissions;
for pending in data {
// if it's 0, we don't need to do anything - we just got that ack!
if Arc::strong_count(&pending) > 1 {
let id = pending.inner_fragment_identifier();
let downgraded = Arc::downgrade(&pending);
map_entry.insert(id, downgraded);
}
}
}
fn should_request_more_surbs(&self, target: &AnonymousSenderTag) -> bool {
trace!("checking if we should request more surbs from {target}");
let total_queue = self
.surb_senders
.get(target)
.map(|pending| pending.total_pending())
.unwrap_or_default();
// only consider 'fresh' surbs
let available_surbs = self.surbs_storage.available_fresh_surbs(target);
let pending_surbs = self.surbs_storage.pending_reception(target) as usize;
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_surbs_threshold = self.surbs_storage.max_surb_threshold();
let min_surbs_threshold_buffer =
self.config.reply_surbs.minimum_reply_surb_threshold_buffer;
// After clearing the queue, we want to have at least `min_surbs_threshold` surbs available
// and reserved for requesting additional surbs, and in addition to that we also want to
// have `min_surbs_threshold_buffer` surbs available proactively.
let target_surbs_after_clearing_queue = min_surbs_threshold + min_surbs_threshold_buffer;
// Check if we have enough surbs to handle the total queue and maintain minimum thresholds
let total_required_surbs = total_queue + target_surbs_after_clearing_queue;
let total_available_surbs = pending_surbs + available_surbs;
debug!("available surbs: {available_surbs} pending surbs: {pending_surbs} threshold range: {min_surbs_threshold}..+{min_surbs_threshold_buffer}..{max_surbs_threshold}");
// We should request more surbs if:
// 1. We haven't hit the maximum surb threshold, and
// 2. We don't have enough surbs to handle the queue plus minimum thresholds
let is_below_max_threshold = total_available_surbs < max_surbs_threshold;
let is_below_required_surbs = total_available_surbs < total_required_surbs;
is_below_max_threshold && is_below_required_surbs
}
pub(crate) async fn handle_send_reply(
&mut self,
recipient_tag: AnonymousSenderTag,
data: Vec<u8>,
lane: TransmissionLane,
max_retransmissions: Option<u32>,
) {
if !self.surbs_storage.contains_surbs_for(&recipient_tag) {
if self
.unavailable
.insert(recipient_tag, OffsetDateTime::now_utc())
.is_none()
{
// don't report it every single time
warn!("received reply request for {recipient_tag} but we don't have any surbs stored for that recipient!");
} else {
trace!("received reply request for {recipient_tag} but we don't have any surbs stored for that recipient!");
}
return;
}
trace!("handling reply to {recipient_tag}");
let mut fragments = self.message_handler.split_reply_message(data);
let total_size = fragments.len();
trace!("This reply requires {total_size} SURBs");
// for the purposes of sending reply, do allow using possibly stale entries
let available_surbs = self.surbs_storage.available_surbs(&recipient_tag);
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_to_send = if available_surbs > min_surbs_threshold {
min(fragments.len(), available_surbs - min_surbs_threshold)
} else {
0
};
if max_to_send > 0 {
let (surbs, surbs_left) = self
.surbs_storage
.get_reply_surbs(&recipient_tag, max_to_send);
debug!(
"retrieved {} reply surbs. {surbs_left} surbs remaining in storage",
surbs.as_ref().map(|s| s.len()).unwrap_or_default()
);
if let Some(reply_surbs) = surbs {
let to_send = fragments
.drain(..reply_surbs.len())
.map(|f| FragmentWithMaxRetransmissions {
fragment: f,
max_retransmissions,
})
.collect::<Vec<_>>();
if let Err(err) = self
.message_handler
.try_send_reply_chunks_on_lane(
recipient_tag,
to_send.clone(),
reply_surbs,
lane,
)
.await
{
let err = err.return_unused_surbs(&self.surbs_storage, &recipient_tag);
warn!("failed to send reply to {recipient_tag}: {err}");
info!(
"buffering {no_fragments} fragments for {recipient_tag}",
no_fragments = to_send.len()
);
self.insert_pending_replies(&recipient_tag, to_send, lane);
}
}
}
// if there's leftover data we didn't send because we didn't have enough (or any) surbs - buffer it
if !fragments.is_empty() {
// Ideally we should have enough surbs above the minimum threshold to handle sending
// new replies without having to first request more surbs. That's why I'd like to log
// these cases as they might indicate a problem with the surb management.
debug!(
"buffering {no_fragments} fragments for {recipient_tag}",
no_fragments = fragments.len()
);
let fragments: Vec<_> = fragments
.into_iter()
.map(|fragment| FragmentWithMaxRetransmissions {
fragment,
max_retransmissions,
})
.collect();
self.insert_pending_replies(&recipient_tag, fragments, lane);
}
if self.should_request_more_surbs(&recipient_tag) {
self.request_reply_surbs_for_queue_clearing(recipient_tag)
.await;
}
}
async fn request_additional_reply_surbs(
&mut self,
target: AnonymousSenderTag,
amount: u32,
) -> Result<(), PreparationError> {
debug!("requesting {amount} additional reply surbs for {target}");
let (reply_surb, _) = self
.surbs_storage
.get_reply_surb_ignoring_threshold(&target);
let reply_surb = reply_surb.ok_or(PreparationError::NotEnoughSurbs {
available: 0,
required: 1,
})?;
if let Err(err) = self
.message_handler
.try_request_additional_reply_surbs(target, reply_surb, amount)
.await
{
let err = err.return_unused_surbs(&self.surbs_storage, &target);
warn!("failed to request additional surbs from {target}: {err}",);
return Err(err);
} else {
self.surbs_storage
.increment_pending_reception(&target, amount);
}
Ok(())
}
async fn try_clear_pending_retransmission(&mut self, target: AnonymousSenderTag) {
trace!("trying to clear pending retransmission queue");
let available_surbs = self.surbs_storage.available_surbs(&target);
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_to_clear = if available_surbs > min_surbs_threshold {
available_surbs - min_surbs_threshold
} else {
trace!("we don't have enough surbs for retransmission queue clearing...");
return;
};
trace!("we can clear up to {max_to_clear} entries");
let Some(pending) = self.surb_senders.get_mut(&target) else {
trace!("no pending entry for {target}!");
return;
};
let mut to_take = Vec::new();
while to_take.len() < max_to_clear {
if let Some((_, data)) = pending.pending_retransmissions.pop_first() {
// no need to do anything if we failed to upgrade the reference,
// it means we got the ack while the data was waiting in the queue
if let Some(upgraded) = data.upgrade() {
to_take.push(upgraded)
}
} else {
// our map is empty!
break;
}
}
if to_take.is_empty() {
// no need to do anything
return;
}
let (surbs_for_reply, _) = self.surbs_storage.get_reply_surbs(&target, to_take.len());
let Some(surbs_for_reply) = surbs_for_reply else {
error!("somehow different task has stolen our reply surbs! - this should have been impossible");
self.re_insert_pending_retransmission(&target, to_take);
return;
};
let to_send_vec = to_take.iter().map(|ack| ack.fragment_data()).collect();
let prepared_fragments = match self
.message_handler
.prepare_reply_chunks_for_sending(to_send_vec, surbs_for_reply)
.await
{
Ok(prepared) => prepared,
Err(err) => {
let err = err.return_unused_surbs(&self.surbs_storage, &target);
self.re_insert_pending_retransmission(&target, to_take);
warn!("failed to clear pending retransmission queue for {target}: {err}",);
return;
}
};
// we can't fail at this point, so drop all references to acks so that timer updates wouldn't blow up
drop(to_take);
self.message_handler
.send_retransmission_reply_chunks(prepared_fragments, TransmissionLane::Retransmission)
.await;
}
fn pop_at_most_pending_replies(
&mut self,
from: &AnonymousSenderTag,
amount: usize,
) -> Option<Vec<(TransmissionLane, FragmentWithMaxRetransmissions)>> {
// if possible, pop all pending replies, if not, pop only entries for which we'd have a reply surb
let pending = self.surb_senders.get_mut(from)?;
let total = pending.pending_replies.total_size();
trace!("pending queue has {total} elements");
if total == 0 {
return None;
}
pending
.pending_replies
.pop_at_most_n_next_messages_at_random(amount)
}
#[allow(clippy::panic)]
async fn try_clear_pending_queue(&mut self, target: AnonymousSenderTag) {
trace!("trying to clear pending queue");
let available_surbs = self.surbs_storage.available_surbs(&target);
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_to_clear = if available_surbs > min_surbs_threshold {
available_surbs - min_surbs_threshold
} else {
trace!("we don't have enough surbs for queue clearing...");
return;
};
trace!("we can clear up to {max_to_clear} entries");
// we're guaranteed to not get more entries than we have reply surbs for
if let Some(to_send) = self.pop_at_most_pending_replies(&target, max_to_clear) {
let to_send_clone = to_send.clone();
if to_send_clone.is_empty() {
panic!(
"please let the devs know if you ever see this message (reply_controller.rs)"
);
}
let (surbs_for_reply, _) = self
.surbs_storage
.get_reply_surbs(&target, to_send_clone.len());
let Some(surbs_for_reply) = surbs_for_reply else {
error!("somehow different task has stolen our reply surbs! - this should have been impossible");
self.re_insert_pending_replies(&target, to_send);
return;
};
if let Err(err) = self
.message_handler
.try_send_reply_chunks(target, to_send_clone, surbs_for_reply)
.await
{
let err = err.return_unused_surbs(&self.surbs_storage, &target);
self.re_insert_pending_replies(&target, to_send);
warn!("failed to clear pending queue for {target}: {err}");
}
} else {
trace!("the pending queue is empty");
}
}
fn reset_rerequest_counter(&mut self, from: &AnonymousSenderTag) {
if let Some(pending) = self.surb_senders.get_mut(from) {
pending.reset_current_clear_rerequest_counter()
}
}
pub(crate) async fn handle_received_surbs(
&mut self,
from: AnonymousSenderTag,
reply_surbs: Vec<ReplySurbWithKeyRotation>,
from_surb_request: bool,
) {
trace!("handling received surbs");
// clear the requesting flag since we should have been asking for surbs
if from_surb_request {
self.surbs_storage
.decrement_pending_reception(&from, reply_surbs.len() as u32);
}
// store received surbs
self.surbs_storage.insert_fresh_surbs(&from, reply_surbs);
// reset, if applicable, request counter
self.reset_rerequest_counter(&from);
// use as many as we can for clearing pending retransmission queue
self.try_clear_pending_retransmission(from).await;
// use as many as we can for clearing pending 'normal' queue
self.try_clear_pending_queue(from).await;
// if we have to, request more
if self.should_request_more_surbs(&from) {
self.request_reply_surbs_for_queue_clearing(from).await;
}
}
fn buffer_pending_ack(
&mut self,
recipient: AnonymousSenderTag,
ack_ref: Arc<PendingAcknowledgement>,
weak_ack_ref: Weak<PendingAcknowledgement>,
) {
let frag_id = ack_ref.inner_fragment_identifier();
let pending = self.surb_senders.entry(recipient).or_default();
if let Entry::Vacant(e) = pending.pending_retransmissions.entry(frag_id) {
e.insert(weak_ack_ref);
} else {
warn!(
"we're already trying to retransmit {frag_id}. We must be really behind in surbs!"
);
}
}
pub(crate) async fn handle_reply_retransmission(
&mut self,
recipient_tag: AnonymousSenderTag,
timed_out_ack: Weak<PendingAcknowledgement>,
extra_surbs_request: bool,
) {
// seems we got the ack in the end
let ack_ref = match timed_out_ack.upgrade() {
Some(ack) => ack,
None => {
debug!("we received the ack for one of the reply packets as we were putting it in the retransmission queue");
return;
}
};
// if this is retransmission for obtaining additional reply surbs,
// we can dip below the storage threshold
let (maybe_reply_surb, _) = if extra_surbs_request {
self.surbs_storage
.get_reply_surb_ignoring_threshold(&recipient_tag)
} else {
self.surbs_storage.get_reply_surb(&recipient_tag)
};
if let Some(reply_surb) = maybe_reply_surb {
match self
.message_handler
.try_prepare_single_reply_chunk_for_sending(reply_surb, ack_ref.fragment_data())
.await
{
Ok(prepared) => {
// drop the ack ref so that controller would not panic on `UpdateTimer` if that task
// got to handle the action before this function terminated (which is very much
// possible if `forward_messages` takes a while)
drop(ack_ref);
self.message_handler
.update_ack_delay(prepared.fragment_identifier, prepared.total_delay);
self.message_handler
.forward_messages(vec![prepared.into()], TransmissionLane::Retransmission)
.await;
}
Err(err) => {
let err = err.return_unused_surbs(&self.surbs_storage, &recipient_tag);
warn!("failed to prepare message for retransmission - {err}");
// we buffer that packet and to try another day
self.buffer_pending_ack(recipient_tag, ack_ref, timed_out_ack);
if self.should_request_more_surbs(&recipient_tag) {
self.request_reply_surbs_for_queue_clearing(recipient_tag)
.await;
}
}
};
} else {
self.buffer_pending_ack(recipient_tag, ack_ref, timed_out_ack);
if self.should_request_more_surbs(&recipient_tag) {
self.request_reply_surbs_for_queue_clearing(recipient_tag)
.await;
}
}
}
// to be honest this doesn't make a lot of sense in the context of `connection_id`,
// it should really be asked per tag
pub(crate) fn handle_lane_queue_length(
&self,
connection_id: ConnectionId,
response_channel: oneshot::Sender<usize>,
) {
// TODO: if we ever have duplicate ids for different senders, it means our rng is super weak
// thus I don't think we have to worry about it?
let lane = TransmissionLane::ConnectionId(connection_id);
for buf in self.surb_senders.values().map(|p| &p.pending_replies) {
if let Some(length) = buf.lane_length(&lane) {
if response_channel.send(length).is_err() {
error!("the requester for lane queue length has dropped the response channel!")
}
return;
}
}
// make sure that if we didn't find that lane, we reply with 0
if response_channel.send(0).is_err() {
error!("the requester for lane queue length has dropped the response channel!")
}
}
// TODO: modify this method to more accurately determine the amount of surbs it needs to request
// it should take into consideration the average latency, sending rate and queue size.
// it should request as many surbs as it takes to saturate its sending rate before next batch arrives
async fn request_reply_surbs_for_queue_clearing(&mut self, target: AnonymousSenderTag) {
trace!("requesting surbs for queue clearing");
let total_queue = self
.surb_senders
.get(&target)
.map(|pending| pending.total_pending() as u32)
.unwrap_or_default();
let min_surbs_buffer = self.config.reply_surbs.minimum_reply_surb_threshold_buffer as u32;
// To proactively request additional surbs, we aim to have a buffer of extra surbs in our
// storage.
let total_queue_with_buffer = total_queue + min_surbs_buffer;
let request_size = min(
self.config.reply_surbs.maximum_reply_surb_request_size,
max(
total_queue_with_buffer,
self.config.reply_surbs.minimum_reply_surb_request_size,
),
);
if let Err(err) = self
.request_additional_reply_surbs(target, request_size)
.await
{
let now = OffsetDateTime::now_utc();
let sender_info = self.get_or_create_surb_sender(&target);
let last_failure = sender_info.reset_last_request_failure(now);
// only log at higher level if it's the first time this error has occurred in a while
if now - last_failure > time::Duration::seconds(30) {
warn!("failed to request more surbs to clear pending queue of size {total_queue} (attempted to request: {request_size}): {err}")
} else {
debug!("failed to request more surbs to clear pending queue of size {total_queue} (attempted to request: {request_size}): {err}")
}
}
}
pub(crate) async fn inspect_stale_pending_data(&mut self) {
let mut to_request = Vec::new();
let mut to_remove = Vec::new();
let now = OffsetDateTime::now_utc();
for (pending_reply_target, vals) in self.surb_senders.iter_mut() {
// for now recreate old behaviour
let retransmission_buf = &vals.pending_replies;
if retransmission_buf.is_empty() {
continue;
}
let Some(last_received_time) = self
.surbs_storage
.surbs_last_received_at(pending_reply_target)
else {
error!("we have {} pending replies for {pending_reply_target}, but we somehow never received any reply surbs from them!", retransmission_buf.total_size());
to_remove.push(*pending_reply_target);
continue;
};
let diff = now - last_received_time;
let max_rerequest_wait = self
.config
.reply_surbs
.maximum_reply_surb_rerequest_waiting_period;
let max_drop_wait = self
.config
.reply_surbs
.maximum_reply_surb_drop_waiting_period;
let max_rerequests = self.config.reply_surbs.maximum_reply_surbs_rerequests;
// if we have already requested extra surbs because of the stale entry,
// don't do it again (otherwise we'll get stuck in a constant cycle of requesting more surbs
// if client is offline)
if vals.current_clear_rerequest_counter > max_rerequests {
to_remove.push(*pending_reply_target);
debug!("we have reached the maximum threshold of attempting to request surbs from {pending_reply_target}. dropping the sender");
continue;
}
if diff > max_rerequest_wait {
if diff > max_drop_wait {
to_remove.push(*pending_reply_target)
} else {
debug!("We haven't received any surbs in {} from {pending_reply_target}. Going to explicitly ask for more", humantime::format_duration(diff.unsigned_abs()));
vals.increment_current_clear_rerequest_counter();
to_request.push(*pending_reply_target);
}
}
}
for pending_reply_target in to_request {
self.request_reply_surbs_for_queue_clearing(pending_reply_target)
.await;
self.surbs_storage
.reset_pending_reception(&pending_reply_target)
}
for to_remove in to_remove {
// TODO: in the 'old' version we just removed pending messages,
// not retransmissions, but I think those should follow the same logic.
// if something breaks because of that. I guess here is your explanation, future reader
self.surb_senders.remove(&to_remove);
}
}
pub(crate) async fn check_surb_refresh(&mut self) {
let Some(current_rotation_id) = self.topology_access.current_key_rotation_id().await else {
warn!("failed to retrieve current key rotation id from the network topology");
return;
};
if let SurbRefreshState::WaitingForNextRotation { last_known } = self.surb_refresh_state {
if last_known == current_rotation_id {
trace!("no changes in key rotation id");
} else {
// key rotation actually changed and given the polling rate (1/8th epoch) we should have plenty
// of time to perform the upgrade.
// but wait for one more call before doing this so that the clients could also resync
// their topologies and discover new rotation
self.surb_refresh_state = SurbRefreshState::ScheduledForNextInvocation;
}
return;
}
// here we are in `SurbRefreshState::ScheduledForNextInvocation` state
let mut marked_as_stale = HashMap::new();
// 1. mark all existing surbs we have as possibly stale
for mut map_entry in self.surbs_storage.as_raw_iter_mut() {
let (sender, received) = map_entry.pair_mut();
let num_downgraded = received.downgrade_freshness();
trace!("{sender}: {num_downgraded} downgraded");
if num_downgraded != 0 {
marked_as_stale.insert(*sender, num_downgraded);
}
}
// 2. attempt to re-request the equivalent number of fresh surbs
// TODO PROBLEM: if our request gets lost, we might be in trouble...
// we need some sort of retry mechanism
for (sender, num_to_request) in marked_as_stale {
if self
.request_additional_reply_surbs(sender, num_to_request as u32)
.await
.is_err()
{
warn!("surb refresh request failed")
}
}
self.surb_refresh_state = SurbRefreshState::WaitingForNextRotation {
last_known: current_rotation_id,
};
}
pub(crate) async fn inspect_and_clear_stale_data(&mut self, now: OffsetDateTime) {
// technically we don't know if epoch is stuck, but we're flying in blind here,
// so we have to assume the worst and not purge anything depending on proper epoch progression
let is_epoch_stuck = self
.current_topology_metadata()
.await
.map(|m| self.config.key_rotation.epoch_stuck(m))
.unwrap_or(false);
// expected time of when the CURRENT key rotation has begun
let expected_current_key_rotation_start = self
.config
.key_rotation
.expected_current_key_rotation_start(now);
// expected ID of the CURRENT key rotation
let expected_current_key_rotation = self
.config
.key_rotation
.expected_current_key_rotation_id(now);
// time of the start of one epoch BEFORE the CURRENT rotation has begun
// this indicates the starting time of when packets with the current keys might have been constructed
let prior_epoch_start =
expected_current_key_rotation_start - self.config.key_rotation.epoch_duration;
// time of the start of one epoch AFTER the current rotation has begun
// this indicates the end of transition period and any packets constructed with keys different
// from the current one are definitely invalid
let following_epoch_start =
expected_current_key_rotation_start + self.config.key_rotation.epoch_duration;
// define a closure for validating individual surbs
// (we have to run it twice for different piles)
let basic_surb_retention_logic = |received_surb: &ReceivedReplySurb| {
if is_epoch_stuck {
let diff = now - received_surb.received_at();
return diff < self.config.key_rotation.rotation_lifetime();
}
if received_surb.received_at() < prior_epoch_start {
// it's definitely from previous rotation
return false;
}
let surb_rotation = received_surb.key_rotation();
if surb_rotation.is_unknown() {
// can't do anything, so just retain it
return true;
}
// TODO: will this backfire during transition period where we need surbs to refresh surbs
// and we failed to send a request?
if surb_rotation.is_even() && expected_current_key_rotation % 2 == 1 {
return false;
}
if surb_rotation.is_odd() && expected_current_key_rotation % 2 == 0 {
return false;
}
true
};
// 1. purge full old clients data (this applies to RECEIVER)
self.surbs_storage.retain(|_, received| {
if is_epoch_stuck {
// if epoch is stuck, we can't do much (because we don't know for certain if rotation has advanced)
// apart from the basic check of surbs being received more than maximum lifetime of a rotation
// because at that point we know they must be invalid
let diff = now - received.surbs_last_received_at();
return diff < self.config.key_rotation.rotation_lifetime();
}
// if surbs were received more than 1h before the start of the current rotation,
// they're DEFINITELY invalid.
// if it was up until 1h AFTER the start of the current rotation they MIGHT be valid -
// we don't know for sure, unless the client explicitly attached rotation information
// (which only applies to more recent versions of clients so we can't 100% rely on that)
if received.surbs_last_received_at() < prior_epoch_start {
return false;
}
// 1.1. check individual surbs (same basic logic applies)
received.retain_fresh_surbs(&basic_surb_retention_logic);
// 1.2. check the possibly stale entries
// 1.2.1. check if we're beyond the key rotation transition period,
// if so those surbs are definitely unusable
if now > following_epoch_start {
received.drop_possibly_stale_surbs();
}
// 1.2.2. otherwise continue with the same logic as the fresh ones
received.retain_possibly_stale_surbs(&basic_surb_retention_logic);
// no surbs left, we're not expecting any AND we haven't received anything in a while
// (i.e. sender probably abandoned us)
let max_drop_wait = self
.config
.reply_surbs
.maximum_reply_surb_drop_waiting_period;
let last_received = received.surbs_last_received_at();
let possibly_abandoned = last_received + max_drop_wait < now;
if received.is_empty() && received.pending_reception() == 0 && possibly_abandoned {
return false;
}
true
});
// 1.3 inspect old unavailable receivers to clear any stale data
self.unavailable
.retain(|_, last_reported| now - *last_reported < time::Duration::seconds(30));
}
}
@@ -3,12 +3,12 @@
use crate::client::real_messages_control::acknowledgement_control::PendingAcknowledgement;
use futures::channel::{mpsc, oneshot};
use log::error;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::anonymous_replies::ReplySurbWithKeyRotation;
use nym_sphinx::anonymous_replies::ReplySurb;
use nym_task::connections::{ConnectionId, TransmissionLane};
use std::sync::Weak;
use tracing::error;
pub(crate) fn new_control_channels() -> (ReplyControllerSender, ReplyControllerReceiver) {
let (tx, rx) = mpsc::unbounded();
@@ -81,7 +81,7 @@ impl ReplyControllerSender {
pub(crate) fn send_additional_surbs(
&self,
sender_tag: AnonymousSenderTag,
reply_surbs: Vec<ReplySurbWithKeyRotation>,
reply_surbs: Vec<ReplySurb>,
from_surb_request: bool,
) -> Result<(), ReplyControllerSenderError> {
self.0
@@ -167,7 +167,7 @@ pub enum ReplyControllerMessage {
AdditionalSurbs {
sender_tag: AnonymousSenderTag,
reply_surbs: Vec<ReplySurbWithKeyRotation>,
reply_surbs: Vec<ReplySurb>,
from_surb_request: bool,
},
@@ -1,101 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::replies::reply_controller::Config;
use nym_client_core_surb_storage::{CombinedReplyStorage, SentReplyKeys, UsedSenderTags};
use nym_crypto::aes::cipher::crypto_common::rand_core::CryptoRng;
use nym_sphinx::addressing::Recipient;
use rand::Rng;
use std::cmp::min;
use std::time::Duration;
use time::OffsetDateTime;
use tracing::{debug, trace, warn};
/// Reply controller responsible for controlling sender-related part
/// of replies, such as checking if any reply keys are stale
pub struct SenderReplyController<R> {
config: Config,
tags_storage: UsedSenderTags,
sent_reply_keys: SentReplyKeys,
message_handler: MessageHandler<R>,
}
impl<R> SenderReplyController<R>
where
R: CryptoRng + Rng,
{
pub(crate) fn new(
config: Config,
storage: &CombinedReplyStorage,
message_handler: MessageHandler<R>,
) -> Self {
SenderReplyController {
config,
tags_storage: storage.tags_storage(),
sent_reply_keys: storage.key_storage(),
message_handler,
}
}
pub(crate) async fn handle_surb_request(&mut self, recipient: Recipient, mut amount: u32) {
// 1. check whether we sent any surbs in the past to this recipient, otherwise
// they have no business in asking for more
if !self.tags_storage.exists(&recipient) {
warn!("{recipient} asked us for reply SURBs even though we never sent them any anonymous messages before!");
return;
}
// 2. check whether the requested amount is within sane range
if amount
> self
.config
.reply_surbs
.maximum_allowed_reply_surb_request_size
{
warn!("The requested reply surb amount is larger than our maximum allowed ({amount} > {}). Lowering it to a more sane value...", self.config.reply_surbs.maximum_allowed_reply_surb_request_size);
amount = self
.config
.reply_surbs
.maximum_allowed_reply_surb_request_size;
}
// 3. construct and send the surbs away
// (send them in smaller batches to make the experience a bit smoother
let mut remaining = amount;
while remaining > 0 {
let to_send = min(remaining, 100);
if let Err(err) = self
.message_handler
.try_send_additional_reply_surbs(
recipient,
to_send,
nym_sphinx::params::PacketType::Mix,
)
.await
{
warn!("failed to send additional surbs to {recipient} - {err}");
} else {
trace!("sent {to_send} reply SURBs to {recipient}");
}
remaining -= to_send;
}
}
pub(crate) fn inspect_and_clear_stale_data(&self, now: OffsetDateTime) {
// check reply keys (this applies to SENDER)
self.sent_reply_keys.retain(|_, reply_key| {
let diff = now - reply_key.sent_at;
if diff > self.config.reply_surbs.maximum_reply_key_age {
let std_diff = Duration::try_from(diff).unwrap_or_default();
let diff_formatted = humantime::format_duration(std_diff);
debug!("it's been {diff_formatted} since we created this reply key. it's probably never going to get used, so we're going to purge it...");
false
} else {
true
}
});
}
}
@@ -16,17 +16,21 @@
#![warn(clippy::todo)]
#![warn(clippy::dbg_macro)]
use crate::client::inbound_messages::{InputMessage, InputMessageSender};
use futures::StreamExt;
use nym_client_core_config_types::StatsReporting;
use nym_sphinx::addressing::Recipient;
use nym_statistics_common::clients::{
ClientStatsController, ClientStatsReceiver, ClientStatsSender,
};
use nym_task::{connections::TransmissionLane, ShutdownToken, ShutdownTracker};
use nym_task::{connections::TransmissionLane, TaskClient};
use std::time::Duration;
/// Time interval between reporting statistics locally (logging/shutdown_token)
use crate::{
client::inbound_messages::{InputMessage, InputMessageSender},
spawn_future,
};
/// Time interval between reporting statistics locally (logging/task_client)
const LOCAL_REPORT_INTERVAL: Duration = Duration::from_secs(2);
/// Interval for taking snapshots of the statistics
const SNAPSHOT_INTERVAL: Duration = Duration::from_millis(500);
@@ -47,6 +51,9 @@ pub(crate) struct StatisticsControl {
/// Config for stats reporting (enabled, address, interval)
reporting_config: StatsReporting,
/// Task client for listening for shutdown
task_client: TaskClient,
}
impl StatisticsControl {
@@ -55,20 +62,24 @@ impl StatisticsControl {
client_type: String,
client_stats_id: String,
report_tx: InputMessageSender,
shutdown_token: ShutdownToken,
task_client: TaskClient,
) -> (Self, ClientStatsSender) {
let (stats_tx, stats_rx) = tokio::sync::mpsc::unbounded_channel();
let stats = ClientStatsController::new(client_stats_id, client_type);
let mut task_client_stats_sender = task_client.fork("stats_sender");
task_client_stats_sender.disarm();
(
StatisticsControl {
stats,
stats_rx,
report_tx,
reporting_config,
task_client,
},
ClientStatsSender::new(Some(stats_tx), shutdown_token),
ClientStatsSender::new(Some(stats_tx), task_client_stats_sender),
)
}
@@ -82,15 +93,14 @@ impl StatisticsControl {
None,
);
if let Err(err) = self.report_tx.send(report_message).await {
tracing::error!("Failed to report client stats: {err:?}");
log::error!("Failed to report client stats: {:?}", err);
} else {
self.stats.reset();
}
}
// manually control the shutdown mechanism as we don't want to get interrupted mid-snapshot
pub async fn run(&mut self, shutdown_token: ShutdownToken) {
tracing::debug!("Started StatisticsControl with graceful shutdown support");
async fn run(&mut self) {
log::debug!("Started StatisticsControl with graceful shutdown support");
#[cfg(not(target_arch = "wasm32"))]
let mut stats_report_interval = tokio_stream::wrappers::IntervalStream::new(
@@ -119,17 +129,17 @@ impl StatisticsControl {
let mut snapshot_interval =
gloo_timers::future::IntervalStream::new(SNAPSHOT_INTERVAL.as_millis() as u32);
loop {
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = shutdown_token.cancelled() => {
tracing::trace!("StatisticsControl: Received shutdown");
_ = self.task_client.recv() => {
log::trace!("StatisticsControl: Received shutdown");
break;
},
stats_event = self.stats_rx.recv() => match stats_event {
Some(stats_event) => self.stats.handle_event(stats_event),
None => {
tracing::trace!("StatisticsControl: shutting down due to closed stats channel");
log::trace!("StatisticsControl: shutting down due to closed stats channel");
break;
}
},
@@ -147,11 +157,17 @@ impl StatisticsControl {
}
_ = local_report_interval.next() => {
self.stats.local_report();
self.stats.local_report(&mut self.task_client);
}
}
}
tracing::debug!("StatisticsControl: Exiting");
log::debug!("StatisticsControl: Exiting");
}
pub(crate) fn start(mut self) {
spawn_future(async move {
self.run().await;
})
}
pub(crate) fn create_and_start(
@@ -159,22 +175,16 @@ impl StatisticsControl {
client_type: String,
client_stats_id: String,
report_tx: InputMessageSender,
shutdown_tracker: &ShutdownTracker,
task_client: TaskClient,
) -> ClientStatsSender {
let (mut controller, sender) = Self::create(
let (controller, sender) = Self::create(
reporting_config,
client_type,
client_stats_id,
report_tx,
shutdown_tracker.child_shutdown_token(),
);
let shutdown_token = shutdown_tracker.clone_shutdown_token();
shutdown_tracker.try_spawn_named(
async move {
controller.run(shutdown_token).await;
},
"StatisticsControl",
task_client,
);
controller.start();
sender
}
}
@@ -2,8 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use nym_sphinx::addressing::clients::Recipient;
use nym_topology::{NymRouteProvider, NymTopology, NymTopologyError, NymTopologyMetadata};
use nym_validator_client::models::KeyRotationId;
use nym_topology::{NymRouteProvider, NymTopology, NymTopologyError};
use std::ops::Deref;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
@@ -126,7 +125,7 @@ impl TopologyAccessor {
.map(|p| p.topology.clone())
}
pub async fn current_route_provider(&self) -> Option<RwLockReadGuard<'_, NymRouteProvider>> {
pub async fn current_route_provider(&self) -> Option<RwLockReadGuard<NymRouteProvider>> {
let provider = self.inner.topology.read().await;
if provider.topology.is_empty() {
None
@@ -135,21 +134,6 @@ impl TopologyAccessor {
}
}
pub async fn current_mixnet_epoch_id(&self) -> Option<u32> {
let route_provider = self.current_route_provider().await?;
Some(route_provider.absolute_epoch_id())
}
pub async fn current_key_rotation_id(&self) -> Option<KeyRotationId> {
let route_provider = self.current_route_provider().await?;
Some(route_provider.current_key_rotation())
}
pub async fn current_metadata(&self) -> Option<NymTopologyMetadata> {
let route_provider = self.current_route_provider().await?;
Some(route_provider.metadata())
}
pub async fn manually_change_topology(&self, new_topology: NymTopology) {
self.inner.controlled_manually.store(true, Ordering::SeqCst);
self.inner.update(Some(new_topology)).await;
@@ -1,12 +1,14 @@
// Copyright 2021-2023 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::spawn_future;
pub(crate) use accessor::{TopologyAccessor, TopologyReadPermit};
use futures::StreamExt;
use log::*;
use nym_sphinx::addressing::nodes::NodeIdentity;
use nym_task::TaskClient;
use nym_topology::NymTopologyError;
use std::time::Duration;
use tracing::*;
#[cfg(not(target_arch = "wasm32"))]
use tokio::time::sleep;
@@ -18,7 +20,7 @@ mod accessor;
pub mod nym_api_provider;
pub use nym_api_provider::{Config as NymApiTopologyProviderConfig, NymApiTopologyProvider};
pub use nym_topology::provider_trait::{ToTopologyMetadata, TopologyProvider};
pub use nym_topology::provider_trait::TopologyProvider;
// TODO: move it to config later
const MAX_FAILURE_COUNT: usize = 10;
@@ -39,6 +41,8 @@ pub struct TopologyRefresher {
refresh_rate: Duration,
consecutive_failure_count: usize,
task_client: TaskClient,
}
impl TopologyRefresher {
@@ -46,12 +50,14 @@ impl TopologyRefresher {
cfg: TopologyRefresherConfig,
topology_accessor: TopologyAccessor,
topology_provider: Box<dyn TopologyProvider + Send + Sync>,
task_client: TaskClient,
) -> Self {
TopologyRefresher {
topology_provider,
topology_accessor,
refresh_rate: cfg.refresh_rate,
consecutive_failure_count: 0,
task_client,
}
}
@@ -138,30 +144,37 @@ impl TopologyRefresher {
}
}
// it's perfectly fine if task is interrupted mid-refresh
// there's no data to persist or send over
pub async fn run(&mut self) {
debug!("Started TopologyRefresher with graceful shutdown support");
pub fn start(mut self) {
spawn_future(async move {
debug!("Started TopologyRefresher with graceful shutdown support");
#[cfg(not(target_arch = "wasm32"))]
let mut interval =
tokio_stream::wrappers::IntervalStream::new(tokio::time::interval(self.refresh_rate));
#[cfg(not(target_arch = "wasm32"))]
let mut interval = tokio_stream::wrappers::IntervalStream::new(tokio::time::interval(
self.refresh_rate,
));
#[cfg(target_arch = "wasm32")]
let mut interval =
gloo_timers::future::IntervalStream::new(self.refresh_rate.as_millis() as u32);
#[cfg(target_arch = "wasm32")]
let mut interval =
gloo_timers::future::IntervalStream::new(self.refresh_rate.as_millis() as u32);
// We already have an initial topology, so no need to refresh it immediately.
// My understanding is that js setInterval does not fire immediately, so it's not
// needed there.
#[cfg(not(target_arch = "wasm32"))]
interval.next().await;
// We already have an initial topology, so no need to refresh it immediately.
// My understanding is that js setInterval does not fire immediately, so it's not
// needed there.
#[cfg(not(target_arch = "wasm32"))]
interval.next().await;
while interval.next().await.is_some() {
self.try_refresh().await;
}
// this should never get triggered
error!("topology refresher interval has been exhausted!")
while !self.task_client.is_shutdown() {
tokio::select! {
_ = interval.next() => {
self.try_refresh().await;
},
_ = self.task_client.recv() => {
log::trace!("TopologyRefresher: Received shutdown");
},
}
}
self.task_client.recv_timeout().await;
log::debug!("TopologyRefresher: Exiting");
})
}
}
@@ -2,14 +2,13 @@
// SPDX-License-Identifier: Apache-2.0
use async_trait::async_trait;
use nym_mixnet_contract_common::EpochRewardedSet;
use nym_topology::provider_trait::{ToTopologyMetadata, TopologyProvider};
use log::{debug, error, warn};
use nym_topology::provider_trait::TopologyProvider;
use nym_topology::NymTopology;
use nym_validator_client::nym_api::NymApiClientExt;
use nym_validator_client::UserAgent;
use rand::prelude::SliceRandom;
use rand::thread_rng;
use std::cmp::min;
use tracing::{debug, error, warn};
use url::Url;
#[derive(Debug)]
@@ -41,43 +40,38 @@ impl Config {
pub struct NymApiTopologyProvider {
config: Config,
validator_client: nym_http_api_client::Client,
validator_client: nym_validator_client::client::NymApiClient,
nym_api_urls: Vec<Url>,
currently_used_api: usize,
use_bincode: bool,
}
impl NymApiTopologyProvider {
pub fn new(
config: impl Into<Config>,
mut nym_api_urls: Vec<Url>,
validator_client: nym_http_api_client::Client,
user_agent: Option<UserAgent>,
) -> Self {
nym_api_urls.shuffle(&mut thread_rng());
let mut provider = NymApiTopologyProvider {
let validator_client = if let Some(user_agent) = user_agent {
nym_validator_client::client::NymApiClient::new_with_user_agent(
nym_api_urls[0].clone(),
user_agent,
)
} else {
nym_validator_client::client::NymApiClient::new(nym_api_urls[0].clone())
};
NymApiTopologyProvider {
config: config.into(),
validator_client,
nym_api_urls,
currently_used_api: 0,
use_bincode: true,
};
// Set all API URLs - the client will try them in order with automatic failover
provider.validator_client.change_base_urls(
provider
.nym_api_urls
.iter()
.map(|u| u.clone().into())
.collect(),
);
provider
}
}
pub fn disable_bincode(&mut self) {
self.use_bincode = false;
// Note: The unified client doesn't support toggling bincode after creation.
// This would require recreating the client without bincode.
// For now, we'll track the preference but it won't take effect.
warn!("Disabling bincode on existing client is not currently supported");
self.validator_client.use_bincode = false;
}
fn use_next_nym_api(&mut self) {
@@ -87,105 +81,63 @@ impl NymApiTopologyProvider {
}
self.currently_used_api = (self.currently_used_api + 1) % self.nym_api_urls.len();
// Provide all URLs starting from the next one in rotation order
// This enables automatic failover to other endpoints
let rotated_urls: Vec<_> = self
.nym_api_urls
.iter()
.cycle()
.skip(self.currently_used_api)
.take(self.nym_api_urls.len())
.map(|u| u.clone().into())
.collect();
self.validator_client.change_base_urls(rotated_urls)
self.validator_client
.change_nym_api(self.nym_api_urls[self.currently_used_api].clone())
}
async fn get_current_compatible_topology(&mut self) -> Option<NymTopology> {
let rewarded_set_fut = self.validator_client.get_current_rewarded_set();
let topology = if self.config.use_extended_topology {
let all_nodes_fut = self.validator_client.get_all_basic_nodes_with_metadata();
let all_nodes_fut = self.validator_client.get_all_basic_nodes();
// Join rewarded_set_fut and all_nodes_fut concurrently
let (rewarded_set, all_nodes_res) = futures::try_join!(rewarded_set_fut, all_nodes_fut)
let (rewarded_set, all_nodes) = futures::try_join!(rewarded_set_fut, all_nodes_fut)
.inspect_err(|err| error!("failed to get network nodes: {err}"))
.ok()?;
let metadata = all_nodes_res.metadata;
let all_nodes = all_nodes_res.nodes;
debug!(
"there are {} nodes on the network (before filtering)",
all_nodes.len()
);
let nodes_filtered = all_nodes
.into_iter()
.filter(|n| n.performance.round_to_integer() >= self.config.min_node_performance())
.collect::<Vec<_>>();
let mut topology = NymTopology::new_empty(rewarded_set);
topology.add_additional_nodes(all_nodes.iter().filter(|n| {
n.performance.round_to_integer() >= self.config.min_node_performance()
}));
let epoch_rewarded_set: EpochRewardedSet = rewarded_set.into();
NymTopology::new(
metadata.to_topology_metadata(),
epoch_rewarded_set,
Vec::new(),
)
.with_skimmed_nodes(&nodes_filtered)
topology
} else {
// if we're not using extended topology, we're only getting active set mixnodes and gateways
let mixnodes_fut = self
.validator_client
.get_all_basic_active_mixing_assigned_nodes_with_metadata();
.get_all_basic_active_mixing_assigned_nodes();
// TODO: we really should be getting ACTIVE gateways only
let gateways_fut = self
.validator_client
.get_all_basic_entry_assigned_nodes_with_metadata();
let gateways_fut = self.validator_client.get_all_basic_entry_assigned_nodes();
let (rewarded_set, mixnodes_res, gateways_res) =
let (rewarded_set, mixnodes, gateways) =
futures::try_join!(rewarded_set_fut, mixnodes_fut, gateways_fut)
.inspect_err(|err| {
error!("failed to get network nodes: {err}");
})
.ok()?;
let metadata = mixnodes_res.metadata;
let mixnodes = mixnodes_res.nodes;
if !gateways_res.metadata.consistency_check(&metadata) {
warn!("inconsistent nodes metadata between mixnodes and gateways calls! {metadata:?} and {:?}", gateways_res.metadata);
return None;
}
let gateways = gateways_res.nodes;
debug!(
"there are {} mixnodes and {} gateways in total (before performance filtering)",
mixnodes.len(),
gateways.len()
);
let mut nodes = Vec::new();
for mix in mixnodes {
if mix.performance.round_to_integer() >= self.config.min_mixnode_performance {
nodes.push(mix)
}
}
for gateway in gateways {
if gateway.performance.round_to_integer() >= self.config.min_gateway_performance {
nodes.push(gateway)
}
}
let mut topology = NymTopology::new_empty(rewarded_set);
topology.add_additional_nodes(mixnodes.iter().filter(|m| {
m.performance.round_to_integer() >= self.config.min_mixnode_performance
}));
topology.add_additional_nodes(gateways.iter().filter(|m| {
m.performance.round_to_integer() >= self.config.min_gateway_performance
}));
let epoch_rewarded_set: EpochRewardedSet = rewarded_set.into();
NymTopology::new(
metadata.to_topology_metadata(),
epoch_rewarded_set,
Vec::new(),
)
.with_skimmed_nodes(&nodes)
topology
};
if !topology.is_minimally_routable() {

Some files were not shown because too many files have changed in this diff Show More