Compare commits

..

36 Commits

Author SHA1 Message Date
durch 77c4acf602 Add RESET_CONFIG option to entrypoint 2025-06-06 09:38:28 +02:00
durch f4d0ac855c Add periodic route data cleanup to epoch operations
Implements automatic cleanup of route monitoring results to prevent
unbounded storage growth while maintaining data for performance analysis.

- Add purge_old_routes() method to StorageManager following existing patterns
- Integrate route cleanup into purge_old_statuses() wrapper function
- Route data now purged every epoch with 48-hour retention, to facilitate comparisons with legacy data
- Update logging to reflect cleanup of both node statuses and routes
2025-06-06 09:12:21 +02:00
durch eb1c7d649e Client per request 2025-06-05 12:06:15 +02:00
durch 75f34ef51b Add timeout to locust 2025-06-05 11:43:04 +02:00
durch 4f7fa557d5 Optimize database queries by eliminating N+1 patterns in simulation system
This commit addresses critical N+1 query performance issues identified in
the reward simulation system. The optimizations significantly reduce database
round trips and improve performance when processing large datasets.

**Key Optimizations:**

1. **Batch Identity Key Lookups**
   - Added `get_mixnode_identity_keys_batch()` and `get_gateway_identity_keys_batch()`
   - Updated simulation performance conversion to use batch operations
   - Reduced from N individual queries to 2 batch queries

2. **Batch Node Classification**
   - Added `classify_nodes_batch()` method for mixnode/gateway determination
   - Updated reliability calculation methods to use batch classification
   - Reduced from N individual lookups to 2 batch queries

3. **Batch Epoch Metadata Enhancement**
   - Added `count_simulated_node_performance_for_epochs_batch()`
   - Added `get_available_calculation_methods_for_epochs_batch()`
   - Updated API handlers to use batch operations for metadata enhancement
   - Reduced from 2N queries to 2 batch queries for epoch data

4. **Bulk Insert Optimizations**
   - Converted individual INSERT operations to use `sqlx::QueryBuilder::push_values()`
   - Optimized simulation data insertion methods
   - Eliminated transaction overhead from individual inserts

**Performance Impact:**
- Before: N+2N database queries for N nodes/epochs
- After: 2+2 batch queries regardless of dataset size
- Significant performance improvement for large simulation datasets

All changes maintain backward compatibility while providing substantial
performance benefits for the reward simulation system.
2025-06-05 10:45:08 +02:00
durch a96fb098c2 Locust sleep if no clients are available 2025-06-05 09:54:39 +02:00
durch ad5c6ab829 Enhance simulation system with performance comparison framework
Refactors the simulation system to focus on performance methodology comparison
rather than reward amounts, enabling robust analysis of old vs new calculation
methods. Key improvements:

- Replace simulated_rewards table with performance_comparisons for better metrics
- Add performance_rankings table for ranking analysis across methodologies
- Enhance database schema with additional performance tracking fields
- Update simulation coordinator to use performance-focused data structures
- Add comprehensive performance ranking calculations
- Improve API models and handlers for performance comparison workflows
- Update SQLx query cache with new database schema changes

This provides a foundation for data-driven performance methodology evaluation
while maintaining separation from actual reward calculations.
2025-06-04 16:59:42 +02:00
durch b3d07e8832 Tests 2025-06-04 11:28:47 +02:00
durch e761255174 Add complete simulation API layer for reward method comparison
This completes Phase 3 of the simulation system implementation:

- Add comprehensive REST API endpoints for simulation data access
- Implement /v1/simulation/* routes with full CRUD operations
- Support JSON/CSV export for external analysis
- Add statistical comparison between old vs new methods
- Provide node performance history tracking
- Include proper error handling and response formatting
- Simplify simulation coordinator to remove unused complex return types
- Clean up dead code while maintaining all functionality
- Pass clippy with no warnings

The simulation API provides complete access to:
- Simulation epoch listing and details
- Method comparison analytics (old 24h vs new 1h)
- Node performance analysis across epochs
- Route reliability statistics
- Export capabilities for further analysis

All simulation data is persisted and accessible via REST endpoints.
2025-06-03 15:38:58 +02:00
durch e4a20f9cf5 Implement core simulation logic for dual reward calculations
Add complete simulation engine that compares old (24h cache-based) vs new (1h route-based)
reward calculation methodologies with full integration into epoch operations.

Core Simulation Engine:
- Add SimulationCoordinator with configurable time windows and comparison settings
- Implement dual calculation methods with proper Performance type conversions
- Add comprehensive error handling with DatabaseError variant in RewardingError
- Store simulation results in database with proper relationship constraints

Old Method Implementation (24h Cache-Based):
- Wrap existing reliability calculation using get_all_avg_mix_reliability_in_last_24hr()
- Convert reliability percentages to Performance types using from_percentage_value()
- Maintain exact same logic as production for accurate baseline comparison
- Generate simulation data structures with proper metadata

New Method Implementation (1h Route-Based):
- Leverage calculate_corrected_node_reliabilities_for_interval() for route analysis
- Support configurable time windows (default 1 hour vs 24 hours)
- Provide detailed route statistics including success rates and failure analysis
- Convert route reliability data to Performance types with naive_try_from_f64()

Epoch Operations Integration:
- Extend EpochAdvancer struct with optional SimulationConfig field
- Update constructor and start method to accept simulation configuration
- Add simulation trigger in perform_epoch_operations() before real rewarding
- Ensure simulation failures don't break epoch advancement process

CLI Integration:
- Update run.rs to handle both --enable-rewarding and --simulate-rewarding modes
- Create SimulationConfig from rewarding.debug configuration settings
- Implement mutual exclusivity between real rewarding and simulation mode
- Skip permission checks for simulation-only mode (no blockchain transactions)

The simulation system runs in parallel with epoch operations, storing comparative
data for analysis without affecting production reward distribution.
2025-06-03 14:52:13 +02:00
durch 1eefe8a579 Add simulated rewarding system foundation
Implement foundation for simulated reward calculations to compare old (24h cache-based)
vs new (1h route-based) methodologies without blockchain transactions.

Database Changes:
- Add migration with 4 new tables for simulation system
- simulated_reward_epochs: tracks each simulation run
- simulated_node_performance: stores performance calculations
- simulated_rewards: stores reward calculation results
- simulated_route_analysis: metadata for route analysis
- Add comprehensive indexes for efficient querying

Configuration Changes:
- Add simulation_mode flag to rewarding configuration
- Add CLI flag --simulate-rewarding with proper dependencies/conflicts
- Add validation for simulation-specific settings
- Add time window configuration for new method (default: 1 hour)

Storage Layer:
- Add model structs with SQLx FromRow derives for all simulation tables
- Add comprehensive CRUD methods for simulation data management
- Add proper type annotations to fix SQLx compile issues
- Maintain separation between simulation and real rewarding logic

The simulation mode is mutually exclusive with real rewarding and does not
require mnemonic since no blockchain transactions are performed.
2025-06-03 12:39:44 +02:00
durch e9dc848950 Bump nym-api 2025-06-02 12:49:13 +02:00
durch 81162fba7e Allow nym-api init fail 2025-06-02 12:45:21 +02:00
durch be36da68b1 Clap value_delimiter 2025-06-02 11:58:12 +02:00
durch 21a56e307f Bump NNM 2025-06-02 09:31:09 +02:00
durch bd966383be Towards untangling nym-api client 2025-06-02 09:30:36 +02:00
durch 7626785ce4 Bump NM version 2025-05-27 15:40:59 +02:00
durch 6f79d39d48 Filter out non mixnodes 2025-05-27 15:40:33 +02:00
durch 014b5f767a Log APIs used 2025-05-27 15:23:18 +02:00
durch e0966565e6 Mnemonic to run, bump 2025-05-27 13:38:13 +02:00
durch c6aec663b7 Bump nym-api version 2025-05-27 13:18:31 +02:00
durch 7d041ddd44 Explicit mnemonic to entrypoint 2025-05-27 13:02:38 +02:00
durch 5d8bdc6570 Bunch of new query files 2025-05-27 10:28:00 +02:00
durch 06c412b3ba Remove debug logging 2025-05-27 10:25:27 +02:00
durch 356cf00106 Put the monitoring back properly 2025-05-27 10:25:27 +02:00
durch 58493a69aa Fix submission URLs 2025-05-27 10:25:27 +02:00
durch e881da834b More NM logging 2025-05-27 10:25:27 +02:00
durch eee9d8ab0c DEBUG: disable epoch operations, less noisy logging 2025-05-27 10:25:27 +02:00
durch 09026307f4 Debug logging for nym-api 2025-05-27 10:25:27 +02:00
durch 507ddf246c Stagger out route sending 2025-05-27 10:25:26 +02:00
durch 8d8ce29113 Update NM readme, fmt 2025-05-27 10:25:26 +02:00
durch 3be9e06bef sqlx prepare, bunch of nits 2025-05-27 10:25:26 +02:00
durch 770078a9ed Delete test script 2025-05-27 10:25:26 +02:00
durch fcffebfe45 Raw route handling and reliability corrections 2025-05-27 10:25:19 +02:00
durch 9c7d79683b Force routing through all nodes 2025-05-27 10:21:02 +02:00
durch c7f34d04c0 Support submitting to multiple APIs 2025-05-27 10:21:02 +02:00
157 changed files with 6371 additions and 2991 deletions
-1
View File
@@ -13,7 +13,6 @@ on:
- 'nym-network-monitor/**'
- 'nym-node/**'
- 'nym-node-status-api/**'
- 'nym-statistics-api/**'
- 'nym-outfox/**'
- 'nym-validator-rewarder/**'
- 'nyx-chain-watcher/**'
+1 -3
View File
@@ -20,7 +20,6 @@ jobs:
runs-on: ubuntu-22.04
env:
CARGO_TERM_COLOR: always
RUSTUP_PERMIT_COPY_RENAME: 1
steps:
- uses: actions/checkout@v4
@@ -28,8 +27,7 @@ jobs:
uses: actions-rs/toolchain@v1
with:
profile: minimal
# pinned due to issues building contracts
toolchain: 1.86.0
toolchain: stable
target: wasm32-unknown-unknown
override: true
components: rustfmt, clippy
+3 -1
View File
@@ -52,7 +52,7 @@ jobs:
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: 1.86.0
toolchain: stable
override: true
- name: Build all binaries
@@ -66,6 +66,7 @@ jobs:
with:
name: my-artifact
path: |
target/release/explorer-api
target/release/nym-client
target/release/nym-socks5-client
target/release/nym-api
@@ -81,6 +82,7 @@ jobs:
if: github.event_name == 'release'
with:
files: |
target/release/explorer-api
target/release/nym-client
target/release/nym-socks5-client
target/release/nym-api
+1
View File
@@ -62,3 +62,4 @@ nym-api/redocly/formatted-openapi.json
**/settings.sql
**/enter_db.sh
CLAUDE.md
-76
View File
@@ -4,82 +4,6 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
## [Unreleased]
## [2025.11-cheddar] (2025-06-10)
- No autoremoval of peers ([#5831])
- Set cached storage counters to 0 ([#5812])
- hack: temporarily use next.config.js instead of next.config.ts ([#5805])
- chore: resolve 1.87 clippy warnings ([#5802])
- Nym Statistics API ([#5800])
- QoL: RequestPath trait for http-api-client ([#5788])
- Fix contains ticketbook function that always returned true ([#5787])
- swap a decode into a fromrow to please future postgres feature ([#5785])
- Make address cache configurable ([#5784])
- Track wireguard credential retries ([#5783])
[#5831]: https://github.com/nymtech/nym/pull/5831
[#5812]: https://github.com/nymtech/nym/pull/5812
[#5805]: https://github.com/nymtech/nym/pull/5805
[#5802]: https://github.com/nymtech/nym/pull/5802
[#5800]: https://github.com/nymtech/nym/pull/5800
[#5788]: https://github.com/nymtech/nym/pull/5788
[#5787]: https://github.com/nymtech/nym/pull/5787
[#5785]: https://github.com/nymtech/nym/pull/5785
[#5784]: https://github.com/nymtech/nym/pull/5784
[#5783]: https://github.com/nymtech/nym/pull/5783
## [2025.10-brie] (2025-05-27)
- Backport PR 5779 ([#5801])
- Expanded Accept Encoding for `reqwest` ([#5779])
- Teach HttpClientError how to report its status code and timeout ([#5770])
- Skip refreshing the topology on startup as we already have an initial set ([#5768])
- Fetch the topology from the nym-api concurrently ([#5767])
- feat: use bincode by default in NymApiClient + remove feature-lock ([#5761])
- Instrument create_request ([#5760])
- Add node_bonded field to delegations ([#5759])
- build(deps): bump mikefarah/yq from 4.45.1 to 4.45.4 ([#5758])
- Raw route submissions ([#5756])
- feat: expires header for `/active` nym-api responses ([#5755])
- Decrease default average packet delay to 15 ms ([#5754])
- build(deps): bump the patch-updates group across 1 directory with 12 updates ([#5753])
- Remove pretty_env_logger and switch remaining crates to use tracing ([#5749])
- Update pretty_env_logger to latest to not depend on unmaintained crate atty ([#5748])
- Upgrade prometheus crate to fix security warning ([#5747])
- Downgrade deranged crate to 0.4.0 ([#5746])
- feat: nym-api bincode + yaml support ([#5745])
- fix parallel feature in ecash crate with send + sync ([#5744])
- Remove old test directory - Update validator docker ([#5743])
- [Feature] `RememberMe` is the new don't `ForgetMe` ([#5742])
- build(deps): bump ammonia from 4.0.0 to 4.1.0 ([#5739])
- build(deps): bump base-x from 3.0.9 to 3.0.11 in /testnet-faucet ([#5737])
- build(deps): bump http-proxy-middleware from 2.0.8 to 2.0.9 ([#5730])
[#5801]: https://github.com/nymtech/nym/pull/5801
[#5779]: https://github.com/nymtech/nym/pull/5779
[#5770]: https://github.com/nymtech/nym/pull/5770
[#5768]: https://github.com/nymtech/nym/pull/5768
[#5767]: https://github.com/nymtech/nym/pull/5767
[#5761]: https://github.com/nymtech/nym/pull/5761
[#5760]: https://github.com/nymtech/nym/pull/5760
[#5759]: https://github.com/nymtech/nym/pull/5759
[#5758]: https://github.com/nymtech/nym/pull/5758
[#5756]: https://github.com/nymtech/nym/pull/5756
[#5755]: https://github.com/nymtech/nym/pull/5755
[#5754]: https://github.com/nymtech/nym/pull/5754
[#5753]: https://github.com/nymtech/nym/pull/5753
[#5749]: https://github.com/nymtech/nym/pull/5749
[#5748]: https://github.com/nymtech/nym/pull/5748
[#5747]: https://github.com/nymtech/nym/pull/5747
[#5746]: https://github.com/nymtech/nym/pull/5746
[#5745]: https://github.com/nymtech/nym/pull/5745
[#5744]: https://github.com/nymtech/nym/pull/5744
[#5743]: https://github.com/nymtech/nym/pull/5743
[#5742]: https://github.com/nymtech/nym/pull/5742
[#5739]: https://github.com/nymtech/nym/pull/5739
[#5737]: https://github.com/nymtech/nym/pull/5737
[#5730]: https://github.com/nymtech/nym/pull/5730
## [2025.9-appenzeller] (2025-05-13)
- build(deps): bump clap from 4.5.36 to 4.5.37 in the patch-updates group ([#5722])
Generated
+552 -622
View File
File diff suppressed because it is too large Load Diff
-2
View File
@@ -112,7 +112,6 @@ members = [
"nym-node/nym-node-metrics",
"nym-node/nym-node-requests",
"nym-outfox",
"nym-statistics-api",
"nym-validator-rewarder",
"nyx-chain-watcher",
"sdk/ffi/cpp",
@@ -153,7 +152,6 @@ default-members = [
"nym-node",
"nym-node-status-api/nym-node-status-agent",
"nym-node-status-api/nym-node-status-api",
"nym-statistics-api",
"nym-validator-rewarder",
"nyx-chain-watcher",
"service-providers/authenticator",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-client"
version = "1.1.57"
version = "1.1.55"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
description = "Implementation of the Nym Client"
edition = "2021"
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-socks5-client"
version = "1.1.57"
version = "1.1.55"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
description = "A SOCKS5 localhost proxy that converts incoming messages to Sphinx and sends them to a Nym address"
edition = "2021"
@@ -456,7 +456,7 @@ where
log::error!("Could not authenticate and start up the gateway connection - {err}");
ClientCoreError::GatewayClientError {
gateway_id: details.gateway_id.to_base58_string(),
source: Box::new(err),
source: err,
}
};
+5 -10
View File
@@ -18,7 +18,7 @@ pub enum ClientCoreError {
#[error("gateway client error ({gateway_id}): {source}")]
GatewayClientError {
gateway_id: String,
source: Box<GatewayClientError>,
source: GatewayClientError,
},
#[error("custom gateway client error: {source}")]
@@ -88,7 +88,10 @@ pub enum ClientCoreError {
},
#[error("failed to establish connection to gateway: {source}")]
GatewayConnectionFailure { source: Box<tungstenite::Error> },
GatewayConnectionFailure {
#[from]
source: tungstenite::Error,
},
#[cfg(target_arch = "wasm32")]
#[error("failed to establish gateway connection (wasm)")]
@@ -224,14 +227,6 @@ pub enum ClientCoreError {
HkdfDerivationError {},
}
impl From<tungstenite::Error> for ClientCoreError {
fn from(err: tungstenite::Error) -> ClientCoreError {
ClientCoreError::GatewayConnectionFailure {
source: Box::new(err),
}
}
}
/// Set of messages that the client can send to listeners via the task manager
#[derive(Debug)]
pub enum ClientCoreStatusMessage {
+2 -2
View File
@@ -329,7 +329,7 @@ pub(super) async fn register_with_gateway(
log::warn!("Failed to establish connection with gateway!");
ClientCoreError::GatewayClientError {
gateway_id: gateway_id.to_base58_string(),
source: Box::new(err),
source: err,
}
})?;
let auth_response = gateway_client
@@ -339,7 +339,7 @@ pub(super) async fn register_with_gateway(
log::warn!("Failed to register with the gateway {gateway_id}: {err}");
ClientCoreError::GatewayClientError {
gateway_id: gateway_id.to_base58_string(),
source: Box::new(err),
source: err,
}
})?;
+1 -1
View File
@@ -232,7 +232,7 @@ where
} => {
log::debug!("GatewaySetup::ReuseConnection");
Ok(reuse_gateway_connection(
*authenticated_ephemeral_client,
authenticated_ephemeral_client,
*gateway_details,
managed_keys,
))
+2 -2
View File
@@ -218,7 +218,7 @@ pub enum GatewaySetup {
ReuseConnection {
/// The authenticated ephemeral client that was created during `init`
authenticated_ephemeral_client: Box<InitGatewayClient>,
authenticated_ephemeral_client: InitGatewayClient,
// Details of this pre-initialised client (i.e. gateway and keys)
gateway_details: Box<GatewayRegistration>,
@@ -261,7 +261,7 @@ impl GatewaySetup {
pub fn try_reuse_connection(init_res: InitialisationResult) -> Result<Self, ClientCoreError> {
if let Some(authenticated_ephemeral_client) = init_res.authenticated_ephemeral_client {
Ok(GatewaySetup::ReuseConnection {
authenticated_ephemeral_client: Box::new(authenticated_ephemeral_client),
authenticated_ephemeral_client,
gateway_details: Box::new(init_res.gateway_registration),
client_keys: init_res.client_keys,
})
@@ -56,7 +56,7 @@ pub(crate) async fn connect_async(
}
.map_err(|err| GatewayClientError::NetworkConnectionFailed {
address: endpoint.to_owned(),
source: Box::new(tungstenite::Error::from(err)),
source: err.into(),
})?;
#[cfg(unix)]
@@ -72,7 +72,7 @@ pub(crate) async fn connect_async(
Err(err) => {
stream = Err(GatewayClientError::NetworkConnectionFailed {
address: endpoint.to_owned(),
source: Box::new(tungstenite::Error::from(err)),
source: err.into(),
});
continue;
}
@@ -83,6 +83,6 @@ pub(crate) async fn connect_async(
.await
.map_err(|error| GatewayClientError::NetworkConnectionFailed {
address: endpoint.to_owned(),
source: Box::new(error),
source: error,
})
}
+3 -12
View File
@@ -25,7 +25,7 @@ pub enum GatewayClientError {
RequestError(#[from] GatewayRequestsError),
#[error("There was a network error: {0}")]
NetworkError(Box<WsError>),
NetworkError(#[from] WsError),
#[error("failed to upgrade our shared key - the gateway sent malformed response")]
FatalKeyUpgradeFailure,
@@ -41,10 +41,7 @@ pub enum GatewayClientError {
NetworkErrorWasm(#[from] JsError),
#[error("connection failed: {address}: {source}")]
NetworkConnectionFailed {
address: String,
source: Box<WsError>,
},
NetworkConnectionFailed { address: String, source: WsError },
#[error("no socket address for endpoint: {address}")]
NoEndpointForConnection { address: String },
@@ -130,16 +127,10 @@ pub enum GatewayClientError {
ShutdownInProgress,
}
impl From<WsError> for GatewayClientError {
fn from(error: WsError) -> Self {
GatewayClientError::NetworkError(Box::new(error))
}
}
impl GatewayClientError {
pub fn is_closed_connection(&self) -> bool {
match self {
GatewayClientError::NetworkError(ws_err) => match ws_err.as_ref() {
GatewayClientError::NetworkError(ws_err) => match ws_err {
WsError::AlreadyClosed | WsError::ConnectionClosed => true,
WsError::Io(io_err) => matches!(
io_err.kind(),
+2 -2
View File
@@ -28,7 +28,7 @@ pub(crate) fn cleanup_socket_message(
msg: Option<Result<Message, WsError>>,
) -> Result<Message, GatewayClientError> {
match msg {
Some(msg) => msg.map_err(GatewayClientError::from),
Some(msg) => msg.map_err(GatewayClientError::NetworkError),
None => Err(GatewayClientError::ConnectionAbruptlyClosed),
}
}
@@ -39,7 +39,7 @@ pub(crate) fn cleanup_socket_messages(
match msgs {
Some(msgs) => msgs
.into_iter()
.map(|msg| msg.map_err(GatewayClientError::from))
.map(|msg| msg.map_err(GatewayClientError::NetworkError))
.collect(),
None => Err(GatewayClientError::ConnectionAbruptlyClosed),
}
@@ -73,6 +73,7 @@ pub const STAKE_SATURATION: &str = "stake-saturation";
pub const INCLUSION_CHANCE: &str = "inclusion-probability";
pub const SUBMIT_GATEWAY: &str = "submit-gateway-monitoring-results";
pub const SUBMIT_NODE: &str = "submit-node-monitoring-results";
pub const SUBMIT_ROUTE: &str = "submit-route-monitoring-results";
pub const SERVICE_PROVIDERS: &str = "services";
+2 -1
View File
@@ -48,7 +48,8 @@ pub mod nym_config {
log::trace!("Loading from file: {:#?}", filepath.as_ref().to_owned());
let config_contents = fs::read_to_string(filepath)?;
toml::from_str(&config_contents).map_err(io::Error::other)
toml::from_str(&config_contents)
.map_err(|toml_err| io::Error::new(io::ErrorKind::Other, toml_err))
}
}
}
+2 -1
View File
@@ -151,7 +151,8 @@ where
let content = fs::read_to_string(path)?;
// TODO: should we be preserving original error type instead?
deserialize_config_from_toml_str(&content).map_err(io::Error::other)
deserialize_config_from_toml_str(&content)
.map_err(|toml_err| io::Error::new(io::ErrorKind::Other, toml_err))
}
//
+48 -68
View File
@@ -138,12 +138,7 @@
pub use reqwest::{IntoUrl, StatusCode};
use crate::path::RequestPath;
use async_trait::async_trait;
use bytes::Bytes;
use http::header::CONTENT_TYPE;
use http::HeaderMap;
use mime::Mime;
use reqwest::header::HeaderValue;
use reqwest::{RequestBuilder, Response};
use serde::de::DeserializeOwned;
@@ -154,6 +149,10 @@ use thiserror::Error;
use tracing::{debug, instrument, warn};
use url::Url;
use bytes::Bytes;
use http::header::CONTENT_TYPE;
use http::HeaderMap;
use mime::Mime;
#[cfg(not(target_arch = "wasm32"))]
use std::net::SocketAddr;
#[cfg(not(target_arch = "wasm32"))]
@@ -164,8 +163,6 @@ pub use user_agent::UserAgent;
#[cfg(not(target_arch = "wasm32"))]
mod dns;
mod path;
#[cfg(not(target_arch = "wasm32"))]
pub use dns::{HickoryDnsError, HickoryDnsResolver};
@@ -457,15 +454,14 @@ impl Client {
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait ApiClientCore {
/// Create an HTTP request using the host configured in this client.
fn create_request<P, B, K, V>(
fn create_request<B, K, V>(
&self,
method: reqwest::Method,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
json_body: Option<&B>,
) -> RequestBuilder
where
P: RequestPath,
B: Serialize + ?Sized,
K: AsRef<str>,
V: AsRef<str>;
@@ -516,7 +512,7 @@ pub trait ApiClientCore {
};
let params: Vec<(String, String)> = standin_url.query_pairs().into_owned().collect();
self.create_request(method, path.as_slice(), &params, json_body)
self.create_request(method, &path, &params, json_body)
}
/// Send a created HTTP request.
@@ -529,15 +525,14 @@ pub trait ApiClientCore {
E: Display;
/// Create and send a created HTTP request.
async fn send_request<P, B, K, V, E>(
async fn send_request<B, K, V, E>(
&self,
method: reqwest::Method,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
json_body: Option<&B>,
) -> Result<Response, HttpClientError<E>>
where
P: RequestPath + Send + Sync,
B: Serialize + ?Sized + Sync,
K: AsRef<str> + Sync,
V: AsRef<str> + Sync,
@@ -552,15 +547,14 @@ pub trait ApiClientCore {
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
impl ApiClientCore for Client {
#[instrument(level = "debug", skip_all, fields(path=?path))]
fn create_request<P, B, K, V>(
fn create_request<B, K, V>(
&self,
method: reqwest::Method,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
json_body: Option<&B>,
) -> RequestBuilder
where
P: RequestPath,
B: Serialize + ?Sized,
K: AsRef<str>,
V: AsRef<str>,
@@ -603,9 +597,12 @@ impl ApiClientCore for Client {
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait ApiClient: ApiClientCore {
/// Create an HTTP GET Request with the provided path and parameters
fn create_get_request<P, K, V>(&self, path: P, params: Params<'_, K, V>) -> RequestBuilder
fn create_get_request<K, V>(
&self,
path: PathSegments<'_>,
params: Params<'_, K, V>,
) -> RequestBuilder
where
P: RequestPath,
K: AsRef<str>,
V: AsRef<str>,
{
@@ -613,14 +610,13 @@ pub trait ApiClient: ApiClientCore {
}
/// Create an HTTP POST Request with the provided path, parameters, and json body
fn create_post_request<P, B, K, V>(
fn create_post_request<B, K, V>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
json_body: &B,
) -> RequestBuilder
where
P: RequestPath,
B: Serialize + ?Sized,
K: AsRef<str>,
V: AsRef<str>,
@@ -629,9 +625,12 @@ pub trait ApiClient: ApiClientCore {
}
/// Create an HTTP DELETE Request with the provided path and parameters
fn create_delete_request<P, K, V>(&self, path: P, params: Params<'_, K, V>) -> RequestBuilder
fn create_delete_request<K, V>(
&self,
path: PathSegments<'_>,
params: Params<'_, K, V>,
) -> RequestBuilder
where
P: RequestPath,
K: AsRef<str>,
V: AsRef<str>,
{
@@ -639,14 +638,13 @@ pub trait ApiClient: ApiClientCore {
}
/// Create an HTTP PATCH Request with the provided path, parameters, and json body
fn create_patch_request<P, B, K, V>(
fn create_patch_request<B, K, V>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
json_body: &B,
) -> RequestBuilder
where
P: RequestPath,
B: Serialize + ?Sized,
K: AsRef<str>,
V: AsRef<str>,
@@ -656,13 +654,12 @@ pub trait ApiClient: ApiClientCore {
/// Create and send an HTTP GET Request with the provided path and parameters
#[instrument(level = "debug", skip_all, fields(path=?path))]
async fn send_get_request<P, K, V, E>(
async fn send_get_request<K, V, E>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
) -> Result<Response, HttpClientError<E>>
where
P: RequestPath + Send + Sync,
K: AsRef<str> + Sync,
V: AsRef<str> + Sync,
E: Display,
@@ -672,14 +669,13 @@ pub trait ApiClient: ApiClientCore {
}
/// Create and send an HTTP POST Request with the provided path, parameters, and json data
async fn send_post_request<P, B, K, V, E>(
async fn send_post_request<B, K, V, E>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
json_body: &B,
) -> Result<Response, HttpClientError<E>>
where
P: RequestPath + Send + Sync,
B: Serialize + ?Sized + Sync,
K: AsRef<str> + Sync,
V: AsRef<str> + Sync,
@@ -690,13 +686,12 @@ pub trait ApiClient: ApiClientCore {
}
/// Create and send an HTTP DELETE Request with the provided path and parameters
async fn send_delete_request<P, K, V, E>(
async fn send_delete_request<K, V, E>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
) -> Result<Response, HttpClientError<E>>
where
P: RequestPath + Send + Sync,
K: AsRef<str> + Sync,
V: AsRef<str> + Sync,
E: Display,
@@ -706,14 +701,13 @@ pub trait ApiClient: ApiClientCore {
}
/// Create and send an HTTP PATCH Request with the provided path, parameters, and json data
async fn send_patch_request<P, B, K, V, E>(
async fn send_patch_request<B, K, V, E>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
json_body: &B,
) -> Result<Response, HttpClientError<E>>
where
P: RequestPath + Send + Sync,
B: Serialize + ?Sized + Sync,
K: AsRef<str> + Sync,
V: AsRef<str> + Sync,
@@ -728,13 +722,12 @@ pub trait ApiClient: ApiClientCore {
/// into the provided type `T`.
#[instrument(level = "debug", skip_all, fields(path=?path))]
// TODO: deprecate in favour of get_response that works based on mime type in the response
async fn get_json<P, T, K, V, E>(
async fn get_json<T, K, V, E>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
) -> Result<T, HttpClientError<E>>
where
P: RequestPath + Send + Sync,
for<'a> T: Deserialize<'a>,
K: AsRef<str> + Sync,
V: AsRef<str> + Sync,
@@ -746,13 +739,12 @@ pub trait ApiClient: ApiClientCore {
/// 'get' data from the segment-defined path, e.g. `["api", "v1", "mixnodes"]`, with tuple
/// defined key-value parameters, e.g. `[("since", "12345")]`. Attempt to parse the response
/// into the provided type `T` based on the content type header
async fn get_response<P, T, K, V, E>(
async fn get_response<T, K, V, E>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
) -> Result<T, HttpClientError<E>>
where
P: RequestPath + Send + Sync,
for<'a> T: Deserialize<'a>,
K: AsRef<str> + Sync,
V: AsRef<str> + Sync,
@@ -767,14 +759,13 @@ pub trait ApiClient: ApiClientCore {
/// 'post' json data to the segment-defined path, e.g. `["api", "v1", "mixnodes"]`, with tuple
/// defined key-value parameters, e.g. `[("since", "12345")]`. Attempt to parse the response
/// into the provided type `T`.
async fn post_json<P, B, T, K, V, E>(
async fn post_json<B, T, K, V, E>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
json_body: &B,
) -> Result<T, HttpClientError<E>>
where
P: RequestPath + Send + Sync,
B: Serialize + ?Sized + Sync,
for<'a> T: Deserialize<'a>,
K: AsRef<str> + Sync,
@@ -790,13 +781,12 @@ pub trait ApiClient: ApiClientCore {
/// 'delete' json data from the segment-defined path, e.g. `["api", "v1", "mixnodes"]`, with
/// tuple defined key-value parameters, e.g. `[("since", "12345")]`. Attempt to parse the
/// response into the provided type `T`.
async fn delete_json<P, T, K, V, E>(
async fn delete_json<T, K, V, E>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
) -> Result<T, HttpClientError<E>>
where
P: RequestPath + Send + Sync,
for<'a> T: Deserialize<'a>,
K: AsRef<str> + Sync,
V: AsRef<str> + Sync,
@@ -811,14 +801,13 @@ pub trait ApiClient: ApiClientCore {
/// 'patch' json data at the segment-defined path, e.g. `["api", "v1", "mixnodes"]`, with tuple
/// defined key-value parameters, e.g. `[("since", "12345")]`. Attempt to parse the response
/// into the provided type `T`.
async fn patch_json<P, B, T, K, V, E>(
async fn patch_json<B, T, K, V, E>(
&self,
path: P,
path: PathSegments<'_>,
params: Params<'_, K, V>,
json_body: &B,
) -> Result<T, HttpClientError<E>>
where
P: RequestPath + Send + Sync,
B: Serialize + ?Sized + Sync,
for<'a> T: Deserialize<'a>,
K: AsRef<str> + Sync,
@@ -901,7 +890,7 @@ impl<C> ApiClient for C where C: ApiClientCore + Sync {}
/// utility function that should solve the double slash problem in API urls forever.
fn sanitize_url<K: AsRef<str>, V: AsRef<str>>(
base: &Url,
request_path: impl RequestPath,
segments: PathSegments<'_>,
params: Params<'_, K, V>,
) -> Url {
let mut url = base.clone();
@@ -911,7 +900,10 @@ fn sanitize_url<K: AsRef<str>, V: AsRef<str>>(
path_segments.pop_if_empty();
for segment in request_path.to_sanitized_segments() {
for segment in segments {
let segment = segment.strip_prefix('/').unwrap_or(segment);
let segment = segment.strip_suffix('/').unwrap_or(segment);
path_segments.push(segment);
}
@@ -1056,18 +1048,6 @@ mod tests {
fn sanitizing_urls() {
let base_url: Url = "http://foomp.com".parse().unwrap();
// works with a full string
assert_eq!(
"http://foomp.com/foo/bar",
sanitize_url(&base_url, "/foo//bar/", NO_PARAMS).as_str()
);
// (and leading slash doesn't matter)
assert_eq!(
"http://foomp.com/foo/bar",
sanitize_url(&base_url, "foo//bar/", NO_PARAMS).as_str()
);
// works with 1 segment
assert_eq!(
"http://foomp.com/foo",
-59
View File
@@ -1,59 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use std::fmt::Debug;
/// Collection of URL Path Segments
pub type PathSegments<'a> = &'a [&'a str];
fn sanitize_fragment(segment: &str) -> &str {
segment.trim_matches(|c: char| c.is_whitespace() || c == '/')
}
pub trait RequestPath: Debug {
fn to_sanitized_segments(&self) -> Vec<&str>;
}
macro_rules! impl_stringified_sanitized_segments {
($frag_iter:expr) => {{
let mut path_segments = Vec::new();
for segment in $frag_iter {
if !segment.is_empty() {
path_segments.push(sanitize_fragment(segment));
}
}
path_segments
}};
}
impl RequestPath for PathSegments<'_> {
fn to_sanitized_segments(&self) -> Vec<&str> {
impl_stringified_sanitized_segments!(self.iter())
}
}
impl<const N: usize> RequestPath for &[&str; N] {
fn to_sanitized_segments(&self) -> Vec<&str> {
impl_stringified_sanitized_segments!(self.iter())
}
}
impl RequestPath for &str {
fn to_sanitized_segments(&self) -> Vec<&str> {
impl_stringified_sanitized_segments!(self.split('/'))
}
}
impl RequestPath for String {
fn to_sanitized_segments(&self) -> Vec<&str> {
impl_stringified_sanitized_segments!(self.split('/'))
}
}
impl RequestPath for &String {
fn to_sanitized_segments(&self) -> Vec<&str> {
impl_stringified_sanitized_segments!(self.split('/'))
}
}
@@ -124,7 +124,7 @@ impl ReconstructionBuffer {
// TODO: what to do in that case? give up on the message? overwrite it? panic?
// it *might* be due to lock ack-packet, but let's keep the `warn` level in case
// it could be somehow exploited
warn!(
debug!(
"duplicate fragment received! - frag - {} (set id: {})",
fragment.current_fragment(),
fragment.id()
+8 -8
View File
@@ -26,54 +26,54 @@ pub enum ScraperError {
WebSocketConnectionFailure {
url: String,
#[source]
source: Box<tendermint_rpc::Error>,
source: tendermint_rpc::Error,
},
#[error("failed to establish rpc connection to {url}: {source}")]
HttpConnectionFailure {
url: String,
#[source]
source: Box<tendermint_rpc::Error>,
source: tendermint_rpc::Error,
},
#[error("failed to create chain subscription: {source}")]
ChainSubscriptionFailure {
#[source]
source: Box<tendermint_rpc::Error>,
source: tendermint_rpc::Error,
},
#[error("could not obtain basic block information at height: {height}: {source}")]
BlockQueryFailure {
height: u32,
#[source]
source: Box<tendermint_rpc::Error>,
source: tendermint_rpc::Error,
},
#[error("could not obtain block results information at height: {height}: {source}")]
BlockResultsQueryFailure {
height: u32,
#[source]
source: Box<tendermint_rpc::Error>,
source: tendermint_rpc::Error,
},
#[error("could not obtain validators information at height: {height}: {source}")]
ValidatorsQueryFailure {
height: u32,
#[source]
source: Box<tendermint_rpc::Error>,
source: tendermint_rpc::Error,
},
#[error("could not obtain tx results for tx: {hash}: {source}")]
TxResultsQueryFailure {
hash: Hash,
#[source]
source: Box<tendermint_rpc::Error>,
source: tendermint_rpc::Error,
},
#[error("could not obtain current abci info: {source}")]
AbciInfoQueryFailure {
#[source]
source: Box<tendermint_rpc::Error>,
source: tendermint_rpc::Error,
},
#[error("could not parse tx {hash}: {source}")]
+18 -30
View File
@@ -29,7 +29,7 @@ impl RpcClient {
let http_client = HttpClient::new(url.as_str()).map_err(|source| {
ScraperError::HttpConnectionFailure {
url: url.to_string(),
source: Box::new(source),
source,
}
})?;
@@ -90,10 +90,7 @@ impl RpcClient {
self.inner
.block(height)
.await
.map_err(|source| ScraperError::BlockQueryFailure {
height,
source: Box::new(source),
})
.map_err(|source| ScraperError::BlockQueryFailure { height, source })
}
#[instrument(skip(self), err(Display))]
@@ -103,37 +100,31 @@ impl RpcClient {
) -> Result<block_results::Response, ScraperError> {
debug!("getting block results");
self.inner.block_results(height).await.map_err(|source| {
ScraperError::BlockResultsQueryFailure {
height,
source: Box::new(source),
}
})
self.inner
.block_results(height)
.await
.map_err(|source| ScraperError::BlockResultsQueryFailure { height, source })
}
pub(crate) async fn current_block_height(&self) -> Result<u64, ScraperError> {
debug!("getting current block height");
let info =
self.inner
.abci_info()
.await
.map_err(|source| ScraperError::AbciInfoQueryFailure {
source: Box::new(source),
})?;
let info = self
.inner
.abci_info()
.await
.map_err(|source| ScraperError::AbciInfoQueryFailure { source })?;
Ok(info.last_block_height.value())
}
pub(crate) async fn earliest_available_block_height(&self) -> Result<u64, ScraperError> {
debug!("getting earliest available block height");
let status =
self.inner
.status()
.await
.map_err(|source| ScraperError::AbciInfoQueryFailure {
source: Box::new(source),
})?;
let status = self
.inner
.status()
.await
.map_err(|source| ScraperError::AbciInfoQueryFailure { source })?;
Ok(status.sync_info.earliest_block_height.value())
}
@@ -176,7 +167,7 @@ impl RpcClient {
.await
.map_err(|source| ScraperError::TxResultsQueryFailure {
hash: tx_hash,
source: Box::new(source),
source,
})
}
@@ -190,9 +181,6 @@ impl RpcClient {
self.inner
.validators(height, Paging::All)
.await
.map_err(|source| ScraperError::ValidatorsQueryFailure {
height,
source: Box::new(source),
})
.map_err(|source| ScraperError::ValidatorsQueryFailure { height, source })
}
}
@@ -42,7 +42,7 @@ impl ChainSubscriber {
let websocket_url = websocket_endpoint.as_str().try_into().map_err(|source| {
ScraperError::WebSocketConnectionFailure {
url: websocket_endpoint.to_string(),
source: Box::new(source),
source,
}
})?;
@@ -52,7 +52,7 @@ impl ChainSubscriber {
.await
.map_err(|source| ScraperError::WebSocketConnectionFailure {
url: websocket_endpoint.to_string(),
source: Box::new(source),
source,
})?;
Ok(ChainSubscriber {
@@ -83,7 +83,7 @@ impl ChainSubscriber {
.await
.map_err(|source| ScraperError::WebSocketConnectionFailure {
url: self.websocket_endpoint.to_string(),
source: Box::new(source),
source,
})?;
self.websocket_client = client;
self.websocket_driver = Some(driver);
@@ -121,9 +121,7 @@ impl ChainSubscriber {
.websocket_client
.subscribe(EventType::NewBlock.into())
.await
.map_err(|source| ScraperError::ChainSubscriptionFailure {
source: Box::new(source),
})?;
.map_err(|source| ScraperError::ChainSubscriptionFailure { source })?;
let mut failures = 0;
+9 -6
View File
@@ -54,11 +54,14 @@ where
let key_pem = read_pem_file(path)?;
if T::pem_type() != key_pem.tag {
return Err(io::Error::other(format!(
"unexpected key pem tag. Got '{}', expected: '{}'",
key_pem.tag,
T::pem_type()
)));
return Err(io::Error::new(
io::ErrorKind::Other,
format!(
"unexpected key pem tag. Got '{}', expected: '{}'",
key_pem.tag,
T::pem_type()
),
));
}
let key = match T::from_bytes(&key_pem.contents) {
@@ -81,7 +84,7 @@ fn read_pem_file<P: AsRef<Path>>(filepath: P) -> io::Result<Pem> {
let mut pem_bytes = File::open(filepath)?;
let mut buf = Vec::new();
pem_bytes.read_to_end(&mut buf)?;
pem::parse(&buf).map_err(io::Error::other)
pem::parse(&buf).map_err(|e| io::Error::new(io::ErrorKind::Other, e))
}
fn write_pem_file<P: AsRef<Path>>(filepath: P, data: Vec<u8>, tag: &str) -> io::Result<()> {
-6
View File
@@ -28,11 +28,5 @@ nym-credentials-interface = { path = "../credentials-interface" }
nym-metrics = { path = "../nym-metrics" }
nym-task = { path = "../task" }
utoipa = { workspace = true, optional = true }
[target."cfg(target_arch = \"wasm32\")".dependencies.wasmtimer]
workspace = true
[features]
default = []
openapi = ["dep:utoipa"]
+1 -1
View File
@@ -1,7 +1,7 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::report::client::{ClientStatsReport, OsInformation};
use crate::report::{ClientStatsReport, OsInformation};
use nym_task::TaskClient;
use time::{OffsetDateTime, Time};
-5
View File
@@ -26,16 +26,11 @@ pub mod report;
pub mod types;
const CLIENT_ID_PREFIX: &str = "client_stats_id";
const VPN_CLIENT_ID_PREFIX: &str = "vpnclient_stats_id";
pub fn generate_client_stats_id(id_key: ed25519::PublicKey) -> String {
generate_stats_id(CLIENT_ID_PREFIX, id_key.to_base58_string())
}
pub fn generate_vpn_client_stats_id<M: AsRef<[u8]>>(seed: M) -> String {
generate_stats_id(VPN_CLIENT_ID_PREFIX, seed)
}
fn generate_stats_id<M: AsRef<[u8]>>(prefix: &str, id_seed: M) -> String {
let mut hasher = sha2::Sha256::new();
hasher.update(prefix);
@@ -6,7 +6,7 @@ use crate::clients::{
nym_api_statistics::NymApiStats, packet_statistics::PacketStatistics,
};
use crate::error::StatsError;
use super::error::StatsError;
use serde::{Deserialize, Serialize};
use sysinfo::System;
-5
View File
@@ -1,5 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
pub mod client;
pub mod vpn_client;
@@ -1,51 +0,0 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use serde::{Deserialize, Serialize};
const KIND: &str = "vpn_client_stats_report";
const VERSION: &str = "v1";
#[cfg_attr(feature = "openapi", derive(utoipa::ToSchema))]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct VpnClientStatsReport {
pub kind: String,
pub api_version: String,
pub stats_id: String,
pub static_information: StaticInformationReport,
//SW called it basic so we can swap it easily down the line for more data
pub basic_usage: Option<UsageReport>,
}
impl VpnClientStatsReport {
pub fn new(stats_id: String, static_information: StaticInformationReport) -> Self {
VpnClientStatsReport {
kind: KIND.into(),
api_version: VERSION.into(),
stats_id,
static_information,
basic_usage: None,
}
}
#[must_use]
pub fn with_usage_report(mut self, usage_report: UsageReport) -> Self {
self.basic_usage = Some(usage_report);
self
}
}
#[cfg_attr(feature = "openapi", derive(utoipa::ToSchema))]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct StaticInformationReport {
pub os_type: String,
pub os_version: Option<String>,
pub os_arch: String,
pub app_version: String,
}
#[cfg_attr(feature = "openapi", derive(utoipa::ToSchema))]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct UsageReport {
pub connection_time_ms: Option<i32>,
pub two_hop: bool,
}
+4
View File
@@ -224,6 +224,10 @@ impl NymTopology {
serde_json::from_reader(file).map_err(Into::into)
}
pub fn node_details(&self) -> &HashMap<NodeId, RoutingNode> {
&self.node_details
}
pub fn add_skimmed_nodes(&mut self, nodes: &[SkimmedNode]) {
self.add_additional_nodes(nodes.iter())
}
+28 -7
View File
@@ -11,6 +11,27 @@ static NETWORK_MONITORS: LazyLock<HashSet<String>> = LazyLock::new(|| {
nm
});
#[derive(Debug, Serialize, Deserialize, JsonSchema, Clone, ToSchema)]
pub struct RouteResult {
pub layer1: u32,
pub layer2: u32,
pub layer3: u32,
pub gw: u32,
pub success: bool,
}
impl RouteResult {
pub fn new(layer1: u32, layer2: u32, layer3: u32, gw: u32, success: bool) -> Self {
RouteResult {
layer1,
layer2,
layer3,
gw,
success,
}
}
}
#[derive(Debug, Serialize, Deserialize, JsonSchema, Clone, ToSchema)]
pub struct NodeResult {
#[schema(value_type = u32)]
@@ -29,23 +50,23 @@ impl NodeResult {
}
}
#[derive(Serialize, Deserialize, JsonSchema)]
#[derive(Serialize, Deserialize, JsonSchema, ToSchema)]
#[serde(untagged)]
pub enum MonitorResults {
Mixnode(Vec<NodeResult>),
Gateway(Vec<NodeResult>),
Node(Vec<NodeResult>),
Route(Vec<RouteResult>),
}
#[derive(Serialize, Deserialize, JsonSchema, ToSchema)]
pub struct MonitorMessage {
results: Vec<NodeResult>,
results: MonitorResults,
signature: String,
signer: String,
timestamp: i64,
}
impl MonitorMessage {
fn message_to_sign(results: &[NodeResult], timestamp: i64) -> Vec<u8> {
fn message_to_sign(results: &MonitorResults, timestamp: i64) -> Vec<u8> {
let mut msg = serde_json::to_vec(results).unwrap_or_default();
msg.extend_from_slice(&timestamp.to_le_bytes());
msg
@@ -60,7 +81,7 @@ impl MonitorMessage {
now - self.timestamp < 5
}
pub fn new(results: Vec<NodeResult>, private_key: &PrivateKey) -> Self {
pub fn new(results: MonitorResults, private_key: &PrivateKey) -> Self {
let timestamp = SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
.expect("Time went backwards")
@@ -82,7 +103,7 @@ impl MonitorMessage {
NETWORK_MONITORS.contains(&self.signer)
}
pub fn results(&self) -> &[NodeResult] {
pub fn results(&self) -> &MonitorResults {
&self.results
}
+4 -2
View File
@@ -25,8 +25,10 @@ fn map_ws_error(err: WebSocketError) -> WsError {
// TODO: are we preserving correct semantics?
WebSocketError::ConnectionError => WsError::ConnectionClosed,
WebSocketError::ConnectionClose(_event) => WsError::ConnectionClosed,
WebSocketError::MessageSendError(err) => WsError::Io(io::Error::other(err.to_string())),
_ => WsError::Io(io::Error::other("new websocket error")),
WebSocketError::MessageSendError(err) => {
WsError::Io(io::Error::new(io::ErrorKind::Other, err.to_string()))
}
_ => WsError::Io(io::Error::new(io::ErrorKind::Other, "new websocket error")),
}
}
+40 -46
View File
@@ -13,10 +13,12 @@ use nym_gateway_storage::models::WireguardPeer;
use nym_task::TaskClient;
use nym_wireguard_types::DEFAULT_PEER_TIMEOUT_CHECK;
use std::sync::Arc;
use std::time::{Duration, SystemTime};
use tokio::sync::{mpsc, RwLock};
use tokio_stream::{wrappers::IntervalStream, StreamExt};
pub(crate) type SharedBandwidthStorageManager = Arc<RwLock<BandwidthStorageManager>>;
const AUTO_REMOVE_AFTER: Duration = Duration::from_secs(60 * 60); // 1 hour
pub struct PeerHandle {
public_key: Key,
@@ -26,6 +28,7 @@ pub struct PeerHandle {
request_tx: mpsc::Sender<PeerControlRequest>,
timeout_check_interval: IntervalStream,
task_client: TaskClient,
startup_timestamp: SystemTime,
}
impl PeerHandle {
@@ -50,6 +53,7 @@ impl PeerHandle {
request_tx,
timeout_check_interval,
task_client,
startup_timestamp: SystemTime::now(),
}
}
@@ -69,49 +73,26 @@ impl PeerHandle {
Ok(success)
}
fn compute_spent_bandwidth(kernel_peer: &Peer, storage_peer: &WireguardPeer) -> Option<u64> {
let storage_peer_rx_bytes = u64::try_from(storage_peer.rx_bytes)
.inspect_err(|e| tracing::error!("Storage rx bytes could not be converted: {e}"))
.ok()?;
let storage_peer_tx_bytes = u64::try_from(storage_peer.tx_bytes)
.inspect_err(|e| tracing::error!("Storage tx bytes could not be converted: {e}"))
.ok()?;
let kernel_total = kernel_peer
.rx_bytes
.checked_add(kernel_peer.tx_bytes)
.or_else(|| {
tracing::error!(
"Overflow on kernel adding bytes: {} + {}",
kernel_peer.rx_bytes,
kernel_peer.tx_bytes
);
None
})?;
let storage_total = storage_peer_rx_bytes
.checked_add(storage_peer_tx_bytes)
.or_else(|| {
tracing::error!("Overflow on storage adding bytes: {storage_peer_rx_bytes} + {storage_peer_tx_bytes}");
None
})?;
kernel_total.checked_sub(storage_total).or_else(|| {
tracing::error!("Overflow on spent bandwidth subtraction: kernel - storage = {kernel_total} - {storage_total}");
None
})
}
async fn active_peer(&mut self, kernel_peer: &Peer) -> Result<bool, Error> {
let Some(storage_peer) = self.peer_storage_manager.get_peer() else {
log::debug!(
"Peer {:?} not in storage anymore, shutting down handle",
self.public_key
);
return Ok(false);
};
async fn active_peer(
&mut self,
storage_peer: &WireguardPeer,
kernel_peer: &Peer,
) -> Result<bool, Error> {
if let Some(bandwidth_manager) = &self.bandwidth_storage_manager {
let spent_bandwidth = Self::compute_spent_bandwidth(kernel_peer, &storage_peer)
if kernel_peer.last_handshake.is_none()
&& SystemTime::now().duration_since(self.startup_timestamp)? >= AUTO_REMOVE_AFTER
{
let success = self.remove_peer().await?;
self.peer_storage_manager.remove_peer();
tracing::debug!(
"Peer {} has not been active for more then {} seconds, removing it",
kernel_peer.public_key.to_string(),
AUTO_REMOVE_AFTER.as_secs()
);
return Ok(!success);
}
let spent_bandwidth = (kernel_peer.rx_bytes + kernel_peer.tx_bytes)
.checked_sub(storage_peer.rx_bytes as u64 + storage_peer.tx_bytes as u64)
.unwrap_or_else(|| {
// if gateway restarted, the kernel values restart from 0
// and we should restart from 0 in storage as well
@@ -119,10 +100,8 @@ impl PeerHandle {
self.peer_storage_manager.peer_information.as_mut()
{
peer_information.force_sync = true;
peer_information.peer.rx_bytes = kernel_peer.rx_bytes;
peer_information.peer.tx_bytes = kernel_peer.tx_bytes;
}
0
kernel_peer.rx_bytes + kernel_peer.tx_bytes
})
.try_into()
.map_err(|_| Error::InconsistentConsumedBytes)?;
@@ -145,6 +124,14 @@ impl PeerHandle {
}
}
} else {
if SystemTime::now().duration_since(self.startup_timestamp)? >= AUTO_REMOVE_AFTER {
log::debug!(
"Peer {} has been present for 30 days, removing it",
self.public_key
);
let success = self.remove_peer().await?;
return Ok(!success);
}
let spent_bandwidth = kernel_peer.rx_bytes + kernel_peer.tx_bytes;
if spent_bandwidth >= BANDWIDTH_CAP_PER_DAY {
log::debug!(
@@ -171,7 +158,14 @@ impl PeerHandle {
// the host information hasn't beed updated yet
return Ok(true);
};
if !self.active_peer(&kernel_peer).await? {
let Some(storage_peer) = self.peer_storage_manager.get_peer() else {
log::debug!(
"Peer {:?} not in storage anymore, shutting down handle",
self.public_key
);
return Ok(false);
};
if !self.active_peer(&storage_peer, &kernel_peer).await? {
log::debug!(
"Peer {:?} is not active anymore, shutting down handle",
self.public_key
-24
View File
@@ -1,24 +0,0 @@
// @ts-check
/** @type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: true,
basePath: "/explorer",
assetPrefix: "/explorer",
trailingSlash: false,
async redirects() {
return [
// Change the basePath to /explorer
{
source: "/",
destination: "/explorer",
basePath: false,
permanent: true,
},
];
},
};
module.exports = nextConfig
+23
View File
@@ -0,0 +1,23 @@
import type { NextConfig } from "next";
const nextConfig: NextConfig = {
reactStrictMode: true,
basePath: "/explorer",
assetPrefix: "/explorer",
trailingSlash: false,
async redirects() {
return [
// Change the basePath to /explorer
{
source: "/",
destination: "/explorer",
basePath: false,
permanent: true,
},
];
},
};
export default nextConfig;
+1 -1
View File
@@ -50,7 +50,7 @@ nym-api-requests = { path = "../nym-api/nym-api-requests" }
nym-credentials = { path = "../common/credentials" }
nym-credentials-interface = { path = "../common/credentials-interface" }
nym-credential-verification = { path = "../common/credential-verification" }
nym-crypto = { path = "../common/crypto", features = ["sphinx"] }
nym-crypto = { path = "../common/crypto" }
nym-gateway-storage = { path = "../common/gateway-storage" }
nym-gateway-stats-storage = { path = "../common/gateway-stats-storage" }
nym-gateway-requests = { path = "../common/gateway-requests" }
+4 -9
View File
@@ -69,7 +69,10 @@ pub enum GatewayError {
},
#[error("there was an issue with the local authenticator: {source}")]
AuthenticatorFailure { source: Box<AuthenticatorError> },
AuthenticatorFailure {
#[from]
source: AuthenticatorError,
},
#[error("failed to startup local {typ}")]
ServiceProviderStartupFailure { typ: &'static str },
@@ -135,11 +138,3 @@ impl From<ClientCoreError> for GatewayError {
}
}
}
impl From<AuthenticatorError> for GatewayError {
fn from(error: AuthenticatorError) -> Self {
GatewayError::AuthenticatorFailure {
source: Box::new(error),
}
}
}
@@ -22,7 +22,7 @@ enum ActiveClient {
Remote(RemoteClientData),
/// Handle to a locally (inside the same process) running client.
Embedded(Box<LocalEmbeddedClientHandle>),
Embedded(LocalEmbeddedClientHandle),
}
impl ActiveClient {
@@ -163,7 +163,7 @@ impl ActiveClientsStore {
/// Inserts a handle to the embedded client
pub fn insert_embedded(&self, local_client_handle: LocalEmbeddedClientHandle) {
let key = local_client_handle.client_destination();
let entry = ActiveClient::Embedded(Box::new(local_client_handle));
let entry = ActiveClient::Embedded(local_client_handle);
if self.inner.insert(key, entry).is_some() {
// this is literally impossible since we're starting the local embedded client before
// even spawning the websocket listener task
@@ -83,7 +83,7 @@ pub(crate) enum InitialAuthenticationError {
UnexpectedMessageType { typ: String },
#[error("Experienced connection error: {0}")]
ConnectionError(Box<WsError>),
ConnectionError(#[from] WsError),
#[error("Attempted to negotiate connection with client using incompatible protocol version. Ours is {current} and the client reports {client:?}")]
IncompatibleProtocol { client: Option<u8>, current: u8 },
@@ -91,7 +91,7 @@ pub(crate) enum InitialAuthenticationError {
#[error("failed to send authentication response: {source}")]
ResponseSendFailure {
#[source]
source: Box<WsError>,
source: WsError,
},
#[error("possibly received a sphinx packet without prior authentication. Request is going to be ignored")]
@@ -103,7 +103,7 @@ pub(crate) enum InitialAuthenticationError {
#[error("failed to obtain message from websocket stream: {source}")]
FailedToReadMessage {
#[source]
source: Box<WsError>,
source: WsError,
},
#[error("timed out while waiting for initial data")]
@@ -113,12 +113,6 @@ pub(crate) enum InitialAuthenticationError {
EmptyClientDetails,
}
impl From<WsError> for InitialAuthenticationError {
fn from(error: WsError) -> Self {
InitialAuthenticationError::ConnectionError(Box::new(error))
}
}
pub(crate) struct FreshHandler<R, S> {
rng: R,
pub(crate) shared_state: CommonHandlerState,
@@ -169,7 +163,7 @@ impl<R, S> FreshHandler<R, S> {
SocketStream::RawTcp(conn) => {
// TODO: perhaps in the future, rather than panic here (and uncleanly shut tcp stream)
// return a result with an error?
let ws_stream = Box::new(tokio_tungstenite::accept_async(conn).await?);
let ws_stream = tokio_tungstenite::accept_async(conn).await?;
SocketStream::UpgradedWebSocket(ws_stream)
}
other => other,
@@ -348,7 +342,7 @@ impl<R, S> FreshHandler<R, S> {
// push them to the client
if let Err(err) = self.push_packets_to_client(shared_keys, messages).await {
warn!("We failed to send stored messages to fresh client - {err}",);
return Err(InitialAuthenticationError::ConnectionError(Box::new(err)));
return Err(InitialAuthenticationError::ConnectionError(err));
} else {
// if it was successful - remove them from the store
self.shared_state.storage.remove_messages(ids).await?;
@@ -886,9 +880,7 @@ impl<R, S> FreshHandler<R, S> {
.await
{
debug!("failed to send authentication response: {source}");
return Err(InitialAuthenticationError::ResponseSendFailure {
source: Box::new(source),
});
return Err(InitialAuthenticationError::ResponseSendFailure { source });
}
let Some(client_details) = auth_result.client_details else {
@@ -971,9 +963,7 @@ impl<R, S> FreshHandler<R, S> {
Ok(Some(Ok(msg))) => msg,
Ok(Some(Err(source))) => {
debug!("failed to obtain message from websocket stream! stopping connection handler: {source}");
return Err(InitialAuthenticationError::FailedToReadMessage {
source: Box::new(source),
});
return Err(InitialAuthenticationError::FailedToReadMessage { source });
}
Ok(None) => return Err(InitialAuthenticationError::ClosedConnection),
Err(_timeout) => return Err(InitialAuthenticationError::Timeout),
@@ -31,7 +31,7 @@ const INITIAL_MESSAGE_TIMEOUT: Duration = Duration::from_millis(10_000);
pub(crate) enum SocketStream<S> {
RawTcp(S),
UpgradedWebSocket(Box<WebSocketStream<S>>),
UpgradedWebSocket(WebSocketStream<S>),
Invalid,
}
@@ -1,32 +0,0 @@
{
"db_name": "SQLite",
"query": "\n SELECT epoch_id as \"epoch_id: u32\", serialised_signatures, serialization_revision as \"serialization_revision: u8\"\n FROM expiration_date_signatures\n WHERE expiration_date = ?\n ",
"describe": {
"columns": [
{
"name": "epoch_id: u32",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "serialised_signatures",
"ordinal": 1,
"type_info": "Blob"
},
{
"name": "serialization_revision: u8",
"ordinal": 2,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false
]
},
"hash": "00d857b624e7edab1198114b17cbad1e16988a3f9989d135840500e1143ce5e5"
}
@@ -1,32 +0,0 @@
{
"db_name": "SQLite",
"query": "\n SELECT epoch_id as \"epoch_id: u32\", serialised_key, serialization_revision as \"serialization_revision: u8\"\n FROM master_verification_key WHERE epoch_id = ?\n ",
"describe": {
"columns": [
{
"name": "epoch_id: u32",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "serialised_key",
"ordinal": 1,
"type_info": "Blob"
},
{
"name": "serialization_revision: u8",
"ordinal": 2,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false
]
},
"hash": "0112296b190328a3856d1adf51aafa2525da6c0b871633aad80ad555db9cf47c"
}
@@ -0,0 +1,20 @@
{
"db_name": "SQLite",
"query": "SELECT COUNT(*) as count FROM simulated_node_performance WHERE simulated_epoch_id = ?",
"describe": {
"columns": [
{
"name": "count",
"ordinal": 0,
"type_info": "Int"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "0557e64c547e147ef7eef713b49c62afde62b3b04ff817ae372ffe9fbeaee6b8"
}
@@ -0,0 +1,20 @@
{
"db_name": "SQLite",
"query": "SELECT COUNT(*) as count FROM simulated_reward_epochs",
"describe": {
"columns": [
{
"name": "count",
"ordinal": 0,
"type_info": "Int"
}
],
"parameters": {
"Right": 0
},
"nullable": [
false
]
},
"hash": "1411660a6456f6f5ca141db10d9a54857fc8b4a661e09ae6719d84ac2e77cf6d"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n INSERT OR IGNORE INTO expiration_date_signatures(expiration_date, epoch_id, serialised_signatures, serialization_revision)\n VALUES (?, ?, ?, ?);\n UPDATE expiration_date_signatures\n SET\n serialised_signatures = ?,\n serialization_revision = ?\n WHERE expiration_date = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 7
},
"nullable": []
},
"hash": "16d10f0ac0ed9ce4239937f46df3797a6a9ee7db2aab9f1b5e55f7c13c53bcc1"
}
@@ -0,0 +1,56 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", epoch_id as \"epoch_id!: u32\", calculation_method as \"calculation_method!\", \n start_timestamp as \"start_timestamp!\", end_timestamp as \"end_timestamp!\", \n description, created_at as \"created_at!\"\n FROM simulated_reward_epochs\n WHERE id = ?\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "epoch_id!: u32",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 2,
"type_info": "Text"
},
{
"name": "start_timestamp!",
"ordinal": 3,
"type_info": "Int64"
},
{
"name": "end_timestamp!",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "description",
"ordinal": 5,
"type_info": "Text"
},
{
"name": "created_at!",
"ordinal": 6,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
false,
true,
false
]
},
"hash": "1e4daea4d5f87938cf98cb2b6facc5802e79b06f3d0d41f6467844265b26d0f7"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n INSERT INTO ecash_ticketbook\n (serialization_revision, ticketbook_data, expiration_date, ticketbook_type, epoch_id, total_tickets, used_tickets)\n VALUES (?, ?, ?, ?, ?, ?, ?)\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 7
},
"nullable": []
},
"hash": "284b3ceae42f9320c30323dde47765854899103fd3c0fa670eb6809492270e02"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "\n INSERT INTO simulated_reward_epochs \n (epoch_id, calculation_method, start_timestamp, end_timestamp, description)\n VALUES (?, ?, ?, ?, ?)\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 5
},
"nullable": []
},
"hash": "2bb062a25129bfc6b060f68592e74aa91d8c82b8b912009fce14f9c19fd4464c"
}
@@ -0,0 +1,56 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", \n node_id as \"node_id!: NodeId\", calculation_method as \"calculation_method!\", \n performance_rank as \"performance_rank!\", performance_percentile as \"performance_percentile!\",\n calculated_at as \"calculated_at!\"\n FROM simulated_performance_rankings\n WHERE simulated_epoch_id = ? AND calculation_method = ?\n ORDER BY performance_rank\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "performance_rank!",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "performance_percentile!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "calculated_at!",
"ordinal": 6,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
false,
false,
false
]
},
"hash": "3024d5e589b1f50f95f5e7bf09e31161beca74e137e340d2a6a02a589ad251cb"
}
@@ -0,0 +1,56 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", \n node_id as \"node_id!: NodeId\", calculation_method as \"calculation_method!\", \n performance_rank as \"performance_rank!\", performance_percentile as \"performance_percentile!\",\n calculated_at as \"calculated_at!\"\n FROM simulated_performance_rankings\n WHERE simulated_epoch_id = ?\n ORDER BY calculation_method, performance_rank\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "performance_rank!",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "performance_percentile!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "calculated_at!",
"ordinal": 6,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
true,
false,
false,
false,
false,
false,
false
]
},
"hash": "33108405de0328977ac5d05d89f484bdb127f9cba0ba5370c931e1225983d74a"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "DELETE FROM ecash_ticketbook WHERE expiration_date <= ?",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "37f82c9ec26b53d01601a2d6df82038a77ec37cca9f9aef18008dcd03030c2c4"
}
@@ -0,0 +1,74 @@
{
"db_name": "SQLite",
"query": "\n SELECT sra.id as \"id!\", sra.simulated_epoch_id as \"simulated_epoch_id!\", \n sra.calculation_method as \"calculation_method!\", sra.total_routes_analyzed as \"total_routes_analyzed!: u32\", \n sra.successful_routes as \"successful_routes!: u32\", sra.failed_routes as \"failed_routes!: u32\", \n sra.average_route_reliability, sra.time_window_hours as \"time_window_hours!: u32\", \n sra.analysis_parameters, sra.calculated_at as \"calculated_at!\"\n FROM simulated_route_analysis sra\n JOIN simulated_reward_epochs sre ON sra.simulated_epoch_id = sre.id\n WHERE sre.epoch_id = ? AND sra.calculation_method = ?\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 2,
"type_info": "Text"
},
{
"name": "total_routes_analyzed!: u32",
"ordinal": 3,
"type_info": "Int64"
},
{
"name": "successful_routes!: u32",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "failed_routes!: u32",
"ordinal": 5,
"type_info": "Int64"
},
{
"name": "average_route_reliability",
"ordinal": 6,
"type_info": "Float"
},
{
"name": "time_window_hours!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "analysis_parameters",
"ordinal": 8,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 9,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
false,
false,
true,
false,
true,
false
]
},
"hash": "4e2338362058e4356ac639fdf806e827761f879cab723cb2a649bc76b176fdad"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "DELETE FROM pending_issuance WHERE deposit_id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "5c5d4bfabf18bc6fa56e76a9b98e38b7f6ceb8e9191a7b9201922efcf6b07966"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "\n INSERT OR IGNORE INTO routes (layer1, layer2, layer3, gw, success) VALUES (?, ?, ?, ?, ?);\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 5
},
"nullable": []
},
"hash": "66109c1d856e1ca2b5126e4bf4c58c7a27b8c303bfa079cf74909354202dcc49"
}
@@ -1,26 +0,0 @@
{
"db_name": "SQLite",
"query": "\n SELECT\n d.node_id as \"node_id: NodeId\",\n CASE WHEN count(*) > 3 THEN AVG(reliability) ELSE 100 END as \"value: f32\"\n FROM\n gateway_details d\n JOIN\n gateway_status s on d.id = s.gateway_details_id\n WHERE\n timestamp >= ? AND\n timestamp <= ?\n GROUP BY 1\n ",
"describe": {
"columns": [
{
"name": "node_id: NodeId",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "value: f32",
"ordinal": 1,
"type_info": "Int"
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
true
]
},
"hash": "676299beb2004ab89f7b38cf21ffb84ab5e7d7435297573523e2532560c2e302"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", node_id as \"node_id!: NodeId\", \n node_type as \"node_type!\", performance_score as \"performance_score!\", \n work_factor as \"work_factor!\", calculation_method as \"calculation_method!\", \n positive_samples as \"positive_samples?\", negative_samples as \"negative_samples?\",\n route_success_rate as \"route_success_rate?\", calculated_at as \"calculated_at!\"\n FROM simulated_performance_comparisons\n WHERE simulated_epoch_id = ?\n ORDER BY calculation_method, node_id\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "performance_score!",
"ordinal": 4,
"type_info": "Float"
},
{
"name": "work_factor!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 6,
"type_info": "Text"
},
{
"name": "positive_samples?",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "negative_samples?",
"ordinal": 8,
"type_info": "Int64"
},
{
"name": "route_success_rate?",
"ordinal": 9,
"type_info": "Float"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
true,
false,
false,
false,
false,
false,
false,
true,
true,
true,
false
]
},
"hash": "6a7b25d2c580298bd44fcf74e4d2c550537c64dbfb2c17a6c91bd5345b9a3e6a"
}
@@ -0,0 +1,44 @@
{
"db_name": "SQLite",
"query": "\n SELECT\n layer1 as \"layer1\",\n layer2 as \"layer2\",\n layer3 as \"layer3\",\n gw as \"gw\",\n success\n FROM routes\n WHERE timestamp >= ? AND timestamp <= ?\n ORDER BY timestamp ASC\n ",
"describe": {
"columns": [
{
"name": "layer1",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "layer2",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "layer3",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "gw",
"ordinal": 3,
"type_info": "Int64"
},
{
"name": "success",
"ordinal": 4,
"type_info": "Bool"
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
false,
false,
false,
false
]
},
"hash": "6b2479c02cf1ef5ae674ce0ab4d027595b91739f3579e1f289b0c722ea91bbcc"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT snp.id as \"id!\", snp.simulated_epoch_id as \"simulated_epoch_id!\", \n snp.node_id as \"node_id!: NodeId\", snp.node_type as \"node_type!\", \n snp.identity_key, snp.reliability_score as \"reliability_score!\", \n snp.positive_samples as \"positive_samples!: u32\", snp.negative_samples as \"negative_samples!: u32\", \n snp.work_factor, snp.calculation_method as \"calculation_method!\", snp.calculated_at as \"calculated_at!\"\n FROM simulated_node_performance snp\n JOIN simulated_reward_epochs sre ON snp.simulated_epoch_id = sre.id\n WHERE snp.node_id = ?\n ORDER BY sre.epoch_id DESC, snp.calculation_method\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "identity_key",
"ordinal": 4,
"type_info": "Text"
},
{
"name": "reliability_score!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "positive_samples!: u32",
"ordinal": 6,
"type_info": "Int64"
},
{
"name": "negative_samples!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "work_factor",
"ordinal": 8,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 9,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
true,
false,
false,
false,
true,
false,
false,
false,
true,
false,
false
]
},
"hash": "6fa6967e77886b69b049c68b664cdf8f4b97daa3734036db16e49915fd89073a"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n INSERT INTO pending_issuance\n (deposit_id, serialization_revision, pending_ticketbook_data, expiration_date)\n VALUES (?, ?, ?, ?)\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 4
},
"nullable": []
},
"hash": "81a12a8a419c88b1c28a5533fde4d63462e9ea0049e2edafea1dc3f8476b33e4"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "UPDATE ecash_ticketbook SET used_tickets = used_tickets + ? WHERE id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "84cad8b1078a4000830835e6349de3eb76fed954b7336530401db72cd008aff3"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n INSERT OR IGNORE INTO master_verification_key(epoch_id, serialised_key, serialization_revision) VALUES (?, ?, ?);\n UPDATE master_verification_key\n SET\n serialised_key = ?,\n serialization_revision = ?\n WHERE epoch_id = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 6
},
"nullable": []
},
"hash": "a5b18e66d77ff802e274623605e15dcfcffb502ba8398caefd56c481f44eb84e"
}
@@ -1,32 +0,0 @@
{
"db_name": "SQLite",
"query": "\n SELECT epoch_id as \"epoch_id: u32\", serialised_signatures, serialization_revision as \"serialization_revision: u8\"\n FROM coin_indices_signatures WHERE epoch_id = ?\n ",
"describe": {
"columns": [
{
"name": "epoch_id: u32",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "serialised_signatures",
"ordinal": 1,
"type_info": "Blob"
},
{
"name": "serialization_revision: u8",
"ordinal": 2,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false
]
},
"hash": "ba96344db31b0f2155e2af53eaaeafc9b5f64061b6c9a829e2912945b6cffc82"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n UPDATE ecash_ticketbook\n SET used_tickets = used_tickets - ?\n WHERE id = ?\n AND used_tickets = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 3
},
"nullable": []
},
"hash": "bc823c54143e2dc590b91347cd089dde284b38a3a4960afed758206d03ca1cf4"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n INSERT OR IGNORE INTO coin_indices_signatures(epoch_id, serialised_signatures, serialization_revision) VALUES (?, ?, ?);\n UPDATE coin_indices_signatures\n SET\n serialised_signatures = ?,\n serialization_revision = ?\n WHERE epoch_id = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 6
},
"nullable": []
},
"hash": "bd1973696121b6128bd75ae80fab253c071e04eb853d4b0f3b21782ea57c2f68"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", node_id as \"node_id!: NodeId\", \n node_type as \"node_type!\", identity_key, reliability_score as \"reliability_score!\", \n positive_samples as \"positive_samples!: u32\", negative_samples as \"negative_samples!: u32\", \n work_factor, calculation_method as \"calculation_method!\", calculated_at as \"calculated_at!\"\n FROM simulated_node_performance\n WHERE simulated_epoch_id = ?\n ORDER BY calculation_method, node_id\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "identity_key",
"ordinal": 4,
"type_info": "Text"
},
{
"name": "reliability_score!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "positive_samples!: u32",
"ordinal": 6,
"type_info": "Int64"
},
{
"name": "negative_samples!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "work_factor",
"ordinal": 8,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 9,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
true,
false,
false,
false,
true,
false,
false,
false,
true,
false,
false
]
},
"hash": "c04c33631eea6959267dd441f50c2bd876ba951e32e8b8bee398cf98fec5e7bf"
}
@@ -1,26 +0,0 @@
{
"db_name": "SQLite",
"query": "\n SELECT\n d.mix_id as \"mix_id: NodeId\",\n AVG(s.reliability) as \"value: f32\"\n FROM\n mixnode_details d\n JOIN\n mixnode_status s on d.id = s.mixnode_details_id\n WHERE\n timestamp >= ? AND\n timestamp <= ?\n GROUP BY 1\n ",
"describe": {
"columns": [
{
"name": "mix_id: NodeId",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "value: f32",
"ordinal": 1,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
true
]
},
"hash": "c19e1b3768bf2929407599e6e8783ead09f4d7319b7997fa2a9bb628f9404166"
}
@@ -0,0 +1,20 @@
{
"db_name": "SQLite",
"query": "SELECT DISTINCT calculation_method FROM simulated_reward_epochs WHERE epoch_id = ?",
"describe": {
"columns": [
{
"name": "calculation_method",
"ordinal": 0,
"type_info": "Text"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "c9796e61c49a29ebd42dec46fb047f7a7e2f80aa4db4f963146cf56864a62347"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", node_id as \"node_id!: NodeId\", \n node_type as \"node_type!\", identity_key, reliability_score as \"reliability_score!\", \n positive_samples as \"positive_samples!: u32\", negative_samples as \"negative_samples!: u32\", \n work_factor, calculation_method as \"calculation_method!\", calculated_at as \"calculated_at!\"\n FROM simulated_node_performance\n WHERE simulated_epoch_id = ? AND calculation_method = ?\n ORDER BY node_id\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "identity_key",
"ordinal": 4,
"type_info": "Text"
},
{
"name": "reliability_score!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "positive_samples!: u32",
"ordinal": 6,
"type_info": "Int64"
},
{
"name": "negative_samples!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "work_factor",
"ordinal": 8,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 9,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
true,
false,
false,
false,
true,
false,
false
]
},
"hash": "d76e131fff79978284f31509e70d42d20ea2e95e5a2f2d574d33ad8d2a7f2a19"
}
@@ -0,0 +1,74 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", \n calculation_method as \"calculation_method!\", total_routes_analyzed as \"total_routes_analyzed!: u32\", \n successful_routes as \"successful_routes!: u32\", failed_routes as \"failed_routes!: u32\", \n average_route_reliability, time_window_hours as \"time_window_hours!: u32\", \n analysis_parameters, calculated_at as \"calculated_at!\"\n FROM simulated_route_analysis\n WHERE simulated_epoch_id = ?\n ORDER BY calculation_method\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 2,
"type_info": "Text"
},
{
"name": "total_routes_analyzed!: u32",
"ordinal": 3,
"type_info": "Int64"
},
{
"name": "successful_routes!: u32",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "failed_routes!: u32",
"ordinal": 5,
"type_info": "Int64"
},
{
"name": "average_route_reliability",
"ordinal": 6,
"type_info": "Float"
},
{
"name": "time_window_hours!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "analysis_parameters",
"ordinal": 8,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 9,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
true,
false,
false,
false,
false,
false,
true,
false,
true,
false
]
},
"hash": "db2b71d1dd33022d5c960053d6a0f5d7ccf52de6775d4f57744d1401d44fee62"
}
@@ -0,0 +1,56 @@
{
"db_name": "SQLite",
"query": "SELECT id as \"id!\", epoch_id as \"epoch_id!: u32\", calculation_method as \"calculation_method!\", \n start_timestamp as \"start_timestamp!\", end_timestamp as \"end_timestamp!\", \n description, created_at as \"created_at!\"\n FROM simulated_reward_epochs \n ORDER BY created_at DESC \n LIMIT ? OFFSET ?",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "epoch_id!: u32",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 2,
"type_info": "Text"
},
{
"name": "start_timestamp!",
"ordinal": 3,
"type_info": "Int64"
},
{
"name": "end_timestamp!",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "description",
"ordinal": 5,
"type_info": "Text"
},
{
"name": "created_at!",
"ordinal": 6,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
false,
true,
false
]
},
"hash": "dbc52e443ab600516b4f96ab2904d6b6379c06cfa1ff8598c29b67a2fe37055d"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "\n INSERT INTO simulated_route_analysis\n (simulated_epoch_id, calculation_method, total_routes_analyzed, successful_routes,\n failed_routes, average_route_reliability, time_window_hours, analysis_parameters)\n VALUES (?, ?, ?, ?, ?, ?, ?, ?)\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 8
},
"nullable": []
},
"hash": "df9013912c0e4177625267776656cb139cf3ac8dd4cd4c5146005b055aefd228"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT snp.id as \"id!\", snp.simulated_epoch_id as \"simulated_epoch_id!\", \n snp.node_id as \"node_id!: NodeId\", snp.node_type as \"node_type!\", \n snp.identity_key, snp.reliability_score as \"reliability_score!\", \n snp.positive_samples as \"positive_samples!: u32\", snp.negative_samples as \"negative_samples!: u32\", \n snp.work_factor, snp.calculation_method as \"calculation_method!\", snp.calculated_at as \"calculated_at!\"\n FROM simulated_node_performance snp\n JOIN simulated_reward_epochs sre ON snp.simulated_epoch_id = sre.id\n WHERE sre.epoch_id = ? AND snp.calculation_method = ?\n ORDER BY snp.node_id\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "identity_key",
"ordinal": 4,
"type_info": "Text"
},
{
"name": "reliability_score!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "positive_samples!: u32",
"ordinal": 6,
"type_info": "Int64"
},
{
"name": "negative_samples!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "work_factor",
"ordinal": 8,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 9,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
true,
false,
false,
false,
true,
false,
false
]
},
"hash": "e2a8ebf260e0e3dd25d614a3c9f70715d0b593cf19d0a5f5c7972ac5925151c3"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", node_id as \"node_id!: NodeId\", \n node_type as \"node_type!\", performance_score as \"performance_score!\", \n work_factor as \"work_factor!\", calculation_method as \"calculation_method!\", \n positive_samples as \"positive_samples?\", negative_samples as \"negative_samples?\",\n route_success_rate as \"route_success_rate?\", calculated_at as \"calculated_at!\"\n FROM simulated_performance_comparisons\n WHERE simulated_epoch_id = ? AND calculation_method = ?\n ORDER BY node_id\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "performance_score!",
"ordinal": 4,
"type_info": "Float"
},
{
"name": "work_factor!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 6,
"type_info": "Text"
},
{
"name": "positive_samples?",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "negative_samples?",
"ordinal": 8,
"type_info": "Int64"
},
{
"name": "route_success_rate?",
"ordinal": 9,
"type_info": "Float"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
false,
false,
false,
true,
true,
true,
false
]
},
"hash": "e8a88007b46202c6dcd87c2f964e366fa2b6fe0c0fd71ce48a544d51c679a9c8"
}
+1 -1
View File
@@ -4,7 +4,7 @@
[package]
name = "nym-api"
license = "GPL-3.0"
version = "1.1.60"
version = "1.1.62"
authors.workspace = true
edition = "2021"
rust-version.workspace = true
+8 -1
View File
@@ -2,4 +2,11 @@
set -e
/usr/src/nym/target/release/nym-api init && /usr/src/nym/target/release/nym-api run
# Optional: delete existing config and force reinit
if [ "$NYM_API_RESET_CONFIG" = "true" ]; then
echo "RESET_CONFIG enabled - removing existing configuration..."
rm -rf ~/.nym/nym-api
fi
# Init can fail if the mounted volume already has a config
/usr/src/nym/target/release/nym-api init --mnemonic "$MNEMONIC" || true && /usr/src/nym/target/release/nym-api run --mnemonic "$MNEMONIC"
@@ -0,0 +1,17 @@
-- Add routes table for storing route metrics data
CREATE TABLE IF NOT EXISTS routes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
layer1 INTEGER NOT NULL, -- NodeId of layer 1 mixnode
layer2 INTEGER NOT NULL, -- NodeId of layer 2 mixnode
layer3 INTEGER NOT NULL, -- NodeId of layer 3 mixnode
gw INTEGER NOT NULL, -- NodeId of gateway
success BOOLEAN NOT NULL, -- Whether the packet was delivered successfully
timestamp INTEGER NOT NULL DEFAULT (unixepoch()) -- When the measurement was taken
);
-- Add indexes for efficient querying
CREATE INDEX IF NOT EXISTS idx_routes_timestamp ON routes(timestamp);
CREATE INDEX IF NOT EXISTS idx_routes_layer1 ON routes(layer1);
CREATE INDEX IF NOT EXISTS idx_routes_layer2 ON routes(layer2);
CREATE INDEX IF NOT EXISTS idx_routes_layer3 ON routes(layer3);
CREATE INDEX IF NOT EXISTS idx_routes_gw ON routes(gw);
@@ -0,0 +1,100 @@
-- Migration: Add performance methodology comparison tables
-- This migration adds support for comparing performance calculation methodologies:
-- old (24h cache-based) vs new (1h route-based) without affecting actual rewards
-- Simulated reward epochs track each simulation run
CREATE TABLE IF NOT EXISTS simulated_reward_epochs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
epoch_id INTEGER NOT NULL,
calculation_method TEXT NOT NULL CHECK (calculation_method IN ('old', 'new', 'comparison', 'test')),
start_timestamp INTEGER NOT NULL,
end_timestamp INTEGER NOT NULL,
description TEXT,
created_at INTEGER NOT NULL DEFAULT (unixepoch())
);
-- Node performance data calculated from different methodologies
CREATE TABLE IF NOT EXISTS simulated_node_performance (
id INTEGER PRIMARY KEY AUTOINCREMENT,
simulated_epoch_id INTEGER NOT NULL,
node_id INTEGER NOT NULL,
node_type TEXT NOT NULL CHECK (node_type IN ('mixnode', 'gateway')),
identity_key TEXT,
reliability_score REAL NOT NULL CHECK (reliability_score >= 0.0 AND reliability_score <= 100.0),
positive_samples INTEGER NOT NULL DEFAULT 0,
negative_samples INTEGER NOT NULL DEFAULT 0,
work_factor REAL CHECK (work_factor >= 0.0 AND work_factor <= 1.0),
calculation_method TEXT NOT NULL CHECK (calculation_method IN ('old', 'new', 'test')),
calculated_at INTEGER NOT NULL DEFAULT (unixepoch()),
FOREIGN KEY (simulated_epoch_id) REFERENCES simulated_reward_epochs(id) ON DELETE CASCADE
);
-- Performance comparison data for analyzing methodology differences
CREATE TABLE IF NOT EXISTS simulated_performance_comparisons (
id INTEGER PRIMARY KEY AUTOINCREMENT,
simulated_epoch_id INTEGER NOT NULL,
node_id INTEGER NOT NULL,
node_type TEXT NOT NULL CHECK (node_type IN ('mixnode', 'gateway')),
-- Performance scores from each methodology
performance_score REAL NOT NULL CHECK (performance_score >= 0.0 AND performance_score <= 100.0),
work_factor REAL NOT NULL CHECK (work_factor >= 0.0),
calculation_method TEXT NOT NULL CHECK (calculation_method IN ('old', 'new', 'test')),
-- Additional metrics for analysis
positive_samples INTEGER DEFAULT 0,
negative_samples INTEGER DEFAULT 0,
route_success_rate REAL CHECK (route_success_rate >= 0.0 AND route_success_rate <= 100.0),
calculated_at INTEGER NOT NULL DEFAULT (unixepoch()),
FOREIGN KEY (simulated_epoch_id) REFERENCES simulated_reward_epochs(id) ON DELETE CASCADE
);
-- Route analysis metadata for each simulation run
CREATE TABLE IF NOT EXISTS simulated_route_analysis (
id INTEGER PRIMARY KEY AUTOINCREMENT,
simulated_epoch_id INTEGER NOT NULL,
calculation_method TEXT NOT NULL CHECK (calculation_method IN ('old', 'new', 'test')),
total_routes_analyzed INTEGER NOT NULL DEFAULT 0,
successful_routes INTEGER NOT NULL DEFAULT 0,
failed_routes INTEGER NOT NULL DEFAULT 0,
average_route_reliability REAL CHECK (average_route_reliability >= 0.0 AND average_route_reliability <= 100.0),
time_window_hours INTEGER NOT NULL DEFAULT 1, -- New method uses 1 hour, old uses 24
analysis_parameters TEXT, -- JSON with additional analysis configuration
calculated_at INTEGER NOT NULL DEFAULT (unixepoch()),
FOREIGN KEY (simulated_epoch_id) REFERENCES simulated_reward_epochs(id) ON DELETE CASCADE
);
-- Performance rankings and comparison analytics
CREATE TABLE IF NOT EXISTS simulated_performance_rankings (
id INTEGER PRIMARY KEY AUTOINCREMENT,
simulated_epoch_id INTEGER NOT NULL,
node_id INTEGER NOT NULL,
calculation_method TEXT NOT NULL CHECK (calculation_method IN ('old', 'new', 'test')),
performance_rank INTEGER NOT NULL,
performance_percentile REAL NOT NULL CHECK (performance_percentile >= 0.0 AND performance_percentile <= 100.0),
calculated_at INTEGER NOT NULL DEFAULT (unixepoch()),
FOREIGN KEY (simulated_epoch_id) REFERENCES simulated_reward_epochs(id) ON DELETE CASCADE,
UNIQUE(simulated_epoch_id, node_id, calculation_method)
);
-- Indexes for efficient querying
CREATE INDEX IF NOT EXISTS idx_simulated_reward_epochs_epoch_id ON simulated_reward_epochs(epoch_id);
CREATE INDEX IF NOT EXISTS idx_simulated_reward_epochs_calculation_method ON simulated_reward_epochs(calculation_method);
CREATE INDEX IF NOT EXISTS idx_simulated_reward_epochs_created_at ON simulated_reward_epochs(created_at);
CREATE INDEX IF NOT EXISTS idx_simulated_node_performance_simulated_epoch_id ON simulated_node_performance(simulated_epoch_id);
CREATE INDEX IF NOT EXISTS idx_simulated_node_performance_node_id ON simulated_node_performance(node_id);
CREATE INDEX IF NOT EXISTS idx_simulated_node_performance_calculation_method ON simulated_node_performance(calculation_method);
CREATE INDEX IF NOT EXISTS idx_simulated_node_performance_node_type ON simulated_node_performance(node_type);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_comparisons_simulated_epoch_id ON simulated_performance_comparisons(simulated_epoch_id);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_comparisons_node_id ON simulated_performance_comparisons(node_id);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_comparisons_calculation_method ON simulated_performance_comparisons(calculation_method);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_comparisons_node_type ON simulated_performance_comparisons(node_type);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_comparisons_performance_score ON simulated_performance_comparisons(performance_score);
CREATE INDEX IF NOT EXISTS idx_simulated_route_analysis_simulated_epoch_id ON simulated_route_analysis(simulated_epoch_id);
CREATE INDEX IF NOT EXISTS idx_simulated_route_analysis_calculation_method ON simulated_route_analysis(calculation_method);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_rankings_simulated_epoch_id ON simulated_performance_rankings(simulated_epoch_id);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_rankings_node_id ON simulated_performance_rankings(node_id);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_rankings_calculation_method ON simulated_performance_rankings(calculation_method);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_rankings_performance_rank ON simulated_performance_rankings(performance_rank);
+1 -1
View File
@@ -16,7 +16,7 @@ pub enum DkgError {
StatePersistenceFailure {
path: PathBuf,
#[source]
source: Box<EcashError>,
source: EcashError,
},
#[error("failed to query for the current DKG epoch state: {source}")]
+1 -1
View File
@@ -73,7 +73,7 @@ impl<R: RngCore + CryptoRng + Clone> DkgController<R> {
persistent_state.save_to_file(save_path).map_err(|source| {
DkgError::StatePersistenceFailure {
path: save_path.to_path_buf(),
source: Box::new(source),
source,
}
})
}
+1 -1
View File
@@ -1275,7 +1275,7 @@ impl TestFixture {
AppState {
nyxd_client,
chain_status_cache: ChainStatusCache::new(Duration::from_secs(42)),
address_info_cache: AddressInfoCache::new(Duration::from_secs(42), 1000),
address_info_cache: AddressInfoCache::new(),
forced_refresh: ForcedRefresh::new(true),
nym_contract_cache: NymContractCache::new(),
node_status_cache: NodeStatusCache::new(),
+5
View File
@@ -59,6 +59,11 @@ pub enum RewardingError {
#[error("could not obtain the current interval rewarding parameters")]
RewardingParamsRetrievalFailure,
#[error("Database operation failed: {source}")]
DatabaseError {
source: anyhow::Error,
},
#[error("{0}")]
GenericError(#[from] anyhow::Error),
}
+40 -1
View File
@@ -22,6 +22,7 @@ use error::RewardingError;
pub(crate) use helpers::RewardedNodeWithParams;
use nym_mixnet_contract_common::{CurrentIntervalResponse, Interval};
use nym_task::{TaskClient, TaskManager};
use simulation::SimulationConfig;
use std::collections::HashSet;
use std::time::Duration;
use tokio::time::sleep;
@@ -32,6 +33,7 @@ mod event_reconciliation;
mod helpers;
mod rewarded_set_assignment;
mod rewarding;
pub(crate) mod simulation;
mod transition_beginning;
// naming things is difficult, ok?
@@ -42,6 +44,7 @@ pub struct EpochAdvancer {
described_cache: SharedCache<DescribedNodes>,
status_cache: NodeStatusCache,
storage: NymApiStorage,
simulation_config: Option<SimulationConfig>,
}
impl EpochAdvancer {
@@ -57,6 +60,7 @@ impl EpochAdvancer {
status_cache: NodeStatusCache,
described_cache: SharedCache<DescribedNodes>,
storage: NymApiStorage,
simulation_config: Option<SimulationConfig>,
) -> Self {
EpochAdvancer {
nyxd_client,
@@ -64,6 +68,7 @@ impl EpochAdvancer {
described_cache,
status_cache,
storage,
simulation_config,
}
}
@@ -148,6 +153,38 @@ impl EpochAdvancer {
}
}
// Run simulation if enabled (before actual rewarding)
if let Some(simulation_config) = &self.simulation_config {
info!("Running reward simulation for epoch {}", interval.current_epoch_absolute_id());
let rewarded_set = match self.nyxd_client.get_rewarded_set_nodes().await {
Ok(rewarded_set) => rewarded_set,
Err(err) => {
warn!("Failed to obtain current rewarded set for simulation: {err}. Falling back to cached version");
self.nym_contract_cache
.rewarded_set_owned()
.await
.into_inner()
.into()
}
};
if let Some(reward_params) = self
.nym_contract_cache
.interval_reward_params()
.await
.into_inner()
{
let _ = self.run_simulation_if_enabled(
&rewarded_set,
reward_params,
interval.current_epoch_absolute_id(),
simulation_config.clone(),
).await;
} else {
warn!("Could not obtain reward parameters for simulation");
}
}
// Reward all the nodes in the still current, soon to be previous rewarded set
info!("Rewarding the current rewarded set...");
self.reward_current_rewarded_set(rewards, interval).await?;
@@ -158,7 +195,7 @@ impl EpochAdvancer {
self.update_rewarded_set_and_advance_epoch(&nym_nodes)
.await?;
info!("Purging old node statuses from the storage...");
info!("Purging old data (node statuses and routes) from the storage...");
let cutoff = (epoch_end - 2 * ONE_DAY).unix_timestamp();
self.storage.purge_old_statuses(cutoff).await?;
@@ -297,6 +334,7 @@ impl EpochAdvancer {
status_cache: &NodeStatusCache,
described_cache: SharedCache<DescribedNodes>,
storage: &NymApiStorage,
simulation_config: Option<SimulationConfig>,
shutdown: &TaskManager,
) {
let mut rewarded_set_updater = EpochAdvancer::new(
@@ -305,6 +343,7 @@ impl EpochAdvancer {
status_cache.to_owned(),
described_cache,
storage.to_owned(),
simulation_config,
);
let shutdown_listener = shutdown.subscribe();
tokio::spawn(async move { rewarded_set_updater.run(shutdown_listener).await });
+598
View File
@@ -0,0 +1,598 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
//! Performance methodology comparison system
//!
//! This module provides functionality to compare different performance calculation
//! methodologies without affecting actual rewards, enabling analysis of:
//! - Old method: 24-hour cache-based performance calculation
//! - New method: 1-hour route-based performance calculation
//!
//! The system focuses on performance metrics and rankings rather than reward amounts,
//! as actual rewards are calculated on-chain based on factors not available to the API.
use crate::epoch_operations::error::RewardingError;
use crate::epoch_operations::helpers::RewardedNodeWithParams;
use crate::storage::models::{SimulatedNodePerformance, SimulatedPerformanceComparison, SimulatedPerformanceRanking, SimulatedRouteAnalysis};
use crate::support::storage::NymApiStorage;
use crate::EpochAdvancer;
use nym_contracts_common::types::NaiveFloat;
use nym_mixnet_contract_common::reward_params::Performance;
use nym_mixnet_contract_common::{EpochRewardedSet, NodeId, RewardingParams};
use std::collections::HashMap;
use time::OffsetDateTime;
use tracing::{debug, error, info};
/// Configuration for simulation runs
#[derive(Debug, Clone)]
pub struct SimulationConfig {
/// Time window in hours for new method calculation (default: 1)
pub new_method_time_window_hours: u32,
/// Whether to run both old and new methods (default: true)
pub run_both_methods: bool,
/// Description for this simulation run
pub description: Option<String>,
}
impl Default for SimulationConfig {
fn default() -> Self {
Self {
new_method_time_window_hours: 1,
run_both_methods: true,
description: None,
}
}
}
/// Main simulation coordinator
pub struct SimulationCoordinator<'a> {
storage: &'a NymApiStorage,
config: SimulationConfig,
}
impl<'a> SimulationCoordinator<'a> {
pub fn new(storage: &'a NymApiStorage, config: SimulationConfig) -> Self {
Self { storage, config }
}
/// Run a complete simulation comparing old vs new rewarding methods
pub async fn run_simulation(
&self,
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
current_epoch_id: u32,
) -> Result<(), RewardingError> {
let now = OffsetDateTime::now_utc();
let end_timestamp = now.unix_timestamp();
let start_timestamp = end_timestamp - (24 * 3600); // 24 hours ago for baseline
info!(
"Starting simulation for epoch {} with time window {}h",
current_epoch_id, self.config.new_method_time_window_hours
);
// Create simulation epoch record
let epoch_db_id = self.storage.manager
.create_simulated_reward_epoch(
current_epoch_id,
"comparison",
start_timestamp,
end_timestamp,
self.config.description.as_deref(),
)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
// Run old method simulation (24h cache-based)
if self.config.run_both_methods {
match self.run_old_method_simulation(
rewarded_set,
reward_params,
epoch_db_id,
end_timestamp,
).await {
Ok(_) => {
info!("Old method simulation completed successfully");
}
Err(e) => {
error!("Old method simulation failed: {}", e);
// Continue with new method even if old fails
}
}
}
// Run new method simulation (1h route-based)
match self.run_new_method_simulation(
rewarded_set,
reward_params,
epoch_db_id,
end_timestamp,
).await {
Ok(_) => {
info!("New method simulation completed successfully");
}
Err(e) => {
error!("New method simulation failed: {}", e);
}
}
info!("Simulation completed for epoch {}", current_epoch_id);
Ok(())
}
/// Run simulation using old method (24h cache-based)
async fn run_old_method_simulation(
&self,
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
epoch_db_id: i64,
end_timestamp: i64,
) -> Result<(), RewardingError> {
debug!("Running old method simulation (24h cache-based)");
// Get 24h performance data using existing cache-based method
let mixnode_reliabilities = self.storage
.get_all_avg_mix_reliability_in_last_24hr(end_timestamp)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
let gateway_reliabilities = self.storage
.get_all_avg_gateway_reliability_in_last_24hr(end_timestamp)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
// Convert to performance map
let mut performance_map = HashMap::new();
for mix_reliability in mixnode_reliabilities {
performance_map.insert(
mix_reliability.mix_id(),
Performance::from_percentage_value(mix_reliability.value() as u64).unwrap_or_default()
);
}
for gateway_reliability in gateway_reliabilities {
performance_map.insert(
gateway_reliability.node_id(),
Performance::from_percentage_value(gateway_reliability.value() as u64).unwrap_or_default()
);
}
// Calculate rewards using old method logic
let rewarded_nodes = self.calculate_rewards_for_nodes(
rewarded_set,
reward_params,
&performance_map,
);
// Convert to simulation data structures
let node_performance = self.convert_to_simulated_performance(
&rewarded_nodes,
rewarded_set,
reward_params,
epoch_db_id,
"old",
None, // Old method doesn't have route sample data
).await;
let performance_comparisons = self.convert_to_performance_comparisons(
&rewarded_nodes,
rewarded_set,
reward_params,
epoch_db_id,
"old",
).await;
// Create route analysis for old method
let route_analysis = SimulatedRouteAnalysis {
id: 0, // Will be set by database
simulated_epoch_id: epoch_db_id,
calculation_method: "old".to_string(),
total_routes_analyzed: 0, // Old method doesn't use route data
successful_routes: 0,
failed_routes: 0,
average_route_reliability: None,
time_window_hours: 24, // Old method uses 24h
analysis_parameters: Some("{\"method\":\"cache_based\",\"data_source\":\"status_cache\"}".to_string()),
calculated_at: OffsetDateTime::now_utc().unix_timestamp(),
};
// Store results in database
self.storage.manager
.insert_simulated_node_performance(&node_performance)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
self.storage.manager
.insert_simulated_performance_comparisons(&performance_comparisons)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
// Calculate and store performance rankings
let rankings = self.calculate_performance_rankings(&performance_comparisons, epoch_db_id, "old");
self.storage.manager
.insert_simulated_performance_rankings(&rankings)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
self.storage.manager
.insert_simulated_route_analysis(&route_analysis)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
Ok(())
}
/// Run simulation using new method (1h route-based)
async fn run_new_method_simulation(
&self,
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
epoch_db_id: i64,
end_timestamp: i64,
) -> Result<(), RewardingError> {
debug!("Running new method simulation ({}h route-based)", self.config.new_method_time_window_hours);
let time_window_secs = (self.config.new_method_time_window_hours as i64) * 3600;
let start_timestamp = end_timestamp - time_window_secs;
// Get route-based performance data using new method
let corrected_reliabilities = self.storage.manager
.calculate_corrected_node_reliabilities_for_interval(start_timestamp, end_timestamp)
.await
.map_err(|e| RewardingError::DatabaseError { source: e })?;
// Convert to performance map and build route reliability data
let mut performance_map = HashMap::new();
let mut route_reliability_map = HashMap::new();
let mut total_routes = 0u32;
let mut successful_routes = 0u32;
let mut reliability_sum = 0.0;
let mut reliability_count = 0u32;
for node_reliability in &corrected_reliabilities {
let total_samples = node_reliability.pos_samples_in_interval + node_reliability.neg_samples_in_interval;
total_routes += total_samples;
successful_routes += node_reliability.pos_samples_in_interval;
if total_samples > 0 {
reliability_sum += node_reliability.reliability;
reliability_count += 1;
}
performance_map.insert(
node_reliability.node_id,
Performance::naive_try_from_f64(node_reliability.reliability / 100.0).unwrap_or_default()
);
// Store sample counts for detailed performance records
route_reliability_map.insert(
node_reliability.node_id,
(node_reliability.pos_samples_in_interval, node_reliability.neg_samples_in_interval)
);
}
// Calculate rewards using new method logic
let rewarded_nodes = self.calculate_rewards_for_nodes(
rewarded_set,
reward_params,
&performance_map,
);
// Convert to simulation data structures
let node_performance = self.convert_to_simulated_performance(
&rewarded_nodes,
rewarded_set,
reward_params,
epoch_db_id,
"new",
Some(&route_reliability_map), // Pass route sample data for new method
).await;
let performance_comparisons = self.convert_to_performance_comparisons(
&rewarded_nodes,
rewarded_set,
reward_params,
epoch_db_id,
"new",
).await;
// Create route analysis for new method
let route_analysis = SimulatedRouteAnalysis {
id: 0, // Will be set by database
simulated_epoch_id: epoch_db_id,
calculation_method: "new".to_string(),
total_routes_analyzed: total_routes,
successful_routes,
failed_routes: total_routes - successful_routes,
average_route_reliability: if reliability_count > 0 {
Some(reliability_sum / reliability_count as f64)
} else {
None
},
time_window_hours: self.config.new_method_time_window_hours,
analysis_parameters: Some(format!(
"{{\"method\":\"route_based\",\"time_window_hours\":{},\"corrected_routes\":{}}}",
self.config.new_method_time_window_hours,
corrected_reliabilities.len()
)),
calculated_at: OffsetDateTime::now_utc().unix_timestamp(),
};
// Store results in database
self.storage.manager
.insert_simulated_node_performance(&node_performance)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
self.storage.manager
.insert_simulated_performance_comparisons(&performance_comparisons)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
// Calculate and store performance rankings
let rankings = self.calculate_performance_rankings(&performance_comparisons, epoch_db_id, "new");
self.storage.manager
.insert_simulated_performance_rankings(&rankings)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
self.storage.manager
.insert_simulated_route_analysis(&route_analysis)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
Ok(())
}
/// Calculate rewards for nodes using the provided performance data
/// This mirrors the logic from helpers.rs but uses simulation performance data
fn calculate_rewards_for_nodes(
&self,
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
performance_map: &HashMap<NodeId, Performance>,
) -> Vec<RewardedNodeWithParams> {
let nodes = &rewarded_set.assignment;
let active_node_work_factor = reward_params.active_node_work();
let standby_node_work_factor = reward_params.standby_node_work();
let mut rewarded_nodes = Vec::with_capacity(nodes.rewarded_set_size());
// Process active set mixnodes (layers 1, 2, 3)
for &node_id in nodes
.layer1
.iter()
.chain(nodes.layer2.iter())
.chain(nodes.layer3.iter())
{
let performance = performance_map.get(&node_id).copied().unwrap_or_default();
rewarded_nodes.push(RewardedNodeWithParams {
node_id,
params: nym_mixnet_contract_common::reward_params::NodeRewardingParameters {
performance,
work_factor: active_node_work_factor,
},
});
}
// Process active set gateways
for &node_id in nodes
.entry_gateways
.iter()
.chain(nodes.exit_gateways.iter())
{
let performance = performance_map.get(&node_id).copied().unwrap_or_default();
rewarded_nodes.push(RewardedNodeWithParams {
node_id,
params: nym_mixnet_contract_common::reward_params::NodeRewardingParameters {
performance,
work_factor: active_node_work_factor,
},
});
}
// Process standby nodes
for &node_id in &nodes.standby {
let performance = performance_map.get(&node_id).copied().unwrap_or_default();
rewarded_nodes.push(RewardedNodeWithParams {
node_id,
params: nym_mixnet_contract_common::reward_params::NodeRewardingParameters {
performance,
work_factor: standby_node_work_factor,
},
});
}
rewarded_nodes
}
/// Determine node type and work factor from rewarded set position
fn determine_node_info(&self, node_id: NodeId, rewarded_set: &EpochRewardedSet, reward_params: RewardingParams) -> (String, f64) {
let nodes = &rewarded_set.assignment;
// Check if node is in active mixnode layers
if nodes.layer1.contains(&node_id) || nodes.layer2.contains(&node_id) || nodes.layer3.contains(&node_id) {
return ("mixnode".to_string(), reward_params.active_node_work().naive_to_f64());
}
// Check if node is in active gateways
if nodes.entry_gateways.contains(&node_id) || nodes.exit_gateways.contains(&node_id) {
return ("gateway".to_string(), reward_params.active_node_work().naive_to_f64());
}
// Check if node is in standby (could be mixnode or gateway)
if nodes.standby.contains(&node_id) {
// Note: We cannot determine if standby nodes are mixnodes or gateways from the
// rewarded set data alone. This limitation exists in both old and new calculation
// methods and doesn't significantly impact the simulation since:
// 1. All standby nodes receive the same work factor regardless of type
// 2. The gateway 3-sample rule can't be applied to standby nodes in either method
// For consistency, we label all standby nodes as "mixnode" in the simulation data
return ("mixnode".to_string(), reward_params.standby_node_work().naive_to_f64());
}
// Default case (shouldn't happen)
("unknown".to_string(), 0.0)
}
/// Convert rewarded nodes to simulated performance records
async fn convert_to_simulated_performance(
&self,
rewarded_nodes: &[RewardedNodeWithParams],
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
epoch_db_id: i64,
method: &str,
route_reliability_map: Option<&HashMap<NodeId, (u32, u32)>>, // (positive_samples, negative_samples)
) -> Vec<SimulatedNodePerformance> {
let now = OffsetDateTime::now_utc().unix_timestamp();
let mut performance_records = Vec::with_capacity(rewarded_nodes.len());
// First collect all node IDs for batch identity key lookups
let all_node_ids: Vec<NodeId> = rewarded_nodes.iter().map(|node| node.node_id).collect();
// Batch fetch identity keys for both mixnodes and gateways
let mixnode_identities = self.storage
.manager
.get_mixnode_identity_keys_batch(&all_node_ids)
.await
.unwrap_or_default();
let gateway_identities = self.storage
.manager
.get_gateway_identity_keys_batch(&all_node_ids)
.await
.unwrap_or_default();
for node in rewarded_nodes {
let (node_type, _) = self.determine_node_info(node.node_id, rewarded_set, reward_params);
// Get identity key from our batch results
let identity_key = match node_type.as_str() {
"mixnode" => mixnode_identities.get(&node.node_id).cloned(),
"gateway" => gateway_identities.get(&node.node_id).cloned(),
_ => None,
};
// Extract sample counts from route reliability if available
let (positive_samples, negative_samples) = route_reliability_map
.and_then(|map| map.get(&node.node_id))
.copied()
.unwrap_or((0, 0));
performance_records.push(SimulatedNodePerformance {
id: 0, // Will be set by database
simulated_epoch_id: epoch_db_id,
node_id: node.node_id,
node_type,
identity_key,
reliability_score: node.params.performance.naive_to_f64() * 100.0,
positive_samples,
negative_samples,
work_factor: Some(node.params.work_factor.naive_to_f64()),
calculation_method: method.to_string(),
calculated_at: now,
});
}
performance_records
}
/// Convert rewarded nodes to performance comparison records
async fn convert_to_performance_comparisons(
&self,
rewarded_nodes: &[RewardedNodeWithParams],
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
epoch_db_id: i64,
method: &str,
) -> Vec<SimulatedPerformanceComparison> {
let now = OffsetDateTime::now_utc().unix_timestamp();
let mut performance_records = Vec::with_capacity(rewarded_nodes.len());
for node in rewarded_nodes {
let (node_type, _) = self.determine_node_info(node.node_id, rewarded_set, reward_params);
performance_records.push(SimulatedPerformanceComparison {
id: 0, // Will be set by database
simulated_epoch_id: epoch_db_id,
node_id: node.node_id,
node_type,
performance_score: node.params.performance.naive_to_f64() * 100.0,
work_factor: node.params.work_factor.naive_to_f64(),
calculation_method: method.to_string(),
positive_samples: None, // Will be populated from node performance data
negative_samples: None,
route_success_rate: None,
calculated_at: now,
});
}
performance_records
}
/// Calculate performance rankings for a set of performance comparisons
fn calculate_performance_rankings(
&self,
performance_comparisons: &[SimulatedPerformanceComparison],
epoch_db_id: i64,
method: &str,
) -> Vec<SimulatedPerformanceRanking> {
let now = OffsetDateTime::now_utc().unix_timestamp();
let mut rankings = Vec::with_capacity(performance_comparisons.len());
// Sort by performance score descending
let mut sorted_comparisons: Vec<_> = performance_comparisons.iter()
.enumerate()
.map(|(idx, comp)| (idx, comp.performance_score))
.collect();
sorted_comparisons.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal));
let total_nodes = sorted_comparisons.len() as f64;
for (rank, (original_idx, _score)) in sorted_comparisons.iter().enumerate() {
let comparison = &performance_comparisons[*original_idx];
let percentile = ((total_nodes - rank as f64 - 1.0) / total_nodes) * 100.0;
rankings.push(SimulatedPerformanceRanking {
id: 0, // Will be set by database
simulated_epoch_id: epoch_db_id,
node_id: comparison.node_id,
calculation_method: method.to_string(),
performance_rank: (rank + 1) as i64,
performance_percentile: percentile,
calculated_at: now,
});
}
rankings
}
}
impl EpochAdvancer {
/// Run simulation during epoch operations if simulation mode is enabled
pub async fn run_simulation_if_enabled(
&self,
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
current_epoch_id: u32,
simulation_config: SimulationConfig,
) -> Result<(), RewardingError> {
let coordinator = SimulationCoordinator::new(&self.storage, simulation_config);
match coordinator.run_simulation(rewarded_set, reward_params, current_epoch_id).await {
Ok(()) => {
info!("Simulation completed successfully for epoch {}", current_epoch_id);
Ok(())
}
Err(e) => {
error!("Simulation failed for epoch {}: {}", current_epoch_id, e);
// Don't fail the entire epoch operation due to simulation failure
Ok(())
}
}
}
}
+1
View File
@@ -24,6 +24,7 @@ pub(crate) mod node_describe_cache;
pub(crate) mod node_status_api;
pub(crate) mod nym_contract_cache;
pub(crate) mod nym_nodes;
pub(crate) mod simulation_api;
mod status;
pub(crate) mod support;
mod unstable_routes;
@@ -20,7 +20,7 @@ use nym_api_requests::models::{
};
use nym_http_api_common::{FormattedResponse, OutputParams};
use nym_mixnet_contract_common::NodeId;
use nym_types::monitoring::MonitorMessage;
use nym_types::monitoring::{MonitorMessage, MonitorResults};
use tracing::error;
pub(super) fn mandatory_routes() -> Router<AppState> {
@@ -33,6 +33,10 @@ pub(super) fn mandatory_routes() -> Router<AppState> {
"/submit-node-monitoring-results",
post(submit_node_monitoring_results),
)
.route(
"/submit-route-monitoring-results",
post(submit_route_monitoring_results),
)
.nest(
"/mixnode/:mix_id",
Router::new()
@@ -58,6 +62,53 @@ pub(super) fn mandatory_routes() -> Router<AppState> {
)
}
#[utoipa::path(
tag = "status",
post,
path = "/v1/status/submit-route-monitoring-results",
responses(
(status = 200),
(status = 400, body = String, description = "TBD"),
(status = 403, body = String, description = "TBD"),
(status = 500, body = String, description = "TBD"),
),
)]
pub(crate) async fn submit_route_monitoring_results(
State(state): State<AppState>,
Json(message): Json<MonitorMessage>,
) -> AxumResult<()> {
if !message.is_in_allowed() {
return Err(AxumErrorResponse::forbidden(
"Monitor not registered to submit results",
));
}
if !message.timely() {
return Err(AxumErrorResponse::bad_request("Message is too old"));
}
if !message.verify() {
return Err(AxumErrorResponse::bad_request("invalid signature"));
}
match message.results() {
MonitorResults::Route(results) => {
match state.storage.submit_route_monitoring_results(results).await {
Ok(_) => Ok(()),
Err(err) => {
error!("failed to submit node monitoring results: {err}");
Err(AxumErrorResponse::internal_msg(
"failed to submit node monitoring results",
))
}
}
}
MonitorResults::Node(_results) => Err(AxumErrorResponse::bad_request(
"Node monitoring results not supported for this endpoint",
)),
}
}
#[utoipa::path(
tag = "status",
post,
@@ -87,18 +138,21 @@ pub(crate) async fn submit_gateway_monitoring_results(
return Err(AxumErrorResponse::bad_request("invalid signature"));
}
match state
.storage
.submit_gateway_statuses_v2(message.results())
.await
{
Ok(_) => Ok(()),
Err(err) => {
error!("failed to submit gateway monitoring results: {err}");
Err(AxumErrorResponse::internal_msg(
"failed to submit gateway monitoring results",
))
match message.results() {
MonitorResults::Node(results) => {
match state.storage.submit_gateway_statuses_v2(results).await {
Ok(_) => Ok(()),
Err(err) => {
error!("failed to submit node monitoring results: {err}");
Err(AxumErrorResponse::internal_msg(
"failed to submit node monitoring results",
))
}
}
}
MonitorResults::Route(_results) => Err(AxumErrorResponse::bad_request(
"Gateway monitoring results not supported for this endpoint",
)),
}
}
@@ -131,18 +185,21 @@ pub(crate) async fn submit_node_monitoring_results(
return Err(AxumErrorResponse::bad_request("invalid signature"));
}
match state
.storage
.submit_mixnode_statuses_v2(message.results())
.await
{
Ok(_) => Ok(()),
Err(err) => {
error!("failed to submit node monitoring results: {err}");
Err(AxumErrorResponse::internal_msg(
"failed to submit node monitoring results",
))
match message.results() {
MonitorResults::Node(results) => {
match state.storage.submit_mixnode_statuses_v2(results).await {
Ok(_) => Ok(()),
Err(err) => {
error!("failed to submit node monitoring results: {err}");
Err(AxumErrorResponse::internal_msg(
"failed to submit node monitoring results",
))
}
}
}
MonitorResults::Route(_results) => Err(AxumErrorResponse::bad_request(
"Node monitoring results not supported for this endpoint",
)),
}
}
File diff suppressed because it is too large Load Diff
+10
View File
@@ -0,0 +1,10 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
//! Simulation API for reward calculation analysis
//!
//! This module provides REST endpoints for accessing and analyzing
//! simulated reward calculation data comparing old vs new methodologies.
pub(crate) mod handlers;
pub(crate) mod models;
+255
View File
@@ -0,0 +1,255 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
//! API models for simulation data responses
use crate::storage::models::{SimulatedNodePerformance, SimulatedPerformanceComparison, SimulatedRewardEpoch, SimulatedRouteAnalysis};
use nym_mixnet_contract_common::NodeId;
use serde::{Deserialize, Serialize};
use time::OffsetDateTime;
use utoipa::ToSchema;
/// Response for listing simulation epochs
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct SimulationEpochsResponse {
pub epochs: Vec<SimulationEpochSummary>,
pub total_count: usize,
}
/// Summary information about a simulation epoch
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct SimulationEpochSummary {
pub id: i64,
pub epoch_id: u32,
pub calculation_method: String,
pub start_timestamp: i64,
pub end_timestamp: i64,
pub description: Option<String>,
pub created_at: i64,
/// Number of nodes that had performance calculated
pub nodes_analyzed: usize,
/// Available calculation methods for this epoch
pub available_methods: Vec<String>,
}
impl From<SimulatedRewardEpoch> for SimulationEpochSummary {
fn from(epoch: SimulatedRewardEpoch) -> Self {
Self {
id: epoch.id,
epoch_id: epoch.epoch_id,
calculation_method: epoch.calculation_method,
start_timestamp: epoch.start_timestamp,
end_timestamp: epoch.end_timestamp,
description: epoch.description,
created_at: epoch.created_at,
nodes_analyzed: 0, // Will be populated by handler
available_methods: vec![], // Will be populated by handler
}
}
}
/// Detailed simulation epoch with all data
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct SimulationEpochDetails {
pub epoch: SimulationEpochSummary,
pub node_performance: Vec<NodePerformanceData>,
pub performance_comparisons: Vec<PerformanceComparisonData>,
pub route_analysis: Vec<RouteAnalysisData>,
}
/// Node performance data for API responses
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct NodePerformanceData {
pub node_id: NodeId,
pub node_type: String,
pub identity_key: Option<String>,
pub reliability_score: f64,
pub positive_samples: u32,
pub negative_samples: u32,
pub work_factor: Option<f64>,
pub calculation_method: String,
pub calculated_at: i64,
}
impl From<SimulatedNodePerformance> for NodePerformanceData {
fn from(perf: SimulatedNodePerformance) -> Self {
Self {
node_id: perf.node_id,
node_type: perf.node_type,
identity_key: perf.identity_key,
reliability_score: perf.reliability_score,
positive_samples: perf.positive_samples,
negative_samples: perf.negative_samples,
work_factor: perf.work_factor,
calculation_method: perf.calculation_method,
calculated_at: perf.calculated_at,
}
}
}
/// Performance comparison data for API responses
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct PerformanceComparisonData {
pub node_id: NodeId,
pub node_type: String,
pub performance_score: f64,
pub work_factor: f64,
pub calculation_method: String,
pub positive_samples: Option<i64>,
pub negative_samples: Option<i64>,
pub route_success_rate: Option<f64>,
pub calculated_at: i64,
}
impl From<SimulatedPerformanceComparison> for PerformanceComparisonData {
fn from(comparison: SimulatedPerformanceComparison) -> Self {
Self {
node_id: comparison.node_id,
node_type: comparison.node_type,
performance_score: comparison.performance_score,
work_factor: comparison.work_factor,
calculation_method: comparison.calculation_method,
positive_samples: comparison.positive_samples,
negative_samples: comparison.negative_samples,
route_success_rate: comparison.route_success_rate,
calculated_at: comparison.calculated_at,
}
}
}
/// Route analysis data for API responses
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct RouteAnalysisData {
pub calculation_method: String,
pub total_routes_analyzed: u32,
pub successful_routes: u32,
pub failed_routes: u32,
pub average_route_reliability: Option<f64>,
pub time_window_hours: u32,
pub analysis_parameters: Option<String>,
pub calculated_at: i64,
}
impl From<SimulatedRouteAnalysis> for RouteAnalysisData {
fn from(analysis: SimulatedRouteAnalysis) -> Self {
Self {
calculation_method: analysis.calculation_method,
total_routes_analyzed: analysis.total_routes_analyzed,
successful_routes: analysis.successful_routes,
failed_routes: analysis.failed_routes,
average_route_reliability: analysis.average_route_reliability,
time_window_hours: analysis.time_window_hours,
analysis_parameters: analysis.analysis_parameters,
calculated_at: analysis.calculated_at,
}
}
}
/// Comparison between old and new methods for a specific epoch
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct MethodComparisonResponse {
pub epoch_id: u32,
pub simulation_epoch_id: i64,
pub node_comparisons: Vec<NodeMethodComparison>,
pub summary_statistics: ComparisonSummaryStats,
pub route_analysis_comparison: RouteAnalysisComparison,
}
/// Comparison data for a single node between old and new methods
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct NodeMethodComparison {
pub node_id: NodeId,
pub node_type: String,
pub identity_key: Option<String>,
pub old_method: Option<NodePerformanceData>,
pub new_method: Option<NodePerformanceData>,
pub reliability_difference: Option<f64>, // new - old
pub performance_delta_percentage: Option<f64>, // (new - old) / old * 100
pub ranking_old_method: Option<i64>,
pub ranking_new_method: Option<i64>,
pub ranking_delta: Option<i64>, // new ranking - old ranking (negative is improvement)
}
/// Summary statistics comparing old vs new methods
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct ComparisonSummaryStats {
pub total_nodes_compared: usize,
pub nodes_improved: usize, // nodes with better performance in new method
pub nodes_degraded: usize, // nodes with worse performance in new method
pub nodes_unchanged: usize, // nodes with same performance
pub average_reliability_old: f64,
pub average_reliability_new: f64,
pub median_reliability_old: f64,
pub median_reliability_new: f64,
pub reliability_std_dev_old: f64,
pub reliability_std_dev_new: f64,
pub max_improvement: f64, // highest positive delta
pub max_degradation: f64, // highest negative delta
}
/// Comparison of route analysis between methods
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct RouteAnalysisComparison {
pub old_method: Option<RouteAnalysisData>,
pub new_method: Option<RouteAnalysisData>,
pub time_window_difference_hours: i32, // new - old
pub route_coverage_difference: i32, // new total routes - old total routes
pub success_rate_difference: Option<f64>, // new success rate - old success rate
}
/// Export format options
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub enum ExportFormat {
#[serde(rename = "json")]
Json,
#[serde(rename = "csv")]
Csv,
}
/// Query parameters for simulation listings
#[derive(Deserialize, ToSchema, Debug, utoipa::IntoParams)]
pub struct SimulationListQuery {
/// Limit number of results (default: 50, max: 1000)
pub limit: Option<usize>,
/// Offset for pagination (default: 0)
pub offset: Option<usize>,
}
/// Query parameters for node-specific performance comparison
#[derive(Deserialize, ToSchema, Debug, utoipa::IntoParams)]
pub struct NodeComparisonQuery {
/// Specific node ID to analyze
pub node_id: Option<NodeId>,
/// Node type filter (mixnode, gateway)
pub node_type: Option<String>,
/// Minimum reliability difference threshold for filtering
pub min_delta: Option<f64>,
/// Maximum reliability difference threshold for filtering
pub max_delta: Option<f64>,
}
/// Error response for simulation API
#[derive(Serialize, Deserialize, ToSchema, Debug)]
pub struct SimulationApiError {
pub error: String,
pub details: Option<String>,
pub timestamp: i64,
}
impl SimulationApiError {
pub fn new(error: &str) -> Self {
Self {
error: error.to_string(),
details: None,
timestamp: OffsetDateTime::now_utc().unix_timestamp(),
}
}
pub fn with_details(error: &str, details: &str) -> Self {
Self {
error: error.to_string(),
details: Some(details.to_string()),
timestamp: OffsetDateTime::now_utc().unix_timestamp(),
}
}
}
+33 -17
View File
@@ -9,7 +9,7 @@ use crate::ecash::dkg::controller::keys::{
};
use crate::ecash::dkg::controller::DkgController;
use crate::ecash::state::EcashState;
use crate::epoch_operations::EpochAdvancer;
use crate::epoch_operations::{EpochAdvancer, simulation::SimulationConfig};
use crate::network::models::NetworkDetails;
use crate::node_describe_cache::DescribedNodes;
use crate::node_status_api::handlers::unstable;
@@ -61,10 +61,22 @@ pub(crate) struct Args {
long,
requires = "enable_monitor",
requires = "mnemonic",
conflicts_with = "simulate_rewarding",
env = "NYMAPI_ENABLE_REWARDING_ARG"
)]
pub(crate) enable_rewarding: Option<bool>,
/// Enable simulated rewarding mode to compare old vs new reward calculations
/// Requires monitoring but does NOT require mnemonic (no blockchain transactions)
/// default: None - config value will be used instead
#[clap(
long,
requires = "enable_monitor",
conflicts_with = "enable_rewarding",
env = "NYMAPI_SIMULATE_REWARDING_ARG"
)]
pub(crate) simulate_rewarding: Option<bool>,
/// Endpoint to nyxd instance used for contract information.
/// default: None - config value will be used instead
#[clap(long, env = "NYMAPI_NYXD_VALIDATOR_ARG")]
@@ -102,15 +114,6 @@ pub(crate) struct Args {
#[clap(long)]
pub(crate) bind_address: Option<SocketAddr>,
/// account/address cache TTL: should be lower than epoch length (1 hour)
/// because, at worst, data will be stale for <epoch_length> + <cache_ttl> seconds
#[clap(long, env = "ADDRESS_CACHE_REFRESH_INTERVAL_S")]
pub(crate) address_cache_ttl_seconds: Option<u64>,
/// number of addresses that are cached on account/address endpoint
#[clap(long, env = "ADDRESS_CACHE_CAPACITY")]
pub(crate) address_cache_capacity: Option<u64>,
#[clap(hide = true, long, default_value_t = false)]
pub(crate) allow_illegal_ips: bool,
}
@@ -203,10 +206,7 @@ async fn start_nym_api_tasks_axum(config: &Config) -> anyhow::Result<ShutdownHan
let router = router.with_state(AppState {
nyxd_client: nyxd_client.clone(),
chain_status_cache: ChainStatusCache::new(DEFAULT_CHAIN_STATUS_CACHE_TTL),
address_info_cache: AddressInfoCache::new(
config.address_cache.time_to_live,
config.address_cache.capacity,
),
address_info_cache: AddressInfoCache::new(),
forced_refresh: ForcedRefresh::new(
config.topology_cacher.debug.node_describe_allow_illegal_ips,
),
@@ -288,15 +288,31 @@ async fn start_nym_api_tasks_axum(config: &Config) -> anyhow::Result<ShutdownHan
HistoricalUptimeUpdater::start(storage.to_owned(), &task_manager);
// start 'rewarding' if its enabled
if config.rewarding.enabled {
epoch_operations::ensure_rewarding_permission(&nyxd_client).await?;
// start 'rewarding' or 'simulation' if enabled
if config.rewarding.enabled || config.rewarding.simulation_mode {
// Only check rewarding permission for real rewarding, not simulation
if config.rewarding.enabled && !config.rewarding.simulation_mode {
epoch_operations::ensure_rewarding_permission(&nyxd_client).await?;
}
// Create simulation config if simulation mode is enabled
let simulation_config = if config.rewarding.simulation_mode {
Some(SimulationConfig {
new_method_time_window_hours: config.rewarding.debug.simulation_new_method_time_window_hours,
run_both_methods: config.rewarding.debug.simulation_run_both_methods,
description: Some("Simulation run at epoch advancement".to_string()),
})
} else {
None
};
EpochAdvancer::start(
nyxd_client,
&nym_contract_cache_state,
&node_status_cache_state,
described_nodes_cache.clone(),
&storage,
simulation_config,
&task_manager,
);
}
+33 -29
View File
@@ -54,9 +54,6 @@ const DEFAULT_TOPOLOGY_CACHE_INTERVAL: Duration = Duration::from_secs(30);
const DEFAULT_NODE_STATUS_CACHE_INTERVAL: Duration = Duration::from_secs(120);
const DEFAULT_CIRCULATING_SUPPLY_CACHE_INTERVAL: Duration = Duration::from_secs(3600);
pub(crate) const DEFAULT_ADDRESS_CACHE_TTL: Duration = Duration::from_secs(60 * 15);
pub(crate) const DEFAULT_ADDRESS_CACHE_CAPACITY: u64 = 1000;
pub(crate) const DEFAULT_NODE_DESCRIBE_CACHE_INTERVAL: Duration = Duration::from_secs(4500);
pub(crate) const DEFAULT_NODE_DESCRIBE_BATCH_SIZE: usize = 50;
@@ -114,9 +111,6 @@ pub struct Config {
#[serde(alias = "coconut_signer")]
pub ecash_signer: EcashSigner,
#[serde(skip)]
pub address_cache: AddressCacheConfig,
}
impl NymConfigTemplate for Config {
@@ -136,21 +130,36 @@ impl Config {
circulating_supply_cacher: Default::default(),
rewarding: Default::default(),
ecash_signer: EcashSigner::new_default(id.as_ref()),
address_cache: Default::default(),
}
}
pub fn validate(&self) -> anyhow::Result<()> {
let can_sign = self.base.mnemonic.is_some();
if !can_sign && self.rewarding.enabled {
// Real rewarding requires mnemonic, but simulation mode does not
if !can_sign && self.rewarding.enabled && !self.rewarding.simulation_mode {
bail!("can't enable rewarding without providing a mnemonic")
}
// Simulation mode and real rewarding are mutually exclusive
if self.rewarding.enabled && self.rewarding.simulation_mode {
bail!("cannot enable both real rewarding and simulation mode simultaneously")
}
if !can_sign && self.ecash_signer.enabled {
bail!("can't enable coconut signer without providing a mnemonic")
}
// Validate simulation-specific settings
if self.rewarding.simulation_mode {
if self.rewarding.debug.simulation_new_method_time_window_hours == 0 {
bail!("simulation_new_method_time_window_hours must be greater than 0")
}
if self.rewarding.debug.simulation_new_method_time_window_hours > 24 {
bail!("simulation_new_method_time_window_hours should not exceed 24 hours")
}
}
self.ecash_signer.validate()?;
Ok(())
@@ -165,6 +174,9 @@ impl Config {
if let Some(enable_rewarding) = args.enable_rewarding {
self.rewarding.enabled = enable_rewarding;
}
if let Some(simulate_rewarding) = args.simulate_rewarding {
self.rewarding.simulation_mode = simulate_rewarding;
}
if let Some(nyxd_upstream) = args.nyxd_validator {
self.base.local_validator = nyxd_upstream;
}
@@ -186,12 +198,6 @@ impl Config {
if args.allow_illegal_ips {
self.topology_cacher.debug.node_describe_allow_illegal_ips = true
}
if let Some(address_cache_ttl) = args.address_cache_ttl {
self.address_cache.time_to_live = address_cache_ttl;
}
if let Some(address_cache_capacity) = args.address_cache_capacity {
self.address_cache.capacity = address_cache_capacity;
}
self
}
@@ -305,21 +311,6 @@ impl Base {
}
}
#[derive(Debug, PartialEq, Eq)]
pub struct AddressCacheConfig {
pub time_to_live: Duration,
pub capacity: u64,
}
impl Default for AddressCacheConfig {
fn default() -> Self {
Self {
time_to_live: DEFAULT_ADDRESS_CACHE_TTL,
capacity: DEFAULT_ADDRESS_CACHE_CAPACITY,
}
}
}
// this got separated into 2 structs so that we could have a sane `default` implementation for the latter
#[derive(Debug, Deserialize, PartialEq, Eq, Serialize)]
pub struct NetworkMonitor {
@@ -527,6 +518,9 @@ pub struct Rewarding {
/// Specifies whether rewarding service is enabled in this process.
pub enabled: bool,
/// Specifies whether to run in simulation mode (compare old vs new calculations without blockchain transactions)
pub simulation_mode: bool,
// this should really be a thing too...
// pub paths: RewardingPathfinder,
#[serde(default)]
@@ -538,6 +532,7 @@ impl Default for Rewarding {
fn default() -> Self {
Rewarding {
enabled: false,
simulation_mode: false,
debug: Default::default(),
}
}
@@ -550,12 +545,21 @@ pub struct RewardingDebug {
/// distribute rewards for given interval.
/// Note, only values in range 0-100 are valid
pub minimum_interval_monitor_threshold: u8,
/// Time window in hours for new route-based performance calculation (default: 1 hour)
/// Old method always uses 24 hours for comparison
pub simulation_new_method_time_window_hours: u32,
/// Whether to run both old and new calculations in simulation mode
pub simulation_run_both_methods: bool,
}
impl Default for RewardingDebug {
fn default() -> Self {
RewardingDebug {
minimum_interval_monitor_threshold: DEFAULT_MONITOR_THRESHOLD,
simulation_new_method_time_window_hours: 1, // New method uses 1 hour
simulation_run_both_methods: true, // Default to running both for comparison
}
}
}
+6 -9
View File
@@ -2,7 +2,7 @@
// SPDX-License-Identifier: GPL-3.0-only
use crate::support::cli::{init, run};
use std::{net::SocketAddr, time::Duration};
use std::net::SocketAddr;
// Configuration that can be overridden.
pub(crate) struct OverrideConfig {
@@ -12,6 +12,9 @@ pub(crate) struct OverrideConfig {
/// Specifies whether network rewarding is enabled on this API
pub(crate) enable_rewarding: Option<bool>,
/// Specifies whether simulated rewarding is enabled on this API
pub(crate) simulate_rewarding: Option<bool>,
/// Endpoint to nyxd instance used for contract information.
pub(crate) nyxd_validator: Option<url::Url>,
@@ -32,9 +35,6 @@ pub(crate) struct OverrideConfig {
/// default: `127.0.0.1:8080` in `debug` builds and `0.0.0.0:8080` in `release`
pub(crate) bind_address: Option<SocketAddr>,
pub(crate) address_cache_ttl: Option<Duration>,
pub(crate) address_cache_capacity: Option<u64>,
pub(crate) allow_illegal_ips: bool,
}
@@ -43,15 +43,13 @@ impl From<init::Args> for OverrideConfig {
OverrideConfig {
enable_monitor: Some(args.enable_monitor),
enable_rewarding: Some(args.enable_rewarding),
simulate_rewarding: None, // Not available during initialization
nyxd_validator: args.nyxd_validator,
mnemonic: args.mnemonic,
enable_zk_nym: Some(args.enable_zk_nym),
announce_address: args.announce_address,
monitor_credentials_mode: Some(args.monitor_credentials_mode),
bind_address: args.bind_address,
// irrelevant for --init command because we set the value in --run
address_cache_ttl: None,
address_cache_capacity: None,
allow_illegal_ips: args.allow_illegal_ips,
}
}
@@ -62,14 +60,13 @@ impl From<run::Args> for OverrideConfig {
OverrideConfig {
enable_monitor: args.enable_monitor,
enable_rewarding: args.enable_rewarding,
simulate_rewarding: args.simulate_rewarding,
nyxd_validator: args.nyxd_validator,
mnemonic: args.mnemonic,
enable_zk_nym: args.enable_zk_nym,
announce_address: args.announce_address,
monitor_credentials_mode: args.monitor_credentials_mode,
bind_address: args.bind_address,
address_cache_ttl: args.address_cache_ttl_seconds.map(Duration::from_secs),
address_cache_capacity: args.address_cache_capacity,
allow_illegal_ips: args.allow_illegal_ips,
}
}
+2
View File
@@ -8,6 +8,7 @@ use crate::node_status_api::handlers::status_routes;
use crate::nym_contract_cache::handlers::nym_contract_cache_routes;
use crate::nym_nodes::handlers::legacy::legacy_nym_node_routes;
use crate::nym_nodes::handlers::nym_node_routes;
use crate::simulation_api::handlers::simulation_routes;
use crate::status;
use crate::support::http::openapi::ApiDoc;
use crate::support::http::state::AppState;
@@ -64,6 +65,7 @@ impl RouterBuilder {
.nest("/api-status", status::handlers::api_status_routes())
.nest("/nym-nodes", nym_node_routes())
.nest("/ecash", ecash_routes())
.nest("/simulation", simulation_routes())
.nest("/unstable", unstable_routes()), // CORS layer needs to be "outside" of routes
);
File diff suppressed because it is too large Load Diff
+34 -2
View File
@@ -17,7 +17,7 @@ use crate::support::storage::models::{
};
use dashmap::DashMap;
use nym_mixnet_contract_common::NodeId;
use nym_types::monitoring::NodeResult;
use nym_types::monitoring::{NodeResult, RouteResult};
use sqlx::sqlite::{SqliteAutoVacuum, SqliteSynchronous};
use sqlx::ConnectOptions;
use std::path::Path;
@@ -31,6 +31,9 @@ pub(crate) mod manager;
pub(crate) mod models;
pub(crate) mod runtime_migrations;
#[cfg(test)]
mod tests;
#[derive(Default)]
pub(crate) struct DbIdCache {
pub mixnodes_v1: DashMap<NodeId, i64>,
@@ -835,6 +838,16 @@ impl NymApiStorage {
Ok(())
}
pub(crate) async fn submit_route_monitoring_results(
&self,
route_results: &[RouteResult],
) -> Result<(), NymApiStorageError> {
self.manager
.submit_route_monitoring_results(route_results)
.await?;
Ok(())
}
/// Obtains number of network monitor test runs that have occurred within the specified interval.
///
/// # Arguments
@@ -930,8 +943,9 @@ impl NymApiStorage {
/// * `until`: timestamp specifying the purge cutoff.
pub(crate) async fn purge_old_statuses(&self, until: i64) -> Result<(), NymApiStorageError> {
self.manager.purge_old_mixnode_statuses(until).await?;
self.manager.purge_old_gateway_statuses(until).await?;
self.manager
.purge_old_gateway_statuses(until)
.purge_old_routes(until)
.await
.map_err(|err| err.into())
}
@@ -1035,5 +1049,23 @@ pub(crate) mod v3_migration {
pub(crate) async fn make_node_id_not_null(&self) -> Result<(), NymApiStorageError> {
Ok(self.manager.make_node_id_not_null().await?)
}
/// Get mixnode identity key by node ID
#[allow(dead_code)]
pub(crate) async fn get_mixnode_identity_key(
&self,
mix_id: NodeId,
) -> Result<Option<String>, NymApiStorageError> {
Ok(self.manager.get_mixnode_identity_key(mix_id).await?)
}
/// Get gateway identity key by node ID
#[allow(dead_code)]
pub(crate) async fn get_gateway_identity_key(
&self,
node_id: NodeId,
) -> Result<Option<String>, NymApiStorageError> {
Ok(self.manager.get_gateway_identity_key(node_id).await?)
}
}
}
+79
View File
@@ -146,3 +146,82 @@ pub struct HistoricalUptime {
pub date: Date,
pub uptime: i64,
}
// Simulated Rewarding System Models
// These models support comparison between old (24h cache-based) and new (1h route-based) rewarding
/// Represents a simulated reward epoch run
#[derive(FromRow, Debug, Clone)]
pub struct SimulatedRewardEpoch {
pub id: i64,
pub epoch_id: u32,
pub calculation_method: String, // 'old', 'new', or 'comparison'
pub start_timestamp: i64,
pub end_timestamp: i64,
pub description: Option<String>,
pub created_at: i64,
}
/// Node performance calculated using different methodologies
#[derive(FromRow, Debug, Clone)]
pub struct SimulatedNodePerformance {
#[allow(dead_code)]
pub id: i64,
pub simulated_epoch_id: i64,
pub node_id: NodeId,
pub node_type: String, // 'mixnode' or 'gateway'
pub identity_key: Option<String>,
pub reliability_score: f64, // 0.0 to 100.0
pub positive_samples: u32,
pub negative_samples: u32,
pub work_factor: Option<f64>, // 0.0 to 1.0
pub calculation_method: String, // 'old' or 'new'
pub calculated_at: i64,
}
/// Performance comparison data for analyzing methodology differences
#[derive(FromRow, Debug, Clone)]
pub struct SimulatedPerformanceComparison {
#[allow(dead_code)]
pub id: i64,
pub simulated_epoch_id: i64,
pub node_id: NodeId,
pub node_type: String, // 'mixnode' or 'gateway'
pub performance_score: f64, // 0.0 to 100.0
pub work_factor: f64, // Work factor applied (e.g., 10.0 for active, 1.0 for standby)
pub calculation_method: String, // 'old' or 'new'
pub positive_samples: Option<i64>,
pub negative_samples: Option<i64>,
pub route_success_rate: Option<f64>, // 0.0 to 100.0, mainly for new method
pub calculated_at: i64,
}
/// Performance ranking data for nodes
#[derive(FromRow, Debug, Clone)]
pub struct SimulatedPerformanceRanking {
#[allow(dead_code)]
pub id: i64,
pub simulated_epoch_id: i64,
pub node_id: NodeId,
pub calculation_method: String,
pub performance_rank: i64,
pub performance_percentile: f64,
#[allow(dead_code)]
pub calculated_at: i64,
}
/// Route analysis metadata for simulation runs
#[derive(FromRow, Debug, Clone)]
pub struct SimulatedRouteAnalysis {
#[allow(dead_code)]
pub id: i64,
pub simulated_epoch_id: i64,
pub calculation_method: String, // 'old' or 'new'
pub total_routes_analyzed: u32,
pub successful_routes: u32,
pub failed_routes: u32,
pub average_route_reliability: Option<f64>, // 0.0 to 100.0
pub time_window_hours: u32, // 1 for new method, 24 for old method
pub analysis_parameters: Option<String>, // JSON with analysis config
pub calculated_at: i64,
}
+432
View File
@@ -0,0 +1,432 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
#[cfg(test)]
mod simulation_storage_tests {
use super::super::models::{
SimulatedNodePerformance, SimulatedPerformanceComparison, SimulatedRouteAnalysis,
};
use super::super::NymApiStorage;
use tempfile::NamedTempFile;
async fn create_test_storage() -> NymApiStorage {
let temp_file = NamedTempFile::new().unwrap();
let temp_path = temp_file.path().to_path_buf();
// Keep the temp file alive
Box::leak(Box::new(temp_file));
NymApiStorage::init(temp_path).await.unwrap()
}
#[tokio::test]
async fn test_simulated_reward_epoch_crud() {
let storage = create_test_storage().await;
// Test create
let epoch_id = storage
.manager
.create_simulated_reward_epoch(
100,
"test",
1234567890,
1234571490,
Some("Test description"),
)
.await
.unwrap();
assert!(epoch_id > 0);
// Test retrieve
let retrieved = storage
.manager
.get_simulated_reward_epoch(epoch_id)
.await
.unwrap();
assert!(retrieved.is_some());
let epoch = retrieved.unwrap();
assert_eq!(epoch.epoch_id, 100);
assert_eq!(epoch.calculation_method, "test");
assert_eq!(epoch.start_timestamp, 1234567890);
assert_eq!(epoch.end_timestamp, 1234571490);
assert_eq!(epoch.description, Some("Test description".to_string()));
}
#[tokio::test]
async fn test_simulated_node_performance_crud() {
let storage = create_test_storage().await;
// Create parent epoch first
let epoch_id = storage
.manager
.create_simulated_reward_epoch(100, "test", 1234567890, 1234571490, None)
.await
.unwrap();
// Test insert node performance
let performance = SimulatedNodePerformance {
id: 0, // Will be set by database
simulated_epoch_id: epoch_id,
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: Some("test_key".to_string()),
reliability_score: 85.5,
positive_samples: 100,
negative_samples: 15,
work_factor: Some(0.95),
calculation_method: "new".to_string(),
calculated_at: 1234567890,
};
storage
.manager
.insert_simulated_node_performance(&[performance])
.await
.unwrap();
// Test retrieve
let retrieved = storage
.manager
.get_simulated_node_performance_for_epoch(epoch_id)
.await
.unwrap();
assert_eq!(retrieved.len(), 1);
let perf = &retrieved[0];
assert_eq!(perf.node_id, 42);
assert_eq!(perf.node_type, "mixnode");
assert_eq!(perf.reliability_score, 85.5);
assert_eq!(perf.calculation_method, "new");
}
#[tokio::test]
async fn test_simulated_performance_comparisons_crud() {
let storage = create_test_storage().await;
// Create parent epoch first
let epoch_id = storage
.manager
.create_simulated_reward_epoch(100, "test", 1234567890, 1234571490, None)
.await
.unwrap();
// Test insert performance comparison
let comparison = SimulatedPerformanceComparison {
id: 0,
simulated_epoch_id: epoch_id,
node_id: 42,
node_type: "mixnode".to_string(),
performance_score: 85.0,
work_factor: 10.0,
calculation_method: "new".to_string(),
positive_samples: Some(100),
negative_samples: Some(15),
route_success_rate: Some(85.0),
calculated_at: 1234567890,
};
storage
.manager
.insert_simulated_performance_comparisons(&[comparison])
.await
.unwrap();
// Test retrieve
let retrieved = storage
.manager
.get_simulated_performance_comparisons_for_epoch(epoch_id)
.await
.unwrap();
assert_eq!(retrieved.len(), 1);
let comp = &retrieved[0];
assert_eq!(comp.node_id, 42);
assert_eq!(comp.performance_score, 85.0);
assert_eq!(comp.work_factor, 10.0);
assert_eq!(comp.calculation_method, "new");
}
#[tokio::test]
async fn test_simulated_route_analysis_crud() {
let storage = create_test_storage().await;
// Create parent epoch first
let epoch_id = storage
.manager
.create_simulated_reward_epoch(100, "test", 1234567890, 1234571490, None)
.await
.unwrap();
// Test insert route analysis
let analysis = SimulatedRouteAnalysis {
id: 0,
simulated_epoch_id: epoch_id,
calculation_method: "new".to_string(),
total_routes_analyzed: 1000,
successful_routes: 950,
failed_routes: 50,
average_route_reliability: Some(95.0),
time_window_hours: 1,
analysis_parameters: Some("{\"threshold\": 0.8}".to_string()),
calculated_at: 1234567890,
};
storage
.manager
.insert_simulated_route_analysis(&analysis)
.await
.unwrap();
// Test retrieve
let retrieved = storage
.manager
.get_simulated_route_analysis_for_epoch(epoch_id)
.await
.unwrap();
assert_eq!(retrieved.len(), 1);
let route = &retrieved[0];
assert_eq!(route.calculation_method, "new");
assert_eq!(route.total_routes_analyzed, 1000);
assert_eq!(route.successful_routes, 950);
assert_eq!(route.average_route_reliability, Some(95.0));
assert_eq!(route.time_window_hours, 1);
}
#[tokio::test]
async fn test_foreign_key_constraints() {
let storage = create_test_storage().await;
// Try to insert node performance without valid epoch - should fail
let performance = SimulatedNodePerformance {
id: 0,
simulated_epoch_id: 999999, // Non-existent epoch
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: None,
reliability_score: 85.5,
positive_samples: 100,
negative_samples: 15,
work_factor: Some(0.95),
calculation_method: "new".to_string(),
calculated_at: 1234567890,
};
let result = storage
.manager
.insert_simulated_node_performance(&[performance])
.await;
assert!(result.is_err()); // Should fail due to foreign key constraint
}
#[tokio::test]
async fn test_performance_by_method_queries() {
let storage = create_test_storage().await;
// Create epoch
let epoch_id = storage
.manager
.create_simulated_reward_epoch(100, "comparison", 1234567890, 1234571490, None)
.await
.unwrap();
// Insert performance data for both methods
let old_performance = SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch_id,
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: Some("key42".to_string()),
reliability_score: 80.0,
positive_samples: 100,
negative_samples: 20,
work_factor: Some(0.9),
calculation_method: "old".to_string(),
calculated_at: 1234567890,
};
let new_performance = SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch_id,
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: Some("key42".to_string()),
reliability_score: 90.0,
positive_samples: 95,
negative_samples: 5,
work_factor: Some(0.95),
calculation_method: "new".to_string(),
calculated_at: 1234567890,
};
storage
.manager
.insert_simulated_node_performance(&[old_performance])
.await
.unwrap();
storage
.manager
.insert_simulated_node_performance(&[new_performance])
.await
.unwrap();
// Test querying by method
let old_results = storage
.manager
.get_simulated_node_performance_by_method(100, "old")
.await
.unwrap();
let new_results = storage
.manager
.get_simulated_node_performance_by_method(100, "new")
.await
.unwrap();
assert_eq!(old_results.len(), 1);
assert_eq!(new_results.len(), 1);
assert_eq!(old_results[0].reliability_score, 80.0);
assert_eq!(new_results[0].reliability_score, 90.0);
assert_eq!(old_results[0].calculation_method, "old");
assert_eq!(new_results[0].calculation_method, "new");
}
#[tokio::test]
async fn test_node_performance_history() {
let storage = create_test_storage().await;
// Create multiple epochs
let epoch1_id = storage
.manager
.create_simulated_reward_epoch(100, "comparison", 1234567890, 1234571490, None)
.await
.unwrap();
let epoch2_id = storage
.manager
.create_simulated_reward_epoch(101, "comparison", 1234571490, 1234575090, None)
.await
.unwrap();
// Insert performance data for same node across epochs
let performances = vec![
SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch1_id,
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: Some("key42".to_string()),
reliability_score: 80.0,
positive_samples: 100,
negative_samples: 20,
work_factor: Some(0.9),
calculation_method: "old".to_string(),
calculated_at: 1234567890,
},
SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch2_id,
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: Some("key42".to_string()),
reliability_score: 85.0,
positive_samples: 105,
negative_samples: 15,
work_factor: Some(0.92),
calculation_method: "old".to_string(),
calculated_at: 1234571490,
},
];
for perf in performances {
storage
.manager
.insert_simulated_node_performance(&[perf])
.await
.unwrap();
}
// Test node history query
let history = storage
.manager
.get_simulated_node_performance_history(42)
.await
.unwrap();
assert_eq!(history.len(), 2);
// Should be ordered by epoch_id DESC
assert_eq!(history[0].reliability_score, 85.0); // epoch 101
assert_eq!(history[1].reliability_score, 80.0); // epoch 100
}
#[tokio::test]
async fn test_count_operations() {
let storage = create_test_storage().await;
// Create epoch
let epoch_id = storage
.manager
.create_simulated_reward_epoch(100, "test", 1234567890, 1234571490, None)
.await
.unwrap();
// Initially should be 0
let count = storage
.manager
.count_simulated_node_performance_for_epoch(epoch_id)
.await
.unwrap();
assert_eq!(count, 0);
// Insert some performance data
let performances = vec![
SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch_id,
node_id: 1,
node_type: "mixnode".to_string(),
identity_key: None,
reliability_score: 80.0,
positive_samples: 100,
negative_samples: 20,
work_factor: Some(0.9),
calculation_method: "old".to_string(),
calculated_at: 1234567890,
},
SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch_id,
node_id: 2,
node_type: "gateway".to_string(),
identity_key: None,
reliability_score: 90.0,
positive_samples: 120,
negative_samples: 10,
work_factor: None,
calculation_method: "new".to_string(),
calculated_at: 1234567890,
},
];
for perf in performances {
storage
.manager
.insert_simulated_node_performance(&[perf])
.await
.unwrap();
}
// Should now be 2
let count = storage
.manager
.count_simulated_node_performance_for_epoch(epoch_id)
.await
.unwrap();
assert_eq!(count, 2);
}
}

Some files were not shown because too many files have changed in this diff Show More