Compare commits

...

56 Commits

Author SHA1 Message Date
durch 688660a7d5 Remove OR ignore 2025-06-25 11:44:44 +02:00
durch e1a0235556 Filter only on tested nodes 2025-06-25 10:59:39 +02:00
durch 29227f452d fix(nym-api): prevent duplicate simulations for the same epoch
- Add create_or_get_simulated_reward_epoch to check for existing simulations
- Skip simulation if already exists for the epoch and calculation method
- Add tests to verify duplicate prevention works correctly
- Update all test calls to use the new method

This fixes the issue where multiple simulations were created for the same
epoch when epoch operations were retried or the service restarted.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-25 10:50:15 +02:00
durch c5c64327da Fix simulated epoch generation 2025-06-25 08:31:14 +02:00
durch 1364316f95 FMT, fix typo 2025-06-25 00:43:53 +02:00
durch 55f0471481 fix(nym-api): refactor simulation API to remove redundant data structures
- Remove old_method/new_method nested objects from NodeMethodComparison
- Use direct fields for production_performance and simulated_performance
- Update build_node_comparisons_from_single_dataset to create cleaner structure
- Fix calculate_summary_statistics to work with new data model
- Update all tests to match new structure

The API now returns a single dataset with production performance values
included directly, rather than separate old/new method objects.

- Deduplicate routes, to control for deterministic routing
2025-06-25 00:40:52 +02:00
durch a127a303f1 refactor(nym-api): simplify simulation to only run new method
Since old method data is already available in production database,
simulation now only calculates new method performance. The API combines
production data with simulated data at query time.

Key changes:
- Remove old_method_simulation and run_both_methods config
- Add production_performance field to NodePerformanceData model
- Update API endpoints to fetch production data from node annotations cache
- Refactor compare_methods to use single performance dataset
- Fix route analysis comparison to handle missing old method data
- Update all tests to use new single-dataset approach

This simplifies the simulation code and ensures consistency with
production calculations.
2025-06-24 23:43:24 +02:00
durch 730c88b0d2 feat(nym-api): add reliability distribution categories to simulation comparison
- Added ReliabilityDistribution struct with 6 categories: excellent (>95%), very_good (90-95%), good (75-90%), moderate (50-75%), poor (25-50%), very_poor (<25%)
- Updated ComparisonSummaryStats to include distribution breakdowns for both old and new methods
- Provides better insight into performance distribution beyond averages
- Added tests for distribution calculation
2025-06-24 21:53:29 +02:00
durch 9cb18cd163 Bump nnm 2025-06-24 21:35:06 +02:00
durch 6993ef0dc8 Bump nym-api 2025-06-24 21:31:50 +02:00
durch 0e53562ce2 Fmt 2025-06-24 21:31:15 +02:00
Drazen Urch c1acef9bc8 Check gateway supported versions (#5860)
* Check gateway supported versions

* Fix boxed errors

* Fmt

* feat(client-core): integrate gateway protocol validation into SDK

Enable protocol version checking for all SDK-based clients by using
gateways_for_init_with_protocol_validation in MixnetClientBuilder.
This ensures clients only connect to gateways with compatible protocol
versions, preventing potential communication issues.

- Replace gateways_for_init with gateways_for_init_with_protocol_validation
- Add import for the new validation function
- Protocol validation now active for network monitor and all SDK users

* refactor(client-core): allow gateways with newer protocol versions

Change protocol validation to be more permissive - instead of rejecting
gateways with newer protocol versions, now logs a warning and continues.
This enables graceful degradation when gateways upgrade, relying on their
backward compatibility while signaling users to update their clients.

Changes:
- Accept gateways with protocol version > client version
- Log warning about version mismatch suggesting client update
- Update log messages from "validation" to "check" for clarity
- Add trace logging showing both gateway and client versions

This prevents connectivity issues when gateways upgrade before clients,
improving overall network resilience.

* feat(nym-network-monitor): add diagnostic logging for ConnectionRefused debugging

- Add detailed logging for client rotation lifecycle with [CLIENT_ROTATION_*] tags
- Add request tracking with unique IDs using the random message as identifier
- Add HTTP connection acceptance logging with [HTTP_REQUEST] tags
- Improve locust script with connection error handling and backoff strategy
- Add TCP backlog configuration support (default 1024)
- Add socket configuration with SO_REUSEADDR and configurable backlog
- Add HTTP timeout and tracing layers for better observability

These changes help diagnose when and why ConnectionRefused errors occur during load testing,
particularly around client rotation periods.

* fix(nym-api): calculate route average reliability as node mean instead of route-based

- Changed route average reliability calculation to use mean of all node reliabilities
- Added median calculation alongside mean for better statistical representation
- Fixed null average reliability for old method which doesn't analyze routes
- Rounded all float values to 2 decimal places in API responses for cleaner display
- Store median values in analysis_parameters JSON field
2025-06-24 21:28:07 +02:00
durch d67a968e76 fix(nym-api): correct reliability value conversions in performance simulation
The old method simulation was incorrectly treating reliability values (0-1 fractions)
as percentages, causing all performance scores to be truncated to 0 when cast to u64.

Changes:
- Old method: multiply reliability by 100 before passing to Performance::from_percentage_value()
- New method: remove division by 100 since Performance::naive_try_from_f64() expects fractions
- Route analysis: multiply average reliability by 100 for percentage display

This ensures consistent handling where internal calculations use fractions (0-1)
while storage and API responses use percentages (0-100).
2025-06-24 19:05:07 +02:00
durch 8ad641f8f8 fix(nym-api): allow simulation mode to run when another API is advancing epoch
In simulation mode, the nym-api should be able to run performance
simulations regardless of which instance is handling epoch advancement,
since simulations don't perform any blockchain transactions.

This fix:
- Allows simulation mode to proceed when another API is advancing the epoch
- Skips epoch transition attempts in simulation mode
- Skips actual reward distribution blockchain operations in simulation mode
2025-06-24 17:44:25 +02:00
durch b45bb9e7e9 Add socket-level websocket connection health checking
Implements proper socket-level health checking for gateway websocket connections:

- Add is_connection_alive() method to GatewayTransceiver trait
- Implement socket-level health check using TcpStream::peek() on Unix systems
- Add is_gateway_connection_alive() method to MixnetClient for easy access
- Update network monitor to use connection health checks before sending
- Fix axum router state configuration in network monitor

The health check performs actual OS-level socket validation rather than just
checking file descriptor existence, providing reliable connection status.
2025-06-06 12:15:58 +02:00
durch cab072f2d0 Fix client drop loop hanging in network monitor
Replace infinite busy-wait loop with timeout-based client cleanup to prevent potential system hangs during client lifecycle management.
2025-06-06 11:02:06 +02:00
durch c389f43dd0 Put client factory back 2025-06-06 10:51:32 +02:00
durch 71c24d8c81 Respond with 504 to timeouts 2025-06-06 10:49:03 +02:00
durch e336e02df2 Dont use hickory dns for gateway client 2025-06-06 10:42:43 +02:00
durch 74db9be819 Fix config cleanup 2025-06-06 10:22:55 +02:00
durch 77c4acf602 Add RESET_CONFIG option to entrypoint 2025-06-06 09:38:28 +02:00
durch f4d0ac855c Add periodic route data cleanup to epoch operations
Implements automatic cleanup of route monitoring results to prevent
unbounded storage growth while maintaining data for performance analysis.

- Add purge_old_routes() method to StorageManager following existing patterns
- Integrate route cleanup into purge_old_statuses() wrapper function
- Route data now purged every epoch with 48-hour retention, to facilitate comparisons with legacy data
- Update logging to reflect cleanup of both node statuses and routes
2025-06-06 09:12:21 +02:00
durch eb1c7d649e Client per request 2025-06-05 12:06:15 +02:00
durch 75f34ef51b Add timeout to locust 2025-06-05 11:43:04 +02:00
durch 4f7fa557d5 Optimize database queries by eliminating N+1 patterns in simulation system
This commit addresses critical N+1 query performance issues identified in
the reward simulation system. The optimizations significantly reduce database
round trips and improve performance when processing large datasets.

**Key Optimizations:**

1. **Batch Identity Key Lookups**
   - Added `get_mixnode_identity_keys_batch()` and `get_gateway_identity_keys_batch()`
   - Updated simulation performance conversion to use batch operations
   - Reduced from N individual queries to 2 batch queries

2. **Batch Node Classification**
   - Added `classify_nodes_batch()` method for mixnode/gateway determination
   - Updated reliability calculation methods to use batch classification
   - Reduced from N individual lookups to 2 batch queries

3. **Batch Epoch Metadata Enhancement**
   - Added `count_simulated_node_performance_for_epochs_batch()`
   - Added `get_available_calculation_methods_for_epochs_batch()`
   - Updated API handlers to use batch operations for metadata enhancement
   - Reduced from 2N queries to 2 batch queries for epoch data

4. **Bulk Insert Optimizations**
   - Converted individual INSERT operations to use `sqlx::QueryBuilder::push_values()`
   - Optimized simulation data insertion methods
   - Eliminated transaction overhead from individual inserts

**Performance Impact:**
- Before: N+2N database queries for N nodes/epochs
- After: 2+2 batch queries regardless of dataset size
- Significant performance improvement for large simulation datasets

All changes maintain backward compatibility while providing substantial
performance benefits for the reward simulation system.
2025-06-05 10:45:08 +02:00
durch a96fb098c2 Locust sleep if no clients are available 2025-06-05 09:54:39 +02:00
durch ad5c6ab829 Enhance simulation system with performance comparison framework
Refactors the simulation system to focus on performance methodology comparison
rather than reward amounts, enabling robust analysis of old vs new calculation
methods. Key improvements:

- Replace simulated_rewards table with performance_comparisons for better metrics
- Add performance_rankings table for ranking analysis across methodologies
- Enhance database schema with additional performance tracking fields
- Update simulation coordinator to use performance-focused data structures
- Add comprehensive performance ranking calculations
- Improve API models and handlers for performance comparison workflows
- Update SQLx query cache with new database schema changes

This provides a foundation for data-driven performance methodology evaluation
while maintaining separation from actual reward calculations.
2025-06-04 16:59:42 +02:00
durch b3d07e8832 Tests 2025-06-04 11:28:47 +02:00
durch e761255174 Add complete simulation API layer for reward method comparison
This completes Phase 3 of the simulation system implementation:

- Add comprehensive REST API endpoints for simulation data access
- Implement /v1/simulation/* routes with full CRUD operations
- Support JSON/CSV export for external analysis
- Add statistical comparison between old vs new methods
- Provide node performance history tracking
- Include proper error handling and response formatting
- Simplify simulation coordinator to remove unused complex return types
- Clean up dead code while maintaining all functionality
- Pass clippy with no warnings

The simulation API provides complete access to:
- Simulation epoch listing and details
- Method comparison analytics (old 24h vs new 1h)
- Node performance analysis across epochs
- Route reliability statistics
- Export capabilities for further analysis

All simulation data is persisted and accessible via REST endpoints.
2025-06-03 15:38:58 +02:00
durch e4a20f9cf5 Implement core simulation logic for dual reward calculations
Add complete simulation engine that compares old (24h cache-based) vs new (1h route-based)
reward calculation methodologies with full integration into epoch operations.

Core Simulation Engine:
- Add SimulationCoordinator with configurable time windows and comparison settings
- Implement dual calculation methods with proper Performance type conversions
- Add comprehensive error handling with DatabaseError variant in RewardingError
- Store simulation results in database with proper relationship constraints

Old Method Implementation (24h Cache-Based):
- Wrap existing reliability calculation using get_all_avg_mix_reliability_in_last_24hr()
- Convert reliability percentages to Performance types using from_percentage_value()
- Maintain exact same logic as production for accurate baseline comparison
- Generate simulation data structures with proper metadata

New Method Implementation (1h Route-Based):
- Leverage calculate_corrected_node_reliabilities_for_interval() for route analysis
- Support configurable time windows (default 1 hour vs 24 hours)
- Provide detailed route statistics including success rates and failure analysis
- Convert route reliability data to Performance types with naive_try_from_f64()

Epoch Operations Integration:
- Extend EpochAdvancer struct with optional SimulationConfig field
- Update constructor and start method to accept simulation configuration
- Add simulation trigger in perform_epoch_operations() before real rewarding
- Ensure simulation failures don't break epoch advancement process

CLI Integration:
- Update run.rs to handle both --enable-rewarding and --simulate-rewarding modes
- Create SimulationConfig from rewarding.debug configuration settings
- Implement mutual exclusivity between real rewarding and simulation mode
- Skip permission checks for simulation-only mode (no blockchain transactions)

The simulation system runs in parallel with epoch operations, storing comparative
data for analysis without affecting production reward distribution.
2025-06-03 14:52:13 +02:00
durch 1eefe8a579 Add simulated rewarding system foundation
Implement foundation for simulated reward calculations to compare old (24h cache-based)
vs new (1h route-based) methodologies without blockchain transactions.

Database Changes:
- Add migration with 4 new tables for simulation system
- simulated_reward_epochs: tracks each simulation run
- simulated_node_performance: stores performance calculations
- simulated_rewards: stores reward calculation results
- simulated_route_analysis: metadata for route analysis
- Add comprehensive indexes for efficient querying

Configuration Changes:
- Add simulation_mode flag to rewarding configuration
- Add CLI flag --simulate-rewarding with proper dependencies/conflicts
- Add validation for simulation-specific settings
- Add time window configuration for new method (default: 1 hour)

Storage Layer:
- Add model structs with SQLx FromRow derives for all simulation tables
- Add comprehensive CRUD methods for simulation data management
- Add proper type annotations to fix SQLx compile issues
- Maintain separation between simulation and real rewarding logic

The simulation mode is mutually exclusive with real rewarding and does not
require mnemonic since no blockchain transactions are performed.
2025-06-03 12:39:44 +02:00
durch e9dc848950 Bump nym-api 2025-06-02 12:49:13 +02:00
durch 81162fba7e Allow nym-api init fail 2025-06-02 12:45:21 +02:00
durch be36da68b1 Clap value_delimiter 2025-06-02 11:58:12 +02:00
durch 21a56e307f Bump NNM 2025-06-02 09:31:09 +02:00
durch bd966383be Towards untangling nym-api client 2025-06-02 09:30:36 +02:00
durch 7626785ce4 Bump NM version 2025-05-27 15:40:59 +02:00
durch 6f79d39d48 Filter out non mixnodes 2025-05-27 15:40:33 +02:00
durch 014b5f767a Log APIs used 2025-05-27 15:23:18 +02:00
durch e0966565e6 Mnemonic to run, bump 2025-05-27 13:38:13 +02:00
durch c6aec663b7 Bump nym-api version 2025-05-27 13:18:31 +02:00
durch 7d041ddd44 Explicit mnemonic to entrypoint 2025-05-27 13:02:38 +02:00
durch 5d8bdc6570 Bunch of new query files 2025-05-27 10:28:00 +02:00
durch 06c412b3ba Remove debug logging 2025-05-27 10:25:27 +02:00
durch 356cf00106 Put the monitoring back properly 2025-05-27 10:25:27 +02:00
durch 58493a69aa Fix submission URLs 2025-05-27 10:25:27 +02:00
durch e881da834b More NM logging 2025-05-27 10:25:27 +02:00
durch eee9d8ab0c DEBUG: disable epoch operations, less noisy logging 2025-05-27 10:25:27 +02:00
durch 09026307f4 Debug logging for nym-api 2025-05-27 10:25:27 +02:00
durch 507ddf246c Stagger out route sending 2025-05-27 10:25:26 +02:00
durch 8d8ce29113 Update NM readme, fmt 2025-05-27 10:25:26 +02:00
durch 3be9e06bef sqlx prepare, bunch of nits 2025-05-27 10:25:26 +02:00
durch 770078a9ed Delete test script 2025-05-27 10:25:26 +02:00
durch fcffebfe45 Raw route handling and reliability corrections 2025-05-27 10:25:19 +02:00
durch 9c7d79683b Force routing through all nodes 2025-05-27 10:21:02 +02:00
durch c7f34d04c0 Support submitting to multiple APIs 2025-05-27 10:21:02 +02:00
77 changed files with 6814 additions and 1297 deletions
+1
View File
@@ -62,3 +62,4 @@ nym-api/redocly/formatted-openapi.json
**/settings.sql
**/enter_db.sh
CLAUDE.md
Generated
+794 -797
View File
File diff suppressed because it is too large Load Diff
@@ -36,6 +36,9 @@ pub trait GatewayTransceiver: GatewaySender + GatewayReceiver {
&mut self,
message: ClientRequest,
) -> Result<(), GatewayClientError>;
/// Check if the websocket connection to the gateway is alive
fn is_connection_alive(&self) -> bool;
}
/// This trait defines the functionality of sending `MixPacket` into the mixnet,
@@ -90,6 +93,11 @@ impl<G: GatewayTransceiver + ?Sized + Send> GatewayTransceiver for Box<G> {
log::debug!("Sent client request: {:?}", message);
Ok(())
}
#[inline]
fn is_connection_alive(&self) -> bool {
(**self).is_connection_alive()
}
}
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
@@ -147,6 +155,10 @@ where
) -> Result<(), GatewayClientError> {
self.gateway_client.send_client_request(message).await
}
fn is_connection_alive(&self) -> bool {
self.gateway_client.is_connection_alive()
}
}
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
@@ -234,6 +246,11 @@ mod nonwasm_sealed {
) -> Result<(), GatewayClientError> {
Ok(())
}
fn is_connection_alive(&self) -> bool {
// LocalGateway is always "connected" since it's in-process
true
}
}
#[async_trait]
@@ -316,4 +333,9 @@ impl GatewayTransceiver for MockGateway {
) -> Result<(), GatewayClientError> {
Ok(())
}
fn is_connection_alive(&self) -> bool {
// MockGateway is always "connected" for testing purposes
true
}
}
+3
View File
@@ -166,6 +166,9 @@ pub enum ClientCoreError {
#[error("there are no gateways supporting the wss protocol available")]
NoWssGateways,
#[error("there are no gateways with compatible protocol versions available")]
NoGatewaysWithCompatibleProtocol,
#[error("the specified gateway '{gateway}' does not support the wss protocol")]
UnsupportedWssProtocol { gateway: String },
+184
View File
@@ -7,6 +7,7 @@ use futures::{SinkExt, StreamExt};
use log::{debug, info, trace, warn};
use nym_crypto::asymmetric::ed25519;
use nym_gateway_client::GatewayClient;
use nym_gateway_requests::{ClientControlRequest, ServerResponse, CURRENT_PROTOCOL_VERSION};
use nym_topology::node::RoutingNode;
use nym_validator_client::client::IdentityKeyRef;
use nym_validator_client::UserAgent;
@@ -131,6 +132,63 @@ pub async fn gateways_for_init<R: Rng>(
Ok(valid_gateways)
}
pub async fn gateways_for_init_with_protocol_validation<R: Rng>(
rng: &mut R,
nym_apis: &[Url],
user_agent: Option<UserAgent>,
minimum_performance: u8,
ignore_epoch_roles: bool,
) -> Result<Vec<RoutingNode>, ClientCoreError> {
// First get the initial list of gateways
let gateways = gateways_for_init(
rng,
nym_apis,
user_agent,
minimum_performance,
ignore_epoch_roles,
)
.await?;
info!(
"Checking protocol compatibility for {} gateways...",
gateways.len()
);
// Filter out gateways with invalid protocols concurrently
let validated_gateways = Arc::new(tokio::sync::Mutex::new(Vec::new()));
futures::stream::iter(&gateways)
.for_each_concurrent(CONCURRENT_GATEWAYS_MEASURED, |gateway| async {
let id = gateway.identity();
trace!("validating protocol compatibility with {id}...");
match validate_gateway_protocol(gateway).await {
Ok(()) => {
debug!("{id}: protocol check successful");
validated_gateways.lock().await.push(gateway.clone());
}
Err(err) => {
warn!("failed to check protocol for {id}: {err}");
}
}
})
.await;
let validated_gateways = validated_gateways.lock().await;
info!(
"Protocol check complete: {}/{} gateways responded successfully",
validated_gateways.len(),
gateways.len()
);
if validated_gateways.is_empty() {
return Err(ClientCoreError::NoGatewaysWithCompatibleProtocol);
}
Ok(validated_gateways.clone())
}
#[cfg(not(target_arch = "wasm32"))]
async fn connect(endpoint: &str) -> Result<WsConn, ClientCoreError> {
match tokio::time::timeout(CONN_TIMEOUT, connect_async(endpoint)).await {
@@ -210,6 +268,132 @@ where
Ok(GatewayWithLatency::new(gateway, avg))
}
async fn validate_gateway_protocol<G>(gateway: &G) -> Result<(), ClientCoreError>
where
G: ConnectableGateway,
{
let Some(addr) = gateway.clients_address(false) else {
return Err(ClientCoreError::UnsupportedEntry {
id: gateway.node_id(),
identity: gateway.identity().to_string(),
});
};
trace!(
"validating protocol compatibility with {} ({addr})...",
gateway.identity(),
);
let mut stream = connect(&addr).await?;
// Send protocol version request
let protocol_request = ClientControlRequest::SupportedProtocol {};
// Send the request as JSON text message
stream.send(Message::from(protocol_request)).await?;
// Wait for response with timeout
let protocol_timeout = Duration::from_millis(2000);
let response_future = stream.next();
match tokio::time::timeout(protocol_timeout, response_future).await {
Err(_) => {
warn!("Gateway {} protocol check timed out", gateway.identity());
Err(ClientCoreError::GatewayConnectionTimeout)
}
Ok(Some(Ok(Message::Text(response_text)))) => {
// Try to deserialize the response
let response = ServerResponse::try_from(response_text).map_err(|_| {
ClientCoreError::GatewayClientError {
gateway_id: gateway.identity().to_base58_string(),
source: *Box::new(
nym_gateway_client::error::GatewayClientError::MalformedResponse,
),
}
})?;
match response {
ServerResponse::SupportedProtocol { version } => {
debug!(
"Gateway {} supports protocol version {}, ours: {}",
gateway.identity(),
version,
CURRENT_PROTOCOL_VERSION
);
// Check protocol compatibility
if version > CURRENT_PROTOCOL_VERSION {
warn!(
"Gateway {} uses newer protocol version {} (client supports {}). \
Gateway should gracefully degrade, but consider updating your client.",
gateway.identity(),
version,
CURRENT_PROTOCOL_VERSION
);
}
trace!(
"Gateway {} protocol validation successful (gateway: v{}, client: v{})",
gateway.identity(),
version,
CURRENT_PROTOCOL_VERSION
);
Ok(())
}
ServerResponse::Error { message } => {
warn!(
"Gateway {} returned error during protocol check: {}",
gateway.identity(),
message
);
Err(ClientCoreError::GatewayClientError {
gateway_id: gateway.identity().to_base58_string(),
source: *Box::new(
nym_gateway_client::error::GatewayClientError::GatewayError(message),
),
})
}
_ => {
warn!(
"Gateway {} returned unexpected response during protocol check",
gateway.identity()
);
Err(ClientCoreError::GatewayClientError {
gateway_id: gateway.identity().to_base58_string(),
source: *Box::new(
nym_gateway_client::error::GatewayClientError::UnexpectedResponse {
name: response.name().to_string(),
},
),
})
}
}
}
Ok(Some(Ok(_))) => {
warn!(
"Gateway {} sent non-text response during protocol check",
gateway.identity()
);
Err(ClientCoreError::GatewayConnectionAbruptlyClosed)
}
Ok(Some(Err(e))) => {
warn!(
"WebSocket error during protocol check with {}: {}",
gateway.identity(),
e
);
Err(e.into())
}
Ok(None) => {
warn!(
"Gateway {} closed connection during protocol check",
gateway.identity()
);
Err(ClientCoreError::GatewayConnectionAbruptlyClosed)
}
}
}
pub async fn choose_gateway_by_latency<R: Rng, G: ConnectableGateway + Clone>(
rng: &mut R,
gateways: &[G],
@@ -27,7 +27,6 @@ nym-credential-storage = { path = "../../credential-storage" }
nym-credentials-interface = { path = "../../credentials-interface" }
nym-crypto = { path = "../../crypto" }
nym-gateway-requests = { path = "../../gateway-requests" }
nym-http-api-client = { path = "../../http-api-client" }
nym-network-defaults = { path = "../../network-defaults" }
nym-sphinx = { path = "../../nymsphinx" }
nym-statistics-common = { path = "../../statistics" }
@@ -165,6 +165,24 @@ impl<C, St> GatewayClient<C, St> {
self.bandwidth.remaining()
}
pub fn is_connection_established(&self) -> bool {
self.connection.is_established()
}
/// Check if the websocket connection is actually alive at the socket level
pub fn is_connection_alive(&self) -> bool {
// First check if we have an established connection
if !self.connection.is_established() {
return false;
}
// Get the file descriptor and check if the socket is alive
match self.ws_fd() {
Some(fd) => socket_is_alive(fd),
None => false,
}
}
#[cfg(not(target_arch = "wasm32"))]
async fn _close_connection(&mut self) -> Result<(), GatewayClientError> {
match std::mem::replace(&mut self.connection, SocketState::NotConnected) {
@@ -1128,3 +1146,49 @@ impl GatewayClient<InitOnly, EphemeralCredentialStorage> {
}
}
}
/// Check if a socket file descriptor is alive and responsive
///
/// This function performs socket-level checks to determine if the connection is actually alive.
/// It's cross-platform compatible and works on both Unix and non-Unix systems.
fn socket_is_alive(fd: RawFd) -> bool {
#[cfg(unix)]
{
use std::io::ErrorKind;
use std::net::TcpStream;
use std::os::unix::io::FromRawFd;
unsafe {
// Create a TcpStream from the raw fd to perform socket operations
let stream = TcpStream::from_raw_fd(fd);
// Try to peek at the socket to see if it's still connected
// We peek with a zero-length buffer to avoid consuming data
let mut buf = [0u8; 0];
let result = match stream.peek(&mut buf) {
Ok(_) => true, // Socket is alive and readable
Err(e) => match e.kind() {
ErrorKind::WouldBlock => true, // Socket is alive but no data available
ErrorKind::ConnectionReset
| ErrorKind::ConnectionAborted
| ErrorKind::BrokenPipe
| ErrorKind::NotConnected => false, // Socket is clearly dead
_ => true, // Other errors might be temporary, assume alive
},
};
// Prevent the TcpStream from closing the fd when it's dropped
// since we don't own the fd
std::mem::forget(stream);
result
}
}
#[cfg(not(unix))]
{
// On non-Unix systems, we can't easily check socket state
// Fall back to assuming the connection is alive if we have an fd
fd != 0
}
}
@@ -1,6 +1,5 @@
use crate::error::GatewayClientError;
use nym_http_api_client::HickoryDnsResolver;
#[cfg(unix)]
use std::{
os::fd::{AsRawFd, RawFd},
@@ -20,7 +19,6 @@ pub(crate) async fn connect_async(
) -> Result<(WebSocketStream<MaybeTlsStream<TcpStream>>, Response), GatewayClientError> {
use tokio::net::TcpSocket;
let resolver = HickoryDnsResolver::default();
let uri =
Url::parse(endpoint).map_err(|_| GatewayClientError::InvalidUrl(endpoint.to_owned()))?;
let port: u16 = uri.port_or_known_default().unwrap_or(443);
@@ -29,18 +27,18 @@ pub(crate) async fn connect_async(
.host()
.ok_or(GatewayClientError::InvalidUrl(endpoint.to_owned()))?;
// Get address for tcp connection, if a domain is provided use our preferred resolver rather than
// the default std resolve
// Get address for tcp connection, using system DNS resolver
let sock_addrs: Vec<SocketAddr> = match host {
Host::Ipv4(addr) => vec![SocketAddr::new(addr.into(), port)],
Host::Ipv6(addr) => vec![SocketAddr::new(addr.into(), port)],
Host::Domain(domain) => {
// Do a DNS lookup for the domain using our custom DNS resolver
resolver
.resolve_str(domain)
.await?
.into_iter()
.map(|a| SocketAddr::new(a, port))
// Do a DNS lookup for the domain using system DNS resolver
tokio::net::lookup_host((domain, port))
.await
.map_err(|err| GatewayClientError::NetworkConnectionFailed {
address: endpoint.to_owned(),
source: err.into(),
})?
.collect()
}
};
@@ -49,10 +49,6 @@ pub enum GatewayClientError {
#[error("Invalid URL: {0}")]
InvalidUrl(String),
#[cfg(not(target_arch = "wasm32"))]
#[error("resolution failed: {0}")]
ResolutionFailed(#[from] nym_http_api_client::HickoryDnsError),
#[error("No shared key was provided or obtained")]
NoSharedKeyAvailable,
@@ -73,6 +73,7 @@ pub const STAKE_SATURATION: &str = "stake-saturation";
pub const INCLUSION_CHANCE: &str = "inclusion-probability";
pub const SUBMIT_GATEWAY: &str = "submit-gateway-monitoring-results";
pub const SUBMIT_NODE: &str = "submit-node-monitoring-results";
pub const SUBMIT_ROUTE: &str = "submit-route-monitoring-results";
pub const SERVICE_PROVIDERS: &str = "services";
@@ -124,7 +124,7 @@ impl ReconstructionBuffer {
// TODO: what to do in that case? give up on the message? overwrite it? panic?
// it *might* be due to lock ack-packet, but let's keep the `warn` level in case
// it could be somehow exploited
warn!(
debug!(
"duplicate fragment received! - frag - {} (set id: {})",
fragment.current_fragment(),
fragment.id()
+4
View File
@@ -224,6 +224,10 @@ impl NymTopology {
serde_json::from_reader(file).map_err(Into::into)
}
pub fn node_details(&self) -> &HashMap<NodeId, RoutingNode> {
&self.node_details
}
pub fn add_skimmed_nodes(&mut self, nodes: &[SkimmedNode]) {
self.add_additional_nodes(nodes.iter())
}
+28 -7
View File
@@ -11,6 +11,27 @@ static NETWORK_MONITORS: LazyLock<HashSet<String>> = LazyLock::new(|| {
nm
});
#[derive(Debug, Serialize, Deserialize, JsonSchema, Clone, ToSchema)]
pub struct RouteResult {
pub layer1: u32,
pub layer2: u32,
pub layer3: u32,
pub gw: u32,
pub success: bool,
}
impl RouteResult {
pub fn new(layer1: u32, layer2: u32, layer3: u32, gw: u32, success: bool) -> Self {
RouteResult {
layer1,
layer2,
layer3,
gw,
success,
}
}
}
#[derive(Debug, Serialize, Deserialize, JsonSchema, Clone, ToSchema)]
pub struct NodeResult {
#[schema(value_type = u32)]
@@ -29,23 +50,23 @@ impl NodeResult {
}
}
#[derive(Serialize, Deserialize, JsonSchema)]
#[derive(Serialize, Deserialize, JsonSchema, ToSchema)]
#[serde(untagged)]
pub enum MonitorResults {
Mixnode(Vec<NodeResult>),
Gateway(Vec<NodeResult>),
Node(Vec<NodeResult>),
Route(Vec<RouteResult>),
}
#[derive(Serialize, Deserialize, JsonSchema, ToSchema)]
pub struct MonitorMessage {
results: Vec<NodeResult>,
results: MonitorResults,
signature: String,
signer: String,
timestamp: i64,
}
impl MonitorMessage {
fn message_to_sign(results: &[NodeResult], timestamp: i64) -> Vec<u8> {
fn message_to_sign(results: &MonitorResults, timestamp: i64) -> Vec<u8> {
let mut msg = serde_json::to_vec(results).unwrap_or_default();
msg.extend_from_slice(&timestamp.to_le_bytes());
msg
@@ -60,7 +81,7 @@ impl MonitorMessage {
now - self.timestamp < 5
}
pub fn new(results: Vec<NodeResult>, private_key: &PrivateKey) -> Self {
pub fn new(results: MonitorResults, private_key: &PrivateKey) -> Self {
let timestamp = SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
.expect("Time went backwards")
@@ -82,7 +103,7 @@ impl MonitorMessage {
NETWORK_MONITORS.contains(&self.signer)
}
pub fn results(&self) -> &[NodeResult] {
pub fn results(&self) -> &MonitorResults {
&self.results
}
@@ -1,32 +0,0 @@
{
"db_name": "SQLite",
"query": "\n SELECT epoch_id as \"epoch_id: u32\", serialised_signatures, serialization_revision as \"serialization_revision: u8\"\n FROM expiration_date_signatures\n WHERE expiration_date = ?\n ",
"describe": {
"columns": [
{
"name": "epoch_id: u32",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "serialised_signatures",
"ordinal": 1,
"type_info": "Blob"
},
{
"name": "serialization_revision: u8",
"ordinal": 2,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false
]
},
"hash": "00d857b624e7edab1198114b17cbad1e16988a3f9989d135840500e1143ce5e5"
}
@@ -1,32 +0,0 @@
{
"db_name": "SQLite",
"query": "\n SELECT epoch_id as \"epoch_id: u32\", serialised_key, serialization_revision as \"serialization_revision: u8\"\n FROM master_verification_key WHERE epoch_id = ?\n ",
"describe": {
"columns": [
{
"name": "epoch_id: u32",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "serialised_key",
"ordinal": 1,
"type_info": "Blob"
},
{
"name": "serialization_revision: u8",
"ordinal": 2,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false
]
},
"hash": "0112296b190328a3856d1adf51aafa2525da6c0b871633aad80ad555db9cf47c"
}
@@ -0,0 +1,20 @@
{
"db_name": "SQLite",
"query": "SELECT COUNT(*) as count FROM simulated_node_performance WHERE simulated_epoch_id = ?",
"describe": {
"columns": [
{
"name": "count",
"ordinal": 0,
"type_info": "Int"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "0557e64c547e147ef7eef713b49c62afde62b3b04ff817ae372ffe9fbeaee6b8"
}
@@ -0,0 +1,20 @@
{
"db_name": "SQLite",
"query": "SELECT COUNT(*) as count FROM simulated_reward_epochs",
"describe": {
"columns": [
{
"name": "count",
"ordinal": 0,
"type_info": "Int"
}
],
"parameters": {
"Right": 0
},
"nullable": [
false
]
},
"hash": "1411660a6456f6f5ca141db10d9a54857fc8b4a661e09ae6719d84ac2e77cf6d"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n INSERT OR IGNORE INTO expiration_date_signatures(expiration_date, epoch_id, serialised_signatures, serialization_revision)\n VALUES (?, ?, ?, ?);\n UPDATE expiration_date_signatures\n SET\n serialised_signatures = ?,\n serialization_revision = ?\n WHERE expiration_date = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 7
},
"nullable": []
},
"hash": "16d10f0ac0ed9ce4239937f46df3797a6a9ee7db2aab9f1b5e55f7c13c53bcc1"
}
@@ -0,0 +1,56 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", epoch_id as \"epoch_id!: u32\", calculation_method as \"calculation_method!\", \n start_timestamp as \"start_timestamp!\", end_timestamp as \"end_timestamp!\", \n description, created_at as \"created_at!\"\n FROM simulated_reward_epochs\n WHERE id = ?\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "epoch_id!: u32",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 2,
"type_info": "Text"
},
{
"name": "start_timestamp!",
"ordinal": 3,
"type_info": "Int64"
},
{
"name": "end_timestamp!",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "description",
"ordinal": 5,
"type_info": "Text"
},
{
"name": "created_at!",
"ordinal": 6,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false,
false,
false,
true,
false
]
},
"hash": "1e4daea4d5f87938cf98cb2b6facc5802e79b06f3d0d41f6467844265b26d0f7"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n INSERT INTO ecash_ticketbook\n (serialization_revision, ticketbook_data, expiration_date, ticketbook_type, epoch_id, total_tickets, used_tickets)\n VALUES (?, ?, ?, ?, ?, ?, ?)\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 7
},
"nullable": []
},
"hash": "284b3ceae42f9320c30323dde47765854899103fd3c0fa670eb6809492270e02"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "\n INSERT INTO simulated_reward_epochs \n (epoch_id, calculation_method, start_timestamp, end_timestamp, description)\n VALUES (?, ?, ?, ?, ?)\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 5
},
"nullable": []
},
"hash": "2bb062a25129bfc6b060f68592e74aa91d8c82b8b912009fce14f9c19fd4464c"
}
@@ -0,0 +1,56 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", \n node_id as \"node_id!: NodeId\", calculation_method as \"calculation_method!\", \n performance_rank as \"performance_rank!\", performance_percentile as \"performance_percentile!\",\n calculated_at as \"calculated_at!\"\n FROM simulated_performance_rankings\n WHERE simulated_epoch_id = ? AND calculation_method = ?\n ORDER BY performance_rank\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "performance_rank!",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "performance_percentile!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "calculated_at!",
"ordinal": 6,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
false,
false,
false
]
},
"hash": "3024d5e589b1f50f95f5e7bf09e31161beca74e137e340d2a6a02a589ad251cb"
}
@@ -0,0 +1,56 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", \n node_id as \"node_id!: NodeId\", calculation_method as \"calculation_method!\", \n performance_rank as \"performance_rank!\", performance_percentile as \"performance_percentile!\",\n calculated_at as \"calculated_at!\"\n FROM simulated_performance_rankings\n WHERE simulated_epoch_id = ?\n ORDER BY calculation_method, performance_rank\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "performance_rank!",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "performance_percentile!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "calculated_at!",
"ordinal": 6,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
true,
false,
false,
false,
false,
false,
false
]
},
"hash": "33108405de0328977ac5d05d89f484bdb127f9cba0ba5370c931e1225983d74a"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "DELETE FROM ecash_ticketbook WHERE expiration_date <= ?",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "37f82c9ec26b53d01601a2d6df82038a77ec37cca9f9aef18008dcd03030c2c4"
}
@@ -0,0 +1,74 @@
{
"db_name": "SQLite",
"query": "\n SELECT sra.id as \"id!\", sra.simulated_epoch_id as \"simulated_epoch_id!\", \n sra.calculation_method as \"calculation_method!\", sra.total_routes_analyzed as \"total_routes_analyzed!: u32\", \n sra.successful_routes as \"successful_routes!: u32\", sra.failed_routes as \"failed_routes!: u32\", \n sra.average_route_reliability, sra.time_window_hours as \"time_window_hours!: u32\", \n sra.analysis_parameters, sra.calculated_at as \"calculated_at!\"\n FROM simulated_route_analysis sra\n JOIN simulated_reward_epochs sre ON sra.simulated_epoch_id = sre.id\n WHERE sre.epoch_id = ? AND sra.calculation_method = ?\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 2,
"type_info": "Text"
},
{
"name": "total_routes_analyzed!: u32",
"ordinal": 3,
"type_info": "Int64"
},
{
"name": "successful_routes!: u32",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "failed_routes!: u32",
"ordinal": 5,
"type_info": "Int64"
},
{
"name": "average_route_reliability",
"ordinal": 6,
"type_info": "Float"
},
{
"name": "time_window_hours!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "analysis_parameters",
"ordinal": 8,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 9,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
false,
false,
true,
false,
true,
false
]
},
"hash": "4e2338362058e4356ac639fdf806e827761f879cab723cb2a649bc76b176fdad"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "DELETE FROM pending_issuance WHERE deposit_id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 1
},
"nullable": []
},
"hash": "5c5d4bfabf18bc6fa56e76a9b98e38b7f6ceb8e9191a7b9201922efcf6b07966"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "\n INSERT OR IGNORE INTO routes (layer1, layer2, layer3, gw, success) VALUES (?, ?, ?, ?, ?);\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 5
},
"nullable": []
},
"hash": "66109c1d856e1ca2b5126e4bf4c58c7a27b8c303bfa079cf74909354202dcc49"
}
@@ -1,26 +0,0 @@
{
"db_name": "SQLite",
"query": "\n SELECT\n d.node_id as \"node_id: NodeId\",\n CASE WHEN count(*) > 3 THEN AVG(reliability) ELSE 100 END as \"value: f32\"\n FROM\n gateway_details d\n JOIN\n gateway_status s on d.id = s.gateway_details_id\n WHERE\n timestamp >= ? AND\n timestamp <= ?\n GROUP BY 1\n ",
"describe": {
"columns": [
{
"name": "node_id: NodeId",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "value: f32",
"ordinal": 1,
"type_info": "Int"
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
true
]
},
"hash": "676299beb2004ab89f7b38cf21ffb84ab5e7d7435297573523e2532560c2e302"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", node_id as \"node_id!: NodeId\", \n node_type as \"node_type!\", performance_score as \"performance_score!\", \n work_factor as \"work_factor!\", calculation_method as \"calculation_method!\", \n positive_samples as \"positive_samples?\", negative_samples as \"negative_samples?\",\n route_success_rate as \"route_success_rate?\", calculated_at as \"calculated_at!\"\n FROM simulated_performance_comparisons\n WHERE simulated_epoch_id = ?\n ORDER BY calculation_method, node_id\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "performance_score!",
"ordinal": 4,
"type_info": "Float"
},
{
"name": "work_factor!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 6,
"type_info": "Text"
},
{
"name": "positive_samples?",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "negative_samples?",
"ordinal": 8,
"type_info": "Int64"
},
{
"name": "route_success_rate?",
"ordinal": 9,
"type_info": "Float"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
true,
false,
false,
false,
false,
false,
false,
true,
true,
true,
false
]
},
"hash": "6a7b25d2c580298bd44fcf74e4d2c550537c64dbfb2c17a6c91bd5345b9a3e6a"
}
@@ -0,0 +1,44 @@
{
"db_name": "SQLite",
"query": "\n SELECT\n layer1 as \"layer1\",\n layer2 as \"layer2\",\n layer3 as \"layer3\",\n gw as \"gw\",\n success\n FROM routes\n WHERE timestamp >= ? AND timestamp <= ?\n ORDER BY timestamp ASC\n ",
"describe": {
"columns": [
{
"name": "layer1",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "layer2",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "layer3",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "gw",
"ordinal": 3,
"type_info": "Int64"
},
{
"name": "success",
"ordinal": 4,
"type_info": "Bool"
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
false,
false,
false,
false
]
},
"hash": "6b2479c02cf1ef5ae674ce0ab4d027595b91739f3579e1f289b0c722ea91bbcc"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT snp.id as \"id!\", snp.simulated_epoch_id as \"simulated_epoch_id!\", \n snp.node_id as \"node_id!: NodeId\", snp.node_type as \"node_type!\", \n snp.identity_key, snp.reliability_score as \"reliability_score!\", \n snp.positive_samples as \"positive_samples!: u32\", snp.negative_samples as \"negative_samples!: u32\", \n snp.work_factor, snp.calculation_method as \"calculation_method!\", snp.calculated_at as \"calculated_at!\"\n FROM simulated_node_performance snp\n JOIN simulated_reward_epochs sre ON snp.simulated_epoch_id = sre.id\n WHERE snp.node_id = ?\n ORDER BY sre.epoch_id DESC, snp.calculation_method\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "identity_key",
"ordinal": 4,
"type_info": "Text"
},
{
"name": "reliability_score!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "positive_samples!: u32",
"ordinal": 6,
"type_info": "Int64"
},
{
"name": "negative_samples!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "work_factor",
"ordinal": 8,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 9,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
true,
false,
false,
false,
true,
false,
false,
false,
true,
false,
false
]
},
"hash": "6fa6967e77886b69b049c68b664cdf8f4b97daa3734036db16e49915fd89073a"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n INSERT INTO pending_issuance\n (deposit_id, serialization_revision, pending_ticketbook_data, expiration_date)\n VALUES (?, ?, ?, ?)\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 4
},
"nullable": []
},
"hash": "81a12a8a419c88b1c28a5533fde4d63462e9ea0049e2edafea1dc3f8476b33e4"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "UPDATE ecash_ticketbook SET used_tickets = used_tickets + ? WHERE id = ?",
"describe": {
"columns": [],
"parameters": {
"Right": 2
},
"nullable": []
},
"hash": "84cad8b1078a4000830835e6349de3eb76fed954b7336530401db72cd008aff3"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n INSERT OR IGNORE INTO master_verification_key(epoch_id, serialised_key, serialization_revision) VALUES (?, ?, ?);\n UPDATE master_verification_key\n SET\n serialised_key = ?,\n serialization_revision = ?\n WHERE epoch_id = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 6
},
"nullable": []
},
"hash": "a5b18e66d77ff802e274623605e15dcfcffb502ba8398caefd56c481f44eb84e"
}
@@ -1,32 +0,0 @@
{
"db_name": "SQLite",
"query": "\n SELECT epoch_id as \"epoch_id: u32\", serialised_signatures, serialization_revision as \"serialization_revision: u8\"\n FROM coin_indices_signatures WHERE epoch_id = ?\n ",
"describe": {
"columns": [
{
"name": "epoch_id: u32",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "serialised_signatures",
"ordinal": 1,
"type_info": "Blob"
},
{
"name": "serialization_revision: u8",
"ordinal": 2,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false,
false,
false
]
},
"hash": "ba96344db31b0f2155e2af53eaaeafc9b5f64061b6c9a829e2912945b6cffc82"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n UPDATE ecash_ticketbook\n SET used_tickets = used_tickets - ?\n WHERE id = ?\n AND used_tickets = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 3
},
"nullable": []
},
"hash": "bc823c54143e2dc590b91347cd089dde284b38a3a4960afed758206d03ca1cf4"
}
@@ -1,12 +0,0 @@
{
"db_name": "SQLite",
"query": "\n INSERT OR IGNORE INTO coin_indices_signatures(epoch_id, serialised_signatures, serialization_revision) VALUES (?, ?, ?);\n UPDATE coin_indices_signatures\n SET\n serialised_signatures = ?,\n serialization_revision = ?\n WHERE epoch_id = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 6
},
"nullable": []
},
"hash": "bd1973696121b6128bd75ae80fab253c071e04eb853d4b0f3b21782ea57c2f68"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", node_id as \"node_id!: NodeId\", \n node_type as \"node_type!\", identity_key, reliability_score as \"reliability_score!\", \n positive_samples as \"positive_samples!: u32\", negative_samples as \"negative_samples!: u32\", \n work_factor, calculation_method as \"calculation_method!\", calculated_at as \"calculated_at!\"\n FROM simulated_node_performance\n WHERE simulated_epoch_id = ?\n ORDER BY calculation_method, node_id\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "identity_key",
"ordinal": 4,
"type_info": "Text"
},
{
"name": "reliability_score!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "positive_samples!: u32",
"ordinal": 6,
"type_info": "Int64"
},
{
"name": "negative_samples!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "work_factor",
"ordinal": 8,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 9,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
true,
false,
false,
false,
true,
false,
false,
false,
true,
false,
false
]
},
"hash": "c04c33631eea6959267dd441f50c2bd876ba951e32e8b8bee398cf98fec5e7bf"
}
@@ -1,26 +0,0 @@
{
"db_name": "SQLite",
"query": "\n SELECT\n d.mix_id as \"mix_id: NodeId\",\n AVG(s.reliability) as \"value: f32\"\n FROM\n mixnode_details d\n JOIN\n mixnode_status s on d.id = s.mixnode_details_id\n WHERE\n timestamp >= ? AND\n timestamp <= ?\n GROUP BY 1\n ",
"describe": {
"columns": [
{
"name": "mix_id: NodeId",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "value: f32",
"ordinal": 1,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
true
]
},
"hash": "c19e1b3768bf2929407599e6e8783ead09f4d7319b7997fa2a9bb628f9404166"
}
@@ -0,0 +1,20 @@
{
"db_name": "SQLite",
"query": "SELECT DISTINCT calculation_method FROM simulated_reward_epochs WHERE epoch_id = ?",
"describe": {
"columns": [
{
"name": "calculation_method",
"ordinal": 0,
"type_info": "Text"
}
],
"parameters": {
"Right": 1
},
"nullable": [
false
]
},
"hash": "c9796e61c49a29ebd42dec46fb047f7a7e2f80aa4db4f963146cf56864a62347"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", node_id as \"node_id!: NodeId\", \n node_type as \"node_type!\", identity_key, reliability_score as \"reliability_score!\", \n positive_samples as \"positive_samples!: u32\", negative_samples as \"negative_samples!: u32\", \n work_factor, calculation_method as \"calculation_method!\", calculated_at as \"calculated_at!\"\n FROM simulated_node_performance\n WHERE simulated_epoch_id = ? AND calculation_method = ?\n ORDER BY node_id\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "identity_key",
"ordinal": 4,
"type_info": "Text"
},
{
"name": "reliability_score!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "positive_samples!: u32",
"ordinal": 6,
"type_info": "Int64"
},
{
"name": "negative_samples!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "work_factor",
"ordinal": 8,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 9,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
true,
false,
false,
false,
true,
false,
false
]
},
"hash": "d76e131fff79978284f31509e70d42d20ea2e95e5a2f2d574d33ad8d2a7f2a19"
}
@@ -0,0 +1,74 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", \n calculation_method as \"calculation_method!\", total_routes_analyzed as \"total_routes_analyzed!: u32\", \n successful_routes as \"successful_routes!: u32\", failed_routes as \"failed_routes!: u32\", \n average_route_reliability, time_window_hours as \"time_window_hours!: u32\", \n analysis_parameters, calculated_at as \"calculated_at!\"\n FROM simulated_route_analysis\n WHERE simulated_epoch_id = ?\n ORDER BY calculation_method\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 2,
"type_info": "Text"
},
{
"name": "total_routes_analyzed!: u32",
"ordinal": 3,
"type_info": "Int64"
},
{
"name": "successful_routes!: u32",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "failed_routes!: u32",
"ordinal": 5,
"type_info": "Int64"
},
{
"name": "average_route_reliability",
"ordinal": 6,
"type_info": "Float"
},
{
"name": "time_window_hours!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "analysis_parameters",
"ordinal": 8,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 9,
"type_info": "Int64"
}
],
"parameters": {
"Right": 1
},
"nullable": [
true,
false,
false,
false,
false,
false,
true,
false,
true,
false
]
},
"hash": "db2b71d1dd33022d5c960053d6a0f5d7ccf52de6775d4f57744d1401d44fee62"
}
@@ -0,0 +1,56 @@
{
"db_name": "SQLite",
"query": "SELECT id as \"id!\", epoch_id as \"epoch_id!: u32\", calculation_method as \"calculation_method!\", \n start_timestamp as \"start_timestamp!\", end_timestamp as \"end_timestamp!\", \n description, created_at as \"created_at!\"\n FROM simulated_reward_epochs \n ORDER BY created_at DESC \n LIMIT ? OFFSET ?",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "epoch_id!: u32",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "calculation_method!",
"ordinal": 2,
"type_info": "Text"
},
{
"name": "start_timestamp!",
"ordinal": 3,
"type_info": "Int64"
},
{
"name": "end_timestamp!",
"ordinal": 4,
"type_info": "Int64"
},
{
"name": "description",
"ordinal": 5,
"type_info": "Text"
},
{
"name": "created_at!",
"ordinal": 6,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
false,
true,
false
]
},
"hash": "dbc52e443ab600516b4f96ab2904d6b6379c06cfa1ff8598c29b67a2fe37055d"
}
@@ -0,0 +1,12 @@
{
"db_name": "SQLite",
"query": "\n INSERT INTO simulated_route_analysis\n (simulated_epoch_id, calculation_method, total_routes_analyzed, successful_routes,\n failed_routes, average_route_reliability, time_window_hours, analysis_parameters)\n VALUES (?, ?, ?, ?, ?, ?, ?, ?)\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 8
},
"nullable": []
},
"hash": "df9013912c0e4177625267776656cb139cf3ac8dd4cd4c5146005b055aefd228"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT snp.id as \"id!\", snp.simulated_epoch_id as \"simulated_epoch_id!\", \n snp.node_id as \"node_id!: NodeId\", snp.node_type as \"node_type!\", \n snp.identity_key, snp.reliability_score as \"reliability_score!\", \n snp.positive_samples as \"positive_samples!: u32\", snp.negative_samples as \"negative_samples!: u32\", \n snp.work_factor, snp.calculation_method as \"calculation_method!\", snp.calculated_at as \"calculated_at!\"\n FROM simulated_node_performance snp\n JOIN simulated_reward_epochs sre ON snp.simulated_epoch_id = sre.id\n WHERE sre.epoch_id = ? AND snp.calculation_method = ?\n ORDER BY snp.node_id\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "identity_key",
"ordinal": 4,
"type_info": "Text"
},
{
"name": "reliability_score!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "positive_samples!: u32",
"ordinal": 6,
"type_info": "Int64"
},
{
"name": "negative_samples!: u32",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "work_factor",
"ordinal": 8,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 9,
"type_info": "Text"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
true,
false,
false,
false,
true,
false,
false
]
},
"hash": "e2a8ebf260e0e3dd25d614a3c9f70715d0b593cf19d0a5f5c7972ac5925151c3"
}
@@ -0,0 +1,80 @@
{
"db_name": "SQLite",
"query": "\n SELECT id as \"id!\", simulated_epoch_id as \"simulated_epoch_id!\", node_id as \"node_id!: NodeId\", \n node_type as \"node_type!\", performance_score as \"performance_score!\", \n work_factor as \"work_factor!\", calculation_method as \"calculation_method!\", \n positive_samples as \"positive_samples?\", negative_samples as \"negative_samples?\",\n route_success_rate as \"route_success_rate?\", calculated_at as \"calculated_at!\"\n FROM simulated_performance_comparisons\n WHERE simulated_epoch_id = ? AND calculation_method = ?\n ORDER BY node_id\n ",
"describe": {
"columns": [
{
"name": "id!",
"ordinal": 0,
"type_info": "Int64"
},
{
"name": "simulated_epoch_id!",
"ordinal": 1,
"type_info": "Int64"
},
{
"name": "node_id!: NodeId",
"ordinal": 2,
"type_info": "Int64"
},
{
"name": "node_type!",
"ordinal": 3,
"type_info": "Text"
},
{
"name": "performance_score!",
"ordinal": 4,
"type_info": "Float"
},
{
"name": "work_factor!",
"ordinal": 5,
"type_info": "Float"
},
{
"name": "calculation_method!",
"ordinal": 6,
"type_info": "Text"
},
{
"name": "positive_samples?",
"ordinal": 7,
"type_info": "Int64"
},
{
"name": "negative_samples?",
"ordinal": 8,
"type_info": "Int64"
},
{
"name": "route_success_rate?",
"ordinal": 9,
"type_info": "Float"
},
{
"name": "calculated_at!",
"ordinal": 10,
"type_info": "Int64"
}
],
"parameters": {
"Right": 2
},
"nullable": [
true,
false,
false,
false,
false,
false,
false,
true,
true,
true,
false
]
},
"hash": "e8a88007b46202c6dcd87c2f964e366fa2b6fe0c0fd71ce48a544d51c679a9c8"
}
+1 -1
View File
@@ -4,7 +4,7 @@
[package]
name = "nym-api"
license = "GPL-3.0"
version = "1.1.57"
version = "1.1.73"
authors.workspace = true
edition = "2021"
rust-version.workspace = true
+8 -1
View File
@@ -2,4 +2,11 @@
set -e
/usr/src/nym/target/release/nym-api init && /usr/src/nym/target/release/nym-api run
# Optional: delete existing config and force reinit
if [ "$NYM_API_RESET_CONFIG" = "true" ]; then
echo "RESET_CONFIG enabled - removing existing configuration..."
rm -rf ~/.nym/nym-api/default/*
fi
# Init can fail if the mounted volume already has a config
/usr/src/nym/target/release/nym-api init --mnemonic "$MNEMONIC" || true && /usr/src/nym/target/release/nym-api run --mnemonic "$MNEMONIC"
@@ -0,0 +1,17 @@
-- Add routes table for storing route metrics data
CREATE TABLE IF NOT EXISTS routes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
layer1 INTEGER NOT NULL, -- NodeId of layer 1 mixnode
layer2 INTEGER NOT NULL, -- NodeId of layer 2 mixnode
layer3 INTEGER NOT NULL, -- NodeId of layer 3 mixnode
gw INTEGER NOT NULL, -- NodeId of gateway
success BOOLEAN NOT NULL, -- Whether the packet was delivered successfully
timestamp INTEGER NOT NULL DEFAULT (unixepoch()) -- When the measurement was taken
);
-- Add indexes for efficient querying
CREATE INDEX IF NOT EXISTS idx_routes_timestamp ON routes(timestamp);
CREATE INDEX IF NOT EXISTS idx_routes_layer1 ON routes(layer1);
CREATE INDEX IF NOT EXISTS idx_routes_layer2 ON routes(layer2);
CREATE INDEX IF NOT EXISTS idx_routes_layer3 ON routes(layer3);
CREATE INDEX IF NOT EXISTS idx_routes_gw ON routes(gw);
@@ -0,0 +1,100 @@
-- Migration: Add performance methodology comparison tables
-- This migration adds support for comparing performance calculation methodologies:
-- old (24h cache-based) vs new (1h route-based) without affecting actual rewards
-- Simulated reward epochs track each simulation run
CREATE TABLE IF NOT EXISTS simulated_reward_epochs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
epoch_id INTEGER NOT NULL,
calculation_method TEXT NOT NULL CHECK (calculation_method IN ('old', 'new', 'comparison', 'test')),
start_timestamp INTEGER NOT NULL,
end_timestamp INTEGER NOT NULL,
description TEXT,
created_at INTEGER NOT NULL DEFAULT (unixepoch())
);
-- Node performance data calculated from different methodologies
CREATE TABLE IF NOT EXISTS simulated_node_performance (
id INTEGER PRIMARY KEY AUTOINCREMENT,
simulated_epoch_id INTEGER NOT NULL,
node_id INTEGER NOT NULL,
node_type TEXT NOT NULL CHECK (node_type IN ('mixnode', 'gateway')),
identity_key TEXT,
reliability_score REAL NOT NULL CHECK (reliability_score >= 0.0 AND reliability_score <= 100.0),
positive_samples INTEGER NOT NULL DEFAULT 0,
negative_samples INTEGER NOT NULL DEFAULT 0,
work_factor REAL CHECK (work_factor >= 0.0 AND work_factor <= 1.0),
calculation_method TEXT NOT NULL CHECK (calculation_method IN ('old', 'new', 'test')),
calculated_at INTEGER NOT NULL DEFAULT (unixepoch()),
FOREIGN KEY (simulated_epoch_id) REFERENCES simulated_reward_epochs(id) ON DELETE CASCADE
);
-- Performance comparison data for analyzing methodology differences
CREATE TABLE IF NOT EXISTS simulated_performance_comparisons (
id INTEGER PRIMARY KEY AUTOINCREMENT,
simulated_epoch_id INTEGER NOT NULL,
node_id INTEGER NOT NULL,
node_type TEXT NOT NULL CHECK (node_type IN ('mixnode', 'gateway')),
-- Performance scores from each methodology
performance_score REAL NOT NULL CHECK (performance_score >= 0.0 AND performance_score <= 100.0),
work_factor REAL NOT NULL CHECK (work_factor >= 0.0),
calculation_method TEXT NOT NULL CHECK (calculation_method IN ('old', 'new', 'test')),
-- Additional metrics for analysis
positive_samples INTEGER DEFAULT 0,
negative_samples INTEGER DEFAULT 0,
route_success_rate REAL CHECK (route_success_rate >= 0.0 AND route_success_rate <= 100.0),
calculated_at INTEGER NOT NULL DEFAULT (unixepoch()),
FOREIGN KEY (simulated_epoch_id) REFERENCES simulated_reward_epochs(id) ON DELETE CASCADE
);
-- Route analysis metadata for each simulation run
CREATE TABLE IF NOT EXISTS simulated_route_analysis (
id INTEGER PRIMARY KEY AUTOINCREMENT,
simulated_epoch_id INTEGER NOT NULL,
calculation_method TEXT NOT NULL CHECK (calculation_method IN ('old', 'new', 'test')),
total_routes_analyzed INTEGER NOT NULL DEFAULT 0,
successful_routes INTEGER NOT NULL DEFAULT 0,
failed_routes INTEGER NOT NULL DEFAULT 0,
average_route_reliability REAL CHECK (average_route_reliability >= 0.0 AND average_route_reliability <= 100.0),
time_window_hours INTEGER NOT NULL DEFAULT 1, -- New method uses 1 hour, old uses 24
analysis_parameters TEXT, -- JSON with additional analysis configuration
calculated_at INTEGER NOT NULL DEFAULT (unixepoch()),
FOREIGN KEY (simulated_epoch_id) REFERENCES simulated_reward_epochs(id) ON DELETE CASCADE
);
-- Performance rankings and comparison analytics
CREATE TABLE IF NOT EXISTS simulated_performance_rankings (
id INTEGER PRIMARY KEY AUTOINCREMENT,
simulated_epoch_id INTEGER NOT NULL,
node_id INTEGER NOT NULL,
calculation_method TEXT NOT NULL CHECK (calculation_method IN ('old', 'new', 'test')),
performance_rank INTEGER NOT NULL,
performance_percentile REAL NOT NULL CHECK (performance_percentile >= 0.0 AND performance_percentile <= 100.0),
calculated_at INTEGER NOT NULL DEFAULT (unixepoch()),
FOREIGN KEY (simulated_epoch_id) REFERENCES simulated_reward_epochs(id) ON DELETE CASCADE,
UNIQUE(simulated_epoch_id, node_id, calculation_method)
);
-- Indexes for efficient querying
CREATE INDEX IF NOT EXISTS idx_simulated_reward_epochs_epoch_id ON simulated_reward_epochs(epoch_id);
CREATE INDEX IF NOT EXISTS idx_simulated_reward_epochs_calculation_method ON simulated_reward_epochs(calculation_method);
CREATE INDEX IF NOT EXISTS idx_simulated_reward_epochs_created_at ON simulated_reward_epochs(created_at);
CREATE INDEX IF NOT EXISTS idx_simulated_node_performance_simulated_epoch_id ON simulated_node_performance(simulated_epoch_id);
CREATE INDEX IF NOT EXISTS idx_simulated_node_performance_node_id ON simulated_node_performance(node_id);
CREATE INDEX IF NOT EXISTS idx_simulated_node_performance_calculation_method ON simulated_node_performance(calculation_method);
CREATE INDEX IF NOT EXISTS idx_simulated_node_performance_node_type ON simulated_node_performance(node_type);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_comparisons_simulated_epoch_id ON simulated_performance_comparisons(simulated_epoch_id);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_comparisons_node_id ON simulated_performance_comparisons(node_id);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_comparisons_calculation_method ON simulated_performance_comparisons(calculation_method);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_comparisons_node_type ON simulated_performance_comparisons(node_type);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_comparisons_performance_score ON simulated_performance_comparisons(performance_score);
CREATE INDEX IF NOT EXISTS idx_simulated_route_analysis_simulated_epoch_id ON simulated_route_analysis(simulated_epoch_id);
CREATE INDEX IF NOT EXISTS idx_simulated_route_analysis_calculation_method ON simulated_route_analysis(calculation_method);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_rankings_simulated_epoch_id ON simulated_performance_rankings(simulated_epoch_id);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_rankings_node_id ON simulated_performance_rankings(node_id);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_rankings_calculation_method ON simulated_performance_rankings(calculation_method);
CREATE INDEX IF NOT EXISTS idx_simulated_performance_rankings_performance_rank ON simulated_performance_rankings(performance_rank);
+3
View File
@@ -59,6 +59,9 @@ pub enum RewardingError {
#[error("could not obtain the current interval rewarding parameters")]
RewardingParamsRetrievalFailure,
#[error("Database operation failed: {source}")]
DatabaseError { source: anyhow::Error },
#[error("{0}")]
GenericError(#[from] anyhow::Error),
}
+73 -14
View File
@@ -22,6 +22,7 @@ use error::RewardingError;
pub(crate) use helpers::RewardedNodeWithParams;
use nym_mixnet_contract_common::{CurrentIntervalResponse, Interval};
use nym_task::{TaskClient, TaskManager};
use simulation::SimulationConfig;
use std::collections::HashSet;
use std::time::Duration;
use tokio::time::sleep;
@@ -32,6 +33,7 @@ mod event_reconciliation;
mod helpers;
mod rewarded_set_assignment;
mod rewarding;
pub(crate) mod simulation;
mod transition_beginning;
// naming things is difficult, ok?
@@ -42,6 +44,7 @@ pub struct EpochAdvancer {
described_cache: SharedCache<DescribedNodes>,
status_cache: NodeStatusCache,
storage: NymApiStorage,
simulation_config: Option<SimulationConfig>,
}
impl EpochAdvancer {
@@ -57,6 +60,7 @@ impl EpochAdvancer {
status_cache: NodeStatusCache,
described_cache: SharedCache<DescribedNodes>,
storage: NymApiStorage,
simulation_config: Option<SimulationConfig>,
) -> Self {
EpochAdvancer {
nyxd_client,
@@ -64,6 +68,7 @@ impl EpochAdvancer {
described_cache,
status_cache,
storage,
simulation_config,
}
}
@@ -136,29 +141,81 @@ impl EpochAdvancer {
if epoch_status.being_advanced_by.as_str() != address.as_ref() {
// another nym-api is already handling
error!("another nym-api ({}) is already advancing the epoch... but we shouldn't have other nym-apis yet!", epoch_status.being_advanced_by);
return Ok(());
// In simulation mode, we want to proceed anyway since we're not actually advancing the epoch
if self.simulation_config.is_some() {
info!("Another nym-api ({}) is advancing the epoch, but we're in simulation mode so proceeding anyway", epoch_status.being_advanced_by);
} else {
error!("another nym-api ({}) is already advancing the epoch... but we shouldn't have other nym-apis yet!", epoch_status.being_advanced_by);
return Ok(());
}
} else {
warn!("we seem to have crashed mid-epoch advancement...");
}
} else {
let should_continue = self.begin_epoch_transition().await?;
if !should_continue {
return Ok(());
// In simulation mode, skip trying to begin epoch transition
if self.simulation_config.is_none() {
let should_continue = self.begin_epoch_transition().await?;
if !should_continue {
return Ok(());
}
} else {
info!("Skipping epoch transition in simulation mode - proceeding to performance calculations");
}
}
// Reward all the nodes in the still current, soon to be previous rewarded set
info!("Rewarding the current rewarded set...");
self.reward_current_rewarded_set(rewards, interval).await?;
// Run simulation if enabled (before actual rewarding)
if let Some(simulation_config) = &self.simulation_config {
info!(
"Running reward simulation for epoch {}",
interval.current_epoch_absolute_id()
);
let rewarded_set = match self.nyxd_client.get_rewarded_set_nodes().await {
Ok(rewarded_set) => rewarded_set,
Err(err) => {
warn!("Failed to obtain current rewarded set for simulation: {err}. Falling back to cached version");
self.nym_contract_cache
.rewarded_set_owned()
.await
.into_inner()
.into()
}
};
// note: those operations don't really have to be atomic, so it's fine to send them
// as separate transactions
self.reconcile_epoch_events().await?;
self.update_rewarded_set_and_advance_epoch(&nym_nodes)
.await?;
if let Some(reward_params) = self
.nym_contract_cache
.interval_reward_params()
.await
.into_inner()
{
let _ = self
.run_simulation_if_enabled(
&rewarded_set,
reward_params,
interval.current_epoch_absolute_id(),
simulation_config.clone(),
)
.await;
} else {
warn!("Could not obtain reward parameters for simulation");
}
}
info!("Purging old node statuses from the storage...");
// Skip actual rewarding operations in simulation mode
if self.simulation_config.is_none() {
// Reward all the nodes in the still current, soon to be previous rewarded set
info!("Rewarding the current rewarded set...");
self.reward_current_rewarded_set(rewards, interval).await?;
// note: those operations don't really have to be atomic, so it's fine to send them
// as separate transactions
self.reconcile_epoch_events().await?;
self.update_rewarded_set_and_advance_epoch(&nym_nodes)
.await?;
} else {
info!("Skipping actual reward distribution in simulation mode");
}
info!("Purging old data (node statuses and routes) from the storage...");
let cutoff = (epoch_end - 2 * ONE_DAY).unix_timestamp();
self.storage.purge_old_statuses(cutoff).await?;
@@ -297,6 +354,7 @@ impl EpochAdvancer {
status_cache: &NodeStatusCache,
described_cache: SharedCache<DescribedNodes>,
storage: &NymApiStorage,
simulation_config: Option<SimulationConfig>,
shutdown: &TaskManager,
) {
let mut rewarded_set_updater = EpochAdvancer::new(
@@ -305,6 +363,7 @@ impl EpochAdvancer {
status_cache.to_owned(),
described_cache,
storage.to_owned(),
simulation_config,
);
let shutdown_listener = shutdown.subscribe();
tokio::spawn(async move { rewarded_set_updater.run(shutdown_listener).await });
+556
View File
@@ -0,0 +1,556 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
//! Performance methodology comparison system
//!
//! This module provides functionality to compare different performance calculation
//! methodologies without affecting actual rewards, enabling analysis of:
//! - Old method: 24-hour cache-based performance calculation
//! - New method: 1-hour route-based performance calculation
//!
//! The system focuses on performance metrics and rankings rather than reward amounts,
//! as actual rewards are calculated on-chain based on factors not available to the API.
use crate::epoch_operations::error::RewardingError;
use crate::epoch_operations::helpers::RewardedNodeWithParams;
use crate::storage::models::{
SimulatedNodePerformance, SimulatedPerformanceComparison, SimulatedPerformanceRanking,
SimulatedRouteAnalysis,
};
use crate::support::storage::NymApiStorage;
use crate::EpochAdvancer;
use nym_contracts_common::types::NaiveFloat;
use nym_mixnet_contract_common::reward_params::Performance;
use nym_mixnet_contract_common::{EpochRewardedSet, NodeId, RewardingParams};
use std::collections::HashMap;
use time::OffsetDateTime;
use tracing::{debug, error, info};
/// Configuration for simulation runs
#[derive(Debug, Clone)]
pub struct SimulationConfig {
/// Time window in hours for new method calculation (default: 1)
pub new_method_time_window_hours: u32,
/// Description for this simulation run
pub description: Option<String>,
}
impl Default for SimulationConfig {
fn default() -> Self {
Self {
new_method_time_window_hours: 1,
description: None,
}
}
}
/// Main simulation coordinator
pub struct SimulationCoordinator<'a> {
storage: &'a NymApiStorage,
config: SimulationConfig,
}
impl<'a> SimulationCoordinator<'a> {
pub fn new(storage: &'a NymApiStorage, config: SimulationConfig) -> Self {
Self { storage, config }
}
/// Run simulation using new rewarding method only
pub async fn run_simulation(
&self,
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
current_epoch_id: u32,
) -> Result<(), RewardingError> {
let now = OffsetDateTime::now_utc();
let end_timestamp = now.unix_timestamp();
let start_timestamp =
end_timestamp - (self.config.new_method_time_window_hours as i64 * 3600);
info!(
"Starting new method simulation for epoch {} with time window {}h",
current_epoch_id, self.config.new_method_time_window_hours
);
// Create simulation epoch record or get existing one
let (epoch_db_id, is_new) = self
.storage
.manager
.create_or_get_simulated_reward_epoch(
current_epoch_id,
"new",
start_timestamp,
end_timestamp,
self.config.description.as_deref(),
)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
if !is_new {
info!(
"Simulation for epoch {} already exists (id: {}), skipping duplicate simulation",
current_epoch_id, epoch_db_id
);
return Ok(());
}
// Run new method simulation only
match self
.run_new_method_simulation(rewarded_set, reward_params, epoch_db_id, end_timestamp)
.await
{
Ok(_) => {
info!("New method simulation completed successfully");
}
Err(e) => {
error!("New method simulation failed: {}", e);
return Err(e);
}
}
info!("Simulation completed for epoch {}", current_epoch_id);
Ok(())
}
/// Run simulation using new method (1h route-based)
async fn run_new_method_simulation(
&self,
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
epoch_db_id: i64,
end_timestamp: i64,
) -> Result<(), RewardingError> {
debug!(
"Running new method simulation ({}h route-based)",
self.config.new_method_time_window_hours
);
let time_window_secs = (self.config.new_method_time_window_hours as i64) * 3600;
let start_timestamp = end_timestamp - time_window_secs;
// Get route-based performance data using new method
let corrected_reliabilities = self
.storage
.manager
.calculate_corrected_node_reliabilities_for_interval(start_timestamp, end_timestamp)
.await
.map_err(|e| RewardingError::DatabaseError { source: e })?;
// Convert to performance map and build route reliability data
let mut performance_map = HashMap::new();
let mut route_reliability_map = HashMap::new();
let mut total_routes = 0u32;
let mut successful_routes = 0u32;
for node_reliability in &corrected_reliabilities {
let total_samples =
node_reliability.pos_samples_in_interval + node_reliability.neg_samples_in_interval;
total_routes += total_samples;
successful_routes += node_reliability.pos_samples_in_interval;
performance_map.insert(
node_reliability.node_id,
Performance::naive_try_from_f64(node_reliability.reliability / 100.0)
.unwrap_or_default(),
);
// Store sample counts for detailed performance records
route_reliability_map.insert(
node_reliability.node_id,
(
node_reliability.pos_samples_in_interval,
node_reliability.neg_samples_in_interval,
),
);
}
// Calculate rewards using new method logic
let rewarded_nodes =
self.calculate_rewards_for_nodes(rewarded_set, reward_params, &performance_map);
// Convert to simulation data structures
let node_performance = self
.convert_to_simulated_performance(
&rewarded_nodes,
rewarded_set,
reward_params,
epoch_db_id,
"new",
Some(&route_reliability_map), // Pass route sample data for new method
)
.await;
let performance_comparisons = self
.convert_to_performance_comparisons(
&rewarded_nodes,
rewarded_set,
reward_params,
epoch_db_id,
"new",
)
.await;
// Calculate average reliability for new method (mean of all node reliabilities)
let node_reliabilities: Vec<f64> = corrected_reliabilities
.iter()
.filter(|n| n.pos_samples_in_interval + n.neg_samples_in_interval > 0)
.map(|n| n.reliability)
.collect();
let (mean_reliability, median_reliability) = if !node_reliabilities.is_empty() {
let mean = node_reliabilities.iter().sum::<f64>() / node_reliabilities.len() as f64;
let median = calculate_median(&node_reliabilities);
(
Some((mean * 100.0).round() / 100.0),
Some((median * 100.0).round() / 100.0),
)
} else {
(None, None)
};
// Create route analysis for new method
let route_analysis = SimulatedRouteAnalysis {
id: 0, // Will be set by database
simulated_epoch_id: epoch_db_id,
calculation_method: "new".to_string(),
total_routes_analyzed: total_routes,
successful_routes,
failed_routes: total_routes - successful_routes,
average_route_reliability: mean_reliability,
time_window_hours: self.config.new_method_time_window_hours,
analysis_parameters: Some(format!(
"{{\"method\":\"route_based\",\"time_window_hours\":{},\"corrected_routes\":{},\"median_reliability\":{},\"nodes_analyzed\":{}}}",
self.config.new_method_time_window_hours,
corrected_reliabilities.len(),
median_reliability.unwrap_or(0.0),
node_reliabilities.len()
)),
calculated_at: OffsetDateTime::now_utc().unix_timestamp(),
};
// Store results in database
self.storage
.manager
.insert_simulated_node_performance(&node_performance)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
self.storage
.manager
.insert_simulated_performance_comparisons(&performance_comparisons)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
// Calculate and store performance rankings
let rankings =
self.calculate_performance_rankings(&performance_comparisons, epoch_db_id, "new");
self.storage
.manager
.insert_simulated_performance_rankings(&rankings)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
self.storage
.manager
.insert_simulated_route_analysis(&route_analysis)
.await
.map_err(|e| RewardingError::DatabaseError { source: e.into() })?;
Ok(())
}
/// Calculate rewards for nodes using the provided performance data
/// This mirrors the logic from helpers.rs but uses simulation performance data
fn calculate_rewards_for_nodes(
&self,
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
performance_map: &HashMap<NodeId, Performance>,
) -> Vec<RewardedNodeWithParams> {
let nodes = &rewarded_set.assignment;
let active_node_work_factor = reward_params.active_node_work();
let standby_node_work_factor = reward_params.standby_node_work();
let mut rewarded_nodes = Vec::with_capacity(nodes.rewarded_set_size());
// Process active set mixnodes (layers 1, 2, 3)
for &node_id in nodes
.layer1
.iter()
.chain(nodes.layer2.iter())
.chain(nodes.layer3.iter())
{
let performance = performance_map.get(&node_id).copied().unwrap_or_default();
rewarded_nodes.push(RewardedNodeWithParams {
node_id,
params: nym_mixnet_contract_common::reward_params::NodeRewardingParameters {
performance,
work_factor: active_node_work_factor,
},
});
}
// Process active set gateways
for &node_id in nodes
.entry_gateways
.iter()
.chain(nodes.exit_gateways.iter())
{
let performance = performance_map.get(&node_id).copied().unwrap_or_default();
rewarded_nodes.push(RewardedNodeWithParams {
node_id,
params: nym_mixnet_contract_common::reward_params::NodeRewardingParameters {
performance,
work_factor: active_node_work_factor,
},
});
}
// Process standby nodes
for &node_id in &nodes.standby {
let performance = performance_map.get(&node_id).copied().unwrap_or_default();
rewarded_nodes.push(RewardedNodeWithParams {
node_id,
params: nym_mixnet_contract_common::reward_params::NodeRewardingParameters {
performance,
work_factor: standby_node_work_factor,
},
});
}
rewarded_nodes
}
/// Determine node type and work factor from rewarded set position
fn determine_node_info(
&self,
node_id: NodeId,
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
) -> (String, f64) {
let nodes = &rewarded_set.assignment;
// Check if node is in active mixnode layers
if nodes.layer1.contains(&node_id)
|| nodes.layer2.contains(&node_id)
|| nodes.layer3.contains(&node_id)
{
return (
"mixnode".to_string(),
reward_params.active_node_work().naive_to_f64(),
);
}
// Check if node is in active gateways
if nodes.entry_gateways.contains(&node_id) || nodes.exit_gateways.contains(&node_id) {
return (
"gateway".to_string(),
reward_params.active_node_work().naive_to_f64(),
);
}
// Check if node is in standby (could be mixnode or gateway)
if nodes.standby.contains(&node_id) {
// Note: We cannot determine if standby nodes are mixnodes or gateways from the
// rewarded set data alone. This limitation exists in both old and new calculation
// methods and doesn't significantly impact the simulation since:
// 1. All standby nodes receive the same work factor regardless of type
// 2. The gateway 3-sample rule can't be applied to standby nodes in either method
// For consistency, we label all standby nodes as "mixnode" in the simulation data
return (
"mixnode".to_string(),
reward_params.standby_node_work().naive_to_f64(),
);
}
// Default case (shouldn't happen)
("unknown".to_string(), 0.0)
}
/// Convert rewarded nodes to simulated performance records
async fn convert_to_simulated_performance(
&self,
rewarded_nodes: &[RewardedNodeWithParams],
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
epoch_db_id: i64,
method: &str,
route_reliability_map: Option<&HashMap<NodeId, (u32, u32)>>, // (positive_samples, negative_samples)
) -> Vec<SimulatedNodePerformance> {
let now = OffsetDateTime::now_utc().unix_timestamp();
let mut performance_records = Vec::with_capacity(rewarded_nodes.len());
// First collect all node IDs for batch identity key lookups
let all_node_ids: Vec<NodeId> = rewarded_nodes.iter().map(|node| node.node_id).collect();
// Batch fetch identity keys for both mixnodes and gateways
let mixnode_identities = self
.storage
.manager
.get_mixnode_identity_keys_batch(&all_node_ids)
.await
.unwrap_or_default();
let gateway_identities = self
.storage
.manager
.get_gateway_identity_keys_batch(&all_node_ids)
.await
.unwrap_or_default();
for node in rewarded_nodes {
let (node_type, _) =
self.determine_node_info(node.node_id, rewarded_set, reward_params);
// Get identity key from our batch results
let identity_key = match node_type.as_str() {
"mixnode" => mixnode_identities.get(&node.node_id).cloned(),
"gateway" => gateway_identities.get(&node.node_id).cloned(),
_ => None,
};
// Extract sample counts from route reliability if available
let (positive_samples, negative_samples) = route_reliability_map
.and_then(|map| map.get(&node.node_id))
.copied()
.unwrap_or((0, 0));
performance_records.push(SimulatedNodePerformance {
id: 0, // Will be set by database
simulated_epoch_id: epoch_db_id,
node_id: node.node_id,
node_type,
identity_key,
reliability_score: node.params.performance.naive_to_f64() * 100.0,
positive_samples,
negative_samples,
work_factor: Some(node.params.work_factor.naive_to_f64()),
calculation_method: method.to_string(),
calculated_at: now,
});
}
performance_records
}
/// Convert rewarded nodes to performance comparison records
async fn convert_to_performance_comparisons(
&self,
rewarded_nodes: &[RewardedNodeWithParams],
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
epoch_db_id: i64,
method: &str,
) -> Vec<SimulatedPerformanceComparison> {
let now = OffsetDateTime::now_utc().unix_timestamp();
let mut performance_records = Vec::with_capacity(rewarded_nodes.len());
for node in rewarded_nodes {
let (node_type, _) =
self.determine_node_info(node.node_id, rewarded_set, reward_params);
performance_records.push(SimulatedPerformanceComparison {
id: 0, // Will be set by database
simulated_epoch_id: epoch_db_id,
node_id: node.node_id,
node_type,
performance_score: node.params.performance.naive_to_f64() * 100.0,
work_factor: node.params.work_factor.naive_to_f64(),
calculation_method: method.to_string(),
positive_samples: None, // Will be populated from node performance data
negative_samples: None,
route_success_rate: None,
calculated_at: now,
});
}
performance_records
}
/// Calculate performance rankings for a set of performance comparisons
fn calculate_performance_rankings(
&self,
performance_comparisons: &[SimulatedPerformanceComparison],
epoch_db_id: i64,
method: &str,
) -> Vec<SimulatedPerformanceRanking> {
let now = OffsetDateTime::now_utc().unix_timestamp();
let mut rankings = Vec::with_capacity(performance_comparisons.len());
// Sort by performance score descending
let mut sorted_comparisons: Vec<_> = performance_comparisons
.iter()
.enumerate()
.map(|(idx, comp)| (idx, comp.performance_score))
.collect();
sorted_comparisons
.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap_or(std::cmp::Ordering::Equal));
let total_nodes = sorted_comparisons.len() as f64;
for (rank, (original_idx, _score)) in sorted_comparisons.iter().enumerate() {
let comparison = &performance_comparisons[*original_idx];
let percentile = ((total_nodes - rank as f64 - 1.0) / total_nodes) * 100.0;
rankings.push(SimulatedPerformanceRanking {
id: 0, // Will be set by database
simulated_epoch_id: epoch_db_id,
node_id: comparison.node_id,
calculation_method: method.to_string(),
performance_rank: (rank + 1) as i64,
performance_percentile: percentile,
calculated_at: now,
});
}
rankings
}
}
impl EpochAdvancer {
/// Run simulation during epoch operations if simulation mode is enabled
pub async fn run_simulation_if_enabled(
&self,
rewarded_set: &EpochRewardedSet,
reward_params: RewardingParams,
current_epoch_id: u32,
simulation_config: SimulationConfig,
) -> Result<(), RewardingError> {
let coordinator = SimulationCoordinator::new(&self.storage, simulation_config);
match coordinator
.run_simulation(rewarded_set, reward_params, current_epoch_id)
.await
{
Ok(()) => {
info!(
"Simulation completed successfully for epoch {}",
current_epoch_id
);
Ok(())
}
Err(e) => {
error!("Simulation failed for epoch {}: {}", current_epoch_id, e);
// Don't fail the entire epoch operation due to simulation failure
Ok(())
}
}
}
}
/// Calculate median of a vector of f64 values
fn calculate_median(values: &[f64]) -> f64 {
if values.is_empty() {
return 0.0;
}
let mut sorted = values.to_vec();
sorted.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
let len = sorted.len();
if len % 2 == 0 {
(sorted[len / 2 - 1] + sorted[len / 2]) / 2.0
} else {
sorted[len / 2]
}
}
+1
View File
@@ -24,6 +24,7 @@ pub(crate) mod node_describe_cache;
pub(crate) mod node_status_api;
pub(crate) mod nym_contract_cache;
pub(crate) mod nym_nodes;
pub(crate) mod simulation_api;
mod status;
pub(crate) mod support;
mod unstable_routes;
@@ -20,7 +20,7 @@ use nym_api_requests::models::{
};
use nym_http_api_common::{FormattedResponse, OutputParams};
use nym_mixnet_contract_common::NodeId;
use nym_types::monitoring::MonitorMessage;
use nym_types::monitoring::{MonitorMessage, MonitorResults};
use tracing::error;
pub(super) fn mandatory_routes() -> Router<AppState> {
@@ -33,6 +33,10 @@ pub(super) fn mandatory_routes() -> Router<AppState> {
"/submit-node-monitoring-results",
post(submit_node_monitoring_results),
)
.route(
"/submit-route-monitoring-results",
post(submit_route_monitoring_results),
)
.nest(
"/mixnode/:mix_id",
Router::new()
@@ -58,6 +62,53 @@ pub(super) fn mandatory_routes() -> Router<AppState> {
)
}
#[utoipa::path(
tag = "status",
post,
path = "/v1/status/submit-route-monitoring-results",
responses(
(status = 200),
(status = 400, body = String, description = "TBD"),
(status = 403, body = String, description = "TBD"),
(status = 500, body = String, description = "TBD"),
),
)]
pub(crate) async fn submit_route_monitoring_results(
State(state): State<AppState>,
Json(message): Json<MonitorMessage>,
) -> AxumResult<()> {
if !message.is_in_allowed() {
return Err(AxumErrorResponse::forbidden(
"Monitor not registered to submit results",
));
}
if !message.timely() {
return Err(AxumErrorResponse::bad_request("Message is too old"));
}
if !message.verify() {
return Err(AxumErrorResponse::bad_request("invalid signature"));
}
match message.results() {
MonitorResults::Route(results) => {
match state.storage.submit_route_monitoring_results(results).await {
Ok(_) => Ok(()),
Err(err) => {
error!("failed to submit node monitoring results: {err}");
Err(AxumErrorResponse::internal_msg(
"failed to submit node monitoring results",
))
}
}
}
MonitorResults::Node(_results) => Err(AxumErrorResponse::bad_request(
"Node monitoring results not supported for this endpoint",
)),
}
}
#[utoipa::path(
tag = "status",
post,
@@ -87,18 +138,21 @@ pub(crate) async fn submit_gateway_monitoring_results(
return Err(AxumErrorResponse::bad_request("invalid signature"));
}
match state
.storage
.submit_gateway_statuses_v2(message.results())
.await
{
Ok(_) => Ok(()),
Err(err) => {
error!("failed to submit gateway monitoring results: {err}");
Err(AxumErrorResponse::internal_msg(
"failed to submit gateway monitoring results",
))
match message.results() {
MonitorResults::Node(results) => {
match state.storage.submit_gateway_statuses_v2(results).await {
Ok(_) => Ok(()),
Err(err) => {
error!("failed to submit node monitoring results: {err}");
Err(AxumErrorResponse::internal_msg(
"failed to submit node monitoring results",
))
}
}
}
MonitorResults::Route(_results) => Err(AxumErrorResponse::bad_request(
"Gateway monitoring results not supported for this endpoint",
)),
}
}
@@ -131,18 +185,21 @@ pub(crate) async fn submit_node_monitoring_results(
return Err(AxumErrorResponse::bad_request("invalid signature"));
}
match state
.storage
.submit_mixnode_statuses_v2(message.results())
.await
{
Ok(_) => Ok(()),
Err(err) => {
error!("failed to submit node monitoring results: {err}");
Err(AxumErrorResponse::internal_msg(
"failed to submit node monitoring results",
))
match message.results() {
MonitorResults::Node(results) => {
match state.storage.submit_mixnode_statuses_v2(results).await {
Ok(_) => Ok(()),
Err(err) => {
error!("failed to submit node monitoring results: {err}");
Err(AxumErrorResponse::internal_msg(
"failed to submit node monitoring results",
))
}
}
}
MonitorResults::Route(_results) => Err(AxumErrorResponse::bad_request(
"Node monitoring results not supported for this endpoint",
)),
}
}
File diff suppressed because it is too large Load Diff
+10
View File
@@ -0,0 +1,10 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
//! Simulation API for reward calculation analysis
//!
//! This module provides REST endpoints for accessing and analyzing
//! simulated reward calculation data comparing old vs new methodologies.
pub(crate) mod handlers;
pub(crate) mod models;
+300
View File
@@ -0,0 +1,300 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
//! API models for simulation data responses
use crate::storage::models::{
SimulatedNodePerformance, SimulatedPerformanceComparison, SimulatedRewardEpoch,
SimulatedRouteAnalysis,
};
use nym_mixnet_contract_common::NodeId;
use serde::{Deserialize, Serialize};
use time::OffsetDateTime;
use utoipa::ToSchema;
/// Response for listing simulation epochs
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct SimulationEpochsResponse {
pub epochs: Vec<SimulationEpochSummary>,
pub total_count: usize,
}
/// Summary information about a simulation epoch
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct SimulationEpochSummary {
pub id: i64,
pub epoch_id: u32,
pub calculation_method: String,
pub start_timestamp: i64,
pub end_timestamp: i64,
pub description: Option<String>,
pub created_at: i64,
/// Number of nodes that had performance calculated
pub nodes_analyzed: usize,
/// Available calculation methods for this epoch
pub available_methods: Vec<String>,
}
impl From<SimulatedRewardEpoch> for SimulationEpochSummary {
fn from(epoch: SimulatedRewardEpoch) -> Self {
Self {
id: epoch.id,
epoch_id: epoch.epoch_id,
calculation_method: epoch.calculation_method,
start_timestamp: epoch.start_timestamp,
end_timestamp: epoch.end_timestamp,
description: epoch.description,
created_at: epoch.created_at,
nodes_analyzed: 0, // Will be populated by handler
available_methods: vec![], // Will be populated by handler
}
}
}
/// Detailed simulation epoch with all data
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct SimulationEpochDetails {
pub epoch: SimulationEpochSummary,
pub node_performance: Vec<NodePerformanceData>,
pub performance_comparisons: Vec<PerformanceComparisonData>,
pub route_analysis: Vec<RouteAnalysisData>,
}
/// Node performance data for API responses
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct NodePerformanceData {
pub node_id: NodeId,
pub node_type: String,
pub identity_key: Option<String>,
pub reliability_score: f64,
pub positive_samples: u32,
pub negative_samples: u32,
pub work_factor: Option<f64>,
pub calculation_method: String,
pub calculated_at: i64,
/// Production performance value (from last 24h cache)
pub production_performance: Option<f64>,
}
impl NodePerformanceData {
pub fn total_samples(&self) -> u32 {
self.positive_samples + self.negative_samples
}
}
impl From<SimulatedNodePerformance> for NodePerformanceData {
fn from(perf: SimulatedNodePerformance) -> Self {
Self {
node_id: perf.node_id,
node_type: perf.node_type,
identity_key: perf.identity_key,
reliability_score: (perf.reliability_score * 100.0).round() / 100.0,
positive_samples: perf.positive_samples,
negative_samples: perf.negative_samples,
work_factor: perf.work_factor.map(|w| (w * 100.0).round() / 100.0),
calculation_method: perf.calculation_method,
calculated_at: perf.calculated_at,
production_performance: None, // Will be populated from node annotations cache
}
}
}
/// Performance comparison data for API responses
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct PerformanceComparisonData {
pub node_id: NodeId,
pub node_type: String,
pub performance_score: f64,
pub work_factor: f64,
pub calculation_method: String,
pub positive_samples: Option<i64>,
pub negative_samples: Option<i64>,
pub route_success_rate: Option<f64>,
pub calculated_at: i64,
}
impl From<SimulatedPerformanceComparison> for PerformanceComparisonData {
fn from(comparison: SimulatedPerformanceComparison) -> Self {
Self {
node_id: comparison.node_id,
node_type: comparison.node_type,
performance_score: (comparison.performance_score * 100.0).round() / 100.0,
work_factor: (comparison.work_factor * 100.0).round() / 100.0,
calculation_method: comparison.calculation_method,
positive_samples: comparison.positive_samples,
negative_samples: comparison.negative_samples,
route_success_rate: comparison
.route_success_rate
.map(|r| (r * 100.0).round() / 100.0),
calculated_at: comparison.calculated_at,
}
}
}
/// Route analysis data for API responses
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct RouteAnalysisData {
pub calculation_method: String,
pub total_routes_analyzed: u32,
pub successful_routes: u32,
pub failed_routes: u32,
pub average_route_reliability: Option<f64>,
pub median_route_reliability: Option<f64>,
pub time_window_hours: u32,
pub analysis_parameters: Option<String>,
pub calculated_at: i64,
}
impl From<SimulatedRouteAnalysis> for RouteAnalysisData {
fn from(analysis: SimulatedRouteAnalysis) -> Self {
// Extract median from analysis_parameters JSON if available
let median_route_reliability = analysis
.analysis_parameters
.as_ref()
.and_then(|params| serde_json::from_str::<serde_json::Value>(params).ok())
.and_then(|json| json.get("median_reliability").and_then(|v| v.as_f64()))
.map(|m| (m * 100.0).round() / 100.0);
Self {
calculation_method: analysis.calculation_method,
total_routes_analyzed: analysis.total_routes_analyzed,
successful_routes: analysis.successful_routes,
failed_routes: analysis.failed_routes,
average_route_reliability: analysis
.average_route_reliability
.map(|a| (a * 100.0).round() / 100.0),
median_route_reliability,
time_window_hours: analysis.time_window_hours,
analysis_parameters: analysis.analysis_parameters,
calculated_at: analysis.calculated_at,
}
}
}
/// Comparison between old and new methods for a specific epoch
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct MethodComparisonResponse {
pub epoch_id: u32,
pub simulation_epoch_id: i64,
pub node_comparisons: Vec<NodeMethodComparison>,
pub summary_statistics: ComparisonSummaryStats,
pub route_analysis_comparison: RouteAnalysisComparison,
}
/// Comparison data for a single node between old and new methods
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct NodeMethodComparison {
pub node_id: NodeId,
pub node_type: String,
pub identity_key: Option<String>,
/// Production performance (old method) value
pub production_performance: Option<f64>,
/// Simulated performance (new method) value
pub simulated_performance: f64,
/// Sample counts from route-based calculation
pub positive_samples: u32,
pub negative_samples: u32,
pub work_factor: Option<f64>,
pub reliability_difference: Option<f64>, // new - old
pub performance_delta_percentage: Option<f64>, // (new - old) / old * 100
pub ranking_old_method: Option<i64>,
pub ranking_new_method: Option<i64>,
pub ranking_delta: Option<i64>, // new ranking - old ranking (negative is improvement)
}
/// Distribution of nodes by reliability score ranges
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct ReliabilityDistribution {
pub excellent: usize, // >95%
pub very_good: usize, // 90-95%
pub good: usize, // 75-90%
pub moderate: usize, // 50-75%
pub poor: usize, // 25-50%
pub very_poor: usize, // <25%
}
/// Summary statistics comparing old vs new methods
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct ComparisonSummaryStats {
pub total_nodes_compared: usize,
pub nodes_improved: usize, // nodes with better performance in new method
pub nodes_degraded: usize, // nodes with worse performance in new method
pub nodes_unchanged: usize, // nodes with same performance
pub average_reliability_old: f64,
pub average_reliability_new: f64,
pub median_reliability_old: f64,
pub median_reliability_new: f64,
pub reliability_std_dev_old: f64,
pub reliability_std_dev_new: f64,
pub max_improvement: f64, // highest positive delta
pub max_degradation: f64, // highest negative delta
pub distribution_old: ReliabilityDistribution,
pub distribution_new: ReliabilityDistribution,
}
/// Comparison of route analysis between methods
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub struct RouteAnalysisComparison {
pub old_method: Option<RouteAnalysisData>,
pub new_method: Option<RouteAnalysisData>,
pub time_window_difference_hours: i32, // new - old
pub route_coverage_difference: i32, // new total routes - old total routes
pub success_rate_difference: Option<f64>, // new average reliability - old average reliability
}
/// Export format options
#[derive(Serialize, Deserialize, ToSchema, Debug, Clone)]
pub enum ExportFormat {
#[serde(rename = "json")]
Json,
#[serde(rename = "csv")]
Csv,
}
/// Query parameters for simulation listings
#[derive(Deserialize, ToSchema, Debug, utoipa::IntoParams)]
pub struct SimulationListQuery {
/// Limit number of results (default: 50, max: 1000)
pub limit: Option<usize>,
/// Offset for pagination (default: 0)
pub offset: Option<usize>,
}
/// Query parameters for node-specific performance comparison
#[derive(Deserialize, ToSchema, Debug, utoipa::IntoParams)]
pub struct NodeComparisonQuery {
/// Specific node ID to analyze
pub node_id: Option<NodeId>,
/// Node type filter (mixnode, gateway)
pub node_type: Option<String>,
/// Minimum reliability difference threshold for filtering
pub min_delta: Option<f64>,
/// Maximum reliability difference threshold for filtering
pub max_delta: Option<f64>,
}
/// Error response for simulation API
#[derive(Serialize, Deserialize, ToSchema, Debug)]
pub struct SimulationApiError {
pub error: String,
pub details: Option<String>,
pub timestamp: i64,
}
impl SimulationApiError {
pub fn new(error: &str) -> Self {
Self {
error: error.to_string(),
details: None,
timestamp: OffsetDateTime::now_utc().unix_timestamp(),
}
}
pub fn with_details(error: &str, details: &str) -> Self {
Self {
error: error.to_string(),
details: Some(details.to_string()),
timestamp: OffsetDateTime::now_utc().unix_timestamp(),
}
}
}
+34 -4
View File
@@ -9,7 +9,7 @@ use crate::ecash::dkg::controller::keys::{
};
use crate::ecash::dkg::controller::DkgController;
use crate::ecash::state::EcashState;
use crate::epoch_operations::EpochAdvancer;
use crate::epoch_operations::{simulation::SimulationConfig, EpochAdvancer};
use crate::network::models::NetworkDetails;
use crate::node_describe_cache::DescribedNodes;
use crate::node_status_api::handlers::unstable;
@@ -61,10 +61,22 @@ pub(crate) struct Args {
long,
requires = "enable_monitor",
requires = "mnemonic",
conflicts_with = "simulate_rewarding",
env = "NYMAPI_ENABLE_REWARDING_ARG"
)]
pub(crate) enable_rewarding: Option<bool>,
/// Enable simulated rewarding mode to compare old vs new reward calculations
/// Requires monitoring but does NOT require mnemonic (no blockchain transactions)
/// default: None - config value will be used instead
#[clap(
long,
requires = "enable_monitor",
conflicts_with = "enable_rewarding",
env = "NYMAPI_SIMULATE_REWARDING_ARG"
)]
pub(crate) simulate_rewarding: Option<bool>,
/// Endpoint to nyxd instance used for contract information.
/// default: None - config value will be used instead
#[clap(long, env = "NYMAPI_NYXD_VALIDATOR_ARG")]
@@ -276,15 +288,33 @@ async fn start_nym_api_tasks_axum(config: &Config) -> anyhow::Result<ShutdownHan
HistoricalUptimeUpdater::start(storage.to_owned(), &task_manager);
// start 'rewarding' if its enabled
if config.rewarding.enabled {
epoch_operations::ensure_rewarding_permission(&nyxd_client).await?;
// start 'rewarding' or 'simulation' if enabled
if config.rewarding.enabled || config.rewarding.simulation_mode {
// Only check rewarding permission for real rewarding, not simulation
if config.rewarding.enabled && !config.rewarding.simulation_mode {
epoch_operations::ensure_rewarding_permission(&nyxd_client).await?;
}
// Create simulation config if simulation mode is enabled
let simulation_config = if config.rewarding.simulation_mode {
Some(SimulationConfig {
new_method_time_window_hours: config
.rewarding
.debug
.simulation_new_method_time_window_hours,
description: Some("Simulation run at epoch advancement".to_string()),
})
} else {
None
};
EpochAdvancer::start(
nyxd_client,
&nym_contract_cache_state,
&node_status_cache_state,
described_nodes_cache.clone(),
&storage,
simulation_config,
&task_manager,
);
}
+29 -1
View File
@@ -136,14 +136,30 @@ impl Config {
pub fn validate(&self) -> anyhow::Result<()> {
let can_sign = self.base.mnemonic.is_some();
if !can_sign && self.rewarding.enabled {
// Real rewarding requires mnemonic, but simulation mode does not
if !can_sign && self.rewarding.enabled && !self.rewarding.simulation_mode {
bail!("can't enable rewarding without providing a mnemonic")
}
// Simulation mode and real rewarding are mutually exclusive
if self.rewarding.enabled && self.rewarding.simulation_mode {
bail!("cannot enable both real rewarding and simulation mode simultaneously")
}
if !can_sign && self.ecash_signer.enabled {
bail!("can't enable coconut signer without providing a mnemonic")
}
// Validate simulation-specific settings
if self.rewarding.simulation_mode {
if self.rewarding.debug.simulation_new_method_time_window_hours == 0 {
bail!("simulation_new_method_time_window_hours must be greater than 0")
}
if self.rewarding.debug.simulation_new_method_time_window_hours > 24 {
bail!("simulation_new_method_time_window_hours should not exceed 24 hours")
}
}
self.ecash_signer.validate()?;
Ok(())
@@ -158,6 +174,9 @@ impl Config {
if let Some(enable_rewarding) = args.enable_rewarding {
self.rewarding.enabled = enable_rewarding;
}
if let Some(simulate_rewarding) = args.simulate_rewarding {
self.rewarding.simulation_mode = simulate_rewarding;
}
if let Some(nyxd_upstream) = args.nyxd_validator {
self.base.local_validator = nyxd_upstream;
}
@@ -499,6 +518,9 @@ pub struct Rewarding {
/// Specifies whether rewarding service is enabled in this process.
pub enabled: bool,
/// Specifies whether to run in simulation mode (compare old vs new calculations without blockchain transactions)
pub simulation_mode: bool,
// this should really be a thing too...
// pub paths: RewardingPathfinder,
#[serde(default)]
@@ -510,6 +532,7 @@ impl Default for Rewarding {
fn default() -> Self {
Rewarding {
enabled: false,
simulation_mode: false,
debug: Default::default(),
}
}
@@ -522,12 +545,17 @@ pub struct RewardingDebug {
/// distribute rewards for given interval.
/// Note, only values in range 0-100 are valid
pub minimum_interval_monitor_threshold: u8,
/// Time window in hours for new route-based performance calculation (default: 1 hour)
/// Old method always uses 24 hours for comparison
pub simulation_new_method_time_window_hours: u32,
}
impl Default for RewardingDebug {
fn default() -> Self {
RewardingDebug {
minimum_interval_monitor_threshold: DEFAULT_MONITOR_THRESHOLD,
simulation_new_method_time_window_hours: 1, // New method uses 1 hour
}
}
}
+5
View File
@@ -12,6 +12,9 @@ pub(crate) struct OverrideConfig {
/// Specifies whether network rewarding is enabled on this API
pub(crate) enable_rewarding: Option<bool>,
/// Specifies whether simulated rewarding is enabled on this API
pub(crate) simulate_rewarding: Option<bool>,
/// Endpoint to nyxd instance used for contract information.
pub(crate) nyxd_validator: Option<url::Url>,
@@ -40,6 +43,7 @@ impl From<init::Args> for OverrideConfig {
OverrideConfig {
enable_monitor: Some(args.enable_monitor),
enable_rewarding: Some(args.enable_rewarding),
simulate_rewarding: None, // Not available during initialization
nyxd_validator: args.nyxd_validator,
mnemonic: args.mnemonic,
enable_zk_nym: Some(args.enable_zk_nym),
@@ -56,6 +60,7 @@ impl From<run::Args> for OverrideConfig {
OverrideConfig {
enable_monitor: args.enable_monitor,
enable_rewarding: args.enable_rewarding,
simulate_rewarding: args.simulate_rewarding,
nyxd_validator: args.nyxd_validator,
mnemonic: args.mnemonic,
enable_zk_nym: args.enable_zk_nym,
+2
View File
@@ -8,6 +8,7 @@ use crate::node_status_api::handlers::status_routes;
use crate::nym_contract_cache::handlers::nym_contract_cache_routes;
use crate::nym_nodes::handlers::legacy::legacy_nym_node_routes;
use crate::nym_nodes::handlers::nym_node_routes;
use crate::simulation_api::handlers::simulation_routes;
use crate::status;
use crate::support::http::openapi::ApiDoc;
use crate::support::http::state::AppState;
@@ -64,6 +65,7 @@ impl RouterBuilder {
.nest("/api-status", status::handlers::api_status_routes())
.nest("/nym-nodes", nym_node_routes())
.nest("/ecash", ecash_routes())
.nest("/simulation", simulation_routes())
.nest("/unstable", unstable_routes()), // CORS layer needs to be "outside" of routes
);
File diff suppressed because it is too large Load Diff
+34 -2
View File
@@ -17,7 +17,7 @@ use crate::support::storage::models::{
};
use dashmap::DashMap;
use nym_mixnet_contract_common::NodeId;
use nym_types::monitoring::NodeResult;
use nym_types::monitoring::{NodeResult, RouteResult};
use sqlx::sqlite::{SqliteAutoVacuum, SqliteSynchronous};
use sqlx::ConnectOptions;
use std::path::Path;
@@ -31,6 +31,9 @@ pub(crate) mod manager;
pub(crate) mod models;
pub(crate) mod runtime_migrations;
#[cfg(test)]
mod tests;
#[derive(Default)]
pub(crate) struct DbIdCache {
pub mixnodes_v1: DashMap<NodeId, i64>,
@@ -835,6 +838,16 @@ impl NymApiStorage {
Ok(())
}
pub(crate) async fn submit_route_monitoring_results(
&self,
route_results: &[RouteResult],
) -> Result<(), NymApiStorageError> {
self.manager
.submit_route_monitoring_results(route_results)
.await?;
Ok(())
}
/// Obtains number of network monitor test runs that have occurred within the specified interval.
///
/// # Arguments
@@ -930,8 +943,9 @@ impl NymApiStorage {
/// * `until`: timestamp specifying the purge cutoff.
pub(crate) async fn purge_old_statuses(&self, until: i64) -> Result<(), NymApiStorageError> {
self.manager.purge_old_mixnode_statuses(until).await?;
self.manager.purge_old_gateway_statuses(until).await?;
self.manager
.purge_old_gateway_statuses(until)
.purge_old_routes(until)
.await
.map_err(|err| err.into())
}
@@ -1035,5 +1049,23 @@ pub(crate) mod v3_migration {
pub(crate) async fn make_node_id_not_null(&self) -> Result<(), NymApiStorageError> {
Ok(self.manager.make_node_id_not_null().await?)
}
/// Get mixnode identity key by node ID
#[allow(dead_code)]
pub(crate) async fn get_mixnode_identity_key(
&self,
mix_id: NodeId,
) -> Result<Option<String>, NymApiStorageError> {
Ok(self.manager.get_mixnode_identity_key(mix_id).await?)
}
/// Get gateway identity key by node ID
#[allow(dead_code)]
pub(crate) async fn get_gateway_identity_key(
&self,
node_id: NodeId,
) -> Result<Option<String>, NymApiStorageError> {
Ok(self.manager.get_gateway_identity_key(node_id).await?)
}
}
}
+79
View File
@@ -146,3 +146,82 @@ pub struct HistoricalUptime {
pub date: Date,
pub uptime: i64,
}
// Simulated Rewarding System Models
// These models support comparison between old (24h cache-based) and new (1h route-based) rewarding
/// Represents a simulated reward epoch run
#[derive(FromRow, Debug, Clone)]
pub struct SimulatedRewardEpoch {
pub id: i64,
pub epoch_id: u32,
pub calculation_method: String, // 'old', 'new', or 'comparison'
pub start_timestamp: i64,
pub end_timestamp: i64,
pub description: Option<String>,
pub created_at: i64,
}
/// Node performance calculated using different methodologies
#[derive(FromRow, Debug, Clone)]
pub struct SimulatedNodePerformance {
#[allow(dead_code)]
pub id: i64,
pub simulated_epoch_id: i64,
pub node_id: NodeId,
pub node_type: String, // 'mixnode' or 'gateway'
pub identity_key: Option<String>,
pub reliability_score: f64, // 0.0 to 100.0
pub positive_samples: u32,
pub negative_samples: u32,
pub work_factor: Option<f64>, // 0.0 to 1.0
pub calculation_method: String, // 'old' or 'new'
pub calculated_at: i64,
}
/// Performance comparison data for analyzing methodology differences
#[derive(FromRow, Debug, Clone)]
pub struct SimulatedPerformanceComparison {
#[allow(dead_code)]
pub id: i64,
pub simulated_epoch_id: i64,
pub node_id: NodeId,
pub node_type: String, // 'mixnode' or 'gateway'
pub performance_score: f64, // 0.0 to 100.0
pub work_factor: f64, // Work factor applied (e.g., 10.0 for active, 1.0 for standby)
pub calculation_method: String, // 'old' or 'new'
pub positive_samples: Option<i64>,
pub negative_samples: Option<i64>,
pub route_success_rate: Option<f64>, // 0.0 to 100.0, mainly for new method
pub calculated_at: i64,
}
/// Performance ranking data for nodes
#[derive(FromRow, Debug, Clone)]
pub struct SimulatedPerformanceRanking {
#[allow(dead_code)]
pub id: i64,
pub simulated_epoch_id: i64,
pub node_id: NodeId,
pub calculation_method: String,
pub performance_rank: i64,
pub performance_percentile: f64,
#[allow(dead_code)]
pub calculated_at: i64,
}
/// Route analysis metadata for simulation runs
#[derive(FromRow, Debug, Clone)]
pub struct SimulatedRouteAnalysis {
#[allow(dead_code)]
pub id: i64,
pub simulated_epoch_id: i64,
pub calculation_method: String, // 'old' or 'new'
pub total_routes_analyzed: u32,
pub successful_routes: u32,
pub failed_routes: u32,
pub average_route_reliability: Option<f64>, // 0.0 to 100.0
pub time_window_hours: u32, // 1 for new method, 24 for old method
pub analysis_parameters: Option<String>, // JSON with analysis config
pub calculated_at: i64,
}
+465
View File
@@ -0,0 +1,465 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
#[cfg(test)]
mod simulation_storage_tests {
use super::super::models::{
SimulatedNodePerformance, SimulatedPerformanceComparison, SimulatedRouteAnalysis,
};
use super::super::NymApiStorage;
use tempfile::NamedTempFile;
async fn create_test_storage() -> NymApiStorage {
let temp_file = NamedTempFile::new().unwrap();
let temp_path = temp_file.path().to_path_buf();
// Keep the temp file alive
Box::leak(Box::new(temp_file));
NymApiStorage::init(temp_path).await.unwrap()
}
#[tokio::test]
async fn test_simulated_reward_epoch_crud() {
let storage = create_test_storage().await;
// Test create
let (epoch_id, is_new) = storage
.manager
.create_or_get_simulated_reward_epoch(
100,
"test",
1234567890,
1234571490,
Some("Test description"),
)
.await
.unwrap();
assert!(is_new);
assert!(epoch_id > 0);
// Test retrieve
let retrieved = storage
.manager
.get_simulated_reward_epoch(epoch_id)
.await
.unwrap();
assert!(retrieved.is_some());
let epoch = retrieved.unwrap();
assert_eq!(epoch.epoch_id, 100);
assert_eq!(epoch.calculation_method, "test");
assert_eq!(epoch.start_timestamp, 1234567890);
assert_eq!(epoch.end_timestamp, 1234571490);
assert_eq!(epoch.description, Some("Test description".to_string()));
// Test duplicate prevention
let (duplicate_epoch_id, is_duplicate_new) = storage
.manager
.create_or_get_simulated_reward_epoch(
100,
"test",
1234567890,
1234571490,
Some("Duplicate attempt"),
)
.await
.unwrap();
assert!(!is_duplicate_new);
assert_eq!(duplicate_epoch_id, epoch_id); // Should return the same ID
// Different method should create new entry
let (different_method_id, is_different_new) = storage
.manager
.create_or_get_simulated_reward_epoch(
100,
"new",
1234567890,
1234571490,
Some("Different method"),
)
.await
.unwrap();
assert!(is_different_new);
assert_ne!(different_method_id, epoch_id); // Should be a different ID
}
#[tokio::test]
async fn test_simulated_node_performance_crud() {
let storage = create_test_storage().await;
// Create parent epoch first
let (epoch_id, _) = storage
.manager
.create_or_get_simulated_reward_epoch(100, "test", 1234567890, 1234571490, None)
.await
.unwrap();
// Test insert node performance
let performance = SimulatedNodePerformance {
id: 0, // Will be set by database
simulated_epoch_id: epoch_id,
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: Some("test_key".to_string()),
reliability_score: 85.5,
positive_samples: 100,
negative_samples: 15,
work_factor: Some(0.95),
calculation_method: "new".to_string(),
calculated_at: 1234567890,
};
storage
.manager
.insert_simulated_node_performance(&[performance])
.await
.unwrap();
// Test retrieve
let retrieved = storage
.manager
.get_simulated_node_performance_for_epoch(epoch_id)
.await
.unwrap();
assert_eq!(retrieved.len(), 1);
let perf = &retrieved[0];
assert_eq!(perf.node_id, 42);
assert_eq!(perf.node_type, "mixnode");
assert_eq!(perf.reliability_score, 85.5);
assert_eq!(perf.calculation_method, "new");
}
#[tokio::test]
async fn test_simulated_performance_comparisons_crud() {
let storage = create_test_storage().await;
// Create parent epoch first
let (epoch_id, _) = storage
.manager
.create_or_get_simulated_reward_epoch(100, "test", 1234567890, 1234571490, None)
.await
.unwrap();
// Test insert performance comparison
let comparison = SimulatedPerformanceComparison {
id: 0,
simulated_epoch_id: epoch_id,
node_id: 42,
node_type: "mixnode".to_string(),
performance_score: 85.0,
work_factor: 10.0,
calculation_method: "new".to_string(),
positive_samples: Some(100),
negative_samples: Some(15),
route_success_rate: Some(85.0),
calculated_at: 1234567890,
};
storage
.manager
.insert_simulated_performance_comparisons(&[comparison])
.await
.unwrap();
// Test retrieve
let retrieved = storage
.manager
.get_simulated_performance_comparisons_for_epoch(epoch_id)
.await
.unwrap();
assert_eq!(retrieved.len(), 1);
let comp = &retrieved[0];
assert_eq!(comp.node_id, 42);
assert_eq!(comp.performance_score, 85.0);
assert_eq!(comp.work_factor, 10.0);
assert_eq!(comp.calculation_method, "new");
}
#[tokio::test]
async fn test_simulated_route_analysis_crud() {
let storage = create_test_storage().await;
// Create parent epoch first
let (epoch_id, _) = storage
.manager
.create_or_get_simulated_reward_epoch(100, "test", 1234567890, 1234571490, None)
.await
.unwrap();
// Test insert route analysis
let analysis = SimulatedRouteAnalysis {
id: 0,
simulated_epoch_id: epoch_id,
calculation_method: "new".to_string(),
total_routes_analyzed: 1000,
successful_routes: 950,
failed_routes: 50,
average_route_reliability: Some(95.0),
time_window_hours: 1,
analysis_parameters: Some("{\"threshold\": 0.8}".to_string()),
calculated_at: 1234567890,
};
storage
.manager
.insert_simulated_route_analysis(&analysis)
.await
.unwrap();
// Test retrieve
let retrieved = storage
.manager
.get_simulated_route_analysis_for_epoch(epoch_id)
.await
.unwrap();
assert_eq!(retrieved.len(), 1);
let route = &retrieved[0];
assert_eq!(route.calculation_method, "new");
assert_eq!(route.total_routes_analyzed, 1000);
assert_eq!(route.successful_routes, 950);
assert_eq!(route.average_route_reliability, Some(95.0));
assert_eq!(route.time_window_hours, 1);
}
#[tokio::test]
async fn test_foreign_key_constraints() {
let storage = create_test_storage().await;
// Try to insert node performance without valid epoch - should fail
let performance = SimulatedNodePerformance {
id: 0,
simulated_epoch_id: 999999, // Non-existent epoch
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: None,
reliability_score: 85.5,
positive_samples: 100,
negative_samples: 15,
work_factor: Some(0.95),
calculation_method: "new".to_string(),
calculated_at: 1234567890,
};
let result = storage
.manager
.insert_simulated_node_performance(&[performance])
.await;
assert!(result.is_err()); // Should fail due to foreign key constraint
}
#[tokio::test]
async fn test_performance_by_method_queries() {
let storage = create_test_storage().await;
// Create epoch
let (epoch_id, _) = storage
.manager
.create_or_get_simulated_reward_epoch(100, "comparison", 1234567890, 1234571490, None)
.await
.unwrap();
// Insert performance data for both methods
let old_performance = SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch_id,
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: Some("key42".to_string()),
reliability_score: 80.0,
positive_samples: 100,
negative_samples: 20,
work_factor: Some(0.9),
calculation_method: "old".to_string(),
calculated_at: 1234567890,
};
let new_performance = SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch_id,
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: Some("key42".to_string()),
reliability_score: 90.0,
positive_samples: 95,
negative_samples: 5,
work_factor: Some(0.95),
calculation_method: "new".to_string(),
calculated_at: 1234567890,
};
storage
.manager
.insert_simulated_node_performance(&[old_performance])
.await
.unwrap();
storage
.manager
.insert_simulated_node_performance(&[new_performance])
.await
.unwrap();
// Test querying by method
let old_results = storage
.manager
.get_simulated_node_performance_by_method(100, "old")
.await
.unwrap();
let new_results = storage
.manager
.get_simulated_node_performance_by_method(100, "new")
.await
.unwrap();
assert_eq!(old_results.len(), 1);
assert_eq!(new_results.len(), 1);
assert_eq!(old_results[0].reliability_score, 80.0);
assert_eq!(new_results[0].reliability_score, 90.0);
assert_eq!(old_results[0].calculation_method, "old");
assert_eq!(new_results[0].calculation_method, "new");
}
#[tokio::test]
async fn test_node_performance_history() {
let storage = create_test_storage().await;
// Create multiple epochs
let (epoch1_id, _) = storage
.manager
.create_or_get_simulated_reward_epoch(100, "comparison", 1234567890, 1234571490, None)
.await
.unwrap();
let (epoch2_id, _) = storage
.manager
.create_or_get_simulated_reward_epoch(101, "comparison", 1234571490, 1234575090, None)
.await
.unwrap();
// Insert performance data for same node across epochs
let performances = vec![
SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch1_id,
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: Some("key42".to_string()),
reliability_score: 80.0,
positive_samples: 100,
negative_samples: 20,
work_factor: Some(0.9),
calculation_method: "old".to_string(),
calculated_at: 1234567890,
},
SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch2_id,
node_id: 42,
node_type: "mixnode".to_string(),
identity_key: Some("key42".to_string()),
reliability_score: 85.0,
positive_samples: 105,
negative_samples: 15,
work_factor: Some(0.92),
calculation_method: "old".to_string(),
calculated_at: 1234571490,
},
];
for perf in performances {
storage
.manager
.insert_simulated_node_performance(&[perf])
.await
.unwrap();
}
// Test node history query
let history = storage
.manager
.get_simulated_node_performance_history(42)
.await
.unwrap();
assert_eq!(history.len(), 2);
// Should be ordered by epoch_id DESC
assert_eq!(history[0].reliability_score, 85.0); // epoch 101
assert_eq!(history[1].reliability_score, 80.0); // epoch 100
}
#[tokio::test]
async fn test_count_operations() {
let storage = create_test_storage().await;
// Create epoch
let (epoch_id, _) = storage
.manager
.create_or_get_simulated_reward_epoch(100, "test", 1234567890, 1234571490, None)
.await
.unwrap();
// Initially should be 0
let count = storage
.manager
.count_simulated_node_performance_for_epoch(epoch_id)
.await
.unwrap();
assert_eq!(count, 0);
// Insert some performance data
let performances = vec![
SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch_id,
node_id: 1,
node_type: "mixnode".to_string(),
identity_key: None,
reliability_score: 80.0,
positive_samples: 100,
negative_samples: 20,
work_factor: Some(0.9),
calculation_method: "old".to_string(),
calculated_at: 1234567890,
},
SimulatedNodePerformance {
id: 0,
simulated_epoch_id: epoch_id,
node_id: 2,
node_type: "gateway".to_string(),
identity_key: None,
reliability_score: 90.0,
positive_samples: 120,
negative_samples: 10,
work_factor: None,
calculation_method: "new".to_string(),
calculated_at: 1234567890,
},
];
for perf in performances {
storage
.manager
.insert_simulated_node_performance(&[perf])
.await
.unwrap();
}
// Should now be 2
let count = storage
.manager
.count_simulated_node_performance_for_epoch(epoch_id)
.await
.unwrap();
assert_eq!(count, 2);
}
}
+3
View File
@@ -24,3 +24,6 @@ mod unstable_nym_nodes;
#[path = "public-api/unstable_status.rs"]
mod unstable_status;
#[path = "public-api/simulation.rs"]
mod simulation;
+393
View File
@@ -0,0 +1,393 @@
use crate::utils::{base_url, make_request, validate_json_response};
use serde_json::Value;
#[tokio::test]
async fn test_simulation_epochs_endpoint() {
let base = match base_url() {
Ok(url) => url,
Err(_) => {
println!("NYM_API not set, skipping simulation API tests");
return;
}
};
let url = format!("{}/v1/simulation/epochs", base);
let response = make_request(&url).await;
match response {
Ok(res) => {
let json = validate_json_response(res).await;
match json {
Ok(data) => {
// Verify response structure
assert!(data.is_object());
assert!(data.get("epochs").is_some());
assert!(data.get("total_count").is_some());
let epochs = data.get("epochs").unwrap();
assert!(epochs.is_array());
// If there are epochs, verify their structure
if let Some(epoch_array) = epochs.as_array() {
if !epoch_array.is_empty() {
let first_epoch = &epoch_array[0];
verify_epoch_summary_structure(first_epoch);
}
}
println!("✓ Simulation epochs endpoint returned valid response");
}
Err(e) => {
println!("✗ Failed to parse JSON response: {}", e);
}
}
}
Err(_) => {
// API might not be running in simulation mode, which is fine for tests
println!("⚠ Simulation API not available (expected if not in simulation mode)");
}
}
}
#[tokio::test]
async fn test_simulation_epochs_with_pagination() {
let base = match base_url() {
Ok(url) => url,
Err(_) => return,
};
let url = format!("{}/v1/simulation/epochs?limit=5&offset=0", base);
let response = make_request(&url).await;
match response {
Ok(res) => {
let json = validate_json_response(res).await;
match json {
Ok(data) => {
assert!(data.is_object());
assert!(data.get("epochs").is_some());
assert!(data.get("total_count").is_some());
let epochs = data.get("epochs").unwrap().as_array().unwrap();
assert!(epochs.len() <= 5); // Should respect limit
println!("✓ Simulation epochs pagination works correctly");
}
Err(_) => {
println!("⚠ Simulation API not available");
}
}
}
Err(_) => {
println!("⚠ Simulation API not available");
}
}
}
#[tokio::test]
async fn test_simulation_epoch_details_structure() {
let base = match base_url() {
Ok(url) => url,
Err(_) => return,
};
// First, get a list of epochs to find a valid ID
let epochs_url = format!("{}/v1/simulation/epochs?limit=1", base);
let epochs_response = make_request(&epochs_url).await;
match epochs_response {
Ok(res) => {
let json = validate_json_response(res).await;
if let Ok(data) = json {
let epochs = data.get("epochs").unwrap().as_array().unwrap();
if !epochs.is_empty() {
let first_epoch = &epochs[0];
let epoch_id = first_epoch.get("id").unwrap().as_i64().unwrap();
// Now test the details endpoint
let details_url = format!("{}/v1/simulation/epochs/{}", base, epoch_id);
let details_response = make_request(&details_url).await;
match details_response {
Ok(res) => {
let json = validate_json_response(res).await;
match json {
Ok(data) => {
verify_epoch_details_structure(&data);
println!("✓ Simulation epoch details endpoint returned valid structure");
}
Err(e) => {
println!("✗ Failed to parse epoch details: {}", e);
}
}
}
Err(e) => {
println!("✗ Failed to fetch epoch details: {}", e);
}
}
}
}
}
Err(_) => {
println!("⚠ Simulation API not available");
}
}
}
#[tokio::test]
async fn test_simulation_comparison_endpoint() {
let base = match base_url() {
Ok(url) => url,
Err(_) => return,
};
// First, get a list of epochs to find a valid ID
let epochs_url = format!("{}/v1/simulation/epochs?limit=1", base);
let epochs_response = make_request(&epochs_url).await;
match epochs_response {
Ok(res) => {
let json = validate_json_response(res).await;
if let Ok(data) = json {
let epochs = data.get("epochs").unwrap().as_array().unwrap();
if !epochs.is_empty() {
let first_epoch = &epochs[0];
let epoch_id = first_epoch.get("id").unwrap().as_i64().unwrap();
// Test the comparison endpoint
let comparison_url =
format!("{}/v1/simulation/epochs/{}/comparison", base, epoch_id);
let comparison_response = make_request(&comparison_url).await;
match comparison_response {
Ok(res) => {
let json = validate_json_response(res).await;
match json {
Ok(data) => {
verify_comparison_structure(&data);
println!(
"✓ Simulation comparison endpoint returned valid structure"
);
}
Err(e) => {
println!("✗ Failed to parse comparison data: {}", e);
}
}
}
Err(e) => {
println!("✗ Failed to fetch comparison data: {}", e);
}
}
}
}
}
Err(_) => {
println!("⚠ Simulation API not available");
}
}
}
#[tokio::test]
async fn test_simulation_export_endpoints() {
let base = match base_url() {
Ok(url) => url,
Err(_) => return,
};
// First, get a list of epochs to find a valid ID
let epochs_url = format!("{}/v1/simulation/epochs?limit=1", base);
let epochs_response = make_request(&epochs_url).await;
match epochs_response {
Ok(res) => {
let json = validate_json_response(res).await;
if let Ok(data) = json {
let epochs = data.get("epochs").unwrap().as_array().unwrap();
if !epochs.is_empty() {
let first_epoch = &epochs[0];
let epoch_id = first_epoch.get("id").unwrap().as_i64().unwrap();
// Test JSON export
let json_export_url = format!(
"{}/v1/simulation/epochs/{}/export?format=json",
base, epoch_id
);
let json_response = make_request(&json_export_url).await;
match json_response {
Ok(res) => {
assert!(res.status().is_success());
let content_type = res.headers().get("content-type");
if let Some(ct) = content_type {
assert!(ct.to_str().unwrap().contains("application/json"));
}
println!("✓ JSON export endpoint works correctly");
}
Err(e) => {
println!("✗ JSON export failed: {}", e);
}
}
// Test CSV export
let csv_export_url = format!(
"{}/v1/simulation/epochs/{}/export?format=csv",
base, epoch_id
);
let csv_response = make_request(&csv_export_url).await;
match csv_response {
Ok(res) => {
assert!(res.status().is_success());
let content_type = res.headers().get("content-type");
if let Some(ct) = content_type {
assert!(ct.to_str().unwrap().contains("text/csv"));
}
println!("✓ CSV export endpoint works correctly");
}
Err(e) => {
println!("✗ CSV export failed: {}", e);
}
}
}
}
}
Err(_) => {
println!("⚠ Simulation API not available");
}
}
}
#[tokio::test]
async fn test_simulation_error_handling() {
let base = match base_url() {
Ok(url) => url,
Err(_) => return,
};
// Test 404 for non-existent epoch
let invalid_url = format!("{}/v1/simulation/epochs/999999", base);
let response = make_request(&invalid_url).await;
match response {
Ok(res) => {
// Should get a successful response (empty or error structure)
assert!(res.status().is_success() || res.status().is_client_error());
println!("✓ Error handling works for invalid epoch IDs");
}
Err(_) => {
// This is also acceptable if the endpoint returns an error
println!("✓ Error handling works for invalid epoch IDs");
}
}
}
// Helper functions to verify response structures
fn verify_epoch_summary_structure(epoch: &Value) {
assert!(epoch.is_object());
assert!(epoch.get("id").is_some());
assert!(epoch.get("epoch_id").is_some());
assert!(epoch.get("calculation_method").is_some());
assert!(epoch.get("start_timestamp").is_some());
assert!(epoch.get("end_timestamp").is_some());
assert!(epoch.get("created_at").is_some());
assert!(epoch.get("nodes_analyzed").is_some());
assert!(epoch.get("available_methods").is_some());
// Verify types
assert!(epoch.get("id").unwrap().is_i64());
assert!(epoch.get("epoch_id").unwrap().is_u64());
assert!(epoch.get("calculation_method").unwrap().is_string());
assert!(epoch.get("nodes_analyzed").unwrap().is_u64());
assert!(epoch.get("available_methods").unwrap().is_array());
}
fn verify_epoch_details_structure(data: &Value) {
assert!(data.is_object());
assert!(data.get("epoch").is_some());
assert!(data.get("node_performance").is_some());
assert!(data.get("rewards").is_some());
assert!(data.get("route_analysis").is_some());
// Verify epoch summary structure
verify_epoch_summary_structure(data.get("epoch").unwrap());
// Verify arrays
assert!(data.get("node_performance").unwrap().is_array());
assert!(data.get("rewards").unwrap().is_array());
assert!(data.get("route_analysis").unwrap().is_array());
// If there's performance data, verify its structure
let performance_array = data.get("node_performance").unwrap().as_array().unwrap();
if !performance_array.is_empty() {
let first_performance = &performance_array[0];
verify_performance_data_structure(first_performance);
}
// If there's reward data, verify its structure
let rewards_array = data.get("rewards").unwrap().as_array().unwrap();
if !rewards_array.is_empty() {
let first_reward = &rewards_array[0];
verify_reward_data_structure(first_reward);
}
}
fn verify_performance_data_structure(performance: &Value) {
assert!(performance.is_object());
assert!(performance.get("node_id").is_some());
assert!(performance.get("node_type").is_some());
assert!(performance.get("reliability_score").is_some());
assert!(performance.get("positive_samples").is_some());
assert!(performance.get("negative_samples").is_some());
assert!(performance.get("calculation_method").is_some());
assert!(performance.get("calculated_at").is_some());
// Verify types
assert!(performance.get("node_id").unwrap().is_u64());
assert!(performance.get("node_type").unwrap().is_string());
assert!(performance.get("reliability_score").unwrap().is_f64());
assert!(performance.get("positive_samples").unwrap().is_u64());
assert!(performance.get("negative_samples").unwrap().is_u64());
assert!(performance.get("calculation_method").unwrap().is_string());
assert!(performance.get("calculated_at").unwrap().is_i64());
}
fn verify_reward_data_structure(reward: &Value) {
assert!(reward.is_object());
assert!(reward.get("node_id").is_some());
assert!(reward.get("node_type").is_some());
assert!(reward.get("calculated_reward_amount").is_some());
assert!(reward.get("reward_currency").is_some());
assert!(reward.get("calculation_method").is_some());
// Verify types
assert!(reward.get("node_id").unwrap().is_u64());
assert!(reward.get("node_type").unwrap().is_string());
assert!(reward.get("calculated_reward_amount").unwrap().is_f64());
assert!(reward.get("reward_currency").unwrap().is_string());
assert!(reward.get("calculation_method").unwrap().is_string());
}
fn verify_comparison_structure(data: &Value) {
assert!(data.is_object());
assert!(data.get("epoch_id").is_some());
assert!(data.get("simulation_epoch_id").is_some());
assert!(data.get("node_comparisons").is_some());
assert!(data.get("summary_statistics").is_some());
assert!(data.get("route_analysis_comparison").is_some());
// Verify types
assert!(data.get("epoch_id").unwrap().is_u64());
assert!(data.get("simulation_epoch_id").unwrap().is_i64());
assert!(data.get("node_comparisons").unwrap().is_array());
assert!(data.get("summary_statistics").unwrap().is_object());
assert!(data.get("route_analysis_comparison").unwrap().is_object());
// Verify summary statistics structure
let stats = data.get("summary_statistics").unwrap();
assert!(stats.get("total_nodes_compared").is_some());
assert!(stats.get("nodes_improved").is_some());
assert!(stats.get("nodes_degraded").is_some());
assert!(stats.get("average_reliability_old").is_some());
assert!(stats.get("average_reliability_new").is_some());
}
+6 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-network-monitor"
version = "1.0.2"
version = "1.1.5"
authors.workspace = true
repository.workspace = true
homepage.workspace = true
@@ -12,7 +12,7 @@ license.workspace = true
[dependencies]
anyhow = { workspace = true }
axum = { workspace = true, features = ["json"] }
axum = { workspace = true, features = ["json", "macros"] }
clap = { workspace = true, features = ["derive", "env"] }
dashmap = { workspace = true }
futures = { workspace = true }
@@ -23,11 +23,15 @@ rand_chacha = { workspace = true }
reqwest = { workspace = true, features = ["json"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
socket2 = "0.5"
tokio = { workspace = true, features = ["macros", "time"] }
tokio-util = { workspace = true }
tower = { workspace = true }
tower-http = { workspace = true, features = ["timeout", "trace"] }
utoipa = { workspace = true, features = ["axum_extras"] }
utoipa-swagger-ui = { workspace = true, features = ["axum"] }
tokio-postgres = { workspace = true }
tracing = { workspace = true }
# internal
nym-bin-common = { path = "../common/bin-common", features = ["basic_tracing"] }
+87
View File
@@ -4,6 +4,14 @@ Monitors the Nym network by sending itself packages across the mixnet.
Network monitor is running two tokio tasks, one manages mixnet clients and another manages monitoring itself. Monitor is designed to be driven externally, via an HTTP api. This means that it does not do any monitoring unless driven by something like [`locust`](https://locust.io/). This allows us to tailor the load externally, potentially distributing it across multiple monitors.
## Features
- **Continuous Monitoring**: Periodically sends test packets through the network
- **Node Performance**: Tracks individual node reliability metrics
- **Route Performance**: Records route-level success rates through specific node combinations
- **Multi-API Submission**: Capable of submitting metrics to multiple API endpoints (fanout)
- **Force Routing**: Can force packets through all mixnet nodes for comprehensive testing
### Client manager
On start network monitor will spawn `C` clients, with 10 being the default. Random client is dropped every `T`, defaults to 60 seconds, and a new one is created. Clients chose a random gateway to connect to the mixnet. Meaning that on average all gateways will be tested in `NUMBER_OF_GATEWAYS/N*T`, assuming at least one request per client per T.
@@ -40,8 +48,87 @@ Options:
-m, --mixnet-timeout <MIXNET_TIMEOUT> [default: 10]
--generate-key-pair
--private-key <PRIVATE_KEY>
--database-url <DATABASE_URL> SQLite database URL
--nym-apis <NYM_APIS> Comma-separated list of Nym API URLs
-h, --help Print help
-V, --version Print version
```
## Metrics Collection & Reporting
### Node Metrics
The Network Monitor tracks performance metrics for individual nodes:
- **Reliability**: Percentage of successful packet handling
- **Failure Sequences**: Tracking consecutive failures
- **Volume**: Number of packets handled
### Route Metrics
Since version 1.1.0, the Network Monitor also tracks route-level metrics:
- **Route Success Rates**: Tracking which specific combinations of nodes have successful packet delivery
- **Layer Analysis**: Identifying weak points in specific network layers
- **Path Correction**: Improved algorithms for attributing failures to specific nodes
### Metrics Fanout
The Network Monitor can submit metrics to multiple API endpoints simultaneously:
1. Metrics are collected during each monitoring cycle
2. The collected metrics are submitted to each configured API endpoint
3. This provides redundancy and allows for distributed metrics collection
To enable metrics fanout, use the `--nym-apis` parameter with a comma-separated list of API URLs:
```bash
cargo run -p nym-network-monitor -- --nym-apis https://api1.example.com,https://api2.example.com
```
## Route Data Structure
Route metrics use the following data structure:
```rust
// Route performance data
pub struct RouteResult {
pub layer1: u32, // NodeId of layer 1 mixnode
pub layer2: u32, // NodeId of layer 2 mixnode
pub layer3: u32, // NodeId of layer 3 mixnode
pub gw: u32, // NodeId of gateway
pub success: bool, // Whether the packet was successfully delivered
}
```
## Forced Routing
To ensure comprehensive testing of all nodes in the network, the Monitor supports forcing packets through all available nodes:
- Each node is assigned to a specific layer (1, 2, or 3) deterministically
- This ensures all nodes participate in route testing
- The routing algorithm cycles through all possible node combinations
Since version 1.1.0, Network Monitor automatically forces all available nodes to be active and distributes them evenly across the three layers (Layer 1, Layer 2, and Layer 3). This ensures every node in the network participates in testing, providing more comprehensive coverage and better metrics for all nodes, not just the popular ones.
## Node Performance Calculation
The Network Monitor uses a sophisticated algorithm for attributing failures to specific nodes:
1. For successful packet deliveries, all nodes in the path receive a positive sample
2. For failed deliveries:
- Nodes with more than 2 consecutive failures are considered "guilty"
- If no node is clearly guilty, all nodes in the path receive negative samples
3. Final node reliability is calculated as: positive_samples / (positive_samples + negative_samples)
## Changelog
### Version 1.1.0
- Added route-level metrics tracking and submission
- Implemented metrics fanout to multiple API endpoints
- Forced routing through all available nodes for comprehensive testing
- Improved reliability corrections with consecutive failure tracking
- Updated data structures for better metrics organization
### Version 1.0.2
- Initial public release with basic monitoring capabilities
+21 -1
View File
@@ -1,7 +1,27 @@
import time
import logging
from locust import HttpUser, task
from requests.exceptions import ConnectionError
# Configure logging to see what's happening
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
class SendMsg(HttpUser):
@task
def hello_world(self):
self.client.post("/v1/send")
try:
response = self.client.post("/v1/send")
if response.status_code == 503:
logger.warning(f"Got 503 Service Unavailable, sleeping for 1 second")
time.sleep(1)
response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx)
except ConnectionError as e:
# This catches ConnectionRefused errors
logger.error(f"Connection refused, backing off for 5 seconds: {e}")
time.sleep(5) # Longer pause for connection errors
except Exception as e:
# Log other errors but don't sleep as long
logger.warning(f"Request failed: {type(e).__name__}: {e}")
time.sleep(0.5) # Brief pause for other errors
+93 -41
View File
@@ -5,11 +5,13 @@ use std::{
use anyhow::Result;
use futures::{pin_mut, stream::FuturesUnordered, StreamExt};
use log::{debug, error, info};
use log::{debug, error, info, warn};
use nym_sphinx::chunking::{monitoring, SentFragment};
use nym_topology::{NymRouteProvider, RoutingNode};
use nym_types::monitoring::{MonitorMessage, NodeResult};
use nym_validator_client::nym_api::routes::{API_VERSION, STATUS, SUBMIT_GATEWAY, SUBMIT_NODE};
use nym_types::monitoring::{MonitorMessage, MonitorResults, NodeResult, RouteResult};
use nym_validator_client::nym_api::routes::{
API_VERSION, STATUS, SUBMIT_GATEWAY, SUBMIT_NODE, SUBMIT_ROUTE,
};
use rand::SeedableRng;
use rand_chacha::ChaCha8Rng;
use serde::{Deserialize, Serialize};
@@ -17,7 +19,7 @@ use tokio::task::JoinHandle;
use tokio_postgres::{binary_copy::BinaryCopyInWriter, types::Type, Client, NoTls};
use utoipa::ToSchema;
use crate::{NYM_API_URL, PRIVATE_KEY, TOPOLOGY};
use crate::{NYM_API_URLS, PRIVATE_KEY, TOPOLOGY};
struct HydratedRoute {
mix_nodes: Vec<RoutingNode>,
@@ -439,6 +441,7 @@ async fn db_connection(database_url: Option<&String>) -> Result<Option<(Client,
Ok(None)
}
}
pub async fn submit_metrics_to_db(database_url: Option<&String>) -> anyhow::Result<()> {
if let Some((client, handle)) = db_connection(database_url).await? {
let client = Arc::new(client);
@@ -491,49 +494,98 @@ pub async fn submit_metrics(database_url: Option<&String>) -> anyhow::Result<()>
}
if let Some(private_key) = PRIVATE_KEY.get() {
let node_stats = monitor_mixnode_results().await?;
let gateway_stats = monitor_gateway_results().await?;
if let Some(nym_api_urls) = NYM_API_URLS.get() {
info!("Submitting metrics to {} nym apis", nym_api_urls.len());
for nym_api_url in nym_api_urls {
info!("Submitting metrics to {}", nym_api_url);
let node_stats = monitor_mixnode_results().await?;
let gateway_stats = monitor_gateway_results().await?;
let client = reqwest::Client::new();
info!("Submitting metrics to {}", *NYM_API_URL);
let client = reqwest::Client::new();
let node_submit_url =
format!("{}/{API_VERSION}/{STATUS}/{SUBMIT_NODE}", nym_api_url);
let gateway_submit_url =
format!("{}/{API_VERSION}/{STATUS}/{SUBMIT_GATEWAY}", nym_api_url);
let route_submit_url =
format!("{}/{API_VERSION}/{STATUS}/{SUBMIT_ROUTE}", nym_api_url);
let node_submit_url = format!("{}/{API_VERSION}/{STATUS}/{SUBMIT_NODE}", &*NYM_API_URL);
let gateway_submit_url =
format!("{}/{API_VERSION}/{STATUS}/{SUBMIT_GATEWAY}", &*NYM_API_URL);
info!("Submitting {} mixnode measurements", node_stats.len());
info!("Submitting {} mixnode measurements", node_stats.len());
node_stats
.chunks(10)
.map(|chunk| {
let monitor_results = MonitorResults::Node(chunk.to_vec());
let monitor_message = MonitorMessage::new(monitor_results, private_key);
client.post(&node_submit_url).json(&monitor_message).send()
})
.collect::<FuturesUnordered<_>>()
.collect::<Vec<Result<_, _>>>()
.await
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
node_stats
.chunks(10)
.map(|chunk| {
let monitor_message = MonitorMessage::new(chunk.to_vec(), private_key);
client.post(&node_submit_url).json(&monitor_message).send()
})
.collect::<FuturesUnordered<_>>()
.collect::<Vec<Result<_, _>>>()
.await
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
info!("Submitting {} gateway measurements", gateway_stats.len());
info!("Submitting {} gateway measurements", gateway_stats.len());
gateway_stats
.chunks(10)
.map(|chunk| {
let monitor_results = MonitorResults::Node(chunk.to_vec());
let monitor_message = MonitorMessage::new(
monitor_results,
PRIVATE_KEY.get().expect("We've set this!"),
);
client
.post(&gateway_submit_url)
.json(&monitor_message)
.send()
})
.collect::<FuturesUnordered<_>>()
.collect::<Vec<Result<_, _>>>()
.await
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
gateway_stats
.chunks(10)
.map(|chunk| {
let monitor_message = MonitorMessage::new(
chunk.to_vec(),
PRIVATE_KEY.get().expect("We've set this!"),
);
client
.post(&gateway_submit_url)
.json(&monitor_message)
.send()
})
.collect::<FuturesUnordered<_>>()
.collect::<Vec<Result<_, _>>>()
.await
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
let network_account = NetworkAccount::finalize()?;
let accounting_routes = network_account.accounting_routes;
info!("Submitting {} accounting routes", accounting_routes.len());
match accounting_routes
.chunks(10)
.map(|chunk| {
let route_results = chunk
.iter()
.map(|route| {
RouteResult::new(
route.mix_nodes.0,
route.mix_nodes.1,
route.mix_nodes.2,
route.gateway_node,
route.success,
)
})
.collect::<Vec<RouteResult>>();
let monitor_results = MonitorResults::Route(route_results);
let monitor_message = MonitorMessage::new(monitor_results, private_key);
client.post(&route_submit_url).json(&monitor_message).send()
})
.collect::<FuturesUnordered<_>>()
.collect::<Vec<Result<_, _>>>()
.await
.into_iter()
.collect::<Result<Vec<_>, _>>()
{
Ok(_) => info!(
"Successfully submitted accounting routes to {}",
nym_api_url
),
Err(e) => error!(
"Error submitting accounting routes to {}: {}",
nym_api_url, e
),
};
}
}
} else {
warn!("No private key or nym api urls found");
}
NetworkAccount::empty_buffers();
+21 -4
View File
@@ -184,6 +184,12 @@ pub async fn mermaid_handler() -> Result<String, StatusCode> {
}
async fn send_receive_mixnet(state: AppState) -> Result<String, StatusCode> {
// let client = Arc::new(RwLock::new(
// make_client(TOPOLOGY.get().expect("Set at the begining").clone())
// .await
// .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?,
// ));
let msg: String = rand::thread_rng()
.sample_iter(&Alphanumeric)
.take(32)
@@ -191,16 +197,23 @@ async fn send_receive_mixnet(state: AppState) -> Result<String, StatusCode> {
.collect();
let sent_msg = msg.clone();
debug!("[REQUEST_START] Processing send request {}", msg);
let client = {
let mut clients = state.clients().write().await;
if let Some(client) = clients.make_contiguous().choose(&mut rand::thread_rng()) {
Arc::clone(client)
} else {
error!("No clients currently available");
return Err(StatusCode::INTERNAL_SERVER_ERROR);
error!("No clients currently available - this may cause connection refused errors");
return Err(StatusCode::SERVICE_UNAVAILABLE);
}
};
if !client.read().await.is_gateway_connection_alive() {
warn!("Client is not connected, waiting for it to connect, trying another one");
return Err(StatusCode::SERVICE_UNAVAILABLE);
}
let recv = Arc::clone(&client);
let sender = Arc::clone(&client);
@@ -238,12 +251,16 @@ async fn send_receive_mixnet(state: AppState) -> Result<String, StatusCode> {
match result {
Ok(_) => {}
Err(e) => {
error!("Failed to send/receive message: {e}");
return Err(StatusCode::INTERNAL_SERVER_ERROR);
error!(
"[REQUEST_ERROR] {} - Failed to send/receive message: {e}",
sent_msg
);
return Err(StatusCode::GATEWAY_TIMEOUT);
}
}
}
debug!("[REQUEST_SUCCESS] {} - Message sent successfully", sent_msg);
Ok(sent_msg)
}
+47 -7
View File
@@ -6,11 +6,14 @@ use crate::handlers::{
};
use axum::routing::{get, post};
use axum::Router;
use log::info;
use log::{debug, info, warn};
use nym_sphinx::chunking::fragment::FragmentHeader;
use nym_sphinx::chunking::{ReceivedFragment, SentFragment};
use std::net::SocketAddr;
use tokio_util::sync::CancellationToken;
use tower::ServiceBuilder;
use tower_http::timeout::TimeoutLayer;
use tower_http::trace::TraceLayer;
use utoipa::OpenApi;
use utoipa_swagger_ui::SwaggerUi;
@@ -19,6 +22,7 @@ use crate::ClientsWrapper;
pub struct HttpServer {
listener: SocketAddr,
cancel: CancellationToken,
tcp_backlog: i32,
}
#[derive(OpenApi)]
@@ -59,15 +63,19 @@ impl AppState {
}
impl HttpServer {
pub fn new(listener: SocketAddr, cancel: CancellationToken) -> Self {
HttpServer { listener, cancel }
pub fn new(listener: SocketAddr, cancel: CancellationToken, tcp_backlog: i32) -> Self {
HttpServer {
listener,
cancel,
tcp_backlog,
}
}
pub async fn run(self, clients: ClientsWrapper) -> anyhow::Result<()> {
let n_clients = clients.read().await.len();
let state = AppState { clients };
let app = Router::new()
.route("/v1/send", post(send_handler).with_state(state))
.route("/v1/send", post(send_handler))
.merge(SwaggerUi::new("/v1/ui").url("/v1/docs/openapi.json", ApiDoc::openapi()))
.route("/v1/accounting", get(accounting_handler))
.route("/v1/sent", get(sent_handler))
@@ -77,15 +85,47 @@ impl HttpServer {
.route("/v1/stats", get(stats_handler))
.route("/v1/node_stats/:mix_id", get(node_stats_handler))
.route("/v1/node_stats", get(all_nodes_stats_handler))
.route("/v1/received", get(recv_handler));
let listener = tokio::net::TcpListener::bind(self.listener).await?;
.route("/v1/received", get(recv_handler))
.layer(
ServiceBuilder::new()
// Add request tracing
.layer(
TraceLayer::new_for_http()
.on_request(|_request: &axum::http::Request<_>, _span: &tracing::Span| {
debug!("[HTTP_REQUEST] New connection accepted");
})
.on_failure(|error: tower_http::classify::ServerErrorsFailureClass, latency: std::time::Duration, _span: &tracing::Span| {
warn!("[HTTP_ERROR] Request failed with error: {:?}, latency: {:?}", error, latency);
})
)
// Add a timeout layer to prevent hanging connections
.layer(TimeoutLayer::new(std::time::Duration::from_secs(30)))
)
.with_state(state);
// Configure socket with higher backlog to handle more concurrent connections
let socket = socket2::Socket::new(
socket2::Domain::for_address(self.listener),
socket2::Type::STREAM,
Some(socket2::Protocol::TCP),
)?;
// Enable SO_REUSEADDR to avoid "Address already in use" errors
socket.set_reuse_address(true)?;
// Set a higher backlog (default is often 128)
socket.bind(&self.listener.into())?;
socket.listen(self.tcp_backlog)?; // Use configurable backlog
let listener = tokio::net::TcpListener::from_std(socket.into())?;
let server_future =
axum::serve(listener, app).with_graceful_shutdown(self.cancel.cancelled_owned());
info!("##########################################################################################");
info!("######################### HTTP server running, with {} clients ############################################", n_clients);
info!("######################### HTTP server running on {} with {} clients ############################################", self.listener, n_clients);
info!("######################### TCP backlog set to {} connections ############################################", self.tcp_backlog);
info!("##########################################################################################");
info!("[HTTP_SERVER] Server started and ready to accept connections");
server_future.await?;
+89 -21
View File
@@ -10,10 +10,9 @@ use nym_network_defaults::setup_env;
use nym_network_defaults::var_names::NYM_API;
use nym_sdk::mixnet::{self, MixnetClient};
use nym_sphinx::chunking::monitoring;
use nym_topology::{HardcodedTopologyProvider, NymTopology};
use nym_topology::{HardcodedTopologyProvider, NymTopology, Role};
use std::fs::File;
use std::io::Write;
use std::sync::LazyLock;
use std::time::Duration;
use std::{
collections::VecDeque,
@@ -25,9 +24,7 @@ use tokio::sync::OnceCell;
use tokio::{signal::ctrl_c, sync::RwLock};
use tokio_util::sync::CancellationToken;
static NYM_API_URL: LazyLock<String> = LazyLock::new(|| {
std::env::var(NYM_API).unwrap_or_else(|_| panic!("{} env var not set", NYM_API))
});
static NYM_API_URLS: OnceCell<Vec<String>> = OnceCell::const_new();
static MIXNET_TIMEOUT: OnceCell<u64> = OnceCell::const_new();
static TOPOLOGY: OnceCell<NymTopology> = OnceCell::const_new();
@@ -53,24 +50,60 @@ async fn make_clients(
if spawned_clients >= n_clients {
info!("New client will be spawned in {} seconds", lifetime);
tokio::time::sleep(tokio::time::Duration::from_secs(lifetime)).await;
info!("Removing oldest client");
info!("[CLIENT_ROTATION_START] Beginning client rotation");
if let Some(dropped_client) = clients.write().await.pop_front() {
loop {
if Arc::strong_count(&dropped_client) == 1 {
if let Some(client) = Arc::into_inner(dropped_client) {
let client_handle = client.into_inner();
client_handle.disconnect().await;
} else {
warn!("Failed to drop client, client had more then one strong ref")
}
break;
info!(
"[CLIENT_ROTATION] Popped client from queue, current ref count: {}",
Arc::strong_count(&dropped_client)
);
const CLIENT_DROP_TIMEOUT: Duration = Duration::from_secs(30);
let start = tokio::time::Instant::now();
// Try immediate unwrap first
match Arc::try_unwrap(dropped_client) {
Ok(client) => {
let client_handle = client.into_inner();
client_handle.disconnect().await;
info!("[CLIENT_ROTATION_END] Successfully disconnected client immediately");
}
Err(dropped_client) => {
// Fallback: wait with timeout for references to drop
info!("[CLIENT_ROTATION_BLOCKING] Client still has {} references, waiting for cleanup with timeout", Arc::strong_count(&dropped_client));
while start.elapsed() < CLIENT_DROP_TIMEOUT {
let elapsed = start.elapsed();
let ref_count = Arc::strong_count(&dropped_client);
if elapsed.as_secs() % 5 == 0 && elapsed.subsec_millis() < 100 {
info!("[CLIENT_ROTATION_WAITING] Still waiting for client cleanup, elapsed: {:?}, ref_count: {}", elapsed, ref_count);
}
if ref_count == 1 {
match Arc::try_unwrap(dropped_client) {
Ok(client) => {
let client_handle = client.into_inner();
client_handle.disconnect().await;
info!("[CLIENT_ROTATION_END] Successfully disconnected client after waiting {:?}", start.elapsed());
break;
}
Err(_) => {
warn!("Failed to unwrap client despite reference count being 1");
break;
}
}
}
tokio::time::sleep(Duration::from_millis(100)).await;
}
if start.elapsed() >= CLIENT_DROP_TIMEOUT {
warn!(
"[CLIENT_ROTATION_TIMEOUT] Client drop timed out after {:?}, forcing drop.",
CLIENT_DROP_TIMEOUT
);
// Client will be dropped when Arc goes out of scope
}
}
info!("Client still in use, waiting 2 seconds");
tokio::time::sleep(tokio::time::Duration::from_secs(2)).await;
}
}
}
info!("Spawning new client");
info!("[CLIENT_SPAWN_START] Spawning new client");
let client = match make_client(topology.clone()).await {
Ok(client) => client,
Err(err) => {
@@ -82,6 +115,7 @@ async fn make_clients(
.write()
.await
.push_back(Arc::new(RwLock::new(client)));
info!("[CLIENT_SPAWN_END] New client added to pool");
}
}
@@ -138,6 +172,12 @@ struct Args {
#[arg(long, env = "DATABASE_URL")]
database_url: Option<String>,
#[arg(long, env = "NYM_APIS", value_delimiter = ',')]
nym_apis: Option<Vec<String>>,
#[arg(long, env = "TCP_BACKLOG", default_value_t = 1024)]
tcp_backlog: i32,
}
fn generate_key_pair() -> Result<()> {
@@ -155,7 +195,7 @@ fn generate_key_pair() -> Result<()> {
Ok(())
}
async fn nym_topology_from_env() -> anyhow::Result<NymTopology> {
async fn nym_topology_forced_all_from_env() -> anyhow::Result<NymTopology> {
let api_url = std::env::var(NYM_API)?;
info!("Generating topology from {api_url}");
@@ -172,6 +212,27 @@ async fn nym_topology_from_env() -> anyhow::Result<NymTopology> {
let mut topology = NymTopology::new_empty(rewarded_set);
topology.add_skimmed_nodes(&nodes);
let node_ids = topology
.node_details()
.iter()
.filter(|(_node_id, node)| {
node.supported_roles.mixnode
&& !node.supported_roles.mixnet_entry
&& !node.supported_roles.mixnet_exit
})
.map(|(node_id, _)| *node_id)
.collect::<Vec<_>>();
// Force all nodes to active to participate in route selection
for (idx, node_id) in node_ids.iter().enumerate() {
match idx % 3 {
0 => topology.force_set_active(*node_id, Role::Layer1),
1 => topology.force_set_active(*node_id, Role::Layer2),
2 => topology.force_set_active(*node_id, Role::Layer3),
_ => unreachable!(), // Unreachable since idx % 3 can only be 0, 1, or 2
}
}
Ok(topology)
}
@@ -200,11 +261,16 @@ async fn main() -> Result<()> {
PRIVATE_KEY.set(pk).ok();
}
if let Some(nym_apis) = args.nym_apis {
info!("Using nym apis: {:?}", nym_apis);
NYM_API_URLS.set(nym_apis).ok();
}
TOPOLOGY
.set(if let Some(topology_file) = args.topology {
NymTopology::new_from_file(topology_file)?
} else {
nym_topology_from_env().await?
nym_topology_forced_all_from_env().await?
})
.ok();
@@ -220,9 +286,10 @@ async fn main() -> Result<()> {
let clients_server = clients.clone();
let tcp_backlog = args.tcp_backlog;
let server_handle = tokio::spawn(async move {
let socket = SocketAddr::new(IpAddr::V4(Ipv4Addr::from_str(&args.host)?), args.port);
let server = HttpServer::new(socket, server_cancel_token);
let server = HttpServer::new(socket, server_cancel_token, tcp_backlog);
server.run(clients_server).await
});
@@ -266,5 +333,6 @@ fn mixnet_debug_config(min_gateway_performance: u8) -> nym_client_core::config::
debug_config.cover_traffic.disable_loop_cover_traffic_stream = true;
debug_config.topology.minimum_gateway_performance = min_gateway_performance;
debug_config.traffic.deterministic_route_selection = true;
debug_config.traffic.average_packet_delay = Duration::from_millis(0);
debug_config
}
+2 -2
View File
@@ -25,7 +25,7 @@ use nym_client_core::client::{
};
use nym_client_core::config::{DebugConfig, ForgetMe, RememberMe, StatsReporting};
use nym_client_core::error::ClientCoreError;
use nym_client_core::init::helpers::gateways_for_init;
use nym_client_core::init::helpers::gateways_for_init_with_protocol_validation;
use nym_client_core::init::setup_gateway;
use nym_client_core::init::types::{GatewaySelectionSpecification, GatewaySetup};
use nym_credentials_interface::TicketType;
@@ -546,7 +546,7 @@ where
let topology_cfg = &self.config.debug_config.topology;
let mut rng = OsRng;
let available_gateways = gateways_for_init(
let available_gateways = gateways_for_init_with_protocol_validation(
&mut rng,
&nym_api_endpoints,
user_agent,
@@ -149,6 +149,17 @@ impl MixnetClient {
self.client_state.gateway_connection
}
/// Check if the websocket connection to the gateway is alive at the socket level
///
/// This performs a real socket-level health check to determine if the connection
/// is actually alive and responsive, not just whether we have a file descriptor.
pub fn is_gateway_connection_alive(&self) -> bool {
match self.client_state.gateway_connection.gateway_ws_fd {
Some(fd) => socket_is_alive(fd),
None => false,
}
}
/// Get a shallow clone of [`MixnetClientSender`]. Useful if you want split the send and
/// receive logic in different locations.
pub fn split_sender(&self) -> MixnetClientSender {
@@ -340,3 +351,49 @@ impl MixnetMessageSender for MixnetClientSender {
.map_err(|_| Error::MessageSendingFailure)
}
}
/// Check if a socket file descriptor is alive and responsive
///
/// This function performs socket-level checks to determine if the connection is actually alive.
/// It's cross-platform compatible and works on both Unix and non-Unix systems.
fn socket_is_alive(fd: std::os::raw::c_int) -> bool {
#[cfg(unix)]
{
use std::io::ErrorKind;
use std::net::TcpStream;
use std::os::unix::io::FromRawFd;
unsafe {
// Create a TcpStream from the raw fd to perform socket operations
let stream = TcpStream::from_raw_fd(fd);
// Try to peek at the socket to see if it's still connected
// We peek with a zero-length buffer to avoid consuming data
let mut buf = [0u8; 0];
let result = match stream.peek(&mut buf) {
Ok(_) => true, // Socket is alive and readable
Err(e) => match e.kind() {
ErrorKind::WouldBlock => true, // Socket is alive but no data available
ErrorKind::ConnectionReset
| ErrorKind::ConnectionAborted
| ErrorKind::BrokenPipe
| ErrorKind::NotConnected => false, // Socket is clearly dead
_ => true, // Other errors might be temporary, assume alive
},
};
// Prevent the TcpStream from closing the fd when it's dropped
// since we don't own the fd
std::mem::forget(stream);
result
}
}
#[cfg(not(unix))]
{
// On non-Unix systems, we can't easily check socket state
// Fall back to assuming the connection is alive if we have an fd
fd != 0
}
}