Implements proper socket-level health checking for gateway websocket connections:
- Add is_connection_alive() method to GatewayTransceiver trait
- Implement socket-level health check using TcpStream::peek() on Unix systems
- Add is_gateway_connection_alive() method to MixnetClient for easy access
- Update network monitor to use connection health checks before sending
- Fix axum router state configuration in network monitor
The health check performs actual OS-level socket validation rather than just
checking file descriptor existence, providing reliable connection status.
Implements automatic cleanup of route monitoring results to prevent
unbounded storage growth while maintaining data for performance analysis.
- Add purge_old_routes() method to StorageManager following existing patterns
- Integrate route cleanup into purge_old_statuses() wrapper function
- Route data now purged every epoch with 48-hour retention, to facilitate comparisons with legacy data
- Update logging to reflect cleanup of both node statuses and routes
This commit addresses critical N+1 query performance issues identified in
the reward simulation system. The optimizations significantly reduce database
round trips and improve performance when processing large datasets.
**Key Optimizations:**
1. **Batch Identity Key Lookups**
- Added `get_mixnode_identity_keys_batch()` and `get_gateway_identity_keys_batch()`
- Updated simulation performance conversion to use batch operations
- Reduced from N individual queries to 2 batch queries
2. **Batch Node Classification**
- Added `classify_nodes_batch()` method for mixnode/gateway determination
- Updated reliability calculation methods to use batch classification
- Reduced from N individual lookups to 2 batch queries
3. **Batch Epoch Metadata Enhancement**
- Added `count_simulated_node_performance_for_epochs_batch()`
- Added `get_available_calculation_methods_for_epochs_batch()`
- Updated API handlers to use batch operations for metadata enhancement
- Reduced from 2N queries to 2 batch queries for epoch data
4. **Bulk Insert Optimizations**
- Converted individual INSERT operations to use `sqlx::QueryBuilder::push_values()`
- Optimized simulation data insertion methods
- Eliminated transaction overhead from individual inserts
**Performance Impact:**
- Before: N+2N database queries for N nodes/epochs
- After: 2+2 batch queries regardless of dataset size
- Significant performance improvement for large simulation datasets
All changes maintain backward compatibility while providing substantial
performance benefits for the reward simulation system.
Refactors the simulation system to focus on performance methodology comparison
rather than reward amounts, enabling robust analysis of old vs new calculation
methods. Key improvements:
- Replace simulated_rewards table with performance_comparisons for better metrics
- Add performance_rankings table for ranking analysis across methodologies
- Enhance database schema with additional performance tracking fields
- Update simulation coordinator to use performance-focused data structures
- Add comprehensive performance ranking calculations
- Improve API models and handlers for performance comparison workflows
- Update SQLx query cache with new database schema changes
This provides a foundation for data-driven performance methodology evaluation
while maintaining separation from actual reward calculations.
This completes Phase 3 of the simulation system implementation:
- Add comprehensive REST API endpoints for simulation data access
- Implement /v1/simulation/* routes with full CRUD operations
- Support JSON/CSV export for external analysis
- Add statistical comparison between old vs new methods
- Provide node performance history tracking
- Include proper error handling and response formatting
- Simplify simulation coordinator to remove unused complex return types
- Clean up dead code while maintaining all functionality
- Pass clippy with no warnings
The simulation API provides complete access to:
- Simulation epoch listing and details
- Method comparison analytics (old 24h vs new 1h)
- Node performance analysis across epochs
- Route reliability statistics
- Export capabilities for further analysis
All simulation data is persisted and accessible via REST endpoints.
Add complete simulation engine that compares old (24h cache-based) vs new (1h route-based)
reward calculation methodologies with full integration into epoch operations.
Core Simulation Engine:
- Add SimulationCoordinator with configurable time windows and comparison settings
- Implement dual calculation methods with proper Performance type conversions
- Add comprehensive error handling with DatabaseError variant in RewardingError
- Store simulation results in database with proper relationship constraints
Old Method Implementation (24h Cache-Based):
- Wrap existing reliability calculation using get_all_avg_mix_reliability_in_last_24hr()
- Convert reliability percentages to Performance types using from_percentage_value()
- Maintain exact same logic as production for accurate baseline comparison
- Generate simulation data structures with proper metadata
New Method Implementation (1h Route-Based):
- Leverage calculate_corrected_node_reliabilities_for_interval() for route analysis
- Support configurable time windows (default 1 hour vs 24 hours)
- Provide detailed route statistics including success rates and failure analysis
- Convert route reliability data to Performance types with naive_try_from_f64()
Epoch Operations Integration:
- Extend EpochAdvancer struct with optional SimulationConfig field
- Update constructor and start method to accept simulation configuration
- Add simulation trigger in perform_epoch_operations() before real rewarding
- Ensure simulation failures don't break epoch advancement process
CLI Integration:
- Update run.rs to handle both --enable-rewarding and --simulate-rewarding modes
- Create SimulationConfig from rewarding.debug configuration settings
- Implement mutual exclusivity between real rewarding and simulation mode
- Skip permission checks for simulation-only mode (no blockchain transactions)
The simulation system runs in parallel with epoch operations, storing comparative
data for analysis without affecting production reward distribution.
Implement foundation for simulated reward calculations to compare old (24h cache-based)
vs new (1h route-based) methodologies without blockchain transactions.
Database Changes:
- Add migration with 4 new tables for simulation system
- simulated_reward_epochs: tracks each simulation run
- simulated_node_performance: stores performance calculations
- simulated_rewards: stores reward calculation results
- simulated_route_analysis: metadata for route analysis
- Add comprehensive indexes for efficient querying
Configuration Changes:
- Add simulation_mode flag to rewarding configuration
- Add CLI flag --simulate-rewarding with proper dependencies/conflicts
- Add validation for simulation-specific settings
- Add time window configuration for new method (default: 1 hour)
Storage Layer:
- Add model structs with SQLx FromRow derives for all simulation tables
- Add comprehensive CRUD methods for simulation data management
- Add proper type annotations to fix SQLx compile issues
- Maintain separation between simulation and real rewarding logic
The simulation mode is mutually exclusive with real rewarding and does not
require mnemonic since no blockchain transactions are performed.
durch