This commit addresses critical N+1 query performance issues identified in
the reward simulation system. The optimizations significantly reduce database
round trips and improve performance when processing large datasets.
**Key Optimizations:**
1. **Batch Identity Key Lookups**
- Added `get_mixnode_identity_keys_batch()` and `get_gateway_identity_keys_batch()`
- Updated simulation performance conversion to use batch operations
- Reduced from N individual queries to 2 batch queries
2. **Batch Node Classification**
- Added `classify_nodes_batch()` method for mixnode/gateway determination
- Updated reliability calculation methods to use batch classification
- Reduced from N individual lookups to 2 batch queries
3. **Batch Epoch Metadata Enhancement**
- Added `count_simulated_node_performance_for_epochs_batch()`
- Added `get_available_calculation_methods_for_epochs_batch()`
- Updated API handlers to use batch operations for metadata enhancement
- Reduced from 2N queries to 2 batch queries for epoch data
4. **Bulk Insert Optimizations**
- Converted individual INSERT operations to use `sqlx::QueryBuilder::push_values()`
- Optimized simulation data insertion methods
- Eliminated transaction overhead from individual inserts
**Performance Impact:**
- Before: N+2N database queries for N nodes/epochs
- After: 2+2 batch queries regardless of dataset size
- Significant performance improvement for large simulation datasets
All changes maintain backward compatibility while providing substantial
performance benefits for the reward simulation system.
Refactors the simulation system to focus on performance methodology comparison
rather than reward amounts, enabling robust analysis of old vs new calculation
methods. Key improvements:
- Replace simulated_rewards table with performance_comparisons for better metrics
- Add performance_rankings table for ranking analysis across methodologies
- Enhance database schema with additional performance tracking fields
- Update simulation coordinator to use performance-focused data structures
- Add comprehensive performance ranking calculations
- Improve API models and handlers for performance comparison workflows
- Update SQLx query cache with new database schema changes
This provides a foundation for data-driven performance methodology evaluation
while maintaining separation from actual reward calculations.
This completes Phase 3 of the simulation system implementation:
- Add comprehensive REST API endpoints for simulation data access
- Implement /v1/simulation/* routes with full CRUD operations
- Support JSON/CSV export for external analysis
- Add statistical comparison between old vs new methods
- Provide node performance history tracking
- Include proper error handling and response formatting
- Simplify simulation coordinator to remove unused complex return types
- Clean up dead code while maintaining all functionality
- Pass clippy with no warnings
The simulation API provides complete access to:
- Simulation epoch listing and details
- Method comparison analytics (old 24h vs new 1h)
- Node performance analysis across epochs
- Route reliability statistics
- Export capabilities for further analysis
All simulation data is persisted and accessible via REST endpoints.
durch