Drazen Urch
af9f6e5ca0
* feat(db): add SQL query wrapper for PostgreSQL placeholder conversion - Created query_wrapper module with functions to automatically convert SQLite ? placeholders to PostgreSQL $1, $2, ... format - Updated build.rs to handle mutually exclusive feature flags - Modified one query in mixnodes.rs as proof of concept - Added type conversions for PostgreSQL compatibility (u32->i64, u16->i32) This is a checkpoint commit before converting all queries to use the wrapper. * feat(nym-node-status-api): add PostgreSQL database support via feature flags Implement dual database support for SQLite and PostgreSQL through Cargo feature flags. The implementation uses a query wrapper that automatically converts SQLite-style ? placeholders to PostgreSQL-style $1, $2, ... placeholders at runtime. Key changes: - Add query wrapper functions that handle placeholder conversion - Convert all sqlx::query\! macros to use wrapper functions - Handle type conversions between databases (i64 vs i32) - Add feature-gated implementations for database-specific SQL syntax - Update Makefile with clippy targets for both database features - Document database support in README * feat(nym-node-status-agent): add multi-API support with random selection Agents can now connect to multiple APIs and randomly select one for each testrun: - Accept multiple --server arguments in format "address:port:auth_key" - Randomly shuffle server list before attempting connections - Try each server until a testrun is obtained - Submit results back only to the API that provided the testrun - Continue to next server if one is down or has no testruns available * feat(nym-node-status): implement primary/secondary server architecture - Agent now requests testruns only from primary server (first in list) - Results are submitted to all configured servers in parallel - Secondary servers accept external testruns via new v2 endpoint - Added auto-creation of gateway and testrun records on secondary servers - New database queries: get_or_create_gateway, insert_external_testrun - Client library enhanced with submit_results_with_context method * Bump Node status API version * Fix build workdir * Bump to 3.1.4 * Fix types and queries * 3.1.6 * Fix gateway perf, bump 3.1.7 * NodeId -> i32, 3.1.8 * Bump agent version * i64 -> i32 * Use image yq * Migration and more types * Update remaining JSONB columns * Simplify server config * Update build path * Change delimiter * bump agent * Split up pg and sqlite builds * More typing fixes, build-and-push script * Fix Dockerfile-pg * Bump node-status-api * TYping * Agent build script * More logging around testruns * Fail loudly on read errors * Cleanup * Debug get gateways query * Fix get_gateways query * Use pg cert, 3.1.16 * Submit regular results to primary server * Bump freshenss cutoff * Update Cargo.lock * fix: resolve rebase conflicts and compilation errors After rebasing onto develop, fixed several issues: - Fixed borrowed data escapes error by using sqlx::query directly in transaction functions - Removed unused imports and cleaned up code - Maintained database-specific implementations for transaction functions * fmt * Make PG default to make lives easier * Performance improvements for Explorer v2 * Fix sqlite build * Fix PG migration * Tests round 1 * DB tests * More tests * And some more tests * And some more, more tests * cargo fmt * Fix some failing lints * Fix lioness version problems * Clippy in tests --------- Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
2.5 KiB
2.5 KiB
Nym Node Status API
The Node Status API serves information about individual nym-nodes in the Mixnet, such as which role they are operating in, statistics about them, services such as Network Requesters, as well as summaries of the state of the Mixnet.
We recommend that developers building applications such as explorers or analytics interfaces about the Mixnet run their own instance of the API, in order to promote a robust network of downstream services, and spread the load of API calls amongst as many endpoints as possible.
You can find build and operation instructions in the docs.
Database Support
The Node Status API supports both SQLite and PostgreSQL databases through Cargo feature flags:
- SQLite (default): Lightweight, file-based database suitable for development and small deployments
- PostgreSQL: Full-featured database recommended for production deployments
Building with Different Database Backends
# Build with SQLite (default)
cargo build --features sqlite --no-default-features
# Build with PostgreSQL
cargo build --features pg --no-default-features
Running Tests
# Test with SQLite
cargo test --features sqlite --no-default-features
# Test with PostgreSQL
make test-db # This sets up a test PostgreSQL instance
Development Commands
The project includes a Makefile with helpful commands for both database backends:
# Check code compilation
make check-sqlite # Check with SQLite
make check-pg # Check with PostgreSQL
# Run clippy linter
make clippy-sqlite # Lint with SQLite
make clippy-pg # Lint with PostgreSQL
make clippy # Run both
# PostgreSQL development
make dev-db # Start a PostgreSQL instance for development
make prepare-pg # Prepare SQLx offline cache for PostgreSQL
Implementation Details
The database abstraction is implemented using a query wrapper that automatically converts SQLite-style ? placeholders to PostgreSQL-style $1, $2, ... placeholders at runtime. This allows writing queries once using SQLite syntax while maintaining compatibility with both databases.
Key differences handled:
- Placeholder syntax (
?vs$1, $2, ...) - Type conversions (SQLite uses i64, PostgreSQL uses i32 for many fields)
- SQL dialect differences (e.g.,
INSERT OR IGNOREvsON CONFLICT DO NOTHING) - RETURNING clause behavior
For more details on PostgreSQL setup, see README_PG.md.