* feat(db): add SQL query wrapper for PostgreSQL placeholder conversion
- Created query_wrapper module with functions to automatically convert
SQLite ? placeholders to PostgreSQL $1, $2, ... format
- Updated build.rs to handle mutually exclusive feature flags
- Modified one query in mixnodes.rs as proof of concept
- Added type conversions for PostgreSQL compatibility (u32->i64, u16->i32)
This is a checkpoint commit before converting all queries to use the wrapper.
* feat(nym-node-status-api): add PostgreSQL database support via feature flags
Implement dual database support for SQLite and PostgreSQL through Cargo feature flags.
The implementation uses a query wrapper that automatically converts SQLite-style ?
placeholders to PostgreSQL-style $1, $2, ... placeholders at runtime.
Key changes:
- Add query wrapper functions that handle placeholder conversion
- Convert all sqlx::query\! macros to use wrapper functions
- Handle type conversions between databases (i64 vs i32)
- Add feature-gated implementations for database-specific SQL syntax
- Update Makefile with clippy targets for both database features
- Document database support in README
* feat(nym-node-status-agent): add multi-API support with random selection
Agents can now connect to multiple APIs and randomly select one for each testrun:
- Accept multiple --server arguments in format "address:port:auth_key"
- Randomly shuffle server list before attempting connections
- Try each server until a testrun is obtained
- Submit results back only to the API that provided the testrun
- Continue to next server if one is down or has no testruns available
* feat(nym-node-status): implement primary/secondary server architecture
- Agent now requests testruns only from primary server (first in list)
- Results are submitted to all configured servers in parallel
- Secondary servers accept external testruns via new v2 endpoint
- Added auto-creation of gateway and testrun records on secondary servers
- New database queries: get_or_create_gateway, insert_external_testrun
- Client library enhanced with submit_results_with_context method
* Bump Node status API version
* Fix build workdir
* Bump to 3.1.4
* Fix types and queries
* 3.1.6
* Fix gateway perf, bump 3.1.7
* NodeId -> i32, 3.1.8
* Bump agent version
* i64 -> i32
* Use image yq
* Migration and more types
* Update remaining JSONB columns
* Simplify server config
* Update build path
* Change delimiter
* bump agent
* Split up pg and sqlite builds
* More typing fixes, build-and-push script
* Fix Dockerfile-pg
* Bump node-status-api
* TYping
* Agent build script
* More logging around testruns
* Fail loudly on read errors
* Cleanup
* Debug get gateways query
* Fix get_gateways query
* Use pg cert, 3.1.16
* Submit regular results to primary server
* Bump freshenss cutoff
* Update Cargo.lock
* fix: resolve rebase conflicts and compilation errors
After rebasing onto develop, fixed several issues:
- Fixed borrowed data escapes error by using sqlx::query directly in transaction functions
- Removed unused imports and cleaned up code
- Maintained database-specific implementations for transaction functions
* fmt
* Make PG default to make lives easier
* Performance improvements for Explorer v2
* Fix sqlite build
* Fix PG migration
* Tests round 1
* DB tests
* More tests
* And some more tests
* And some more, more tests
* cargo fmt
* Fix some failing lints
* Fix lioness version problems
* Clippy in tests
---------
Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
* wip - dvpn directory cache
* Endpoint & cache
* /gateways works
- SkimmedNode data still missing
- need to move probe models to monorepo
* Rest of the data for /gateways
* Revert before merge: pin deps to cheddar release
* Filter gw by country
* Return percent string instead of u8
* Filter by semver
* Bump package version
* Fix probe types
* Reorg
* Add exit, entry endpoints
* Different entry/exit selection criteria
* Date fix migration
* Unpin from cheddar
* Revert "Unpin from cheddar"
This reverts commit f17239075b.
* Validation with celes
* PR feedback
* Fix path
* Bump version
---------
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
* move SessionType into statsitcis common crate
* add RememberMe to clients config
* change stats collection logic to handle remember me
* set up sdk client to send remember me message
* bump NS API version
* introduce 'Bincode' variant for FormattedResponse
* allow nym-api to return responses in bincode (and also yaml)
* client parsing support
* cargo fmt
* missing changes to nym-api tests
* fixed node status api build + adjusted NymApiClient construction
* NMv2 fixes + further api changes
* feature-locking http-api-common to fix wasm build
* Improve swagger definitions
* Sort data in DB
* Improve logging
* Store gw description to nym nodes table
* Move explorer related path to /explorer
* Bump package version
* Expand /v3/nym-nodes
- includes node description and geodata
- expanded scope of included geodata
* Fetch geodata for all nodes
* Bump package version
* Add /v3/nym-nodes
- returns extended node info from local DB
- endpoint caching
- add bond_info & self_described to DB nym_nodes
- update mixnode & gateway bond status on data refresh
- add `active` column to DB nym_nodes
- use only active & bonded nodes in scraping/testrun tasks
* Improve log
* PR feedback
- remove active field from nym_nodes
- delete obsolete nym_nodes
* node-status-api: cargo sqlx prepare
* Remove guardrails in CI file
* Revert "node-status-api: cargo sqlx prepare"
This reverts commit 1fcd895f0d.
* Try to ignore sqlx files
* cargo sqlx prepare
* Repair harbor tag check
* Try without checkout action
* add awk
* Update log
* first pass
* cleanup
* added qu
* add readme
* more verbose err
* reword explainer @ top
* rename private-key.public to public-key
* move instructions to own file + add _meta.json files
* first pass probe
* remove unnecessary doubled notice to developers
* added extra debug log to version()
* include PR suggestions
* remove commented out function
* Disable the test for checking the remaining bandwidth in nym-node-status-api
This check fails almost every time on CI, possibly due to rate limiting?
It's not good to disable the check, but it's blocking CI as it stands
now. Given that we have the check above for locating the ip, we at least
have a little coverage.
* Remove unused
* Use expect in geodata test to give error message on failure
I keep hitting this error on CI, from what I think is network hickup.
But it's hard to tell form the log since the error is swallowed.
Explicitly unwrap the result so we get a more detailed error output
* Add nym-node-status-api to ci-build
Drazen Urch