* squashing work on using cancellation in nym crates
making nym-task wasm compilable
removed sending of status messages
replaced TaskManager with ShutdownManager in the validator rewarder
additional helpers for ShutdownManager
simplified ShutdownToken by removing the name field
TaskClient => ShutdownToken within all client tasks
wip: remove TaskHandle
* track all long-living client tasks
* add task tracking for most top level tasks within nym-node
* improved default builder
* split up cancellation module
* module documentation and unit tests
* nym node fixes and naming consistency
* wasm fixes
* assert_eq => assert
* wasm fixes and made 'run_until_shutdown' take reference instead of ownership
* linux-specific fixes to IpPacketRouter
* post rebasing fixes for signing monitor
* add ShutdownManager constructor to build it from an external token
* applying PR review suggestions
* conditionally enable console-subscriber within nym-node
* Update ci-build-upload-binaries.yml
* Update ci-build-upload-binaries.yml
add features console
* updated feature name
* fixed filtering on tracing layers
* add track_caller when spawning futures for better tokio-console support
* allow [client] tasks to specify their names when used within tokio console
* clippy
* pre-emptively fix wasm clippy
---------
Co-authored-by: Tommy Verrall <60836166+tommyv1987@users.noreply.github.com>
* feat(db): add SQL query wrapper for PostgreSQL placeholder conversion
- Created query_wrapper module with functions to automatically convert
SQLite ? placeholders to PostgreSQL $1, $2, ... format
- Updated build.rs to handle mutually exclusive feature flags
- Modified one query in mixnodes.rs as proof of concept
- Added type conversions for PostgreSQL compatibility (u32->i64, u16->i32)
This is a checkpoint commit before converting all queries to use the wrapper.
* feat(nym-node-status-api): add PostgreSQL database support via feature flags
Implement dual database support for SQLite and PostgreSQL through Cargo feature flags.
The implementation uses a query wrapper that automatically converts SQLite-style ?
placeholders to PostgreSQL-style $1, $2, ... placeholders at runtime.
Key changes:
- Add query wrapper functions that handle placeholder conversion
- Convert all sqlx::query\! macros to use wrapper functions
- Handle type conversions between databases (i64 vs i32)
- Add feature-gated implementations for database-specific SQL syntax
- Update Makefile with clippy targets for both database features
- Document database support in README
* feat(nym-node-status-agent): add multi-API support with random selection
Agents can now connect to multiple APIs and randomly select one for each testrun:
- Accept multiple --server arguments in format "address:port:auth_key"
- Randomly shuffle server list before attempting connections
- Try each server until a testrun is obtained
- Submit results back only to the API that provided the testrun
- Continue to next server if one is down or has no testruns available
* feat(nym-node-status): implement primary/secondary server architecture
- Agent now requests testruns only from primary server (first in list)
- Results are submitted to all configured servers in parallel
- Secondary servers accept external testruns via new v2 endpoint
- Added auto-creation of gateway and testrun records on secondary servers
- New database queries: get_or_create_gateway, insert_external_testrun
- Client library enhanced with submit_results_with_context method
* Bump Node status API version
* Fix build workdir
* Bump to 3.1.4
* Fix types and queries
* 3.1.6
* Fix gateway perf, bump 3.1.7
* NodeId -> i32, 3.1.8
* Bump agent version
* i64 -> i32
* Use image yq
* Migration and more types
* Update remaining JSONB columns
* Simplify server config
* Update build path
* Change delimiter
* bump agent
* Split up pg and sqlite builds
* More typing fixes, build-and-push script
* Fix Dockerfile-pg
* Bump node-status-api
* TYping
* Agent build script
* More logging around testruns
* Fail loudly on read errors
* Cleanup
* Debug get gateways query
* Fix get_gateways query
* Use pg cert, 3.1.16
* Submit regular results to primary server
* Bump freshenss cutoff
* Update Cargo.lock
* fix: resolve rebase conflicts and compilation errors
After rebasing onto develop, fixed several issues:
- Fixed borrowed data escapes error by using sqlx::query directly in transaction functions
- Removed unused imports and cleaned up code
- Maintained database-specific implementations for transaction functions
* fmt
* Make PG default to make lives easier
* Performance improvements for Explorer v2
* Fix sqlite build
* Fix PG migration
* Tests round 1
* DB tests
* More tests
* And some more tests
* And some more, more tests
* cargo fmt
* Fix some failing lints
* Fix lioness version problems
* Clippy in tests
---------
Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
* wip
* wip: wrap node's sphinx key with a manager
* wip: choosing correct key for packet processing
* further propagation of key rotation information
* attaching key rotation information to reply surbs
* added basic key rotation information to mixnet contract
* wip: introducing cached queries for key rotation info from nym api
* unified nym-api contract cache refreshing
* finish packet decoding
* multi api client + retrieving rotation id
* rotating sphinx key files
* logic for migrating config file
* wip: putting new sphinx keys to self described endpoints
* processing loop of KeyRotationController
* fixed sphinx key loading
* rotating bloomfilters
* wired up KeyRotationController
* flushing bloomfilters to disk and loading
* most of nym-node changes
* post rebase fixes
* fixes due to backwards compatible hostkeys
* split http state.rs file
* dont use deprecated fields
* fixed backwards compatible deserialisation of host information
* split up node describe cache
* added a dedicated CacheRefresher listener to perform full refresh outside the set interval
* controlling announced sphinx keys within nym-api
* retrieving rotation id when pulling topology
* split nym-nodes http handlers
* v2 nym-api endpoints to retrieve nodes with additional metadata information
* bug fixes...
* additional bugfixes and guards against stuck epoch
* testnet manager: set first nym-api as the rewarder
* fixed host information deserialisation
* fixed panic during first key rotation
* post rebase fixes
* clippy
* more guards against stuck epochs
* added helper method to reset node's sphinx key
* instantiate mixnet contract with custom key rotation validity
* additional bugfixes and debugging nym-api deadlock
* passing shutdown to nym apis client
* remove dead test
* post rebasing fixes
* missing MixnetQueryClient variants
* remove usage of deprecated methods in sdk example
* fix: incorrect method signature
* post rebasing fixes
* attempt to retrieve key rotation id before doing any config migration work
* ignore tests relying on networking behaviour
* allow networking failures in certain tests
* Clippy in wallet & sdk
* Clippy in wallet
* Pin rust to 1.86 in builder
* apply changes from b7da75a18c
* missing nym-node features
* Box all the things
* additional boxes in the wallet
* post rebasing clippy
---------
Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
* move SessionType into statsitcis common crate
* add RememberMe to clients config
* change stats collection logic to handle remember me
* set up sdk client to send remember me message
* bump NS API version
* remove old packettype + fix: apply routing filter BEFORE delaying
* updated sphinx crate for allow usage of reply tags
* full pipeline for placeholder checking of packet replay
* replay protection with batched insertion
* running background task for clearing/flushing the BF
* allow disabling the replay detection + cleanup
* allow unwrap in bench code
* wip: sending with single client
* tag packets to measure latency
* constantly logging rates
* concurrency
* adjusting some values
* write results to files upon completion
* feature: introduce /load endpoint for self-reported quantised NymNode load
* return Load::Unknown for value of 0 because it means we misread some data
* add additional filtering on 'en...' endpoints
* removed mnemonic from gateway config struct
scaffolding for common mixnet listener
running verloc unconditionally in a nym-node
remove filtering by mixnode
extracted verloc to separate crate
integrated nym-node-http-server more tightly with the binary
most logic for handling forward packets
running all mixnode-related tasks natively inside nymnode
removed gateway storage trait in favour of the only concrete implementation
most logic for handling final hop packets
using nym-node owned socket listener for gateways
utility for sending plain message through mixnet + gateway fix
using common packet forwarding in both modes
nifying nym-node metrics
reproduce behaviour of the console logger
cleaned up cli args
redesigned gateway tasks startup procedure
removing dead code
scaffolding for old config v6
config migration
implemented MixnetMetricsCleaner
* clippy
* require entry/exit for wireguard
* removed dead code in migration code
* updated config template
* use custom user agent for verloc queries
* fixed premature shutdown of gateway tasks
* hidden nym-api flag to allow illegal node ips
* experiment: final hop handing with wireguard
* added additional startup logs
* typo
* fixed legacy stats endpoint data
* additional logs
* apply review comments
* fixed local testnet manager
benedettadavico