* conditionally enable console-subscriber within nym-node
* Update ci-build-upload-binaries.yml
* Update ci-build-upload-binaries.yml
add features console
* updated feature name
* fixed filtering on tracing layers
* add track_caller when spawning futures for better tokio-console support
* allow [client] tasks to specify their names when used within tokio console
* clippy
* pre-emptively fix wasm clippy
---------
Co-authored-by: Tommy Verrall <60836166+tommyv1987@users.noreply.github.com>
* Set cached storage counters to 0 (#5812)
* Set cached storage counters to 0
* u64 to i64 log possible error
* Check addition too
Debug commit
Remove more data from wg storage peer
Put actual ticket type in storage
Simplify add peer
Finish rebase
Pass defguard Peer
Cache less data for consumption
GatewayStorage traits
Wg API trait
Mock test structures
Unit test for peer controller
EcashManager trait
Init test of Authenticator
Remove peer test
* Fix windows different API
* Use make_bincode_serializer like in other places
* Add log_slow_statements to gateway storage
* Use correct LevelFilter
* Fix clippy
* More win fix
* Win clippy
* Use two error variants more
* Use only one Arc<RwLock<T>> instead of many more
* Remove commented test
* Specific trait import
* feat(db): add SQL query wrapper for PostgreSQL placeholder conversion
- Created query_wrapper module with functions to automatically convert
SQLite ? placeholders to PostgreSQL $1, $2, ... format
- Updated build.rs to handle mutually exclusive feature flags
- Modified one query in mixnodes.rs as proof of concept
- Added type conversions for PostgreSQL compatibility (u32->i64, u16->i32)
This is a checkpoint commit before converting all queries to use the wrapper.
* feat(nym-node-status-api): add PostgreSQL database support via feature flags
Implement dual database support for SQLite and PostgreSQL through Cargo feature flags.
The implementation uses a query wrapper that automatically converts SQLite-style ?
placeholders to PostgreSQL-style $1, $2, ... placeholders at runtime.
Key changes:
- Add query wrapper functions that handle placeholder conversion
- Convert all sqlx::query\! macros to use wrapper functions
- Handle type conversions between databases (i64 vs i32)
- Add feature-gated implementations for database-specific SQL syntax
- Update Makefile with clippy targets for both database features
- Document database support in README
* feat(nym-node-status-agent): add multi-API support with random selection
Agents can now connect to multiple APIs and randomly select one for each testrun:
- Accept multiple --server arguments in format "address:port:auth_key"
- Randomly shuffle server list before attempting connections
- Try each server until a testrun is obtained
- Submit results back only to the API that provided the testrun
- Continue to next server if one is down or has no testruns available
* feat(nym-node-status): implement primary/secondary server architecture
- Agent now requests testruns only from primary server (first in list)
- Results are submitted to all configured servers in parallel
- Secondary servers accept external testruns via new v2 endpoint
- Added auto-creation of gateway and testrun records on secondary servers
- New database queries: get_or_create_gateway, insert_external_testrun
- Client library enhanced with submit_results_with_context method
* Bump Node status API version
* Fix build workdir
* Bump to 3.1.4
* Fix types and queries
* 3.1.6
* Fix gateway perf, bump 3.1.7
* NodeId -> i32, 3.1.8
* Bump agent version
* i64 -> i32
* Use image yq
* Migration and more types
* Update remaining JSONB columns
* Simplify server config
* Update build path
* Change delimiter
* bump agent
* Split up pg and sqlite builds
* More typing fixes, build-and-push script
* Fix Dockerfile-pg
* Bump node-status-api
* TYping
* Agent build script
* More logging around testruns
* Fail loudly on read errors
* Cleanup
* Debug get gateways query
* Fix get_gateways query
* Use pg cert, 3.1.16
* Submit regular results to primary server
* Bump freshenss cutoff
* Update Cargo.lock
* fix: resolve rebase conflicts and compilation errors
After rebasing onto develop, fixed several issues:
- Fixed borrowed data escapes error by using sqlx::query directly in transaction functions
- Removed unused imports and cleaned up code
- Maintained database-specific implementations for transaction functions
* fmt
* Make PG default to make lives easier
* Performance improvements for Explorer v2
* Fix sqlite build
* Fix PG migration
* Tests round 1
* DB tests
* More tests
* And some more tests
* And some more, more tests
* cargo fmt
* Fix some failing lints
* Fix lioness version problems
* Clippy in tests
---------
Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
* wip: changes to surb logic + stronger db typing
* surb invalidation logic
* chore: remove unused deps
* resolving todos
* a lot of additional bugfixes
* 1.88 clippy
* wasm fixes
* wasm clippy
* wallet clippy
* wait for epoch end when setting up new network
* split ReplyController into Sender and Receiver for easier reasoning
* additional reply surbs improvements
includes, but is not limited to: unconditionally reseting sender tag on restart, limiting number of surb re-requests, resetting stale surbs on load
* fixed calculation of number of removed surbs
* add additional calculated field to key rotation info
* DBG: 'request_reply_surbs_for_queue_clearing' temp logs
* fixes for silly mistakes
* conditionally reduce log severity
* wip
* wip: wrap node's sphinx key with a manager
* wip: choosing correct key for packet processing
* further propagation of key rotation information
* attaching key rotation information to reply surbs
* added basic key rotation information to mixnet contract
* wip: introducing cached queries for key rotation info from nym api
* unified nym-api contract cache refreshing
* finish packet decoding
* multi api client + retrieving rotation id
* rotating sphinx key files
* logic for migrating config file
* wip: putting new sphinx keys to self described endpoints
* processing loop of KeyRotationController
* fixed sphinx key loading
* rotating bloomfilters
* wired up KeyRotationController
* flushing bloomfilters to disk and loading
* most of nym-node changes
* post rebase fixes
* fixes due to backwards compatible hostkeys
* split http state.rs file
* dont use deprecated fields
* fixed backwards compatible deserialisation of host information
* split up node describe cache
* added a dedicated CacheRefresher listener to perform full refresh outside the set interval
* controlling announced sphinx keys within nym-api
* retrieving rotation id when pulling topology
* split nym-nodes http handlers
* v2 nym-api endpoints to retrieve nodes with additional metadata information
* bug fixes...
* additional bugfixes and guards against stuck epoch
* testnet manager: set first nym-api as the rewarder
* fixed host information deserialisation
* fixed panic during first key rotation
* post rebase fixes
* clippy
* more guards against stuck epochs
* added helper method to reset node's sphinx key
* instantiate mixnet contract with custom key rotation validity
* additional bugfixes and debugging nym-api deadlock
* passing shutdown to nym apis client
* remove dead test
* post rebasing fixes
* missing MixnetQueryClient variants
* remove usage of deprecated methods in sdk example
* fix: incorrect method signature
* post rebasing fixes
* attempt to retrieve key rotation id before doing any config migration work
* ignore tests relying on networking behaviour
* allow networking failures in certain tests
* Clippy in wallet & sdk
* Clippy in wallet
* Pin rust to 1.86 in builder
* apply changes from b7da75a18c
* missing nym-node features
* Box all the things
* additional boxes in the wallet
* post rebasing clippy
---------
Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
* move SessionType into statsitcis common crate
* add RememberMe to clients config
* change stats collection logic to handle remember me
* set up sdk client to send remember me message
* bump NS API version
* introduce 'Bincode' variant for FormattedResponse
* allow nym-api to return responses in bincode (and also yaml)
* client parsing support
* cargo fmt
* missing changes to nym-api tests
* fixed node status api build + adjusted NymApiClient construction
* NMv2 fixes + further api changes
* feature-locking http-api-common to fix wasm build
* Improve swagger definitions
* Sort data in DB
* Improve logging
* Store gw description to nym nodes table
* Move explorer related path to /explorer
* Bump package version
* removed support for legacy packet types from NymCodec
I think nodes had plenty of time to upgrade given versioned variant was introduced in 2022
* temp: use local sphinx packet for development
* introduce new messages that use more efficient reply surbs encoding
* checks for incorrect encoding
* generate correct message depending on config value
* fixed current packet version
* made packet type selection configurable
* updated sphinx packet crate to the published version
* fixed wasm build
* fixes in outfox due to sphinx api changes
* additional tests
* clippy
* fixed log/tracing import
* remove old packettype + fix: apply routing filter BEFORE delaying
* updated sphinx crate for allow usage of reply tags
* full pipeline for placeholder checking of packet replay
* replay protection with batched insertion
* running background task for clearing/flushing the BF
* allow disabling the replay detection + cleanup
* allow unwrap in bench code
* wip: sending with single client
* tag packets to measure latency
* constantly logging rates
* concurrency
* adjusting some values
* write results to files upon completion
* introduced v2 authentication request between clients and gateways
* client to send v2 auth when possible
* added persistence to last used authentication timestamp
* added clients identity to signed plaintext
* new NymNodeTopologyProvider to also keep track of ips of all nodes
* added nym-api endpoint for nodes existence by ip
* change behaviour of updating allowed nodes alongside the topology
* clippy
* license fix
* fix default filtering limit
* introduced v2 authentication request between clients and gateways
* client to send v2 auth when possible
* added persistence to last used authentication timestamp
* added clients identity to signed plaintext
* feature: introduce /load endpoint for self-reported quantised NymNode load
* return Load::Unknown for value of 0 because it means we misread some data
* add additional filtering on 'en...' endpoints
* add timestamp to stored client messages
* removed dead code
* starting node task to remove old messages
* added log for number of removed messages
* debug log on task finishing
* fixed bearer auth for prometheus route
* basic prometheus metrics
* added rates on global values
* improved structure on the prometheus metrics
* added additional metrics for ingress websockets and egress mixnet connections
* some channel business metrics
* fixed metrics registration and added additional variants
* added counter for number of disk persisted packets
* counter for pending egress packets
* counter for pending egress forward packets
* clippy
Jędrzej Stuczyński