* bugfix: retrieve and update ticketbook in the same query
* bump up NS version
* Update Cargo.toml
* remove SKIP LOCKED part of the query
---------
Co-authored-by: benedetta davico <46782255+benedettadavico@users.noreply.github.com>
dont build netstack in CI
additional rust 2024 fixes
fixes
removed temp.rs
first round of cleanup
removed duplicated NS types
moved gateway probe to the monorepo
* ns-api: when `score` is `Offline`, clamp `load` to `Offline`
* ns-api: bump version
* ns-api: add mixnet score field to performance_v2 struct
---------
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
* Debug logging
* Yield based logging
* Reintroduce non-dummy task manager, try add counting for
BatchMessageSender, a couple of compiler target introductions on use
statements.
* Fixed time runtime err
* Uncomment forgetme/rememberme
* remove diffs from debug
* missed commented out forgetme
* yet more forgetme comments
* * Added missing clientreqestsender clone to wasm client to stop
premature drop & busyloop
* Removed hacky mem::forget fix
* Remove debug panic_hook
* Conditional import + use of wasm_utils::console_log
* add wasm_util dep
* Commenting out or removing debug logging
* Remove missed comment
* cleanup gitignore
* clippy
* update go version in ci
* removed unused deps
* add clippy ignore
* remove mixfetch from ci build
* add minifetch fix
* comment out unused ts builds
* stop contract clients killing ci for the moment
* wasm target locking for imports
* Either remove console_log! macro or introduce cfg(debug_assertions)
* downgrade netlink
* debug assertions for console_log import
* modify config logging (debug -> normal)
* remove clone for client_request_sender + grab directly in struct
creation
* reintroduce debug print for config in debug mode
* remove ood / unused custom topology from worker example file
* clippy
* clippy - ignore todo() tests
* modified humantime test in line with new parsing rules
* ns api: add node scraper for bridge information and add to dVPN gateway output
* extra error reporting
* run sqlx-prepare
* fix clippy
---------
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
Co-authored-by: benedettadavico <benedetta.davico@gmail.com>
* ns-api: remove sqlite support
ns-api: add env var to skip migrations for local dev
ns-api: tidy up imports
ns-api: fix deserialisation fo node descriptions
update dockerfile
update README
fix up README and example env
ns-api: bump major version to 4
ns-api: add more geoip data and new performance field in dvpn responses
* ability to import partial ticketbooks
* wip: adding common ecash state to NS API
* buffering ticketbooks
* wip
* distribute tickets when getting testrun assignment
* passing ticketbook data to gateway probe
* wrapped around storage tx
* ticketbook query fixes
* clippy
* modified testrun assignment to always return tickets
* Update version
* Update push-node-status-agent.yaml
* Update Cargo.toml
* add entrypoint for ns agents
* sqlx prepare and cargo fmt
* clippy fixes
* Update ci-check-ns-api-version.yml
---------
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
Co-authored-by: benedetta davico <46782255+benedettadavico@users.noreply.github.com>
Co-authored-by: benedettadavico <benedetta.davico@gmail.com>
* ns-api: remove sqlite support
ns-api: add env var to skip migrations for local dev
ns-api: tidy up imports
ns-api: fix deserialisation fo node descriptions
update dockerfile
update README
fix up README and example env
ns-api: bump major version to 4
ns-api: add more geoip data and new performance field in dvpn responses
* ns-api: polyfill dVPN probe outcomes to make compatible with existing clients
* Use explicit transaction for testrun status change (#6046)
* Use explicit transaction for testrun status change
* Improve run scripts
* Skip locked rows
* bump version 4.0.2
* Fix build.rs
* Fix up .sqlx queries
* Bump agent version and change dockerfile to run the agent in a loop
* Make time between agents configurable by env var SLEEP_TIME
* Update entrypoint.sh
* Update Dockerfile with full path
* Force bigint to avoid postgres numeric cast
* Add override args to agent entry point, bump agent version and NS API version
---------
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
* remove [most of] legacy data from nym-api endpoints
* chore: removed contamination with legacy nodes data
* added /v1/nym-nodes/stake-saturation/{node_id}
* added /v1/legacy/mixnodes and /v1/legacy/gateways
* removed scraping of legacy mixnodes in NS api
* remove export of removed types
* huge warnings on attempting to use removed commands in the wallet
* fixed reference to removed type in tests
* feat: unify HTTP client creation and enable domain fronting
Enhanced the base nym_http_api_client to reduce fragmentation and enable domain fronting:
- Added SerializationFormat enum for explicit JSON/bincode choice (no auto-detection)
- Added from_network() method to create clients from NymNetworkDetails with domain fronting
- Added with_bincode() builder method for explicit serialization configuration
- Set Accept header based on serialization preference
- Added deprecation paths for NymApiClient wrapper and nym_api::Client re-export
- Enabled domain fronting support via network defaults feature
This is part of a broader effort to consolidate HTTP client implementations across the codebase,
reducing ~500 lines of wrapper code and providing automatic domain fronting for censorship resistance.
* feat: migrate NymApiClient usage to unified HTTP client
- Wire up domain fronting configuration in NymNetworkDetails
- Implement NymApiClientExt trait for base nym_http_api_client::Client
- Migrate direct NymApiClient usage in multiple components:
- nym-network-monitor
- verloc measurements
- connection tester
- coconut/ecash client
- validator rewarder
- Add Copy derive to ApiUrlConst to enable iteration
- Update error handling and Display implementations
This enables automatic domain fronting for all Nym API calls via the configured CDN front hosts.
* fix: resolve all compilation errors after NymApiClient migration
- Add missing nym-http-api-client dependencies to multiple crates
- Add NymApiClientExt trait imports where needed
- Fix type mismatches from NymApiClient to unified Client
- Add error conversions for NymAPIError in various error enums
- Implement missing trait methods (get_current_rewarded_set, get_all_basic_nodes_with_metadata, get_all_described_nodes)
- Fix type conversions for RewardedSetResponse in network monitor
- Update all API client instantiation to use new unified HTTP client
* feat: complete migration to unified HTTP client and fix all compilation errors
- Added missing NymApiClientExt trait methods (get_all_expanded_nodes, change_base_urls)
- Fixed all compilation errors across the workspace
- Updated nym-node to use unified client instead of deprecated NymApiClient
- Fixed type conversions for RewardedSetResponse → EpochRewardedSet
- Added nym-http-api-client dependency where needed
- Updated all examples and documentation to use new client API
* fix: provide all API URLs for automatic failover in endpoint rotation
Previously, when rotating API endpoints, only a single URL was provided to the
HTTP client, defeating the purpose of having multiple URLs for resilience.
Changes:
- NymApiTopologyProvider now provides all URLs in rotated order when switching endpoints
- NymApisClient similarly provides all URLs starting from the working endpoint
- Added clarifying comments for broadcast/exhaustive query methods where single URLs are intentionally used
- This enables the HTTP client's built-in failover mechanism while maintaining endpoint rotation behavior
The fix ensures that if the primary endpoint fails, the client can automatically
failover to alternative endpoints without manual intervention, improving overall
network resilience.
* Update common/client-core/src/client/base_client/mod.rs
Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
* Remove error generics, address PR comments
* Explicit warning on missing fronting configuration
* Assorted CI fixes
* Registry proc-macro
* Rename macro
* Syn workspace version
* Where do we need to put inventory
* Ergonomics and call sites, incept the builder
* fix: Address critical issues in client configuration registry implementation
- Fixed HeaderMapInit parsing bug that would cause compilation errors
- Added comprehensive documentation with usage examples and DSL reference
- Improved error handling with better error messages for invalid headers
- Added test coverage for both macro and registry functionality
- Added debug inspection capabilities for registered configurations
- Fixed module name conflicts in tests by using separate modules
All tests now passing:
- 7 macro tests validating DSL parsing and code generation
- 4 registry tests verifying configuration collection and application
* Use default value for the ports until api is deployed
* Feature/improved http error (#6025)
* use display impl for urls
* feat: attempt to add more details to reqwest errors
* temporarily restored GenericRequestFailure variant
* another restoration
* cleanup
* Some debug tooling, and default timeout fix
* Fix user-agent override
* Fix various wasm things
---------
Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
Co-authored-by: Bogdan-Ștefan Neacşu <bogdan@nymtech.net>
* node-status-api: fix missing monikers because of deserialisation issues from unstructured data
* node-status-api: bump version after bug fix monikers
---------
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
* Remove freshness check on testrun submit
- freshness is enforced by a background task
that marks testruns as stale after a
configured amount of time
* Move code around
* Add humantime
* Update launch script
* Fix typo
* Adjust agent run script
* Configure user agent
* Bump version
* feat(db): add SQL query wrapper for PostgreSQL placeholder conversion
- Created query_wrapper module with functions to automatically convert
SQLite ? placeholders to PostgreSQL $1, $2, ... format
- Updated build.rs to handle mutually exclusive feature flags
- Modified one query in mixnodes.rs as proof of concept
- Added type conversions for PostgreSQL compatibility (u32->i64, u16->i32)
This is a checkpoint commit before converting all queries to use the wrapper.
* feat(nym-node-status-api): add PostgreSQL database support via feature flags
Implement dual database support for SQLite and PostgreSQL through Cargo feature flags.
The implementation uses a query wrapper that automatically converts SQLite-style ?
placeholders to PostgreSQL-style $1, $2, ... placeholders at runtime.
Key changes:
- Add query wrapper functions that handle placeholder conversion
- Convert all sqlx::query\! macros to use wrapper functions
- Handle type conversions between databases (i64 vs i32)
- Add feature-gated implementations for database-specific SQL syntax
- Update Makefile with clippy targets for both database features
- Document database support in README
* feat(nym-node-status-agent): add multi-API support with random selection
Agents can now connect to multiple APIs and randomly select one for each testrun:
- Accept multiple --server arguments in format "address:port:auth_key"
- Randomly shuffle server list before attempting connections
- Try each server until a testrun is obtained
- Submit results back only to the API that provided the testrun
- Continue to next server if one is down or has no testruns available
* feat(nym-node-status): implement primary/secondary server architecture
- Agent now requests testruns only from primary server (first in list)
- Results are submitted to all configured servers in parallel
- Secondary servers accept external testruns via new v2 endpoint
- Added auto-creation of gateway and testrun records on secondary servers
- New database queries: get_or_create_gateway, insert_external_testrun
- Client library enhanced with submit_results_with_context method
* Bump Node status API version
* Fix build workdir
* Bump to 3.1.4
* Fix types and queries
* 3.1.6
* Fix gateway perf, bump 3.1.7
* NodeId -> i32, 3.1.8
* Bump agent version
* i64 -> i32
* Use image yq
* Migration and more types
* Update remaining JSONB columns
* Simplify server config
* Update build path
* Change delimiter
* bump agent
* Split up pg and sqlite builds
* More typing fixes, build-and-push script
* Fix Dockerfile-pg
* Bump node-status-api
* TYping
* Agent build script
* More logging around testruns
* Fail loudly on read errors
* Cleanup
* Debug get gateways query
* Fix get_gateways query
* Use pg cert, 3.1.16
* Submit regular results to primary server
* Bump freshenss cutoff
* Update Cargo.lock
* fix: resolve rebase conflicts and compilation errors
After rebasing onto develop, fixed several issues:
- Fixed borrowed data escapes error by using sqlx::query directly in transaction functions
- Removed unused imports and cleaned up code
- Maintained database-specific implementations for transaction functions
* fmt
* Make PG default to make lives easier
* Performance improvements for Explorer v2
* Fix sqlite build
* Fix PG migration
* Tests round 1
* DB tests
* More tests
* And some more tests
* And some more, more tests
* cargo fmt
* Fix some failing lints
* Fix lioness version problems
* Clippy in tests
---------
Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
* wip - dvpn directory cache
* Endpoint & cache
* /gateways works
- SkimmedNode data still missing
- need to move probe models to monorepo
* Rest of the data for /gateways
* Revert before merge: pin deps to cheddar release
* Filter gw by country
* Return percent string instead of u8
* Filter by semver
* Bump package version
* Fix probe types
* Reorg
* Add exit, entry endpoints
* Different entry/exit selection criteria
* Date fix migration
* Unpin from cheddar
* Revert "Unpin from cheddar"
This reverts commit f17239075b.
* Validation with celes
* PR feedback
* Fix path
* Bump version
---------
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
* move SessionType into statsitcis common crate
* add RememberMe to clients config
* change stats collection logic to handle remember me
* set up sdk client to send remember me message
* bump NS API version
* introduce 'Bincode' variant for FormattedResponse
* allow nym-api to return responses in bincode (and also yaml)
* client parsing support
* cargo fmt
* missing changes to nym-api tests
* fixed node status api build + adjusted NymApiClient construction
* NMv2 fixes + further api changes
* feature-locking http-api-common to fix wasm build
* Improve swagger definitions
* Sort data in DB
* Improve logging
* Store gw description to nym nodes table
* Move explorer related path to /explorer
* Bump package version
* Expand /v3/nym-nodes
- includes node description and geodata
- expanded scope of included geodata
* Fetch geodata for all nodes
* Bump package version
* Add /v3/nym-nodes
- returns extended node info from local DB
- endpoint caching
- add bond_info & self_described to DB nym_nodes
- update mixnode & gateway bond status on data refresh
- add `active` column to DB nym_nodes
- use only active & bonded nodes in scraping/testrun tasks
* Improve log
* PR feedback
- remove active field from nym_nodes
- delete obsolete nym_nodes
* node-status-api: cargo sqlx prepare
* Remove guardrails in CI file
* Revert "node-status-api: cargo sqlx prepare"
This reverts commit 1fcd895f0d.
* Try to ignore sqlx files
* cargo sqlx prepare
* Repair harbor tag check
* Try without checkout action
* add awk
* Update log
Jędrzej Stuczyński