170 Commits

Author SHA1 Message Date
benedettadavico dad2d30773 address comment 2026-06-02 13:15:13 +02:00
Jędrzej Stuczyński 8392f7da94 fix(ns-api): lock assignment + ticketbook pre-check in ports-check request
request_ports_check_testrun skipped the lock_testrun_assignment() guard and
has_enough_ticketbooks() pre-check that request_testrun performs, so two
concurrent ports-check requests could race on ticket materials and a depleted
cache left the run InProgress until stale-refresh. Mirror the probe path.
2026-06-01 16:21:27 +01:00
Jędrzej Stuczyński cc405cc802 refactor(ns-api): compile-time-check count_testruns_in_progress_by_kind
Switch from runtime sqlx::query_scalar to the checked query_scalar! macro
(consistent with the rest of the file), returning i64 instead of an always-Some
Option and dropping the unwrap_or_default() at the call site. Regenerates the
.sqlx cache, adding the new query and pruning accumulated orphans.
2026-06-01 16:21:14 +01:00
Jędrzej Stuczyński 2fd26581eb fix(ns-api): guard ports-check migration against non-JSON last_probe_result
The 20260415 migration cast last_probe_result::jsonb guarded only by a
btrim non-empty check, so any row with non-JSON text aborted the whole
migration. Add the same last_probe_result ~ '^[\[{]' guard the follow-up
20260519133000 migration uses, directly to both UPDATEs here.
2026-06-01 16:18:24 +01:00
benedettadavico 639c7f83a4 clippy 2026-06-01 14:02:25 +02:00
benedettadavico 0796e9e0a6 build fix 2026-06-01 12:40:26 +02:00
benedettadavico a98a65c16d addressing coderabbit comments 2026-06-01 11:59:43 +02:00
benedettadavico a52a8c3e81 fix 2026-05-29 11:41:48 +02:00
benedettadavico 23e6169c02 update sqlx 2026-05-29 11:41:39 +02:00
benedettadavico 17d3791b8e typo 2026-05-29 11:41:39 +02:00
benedettadavico c9a9940cb9 migration fix 2026-05-29 11:41:38 +02:00
benedettadavico ff0ecc95fb fix compile error 2026-05-29 11:41:38 +02:00
benedettadavico d791e08fac add port check to dvpn endpoint 2026-05-29 11:41:38 +02:00
benedettadavico 1532c0c16e addressing ai comments 2026-05-29 11:41:38 +02:00
benedettadavico d37b4226d0 testing port checks
add no-log to anywhere

add support for not registered nodes

...
address comments

remove unregistered nodes

testing port checks
add support for not registered nodes

...
address comments

test port check in probe results
migration update

probe arg fix

bump NS versions

cleanup and remove unannounced node option

bugsfixes

Remove in-prove

remove in-probe test, it isn't needed.

add multiple target host options

cleanup

change default target, and use batch only for portquiz

Revert "change default target, and use batch only for portquiz"

This reverts commit 8b38969964e7808b9c4e50a920ee5bc51438c7bf.

ded line

bugfixes

batch fix

batch limits

force ipv4
2026-05-29 11:41:37 +02:00
Jędrzej Stuczyński dd8c0a2521 Bugfix/cherry pick/waterloo ns api (#6833)
* NS: don't return nodes with 0 performance

* reduce concurrency during quorum check tests

* add additional leniency in ticketbook requests
2026-05-28 14:03:18 +01:00
Jędrzej Stuczyński a85256c8c4 Merge branch 'develop' into merge/release/2026.10-waterloo-2 2026-05-27 14:50:51 +01:00
Jędrzej Stuczyński a8cecb1200 additional logs for quorum failures + increase staleness threshold 2026-05-26 14:05:24 +01:00
Jędrzej Stuczyński 4e52e9bf77 fixed invalid ticket rejection 2026-05-26 10:58:05 +01:00
Jędrzej Stuczyński cf55e2fe86 fix nym-api config deserialisation + clippy 2026-05-26 10:18:39 +01:00
Jędrzej Stuczyński 526cb9b8be Merge branch 'develop' into merge/release/2026.10-waterloo 2026-05-26 10:00:43 +01:00
Jędrzej Stuczyński dc0835f1f3 more logs, timeouts and general duct taping 2026-05-26 08:45:46 +01:00
Jędrzej Stuczyński b5a8b9d283 change default netstack download timeout to 30s 2026-05-23 20:50:11 +01:00
Jędrzej Stuczyński a395167139 more logs and going insane 2026-05-23 20:23:39 +01:00
Jędrzej Stuczyński 6b98c168fc dont assign nodes with 0 performance 2026-05-23 20:00:43 +01:00
Jędrzej Stuczyński 4645de3eb5 add logs on failure to submit testrun results 2026-05-23 17:18:45 +01:00
Jędrzej Stuczyński e6dd670b16 quroum checker logs + temp: increase of DefaultBodyLimit 2026-05-23 16:40:43 +01:00
Jędrzej Stuczyński dc48750271 ensure sufficient number of tickets before performing testrun assignment 2026-05-23 14:26:49 +01:00
Jędrzej Stuczyński 28b22f6b22 upgrade axum to 0.8.9 (and side deps) (#6808)
* upgrade axum to 0.8.9 (and side deps)

Bumps axum 0.7.5 → 0.8.9, axum-extra 0.9.4 → 0.12.6,
axum-client-ip 0.6.1 → 1.3.1, axum-test 16.2.0 → 20.0.0,
utoipa-swagger-ui 8.1 → 9.0.2.

* warn upon using fallback ip

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* chore: replace use of deprecated try_next()

* update console-subscriber to ensure single version of axum in the lock file

* removed unused axum-test dev-dep

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-05-22 15:39:33 +01:00
Jędrzej Stuczyński 46c67440bb Mixnode stress testing (#6575)
* Squashing the mix stress testing branch (#6575)

reduced chain watcher per block log severity

update network monitors contract semver to 1.0.0

fix build issues

fix mixnet client dropping initial packet on egress reconnection

adjusted logs for network monitor agent

changed default testing interval to 2h

refresh NM contract information

explicit return type for batch submission

for mixnet listener task to get scheduled before beginning connectivity test

make sure to always use canonical ip for network monitor noise keys

feat: NMv3: make agents decide egress port (#6746)

add config v12->v13 config migration for nym nodes

fix formatting in wallet types

simplified client config creation

remove other swagger redirect

removed swagger redirect on /swagger/ route

log version info on startup

add workflows, contract address, and dockerfile

bugfix: use correct endpoints when setting up orchestrator (#6733)

clippy

adjust DEFAULT_MIN_STRESS_TESTED_NODES ratio

expose route with new performance metrics

fixes and additional docs

use stress testing scores

stub for usage of stress testing scores

stub traits

added new fields to nym-api config controlling usage of stress test data

guard against duplicate packets

prevent usage of chain_authorisation_check_max_attempts with value of 0

make sure duplicate results cant be inserted into the db

submit test results from orchestrator on an interval

docs and fixes

nym-api side of handling result submission

stubs for submitting results

NM orchestrator verifying nym-api result submission permissions

NM orchestrator to update announced key on startup

allow NM orchestrator to announce its identity key to the contract

stubs within nym-api for accepting NMv3 results

added additional metrics

docs

bugfixes + making sure to only assign mixnode testruns

fixed node refresher to only retrieve mixnodes and add additional metrics

topology metrics

defined basic prometheus metrics

authorised endpoint for returning prometheus data

create initial stub for prometheus metrics

post rebasing fixes

adjusted routes

missing implementation for storage getters

a lot of new stubs and db accessors

stubs for results endpoints

update utoipa tags for agent rountes

shared auth between metrics and results

moved stale results eviction into the interval.tick branch

refactor and comments

create background process to evict stale data

include sphinx packet delay as part of the stats

fix mock construction

add median to the calculated latency distribution

remove unused imports

cleanup

performing testrun and submitting the results

assigning testruns to requesting agents

basic stub for http server for the NMv3 orchestrator

chore: rename existing 'NetworkMonitorAgent' to 'NodeStressTester'

make sure to use canonical ips within the noise config

fixed contract tests

cargo fmt

additional comments and unit tests

contract and nym-node support of NM agents being run on the same host

basic unit tests

refactoring

make agents retrieve mix port assignment from the orchestrator

provide sensible defaults to CLI arguments

stub the initial structure for the agent

chore: remove redundant import

missed tick behaviour

removed redundant mutex

removed redundant try_get_client

reuse existing constant for default nymnode port

add node refresher for periodic scraping of bonded nym-node details

- NodeRefresher periodically queries the mixnet contract for all bonded
  nodes and probes each node's HTTP API for host information, sphinx keys,
  noise keys, and key rotation IDs
- Extract NymNodeApiClientRetriever into nym-node-requests with port
  probing, identity verification, and host information signature checking
- Add clone_query_client on NyxdClient so the refresher can hold its own
  query client without locking the signing client
- Batch upsert for nym_node rows (single transaction instead of per-row)
- Reuse the new helpers in nym-api's node_describe_cache

ensure assignment of testrun begins an IMMEDIATE tx

construction of the orchestrator struct

initial set of cli args

make sure to not assign testable nodes too often

very initial database structure and cli

fixed construction of RoutableNetworkMonitors

remove redundant constructor for NoiseNode

forbid 0-nonsense config values

add type safety for test route construction

moved lioness and arrayref to workspace deps

fixed dockerfile build

always use canonical addresses in RoutableNetworkMonitors

fixed old contract formatting issues

removed redundant into() call

network monitor agent fixes

additional logs

config unit tests

more docs

standalone stress testing invocation

further refactoring and changes

refactor testing loop and return valid test result upon completion

initial sending/receiving test loop

generating reusable sphinx headers

additional structure for receiving ingress packets

initial scaffolding for NMv3 agent

added validation of x25519 noise key

removed unstable call to 'is_multiple_of'

remove calls to from_octets as they're unavailable in pre 1.91

additional docs/comments

propagating noise information about NM for mixnet routing

pass full socket address of the agent into the contract storage

feat: store noise keys alongside ip addresses within the contract

removed redundant comment

ensure NM packets can only go to NM

PR review comments

added additional docs

allow NM to replay packets + fix replay prometheus metrics

propagate information about nm agent to connection handler

updated nym-node config migration

feat: introduced nym-node websocket subscription for keeping updated list of NM agents

allow admin to also revoke monitor agents

remove agents upon orchestrator removal

fixed schema generation and regenerated the contract schema

removed rustc restriction on contracts-common

added client methods for interacting with the contract

added unit tests for contract methods

implemented logic of the network monitors contract

create initial structure for network monitors contract

start mix stress testing topic branch

* make nym-node default to the new blockstream rpc/ws node cluster

* reduced mixnet-client log severity

* set network monitors contract address for mainnet
2026-05-22 09:43:20 +01:00
Jędrzej Stuczyński a21a01cf1a node families (#6715)
* start node families topic branch

* start node families topic branch

* initialise node families contract

* define contract storage

* registering new family in storage

* accepting family invitation

* add_pending_invitation

* revoke_pending_invitation

* remove_family_member

* reject_pending_invitation

* disband_family

* added unit tests for the storage methods

* added restriction on uniquness of family names

* update rustc version for node families contract common

* clippy

* basic queries by id

* query_families_paged

* change family membership storage and expose query for all members of a family

* queries for pending invitations

* queries for past invitations

* queries for past data per node

* queries for past family members

* query_past_members_for_node_paged

* queries for family by name and by owner

* fixup family name normalisation

* fixed incorrect lower bound for queries for past data

* implement contract and storage initialisation

* stubbing tx messages that are to be exposed by the contract

* handler for updating config

* removed partial fee return

* wip: create family

* move mixnet contract interaction traits to shared location

* store original family name alongside the normalised variant

* prevent family creation if owner has a node in another family

* try_disband_family

* try_invite_to_family + shared helpers

* try_revoke_family_invitation

* accept_family_invitation

* stub method for node unbonding

* try_reject_family_invitation

* unit tests for family name normalisation

* try_leave_family

* try_kick_from_family

* fix outdated comments and add paid fee event attribute

* feat: NMv3: leave family upon node unbonding

* NF contract handling of unbonding

* lints

* init node families contract when creating performance contract tester

* clippy

* avoid self-dep in the contract dev deps

* introduced client traits for interacting with the node families contract

* add node families contract to cache refresher

* added query for all node family members (globally) and started scaffolding nym-api caches

* docs and cache -> api conversion

* calculating average node age based on individual timestamps

* wire up node families cache

* http stubs

* filled in the implementation

* route tests + extracting shared code

* review fixes

* feat: expose family information for all dvpn gateway endpoints within NS API

* expose family information for explorer v3 route

* clippy

* review comments and optimise db family update

* feat: Node Families: expose stake information inside DVpnGateway

* chore: update lock files after rebase

* chore: sort workspace members

* explicitly require providing node families contract address for mixnet contract migration

* fix missing node families contract address env export

* dont swallow cache overwrite failures in fixture

* pin network-defaults rustc version due to contracts dep

* further version pinning

* chore: update mixnet contract schema
2026-05-19 10:36:20 +01:00
dynco-nym 7140ba4ea9 Fix invalid ticket spend (#6683)
* Fix

* PR feedback

* Bump version

* Update sqlx files
2026-04-17 16:49:17 +02:00
benedetta davico 62962509eb Merge pull request #6686 from nymtech/workflow/fix
Fixes to crates and CI
2026-04-17 16:21:12 +02:00
dynco-nym 534a5068d3 Return ipv6 addresses as well (#6684)
* Return ipv6 addresses as well

* Fix clippy

* Bump NS API version
2026-04-17 13:42:48 +02:00
mfahampshire 924d7d1ccc Enforce ordering of [package] fields in cargo.toml files 2026-04-17 07:49:50 +01:00
dynco-nym 2209e8ac04 Include all gateways in the returned list (#6649)
* Include all gateways in the returned list

* Fix clippy

* Bump API version
2026-04-10 16:55:42 +02:00
dynco-nym 89de989ad1 Optimize GW probe in NS agent (#6636)
* WIP

* NS agent calls probe as lib

* Clippy: ns agent

* Fix submit_v2 on API

* Adjust dockerfile, deployment details for the new flow

* Bump package versions

* PR feedback

* Fix CI

* Final version
2026-04-07 16:35:50 +02:00
benedettadavico 3cdda8fdfd bump NS version 2026-03-20 15:33:16 +01:00
dynco-nym 180802feb8 Fix socks5 GW probe regression (#6576)
* Restore tested gateway into topology

* Bump agent version

* Update .sqlx files

* Clean up code in probe test

* Probe error & logging improvements

* Fix clippy, improve log line

* Improve logging in ns agent

* Better tooling in NS API

* Bump agent

* Bump NS agent version
2026-03-20 10:36:32 +01:00
dynco-nym 8dc3ba4ec3 Add LP to NS UI (#6562)
* Add LP column to gateway view

* Add LP to graphs
2026-03-16 14:07:19 +01:00
dynco-nym 80370b98ec Additional ticket for agent (#6551)
* Additional ticket type for LP tests

* Remove hardcoded comments

* bump cargo version

* Nuke fallback edge case in the probe

* Cleanup unused code

* Bump API & agent versions
- agent bump required due to probe changes
2026-03-12 14:49:03 +01:00
dynco-nym 08b20ac2ab Add LP fields (#6535)
* Add lp field to /dvpn/gateways

* Expand unit tests

* Add lp ports, keys, hashes

* Include the whole struct

* Update Toml version
2026-03-10 13:06:56 +01:00
Jędrzej Stuczyński 6569479083 feat: introduce /v3/unstable/nym-nodes/semi-skimmed to aggregate LP information (#6499)
* feat: introduce /v3/unstable/nym-nodes/semi-skimmed to aggregate LP information

nym-nodes will require this information to establish shared PSQ

* reorganised imports
2026-03-03 14:05:02 +00:00
dependabot[bot] 5f2bc60c2c build(deps): bump next in /nym-node-status-api/nym-node-status-ui (#6385)
Bumps [next](https://github.com/vercel/next.js) from 15.4.10 to 16.1.5.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v15.4.10...v16.1.5)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 16.1.5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-10 15:53:24 +00:00
Jędrzej Stuczyński e2d29f184d Merge pull request #6424 from nymtech/release/2026.3-parmigiano
Syncing parmigiano and develop
2026-02-05 16:49:37 +00:00
dynco-nym 660eff45dc Endpoint for exit GW IPs (#6418) 2026-02-04 22:14:46 +01:00
Simon Wicky cd0881462b [LP Gateway Probe] CLI and behavior improvements (#6400)
* attempt to de-spaghettificationize the gateway probe

* applying suggestions
2026-01-30 16:55:29 +01:00
dynco-nym d3648f13c5 NS API socks5 support (#6361)
* Add conversion from gw_probe crate type

* Move code around
- split 1000+ LoC files into smaller ones

* Add socks5 field
- code improvements in gw_probe crate

* Fix docker build
- install go
- required as build dependency of gw probe

* Add logs to agent

* NS API: configure DB via env

* rebase fix

* socks5 score calc

* Cargo fmt

* use existing div_ceil

* Code improvements

* Bump NS API version

* Rename variables

* Bump API & agent version

* Try to fix CI

* Build only on linux
2026-01-29 20:54:21 +01:00
dynco-nym 9a931b9251 Add socks5 test to gateway-probe (#6393)
* Socks5 in GW probe

Bump NS agent version

Fix bugs
- force route construction
- use same entry = exit

Fix NS API version check workflow

PR feedback

More robust test attempts

CLI arg validation

Fix clippy

PR feedback

* Test provided endpoints in config at startup

Require one valid endpoint

* Bump agent to 1.1.0
2026-01-29 18:20:51 +01:00
Andrej Mihajlov f4b59158df Box reqwest::Url to keep HttpClientError below 128 byte size which triggers clippy 2026-01-29 07:41:43 +01:00