Compare commits

...

170 Commits

Author SHA1 Message Date
Tommy Verrall d39ec4a048 more white space 2025-07-15 16:06:46 +02:00
Tommy Verrall c03c580b1f indentation 2025-07-15 16:05:51 +02:00
Tommy Verrall e0cc094ac8 revert-revert-revert 2025-07-15 15:54:12 +02:00
Tommy Verrall f1fcaa309b try changing the workflow 2025-07-15 15:39:21 +02:00
benedettadavico 8771c1dfa6 bump wallet version 2025-07-15 14:47:49 +02:00
Jack Wampler aea5872ad0 bump h2 dependency to fix DoH connection close logging (#5893) 2025-07-14 12:56:56 -06:00
Mark Sinclair 9e9abd74d7 Update ci-sonar.yml
[skip ci]
2025-07-14 17:34:26 +01:00
Mark Sinclair 3832508af7 Update sonar-project.properties 2025-07-14 17:33:10 +01:00
Mark Sinclair 69a4e33b17 Create sonar-project.properties 2025-07-14 17:25:30 +01:00
Mark Sinclair 83385421ff Create ci-sonar.yml 2025-07-14 17:24:42 +01:00
Jędrzej Stuczyński ec53b570dc listen for shutdown signals during nym-node startup (#5879)
this is to avoid situation where the process can't be killed without 'kill -9' because the logic to listen to shutdown signals hasn't been hit yet
2025-07-14 12:13:40 +01:00
Jędrzej Stuczyński ebcc658f98 chain scraper: ignore precommits from missing validators (#5867) 2025-07-14 08:46:19 +01:00
Mark Sinclair 6a155721c6 Update push-node-status-agent.yaml 2025-07-11 13:51:10 +01:00
Mark Sinclair 1bb8b3a3ec Update push-node-status-api.yaml 2025-07-11 13:50:07 +01:00
Mark Sinclair 8d1a16eb02 Update push-node-status-api.yaml 2025-07-11 11:46:21 +01:00
Mark Sinclair 8d10cf70e9 Update push-node-status-api.yaml 2025-07-11 11:36:16 +01:00
Mark Sinclair e32df10b4d Update push-node-status-api.yaml 2025-07-11 11:30:26 +01:00
Mark Sinclair d1660c01e6 Update push-node-status-api.yaml 2025-07-11 11:12:09 +01:00
Sachin Kamath 14378b1db9 hotfix: fix contract build in Makefile (#5892) 2025-07-11 15:32:49 +05:30
dynco-nym 35bbf5fd84 Batch SQL writes for packet stats (#5874)
* Move stuff around

* Batch SQL operations

* Clippy

* Bump version

* Remove shared queue which was always re-initialized

* Make max_concurrent_tasks configurable

* fixed typo

* clippy

---------

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
2025-07-11 10:53:19 +01:00
Mark Sinclair c374a4935a Update push-node-status-agent.yaml (#5882)
* Update push-node-status-agent.yaml

* Update push-node-status-api.yaml

* Update push-node-status-api.yaml

Fix up typo

* Update push-node-status-agent.yaml

* Update push-node-status-api.yaml
2025-07-11 10:29:05 +01:00
Jędrzej Stuczyński 513f4f652d Merge pull request #5887 from nymtech/merge/release/2025.12-dolcelatte
merge: release/2025.12-dolcelatte into develop
2025-07-10 09:16:58 +01:00
Sachin Kamath 82b9425ca6 chore: build contracts with cw optimizer (#5888) 2025-07-09 21:45:10 +05:30
Jędrzej Stuczyński 615e98b166 Merge branch 'develop' into merge/release/2025.12-dolcelatte 2025-07-09 15:37:41 +01:00
import this b11f6c6c70 [DOCs/operators]: Release notes v2025.12-dolcelatte (#5881)
* initialise release update

* add dev features and bugfixes

* add version

---------

Co-authored-by: mfahampshire <maxhampshire@pm.me>
2025-07-09 13:32:46 +00:00
Jędrzej Stuczyński 2f5e8e0bcd feat: forbid running mixnode + entry on the same node (#5878) 2025-07-09 08:59:55 +01:00
Jędrzej Stuczyński 812a8782b4 ignore 'Send' responses when claiming bandwidth (#5884) 2025-07-08 09:09:18 +01:00
benedettadavico 089c47cce7 update changelog 2025-07-07 15:44:15 +02:00
Jędrzej Stuczyński 833114372a bugfix: key-rotation + reply SURBs (#5876)
* wip: changes to surb logic + stronger db typing

* surb invalidation logic

* chore: remove unused deps

* resolving todos

* a lot of additional bugfixes

* 1.88 clippy

* wasm fixes

* wasm clippy

* wallet clippy

* wait for epoch end when setting up new network

* split ReplyController into Sender and Receiver for easier reasoning

* additional reply surbs improvements

includes, but is not limited to: unconditionally reseting sender tag on restart, limiting number of surb re-requests, resetting stale surbs on load

* fixed calculation of number of removed surbs

* add additional calculated field to key rotation info

* DBG: 'request_reply_surbs_for_queue_clearing' temp logs

* fixes for silly mistakes

* conditionally reduce log severity
2025-07-04 16:29:03 +01:00
Jack Wampler a7b57d7e58 Make Mix hops optional for Mixnet Client SURBs (#5861)
* allow SURBs to be configured without mix hops

* gateways require consistency in surb format so if disabling mixnhops - use updated format
2025-07-03 09:21:50 -06:00
benedettadavico 84e10a654c Revert "Bump ns-api version"
This reverts commit d724f94319.
2025-07-01 15:26:55 +02:00
benedetta davico d724f94319 Bump ns-api version 2025-07-01 15:19:56 +02:00
Jędrzej Stuczyński d0692a567a feat: basic performance contract integration [within Nym API] (#5871)
* renamed nym-api config fields

* decouple rewarder startup from network monitor

* additional sections in nym-api config

* removed vesting queries in circulating supply calculator

* added memoized field for last submitted performance measurement

* wip: performance contract refresher

* cleaned up various contract caches

* modified cache refresher to allow passing update fn

* implement performance cache refreshing

* updated lefthook.yml to run cargo fmt

* impl NodePerformanceProvider trait

* dynamically using specific performance provider

* pre warm up performance contract cache and forbid the mode if its empty

* clippy

* introduce fallback setting for performance contract if value for given epoch is not available

* move some functions around
2025-07-01 11:29:50 +01:00
Jędrzej Stuczyński 2ae38b9e49 chore: 1.88 clippy (#5877)
* 1.88 clippy

* wasm clippy

* wallet clippy
2025-07-01 10:28:57 +01:00
benedetta davico ef5990658a Merge pull request #5873 from nymtech/wallet/fix-link 2025-06-26 13:26:36 +02:00
benedettadavico 658dec8299 fix the broken link 2025-06-26 12:44:47 +02:00
dynco-nym 447352b8d6 Set busy_timeout in sqlx (#5872)
* Set busy_timeout

* Bump version
2025-06-26 10:44:06 +02:00
Tommy Verrall d6bb0979d0 fix imports
- it was not compiling due to this
2025-06-24 16:12:06 +02:00
Simon Wicky eb59615c56 StatsAPI qol : disable swagger try it out and remove debug level from nym_http_api_client (#5868) 2025-06-23 14:58:29 +02:00
Bogdan-Ștefan Neacşu 07c908c497 Return true remaining (#5866) 2025-06-23 11:53:39 +03:00
Jędrzej Stuczyński 6de0c4ce92 feat: initial performance contract (#5833)
* initialised basic structure for the performance contract

* shared code for contract testing

* unified common testing methods between performance and nym pool contracts

* impl of ExecuteMsg for the contract

* impl of QueryMsg for the contract

* setting initial authorised NMs during instantiation

* additional tests and fixes

* ibid

* scaffolding for client traits

* completed client traits

* clippy

* naive add performance contract to testnet manager

* placeholder values for the performance contract address

* introduced admin messages to purge old measurements from the storage

* introduced check ensuring performance data is only added to bonded nodes
2025-06-20 09:06:56 +01:00
Jędrzej Stuczyński fa1d47e941 Bugfix/backwards compat (#5865)
* lowered log severity

* make nodes use legacy encoding for forwarding packets

* note regarding localnet noise
2025-06-19 09:57:46 +01:00
benedettadavico 05d8b31e51 Merge branch 'remove/old-explorer' into develop 2025-06-18 15:34:40 +02:00
Georgio Nicolas 692fbf1392 Merge pull request #5828 from nymtech/georgio/dkg-crypsen-fixes
Security patches for the `dkg` crate
2025-06-18 10:48:37 +02:00
Jędrzej Stuczyński 44ec6d6bc8 bugfix: allow gateways to permit authentication from v4 clients (#5862) 2025-06-18 09:17:54 +01:00
Andrej Mihajlov 0de4aea77b Merge pull request #5796 from nymtech/am/close-sqlite-pool
Close sqlite pool before moving or reopening databases
2025-06-17 19:01:25 +02:00
Georgio Nicolas a7cd8efc04 dkg: fix clippy suggestions 2025-06-17 16:37:50 +02:00
Georgio Nicolas 56aad75220 dkg: verify integrity of ciphertexts during decryption 2025-06-17 16:30:11 +02:00
Georgio Nicolas e2f2ab89ec dkg: add CryptoRng trait requirement 2025-06-17 16:30:11 +02:00
Georgio Nicolas 4d09b6f2e5 bte/proof_chunking.rs: Check for potential arithmentic overflows 2025-06-17 16:30:11 +02:00
Jędrzej Stuczyński 6d47046a38 fixed client route for obtaining v2 list of gateways (#5859) 2025-06-16 14:32:46 +01:00
dynco-nym b9339b8f0c Add /status endpoints (#5857)
* Add /status endpoints

* Bump package version

* pub use instead of import
2025-06-16 13:19:35 +02:00
Andrej Mihajlov 43a7360399 Merge pull request #5856 from nymtech/am/remove-surb-screaming-logs
Clear out screaming logs
2025-06-16 11:39:27 +02:00
Andrej Mihajlov 5f9f7f0fac Clear out screaming logs 2025-06-13 11:00:48 +02:00
Andrej Mihajlov df0e2fe489 Merge pull request #5853 from nymtech/am/path-display
Use display when printing paths
2025-06-13 10:54:12 +02:00
Simon Wicky 5cfd09cd99 fix removal of qa env 2025-06-13 10:03:50 +02:00
benedetta davico bc33cc4c8d Merge pull request #5855 from nymtech/fix-qa-removal 2025-06-13 09:40:56 +02:00
Simon Wicky a31597aca9 fix removal of qa env 2025-06-13 09:30:00 +02:00
Jack Wampler 378229b04e HTTP Discovery objects & network defaults (#5814)
add extended (optional) fields to the NetworkDiscovery and configure fallback hosts
2025-06-12 11:15:36 -06:00
Andrej Mihajlov fec196c097 Use display when printing paths 2025-06-12 17:17:00 +02:00
Andrej Mihajlov 1d7ffc1bb6 test: remove file after closing for a test 2025-06-12 15:39:26 +02:00
Andrej Mihajlov 0caa627960 Fix missing await on self.close_pool_inner() 2025-06-12 15:12:46 +02:00
import this d6b3d7fc0a [DOCs/operators]: Release notes for v2025.11 cheddar (#5852)
* bump up version

* add dev features

* add operator updates

* add updated stats

* update prebuild
2025-06-12 11:19:00 +00:00
benedettadavico 40b4670d80 bump versions 2025-06-12 12:21:02 +02:00
dynco-nym ac273480f8 Fix CI version check (#5851)
* Fix version

* Test .rc version

* Undo cargo.toml version

* Remove comment

* Apply to statistics service
2025-06-12 11:17:56 +02:00
benedettadavico 79603d61d7 fix for QA 2025-06-12 10:02:40 +02:00
dynco-nym e8e9a70ef4 Feature/node status dvpn directory (#5829)
* wip - dvpn directory cache

* Endpoint & cache

* /gateways works
- SkimmedNode data still missing
- need to move probe models to monorepo

* Rest of the data for /gateways

* Revert before merge: pin deps to cheddar release

* Filter gw by country

* Return percent string instead of u8

* Filter by semver

* Bump package version

* Fix probe types

* Reorg

* Add exit, entry endpoints

* Different entry/exit selection criteria

* Date fix migration

* Unpin from cheddar

* Revert "Unpin from cheddar"

This reverts commit f17239075b.

* Validation with celes

* PR feedback

* Fix path

* Bump version

---------

Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
2025-06-12 09:56:31 +02:00
Tommy Verrall 0c52ee89c8 Merge pull request #5821 from nymtech/dependabot/npm_and_yarn/sdk/typescript/tests/integration-tests/mix-fetch/tar-fs-3.0.9
build(deps): bump tar-fs from 3.0.8 to 3.0.9 in /sdk/typescript/tests/integration-tests/mix-fetch
2025-06-12 08:43:47 +01:00
Tommy Verrall f324d45721 Merge pull request #5449 from indmind/patch-1
chore: fixed typo in API endpoint parameter
2025-06-12 08:42:50 +01:00
Tommy Verrall 470e88f46c Merge pull request #5843 from nymtech/dependabot/npm_and_yarn/wasm/mix-fetch/internal-dev/webpack-dev-server-5.2.1
build(deps-dev): bump webpack-dev-server from 4.13.2 to 5.2.1 in /wasm/mix-fetch/internal-dev
2025-06-12 08:41:20 +01:00
Tommy Verrall 42a5016822 Merge pull request #5845 from nymtech/remove/old-mock-nym-api-client
remove not used old mock-api
2025-06-12 08:40:35 +01:00
Tommy Verrall 579cff358d Merge pull request #5849 from nymtech/feature/remove-browser-extension
Updated browser extension piece removal
2025-06-12 08:38:38 +01:00
benedetta davico f95dda0f2f Merge pull request #5844 from nymtech/feature/remove-bity
remove bity dir
2025-06-12 09:37:19 +02:00
benedetta davico fc666fb984 Merge pull request #5848 from nymtech/remove/old-env-references
Remove/old env references
2025-06-12 09:37:08 +02:00
benedetta davico 1264fd9bfb Update ci-build.yml 2025-06-11 17:48:24 +02:00
Tommy Verrall 3e8451f292 updated browser extension piece
- keep all the internal-dev wasm pieces as future examples
- everything previously was going to be removed
- shows functioning wasm interaction with the js
2025-06-11 17:15:20 +02:00
benedetta davico 53f4582202 Merge pull request #5835 from nymtech/benny/node-version-test
Update publish-nym-binaries.yml
2025-06-11 16:39:18 +02:00
benedettadavico c7c6dcab65 remove old env references 2025-06-11 16:13:59 +02:00
benedettadavico 3422c49e85 remove qa env 2025-06-11 16:07:32 +02:00
benedettadavico deee0b8e14 remove bity integration from cargo toml 2025-06-11 16:05:03 +02:00
benedettadavico 3ac58e0c49 Clean up
remove old explorer references
2025-06-11 16:02:19 +02:00
Tommy Verrall 7243cb57b5 remove not used old mock-api 2025-06-11 15:58:01 +02:00
Tommy Verrall 0276bd7b0b Merge pull request #5840 from nymtech/remove-testnet-faucet
Removing test-net faucet
2025-06-11 14:47:08 +01:00
Tommy Verrall 457759bb57 Merge pull request #5841 from nymtech/feature/add-buy-locations
Amended the buy section
2025-06-11 14:20:59 +01:00
dependabot[bot] de0f8ee2d3 build(deps): bump next from 14.2.15 to 14.2.26 in /documentation/docs (#5772)
Bumps [next](https://github.com/vercel/next.js) from 14.2.15 to 14.2.26.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v14.2.15...v14.2.26)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 14.2.26
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-11 13:36:30 +01:00
dependabot[bot] ebf97ece9b build(deps-dev): bump webpack-dev-server in /wasm/mix-fetch/internal-dev
Bumps [webpack-dev-server](https://github.com/webpack/webpack-dev-server) from 4.13.2 to 5.2.1.
- [Release notes](https://github.com/webpack/webpack-dev-server/releases)
- [Changelog](https://github.com/webpack/webpack-dev-server/blob/master/CHANGELOG.md)
- [Commits](https://github.com/webpack/webpack-dev-server/compare/v4.13.2...v5.2.1)

---
updated-dependencies:
- dependency-name: webpack-dev-server
  dependency-version: 5.2.1
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-11 12:23:47 +00:00
dependabot[bot] 50a55f4bfb build(deps-dev): bump webpack-dev-server (#5826)
Bumps [webpack-dev-server](https://github.com/webpack/webpack-dev-server) from 4.15.2 to 5.2.1.
- [Release notes](https://github.com/webpack/webpack-dev-server/releases)
- [Changelog](https://github.com/webpack/webpack-dev-server/blob/master/CHANGELOG.md)
- [Commits](https://github.com/webpack/webpack-dev-server/compare/v4.15.2...v5.2.1)

---
updated-dependencies:
- dependency-name: webpack-dev-server
  dependency-version: 5.2.1
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-11 13:22:07 +01:00
Tommy Verrall 4ee5c6457b remove images dir 2025-06-11 13:18:01 +02:00
Tommy Verrall d7b5fce7aa amended the buy section
- change the wallet to include the buy options for nym
- remove legacy code
2025-06-11 13:15:37 +02:00
benedetta davico c3d9c1131b Merge pull request #5838 from nymtech/release/2025.11-cheddar
merge release/2025.11-cheddar to develop
2025-06-11 13:09:57 +02:00
Tommy Verrall 5bdfb1ba5c removing test-net faucet 2025-06-11 12:00:44 +02:00
benedettadavico 94e51f0047 remove bity dir 2025-06-11 10:28:44 +02:00
benedetta davico f313e95e2f Merge pull request #5837 from nymtech/yana/replace-mintscan
Replace mintscan with ping.pub
2025-06-11 10:14:36 +02:00
Yana 2b13ac99b4 Replace mintscan with ping.pub 2025-06-10 19:34:19 +03:00
benedetta davico ef220882d4 update the workflow file again with a temp fix
reference: https://github.com/softprops/action-gh-release/issues/628
2025-06-10 11:39:20 +02:00
benedetta davico 59e26178ee Update publish-nym-binaries.yml 2025-06-10 11:20:19 +02:00
benedetta davico 0d420fb0a5 remove explorer-api in workflow 2025-06-10 11:01:24 +02:00
benedettadavico fce195fdba update changelog 2025-06-10 10:28:47 +02:00
Jędrzej Stuczyński 554b1eb022 bugfix: fix swapped total and circulating supplies (#5822) 2025-06-09 08:41:21 +01:00
Andrej Mihajlov e52bd918fb Hide tokio behind feature 2025-06-06 15:00:40 +02:00
Andrej Mihajlov 9d82d6d111 Hide tokio and sqlx behind not(wasm32) 2025-06-06 13:34:56 +02:00
Andrej Mihajlov 3593631e4a Exclude sqlx-pool-guard from wasm builds 2025-06-06 13:24:04 +02:00
import this 5b67403fb9 [DOCs/operators]: Add auto scraping of staking_supply_scale_factor & update api outputs (#5832) 2025-06-06 09:57:48 +00:00
Bogdan-Ștefan Neacşu 3a528d3b89 No autoremoval of peers (#5831)
* No autoremoval

* Remove startup_timestamp
2025-06-06 12:48:34 +03:00
Bogdan-Ștefan Neacşu 466bb97bc7 Use the same client bandwidth for top up (#5813)
* Use the same client bandwidth for top up

* Fix clippy
2025-06-06 10:12:50 +03:00
Simon Wicky 0d78416454 [Stats API] IP from nginx headers if available (#5830)
* proper IP handling

* workflow doesn't like fancy versions
2025-06-06 09:08:58 +02:00
Simon Wicky 8ba58ba11e [Feature] Noise XKpsk3 integration (2025 version) (#5692)
* grand Noise squaheroo

* fix merge conflicts and adapt code for the key rotation changes (#5824)

* remove file that should have been ignored
2025-06-05 11:34:55 +02:00
Simon Wicky be16fddc75 [Stats API] Infallible network view (#5825)
* infallible network view and cheddar model for current compatibility

* bump patch version

* typo
2025-06-04 17:08:44 +02:00
benedettadavico e9bb9792ab bump binaries 2025-06-04 14:42:04 +02:00
dependabot[bot] a7d6cba11d build(deps-dev): bump http-proxy-middleware (#5810)
Bumps [http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware) from 2.0.4 to 2.0.9.
- [Release notes](https://github.com/chimurai/http-proxy-middleware/releases)
- [Changelog](https://github.com/chimurai/http-proxy-middleware/blob/v2.0.9/CHANGELOG.md)
- [Commits](https://github.com/chimurai/http-proxy-middleware/compare/v2.0.4...v2.0.9)

---
updated-dependencies:
- dependency-name: http-proxy-middleware
  dependency-version: 2.0.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-04 13:42:41 +02:00
Andrej Mihajlov f5846d5bc2 Log all tracing output just in case 2025-06-04 11:40:56 +02:00
Bogdan-Ștefan Neacşu 88d4a9b111 Set cached storage counters to 0 (#5812)
* Set cached storage counters to 0

* u64 to i64 log possible error

* Check addition too
2025-06-04 12:11:46 +03:00
Andrej Mihajlov d7779df1b7 Include proc_pidinfo on iOS 2025-06-04 11:00:15 +02:00
dependabot[bot] a67ff33054 build(deps): bump tokio from 1.44.2 to 1.45.1 (#5798)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.44.2 to 1.45.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.44.2...tokio-1.45.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.45.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-04 10:21:00 +02:00
dependabot[bot] 61badfdcfe build(deps): bump undici in /.github/actions/nym-hash-releases/src (#5771)
Bumps [undici](https://github.com/nodejs/undici) from 5.28.5 to 5.29.0.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v5.28.5...v5.29.0)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 5.29.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-04 09:40:05 +02:00
dependabot[bot] 19dfbeb2b4 build(deps): bump cargo_metadata from 0.18.1 to 0.19.2 (#5765)
Bumps [cargo_metadata](https://github.com/oli-obk/cargo_metadata) from 0.18.1 to 0.19.2.
- [Release notes](https://github.com/oli-obk/cargo_metadata/releases)
- [Changelog](https://github.com/oli-obk/cargo_metadata/blob/main/CHANGELOG.md)
- [Commits](https://github.com/oli-obk/cargo_metadata/compare/0.18.1...0.19.2)

---
updated-dependencies:
- dependency-name: cargo_metadata
  dependency-version: 0.19.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-04 09:37:54 +02:00
dependabot[bot] 9f13616c24 build(deps): bump tempfile from 3.19.1 to 3.20.0 (#5764)
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.19.1 to 3.20.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.19.1...v3.20.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-version: 3.20.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-04 09:37:39 +02:00
Andrej Mihajlov 7fcc188041 Switch to tracing 2025-06-03 17:19:42 +02:00
Andrej Mihajlov b8c8d33c94 Use log here 2025-06-03 15:13:21 +02:00
Andrej Mihajlov 02909c03dd Expose database path 2025-06-03 14:49:49 +02:00
Jędrzej Stuczyński d8c84cc4d6 feat: key rotation (#5777)
* wip

* wip: wrap node's sphinx key with a manager

* wip: choosing correct key for packet processing

* further propagation of key rotation information

* attaching key rotation information to reply surbs

* added basic key rotation information to mixnet contract

* wip: introducing cached queries for key rotation info from nym api

* unified nym-api contract cache refreshing

* finish packet decoding

* multi api client + retrieving rotation id

* rotating sphinx key files

* logic for migrating config file

* wip: putting new sphinx keys to self described endpoints

* processing loop of KeyRotationController

* fixed sphinx key loading

* rotating bloomfilters

* wired up KeyRotationController

* flushing bloomfilters to disk and loading

* most of nym-node changes

* post rebase fixes

* fixes due to backwards compatible hostkeys

* split http state.rs file

* dont use deprecated fields

* fixed backwards compatible deserialisation of host information

* split up node describe cache

* added a dedicated CacheRefresher listener to perform full refresh outside the set interval

* controlling announced sphinx keys within nym-api

* retrieving rotation id when pulling topology

* split nym-nodes http handlers

* v2 nym-api endpoints to retrieve nodes with additional metadata information

* bug fixes...

* additional bugfixes and guards against stuck epoch

* testnet manager: set first nym-api as the rewarder

* fixed host information deserialisation

* fixed panic during first key rotation

* post rebase fixes

* clippy

* more guards against stuck epochs

* added helper method to reset node's sphinx key

* instantiate mixnet contract with custom key rotation validity

* additional bugfixes and debugging nym-api deadlock

* passing shutdown to nym apis client

* remove dead test

* post rebasing fixes

* missing MixnetQueryClient variants

* remove usage of deprecated methods in sdk example

* fix: incorrect method signature

* post rebasing fixes

* attempt to retrieve key rotation id before doing any config migration work

* ignore tests relying on networking behaviour

* allow networking failures in certain tests
2025-06-03 11:22:51 +01:00
Simon Wicky adbe0392ca Nym-statistics-api : Postgres schema and SSL handling + Dockerfile and GitHub action (#5817)
* add option for ssl mode

* add dockerfile and dev util

* add github workflow for nym-statistics api

* apply review comments

* ci check for version + removed checks from push
2025-06-03 12:06:00 +02:00
windy-ux 3c6567ae64 [DOCs]: redirectsl (#5816)
+ /docs/developers/tutorials/rust-sdk.html
2025-06-03 09:28:55 +00:00
Andrej Mihajlov 11262836d2 Clean up 2025-06-03 09:43:36 +02:00
Andrej Mihajlov f26fd5384d Improve windows 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 085103b333 Cleanup 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 574f7f1abd Revert 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 31e161604a Use sqlite pool guard 2025-06-03 09:43:36 +02:00
Andrej Mihajlov e4e349bea8 Remove logs 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 6391b7ed3a Document 2025-06-03 09:43:36 +02:00
Andrej Mihajlov c225511f95 Add Windows impl 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 4eedbb235a Add Windows implementation 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 548b8717b2 Update Linux impl 2025-06-03 09:43:36 +02:00
Andrej Mihajlov a215b3d0bf Open file watch 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 03d5a133eb Close sqlite pool before erroring 2025-06-03 09:43:36 +02:00
dependabot[bot] b323c62a6e build(deps): bump tar-fs
Bumps [tar-fs](https://github.com/mafintosh/tar-fs) from 3.0.8 to 3.0.9.
- [Commits](https://github.com/mafintosh/tar-fs/compare/v3.0.8...v3.0.9)

---
updated-dependencies:
- dependency-name: tar-fs
  dependency-version: 3.0.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-03 06:50:43 +00:00
Jack Wampler 8384a411df Bug Fix for Wallet build (#5820)
revert url used for connection-tester
2025-06-02 14:19:43 -06:00
Jack Wampler c56ebd9ceb Url scheme warning log (#5819)
fix conditions for logging about url scheme
2025-06-02 09:11:16 -06:00
Jędrzej Stuczyński b081b20a83 chore: adjust heuristic for wireguard peer activity (#5818)
* chore: adjust heuristic for wireguard peer activity

* fixed incorrect delta_tx calculation + typo
2025-06-02 15:37:37 +01:00
Andrej Mihajlov 866d547745 Merge pull request #5795 from nymtech/am/update-sqlx-0.8.6
Update to sqlx 0.8.6
2025-06-02 13:23:09 +02:00
Andrej Mihajlov 64e3f066a7 Use type override to enforce i64 type instead of Option<i64> 2025-05-30 10:17:19 +02:00
Andrej Mihajlov 62520c9308 Update sqlx cache 2025-05-30 09:28:48 +02:00
Andrej Mihajlov e65d455c91 Switch counters to i64 since sqlx started giving it back 2025-05-30 09:28:48 +02:00
Andrej Mihajlov 9b9c82a02a Run unchecked as sqlx does not understand COALESCE on NULL value 2025-05-30 09:28:48 +02:00
Andrej Mihajlov 1a38a2503e Stick to OffsetDateTime 2025-05-30 09:28:48 +02:00
Andrej Mihajlov 318f293983 All count() calls return i64 from now on 2025-05-30 09:28:48 +02:00
Andrej Mihajlov 5f2aba19c2 Update to sqlx 0.8.6 2025-05-30 09:28:48 +02:00
Jack Wampler 814ee45b4d HTTP Client Retries, Fallbacks, and Redirects (#5789)
updates to nym HTTP api client with multiple features relating to request domains
2025-05-29 10:37:07 -06:00
dynco-nym 56ed915626 Replace chrono with time in NS API (#5811)
* Replace chrono with time in NS API

* Replace chrono in client

* Bump package version
2025-05-29 16:33:00 +02:00
Jędrzej Stuczyński 2de8f8bc21 feature: nympool contract (#5464)
* squashed nym-pool commits

initialised nym-pool contract and updated all bls12_381 to make it possible

create scaffolding for tests

ability to control the contract admin

introducing contract grants

grant type validation

basic grant operations + stubs for other messages

added queries

use transaction stubs

added expiration information to grant queries

setting initial grant state based on the current environment

allowance logic for attempting to spend part of a grant

implemented all remaining transactions

made public api for coin locking perform validation

tests for locked tokens storage

nympool storage tests

added messages for changing granter set

tests and fixes for sufficient tokens when inserting grants

tests for initial state + more bugfixes

queries tests

additional tests for transactions and fixes

post rebase fixes

updated contract dependencies

removed redundant wasm constructor

dont ask me why this suddenly became an issue - no clue

removed redundant wasm constructor

dont ask me why this suddenly became an issue - no clue

* missing schema + added nym_pool to the main Makefile
2025-05-29 10:31:01 +01:00
import this f04cb6f6a6 [DOCs/operators]: Release notes v2025.10-brie (#5808)
* finish release notes and operator updates

* add NSL update - ready for merge

* address review comment
2025-05-28 11:59:35 +00:00
dynco-nym 4c67f01efb Make address cache configurable (#5784)
* Make address cache configurable

* TestFixture
2025-05-28 10:41:12 +02:00
Simon Wicky b69c2e1e94 Nym Statistics API (#5800)
* move stats types from vpn-client to here

* base stats api

* change storage schema

* add link to nymAPI for whitelisting

* remove outdated comment

* more comments update

* example of chrono vs time

* Add build.rs
- exports DATABASE_URL so cargo check works
- exports SQLX_OFFLINE for CI
- added pg_up.sh which spawns PG container
  - required for cargo sqlx prepare

* fixes time vs chrono issue and cleaner build with docker

* add correct swagger types, with feature locking where relevant

* apply dynco suggestions

---------

Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
2025-05-28 10:23:11 +02:00
benedetta davico d27e3b49db Merge pull request #5806 from nymtech/release/2025.10-brie 2025-05-28 09:38:36 +02:00
Jędrzej Stuczyński 0b92a59f1a hack: temporarily use next.config.js instead of next.config.ts (#5805) 2025-05-27 11:41:51 +01:00
Jędrzej Stuczyński 474eff67fa chore: adjusted wallet storybook mocks to fix the build (#5804) 2025-05-27 11:38:13 +01:00
Jędrzej Stuczyński 4a1ce8154a chore: resolve 1.87 clippy warnings (#5802)
* Clippy in wallet & sdk

* Clippy in wallet

* Pin rust to 1.86 in builder

* apply changes from b7da75a18c

* missing nym-node features

* Box all the things

* additional boxes in the wallet

* post rebasing clippy

---------

Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
2025-05-27 11:08:36 +01:00
Bogdan-Ștefan Neacşu aca98ab04f Track wireguard credential retries (#5783)
* Add a cache for the credentials seen before on top-up

* Verify seen credentials on top ups

* Add warning log for timestamp subtraction

* Add unit test
2025-05-27 12:35:44 +03:00
Jędrzej Stuczyński f925c6caf0 QoL: RequestPath trait for http-api-client (#5788)
* qol: RequestPath trait for http-api-client

* additional test case

* applied the change to other trait methods
2025-05-27 10:30:13 +01:00
Jon Häggblad 9a62581272 Update codeowners 2025-05-23 08:54:25 +02:00
Drazen Urch ebb8e4ef19 Build and push nym-api action (#5793) 2025-05-22 19:12:29 +02:00
mfahampshire a0057eb223 add notice re sdks (#5792)
* add notice re sdks

* fix borked notice

* fix another borked notice
2025-05-22 10:25:25 +00:00
import this 39195d79f5 [DOCs/operators]: Hotfix - Round decimalds to common convention (#5791) 2025-05-21 16:02:09 +00:00
import this ede5ffaffc [DOCs/operaotrs]: Automate Rewards calculator default state value (#5790) 2025-05-21 09:47:04 +00:00
Bogdan-Ștefan Neacşu ed16505137 Fix contains ticketbook function that always returned true (#5787) 2025-05-20 17:18:06 +03:00
Simon Wicky 03bec90b83 swap a decode into a fromrow to please future postgres feature (#5785)
* swap a decode into a fromrow to please future postgres feature

* add missing feature and missing crate in log filter
2025-05-20 15:48:09 +02:00
import this add57b2c14 [DOCs/operators]: Rewards calculator quick tweak (#5786) 2025-05-20 13:26:55 +00:00
import this 927ca8970c [DOCs/operators]: Tokenomics cleanup (#5782)
* correcting APY to ROI

* cleanup and small edits

* add tooltip on ROI
2025-05-19 11:12:36 +00:00
Jack Wampler 47d222b13d more relaxed usage of reqwest accept-encoding (#5779) 2025-05-16 13:03:24 -06:00
indmind d511aac301 chore: fixed typo in API endpoint parameter 2025-02-11 05:39:00 -06:00
1358 changed files with 49229 additions and 118352 deletions
+1 -11
View File
@@ -14,7 +14,6 @@
# contracts
/contracts/mixnet @durch @jstuczyn
/contracts/vesting @durch @jstuczyn
/contracts/service-provider-directory @octol
# crypto code
/common/crypto/ @jstuczyn
@@ -22,14 +21,5 @@
/common/dkg/ @jstuczyn
/common/nymsphinx/ @jstuczyn
# rust sdk
/sdk/rust/ @octol
# nym-connect (rust)
/nym-connect/desktop/src-tauri/ @octol
# nym-wallet (rust)
/nym-wallet/src-tauri/ @octol
# documentation
/documentation @mfahampshire
/documentation @mfahampshire
+3 -3
View File
@@ -415,9 +415,9 @@
}
},
"node_modules/undici": {
"version": "5.28.5",
"resolved": "https://registry.npmjs.org/undici/-/undici-5.28.5.tgz",
"integrity": "sha512-zICwjrDrcrUE0pyyJc1I2QzBkLM8FINsgOrt6WjA+BgajVq9Nxu2PbFFXUrAggLfDXlZGZBVZYw7WNV5KiBiBA==",
"version": "5.29.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-5.29.0.tgz",
"integrity": "sha512-raqeBD6NQK4SkWhQzeYKd1KmIG6dllBOTt55Rmkt4HtI9mwdWtJljnrXjAFUBLTSN67HWrOIZ3EPF4kjUw80Bg==",
"license": "MIT",
"dependencies": {
"@fastify/busboy": "^2.0.0"
+2 -2
View File
@@ -5,7 +5,6 @@ on:
paths:
- 'clients/**'
- 'common/**'
- 'explorer-api/**'
- 'gateway/**'
- 'integrations/**'
- 'nym-api/**'
@@ -13,6 +12,7 @@ on:
- 'nym-network-monitor/**'
- 'nym-node/**'
- 'nym-node-status-api/**'
- 'nym-statistics-api/**'
- 'nym-outfox/**'
- 'nym-validator-rewarder/**'
- 'nyx-chain-watcher/**'
@@ -38,7 +38,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ arc-ubuntu-22.04, custom-windows-11, custom-runner-mac-m1 ]
os: [ arc-ubuntu-22.04, custom-windows-11, custom-macos-15 ]
runs-on: ${{ matrix.os }}
env:
CARGO_TERM_COLOR: always
@@ -44,8 +44,10 @@ jobs:
echo "Tag is empty"
exit 1
fi
# first, list all tags for logging purposes
curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq --arg tag $TAG '.tags | contains([$tag])' )
# check if there's a matching tag
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq -r --arg tag "$TAG" 'any(.tags[]; . == $tag)' )
if [[ $exists = "true" ]]; then
echo "Version '$TAG' defined in Cargo.toml ALREADY EXISTS as tag in harbor repo"
exit 1
@@ -53,5 +55,5 @@ jobs:
echo "Version '$TAG' doesn't exist on the remote"
else
echo "Unknown output '$exists'"
exit 1
exit 2
fi
@@ -0,0 +1,59 @@
name: ci-check-nym-stats-api-version
on:
pull_request:
paths:
- "nym-statistics-api/**"
env:
WORKING_DIRECTORY: "nym-statistics-api"
jobs:
check-if-tag-exists:
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Checkout repo
uses: actions/checkout@v4
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Check if git tag exists
run: |
TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
if [[ -z "$TAG" ]]; then
echo "Tag is empty"
exit 1
fi
git ls-remote --tags origin | awk '{print $2}'
if git ls-remote --tags origin | awk '{print $2}' | grep -q "refs/tags/$TAG$" ; then
echo "Tag '$TAG' ALREADY EXISTS on the remote"
exit 1
else
echo "Tag '$TAG' does not exist on the remote"
fi
- name: Check if harbor tag exists
run: |
TAG=${{ steps.get_version.outputs.result }}
registry=https://harbor.nymte.ch
repo_name=nym/nym-statistics-api
if [[ -z $TAG ]]; then
echo "Tag is empty"
exit 1
fi
# first, list all tags for logging purposes
curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq
# check if there's a matching tag
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq -r --arg tag "$TAG" 'any(.tags[]; . == $tag)' )
if [[ $exists = "true" ]]; then
echo "Version '$TAG' defined in Cargo.toml ALREADY EXISTS as tag in harbor repo"
exit 1
elif [[ $exists = "false" ]]; then
echo "Version '$TAG' doesn't exist on the remote"
else
echo "Unknown output '$exists'"
exit 2
fi
@@ -31,31 +31,26 @@ jobs:
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
target: wasm32-unknown-unknown
override: true
- name: Install wasm-opt
uses: ./.github/actions/install-wasm-opt
with:
version: '114'
- name: Install cosmwasm-check
run: cargo install cosmwasm-check
- name: Build release contracts
run: make contracts
run: make publish-contracts
- name: Prepare build output
shell: bash
env:
OUTPUT_DIR: ci-contract-builds/${{ github.ref_name }}
run: |
cp contracts/target/wasm32-unknown-unknown/release/mixnet_contract.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/vesting_contract.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_coconut_dkg.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw3_flex_multisig.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw4_group.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_ecash.wasm $OUTPUT_DIR
find contracts/artifacts -maxdepth 1 -type f -name '*.wasm' -exec cp {} $OUTPUT_DIR \;
# Also include the optimizer-generated checksums if present
if [ -f contracts/artifacts/checksums.txt ]; then
cp contracts/artifacts/checksums.txt $OUTPUT_DIR
fi
- name: Deploy branch to CI www
continue-on-error: true
+1
View File
@@ -20,6 +20,7 @@ jobs:
runs-on: ubuntu-22.04
env:
CARGO_TERM_COLOR: always
RUSTUP_PERMIT_COPY_RENAME: 1
steps:
- uses: actions/checkout@v4
+19
View File
@@ -0,0 +1,19 @@
name: Run SonarQube Scan
on:
push:
branches:
- develop
# pull_request:
# types: [opened, synchronize, reopened]
jobs:
sonarqube:
name: SonarQube
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: SonarQube Scan
uses: SonarSource/sonarqube-scan-action@v5
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
+7 -5
View File
@@ -19,7 +19,11 @@ jobs:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-binaries-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
strategy:
fail-fast: false
runs-on: arc-ubuntu-22.04
matrix:
include:
- os: arc-ubuntu-22.04
target: x86_64-unknown-linux-gnu
runs-on: ${{ matrix.os }}
outputs:
release_id: ${{ steps.create-release.outputs.id }}
@@ -66,7 +70,6 @@ jobs:
with:
name: my-artifact
path: |
target/release/explorer-api
target/release/nym-client
target/release/nym-socks5-client
target/release/nym-api
@@ -75,14 +78,13 @@ jobs:
target/release/nymvisor
target/release/nym-node
retention-days: 30
- id: create-release
name: Upload to release based on tag name
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@da05d552573ad5aba039eaac05058a918a7bf631
if: github.event_name == 'release'
with:
files: |
target/release/explorer-api
target/release/nym-client
target/release/nym-socks5-client
target/release/nym-api
+12 -6
View File
@@ -49,6 +49,8 @@ jobs:
run: |
curl -L0 https://www.ssl.com/download/codesigntool-for-linux-and-macos/ -o codesigntool.zip
unzip codesigntool.zip
chmod +x CodeSignTool.sh
- name: Get EV certificate credential id
working-directory: nym-wallet/src-tauri
if: ${{ inputs.sign }}
@@ -56,6 +58,7 @@ jobs:
shell: bash
run: |
echo "SSL_COM_CREDENTIAL_ID=$(./CodeSignTool.sh get_credential_ids -username=${{ secrets.SSL_COM_USERNAME }} -password=${{ secrets.SSL_COM_PASSWORD }} | sed -n '1!p' | sed 's/- //')" >> "$GITHUB_OUTPUT"
- name: Add custom sign command to tauri.conf.json
working-directory: nym-wallet/src-tauri
if: ${{ inputs.sign }}
@@ -79,6 +82,7 @@ jobs:
]
}
}' tauri.conf.json
- name: Install project dependencies
shell: bash
run: cd .. && yarn --network-timeout 100000
@@ -93,12 +97,14 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_KEY_PASSWORD }}
SSL_COM_USERNAME: ${{ inputs.sign && secrets.SSL_COM_USERNAME }}
SSL_COM_PASSWORD: ${{ inputs.sign && secrets.SSL_COM_PASSWORD }}
SSL_COM_CREDENTIAL_ID: ${{ inputs.sign && steps.get_credential_ids.outputs.SSL_COM_CREDENTIAL_ID }}
SSL_COM_TOTP_SECRET: ${{ inputs.sign && secrets.SSL_COM_TOTP_SECRET }}
SSL_COM_USERNAME: ${{ inputs.sign && secrets.SSL_COM_USERNAME || '' }}
SSL_COM_PASSWORD: ${{ inputs.sign && secrets.SSL_COM_PASSWORD || '' }}
SSL_COM_CREDENTIAL_ID: ${{ inputs.sign && steps.get_credential_ids.outputs.SSL_COM_CREDENTIAL_ID || '' }}
SSL_COM_TOTP_SECRET: ${{ inputs.sign && secrets.SSL_COM_TOTP_SECRET || '' }}
CODE_SIGN_TOOL_PATH: ${{ inputs.sign && 'C:\\actions-runner\\_work\\nym\\nym\\nym-wallet\\src-tauri\\' || '' }}
run: |
echo "Starting build process..."
echo "Signing enabled: ${{ inputs.sign }}"
yarn build
- name: Check bundle directory
@@ -147,7 +153,7 @@ jobs:
nym-wallet/${{ env.BUNDLE_PATH }}/msi/*.msi.zip*
nym-wallet/${{ env.BUNDLE_PATH }}/*/nym-wallet*.msi
nym-wallet/src-tauri/target/release/bundle/msi/*.msi
- name: Find MSI path for deployment
id: find-msi
shell: bash
@@ -167,4 +173,4 @@ jobs:
needs: publish-tauri
with:
release_tag: ${{ needs.publish-tauri.outputs.release_tag || github.ref_name }}
secrets: inherit
secrets: inherit
+32 -12
View File
@@ -5,8 +5,15 @@ on:
inputs:
gateway_probe_git_ref:
type: string
default: nym-vpn-core-v1.4.0
required: true
description: Which gateway probe git ref to build the image with
release_image:
description: 'Tag image as a release'
required: true
default: false
type: boolean
env:
WORKING_DIRECTORY: "nym-node-status-api/nym-node-status-agent"
CONTAINER_NAME: "node-status-agent"
@@ -43,19 +50,32 @@ jobs:
GIT_REF_SLUG="${GATEWAY_PROBE_GIT_REF//\//-}"
echo "git_ref=${GIT_REF_SLUG}" >> $GITHUB_OUTPUT
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
fi
- name: Set GIT_TAG variable
run: echo "GIT_TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}" >> $GITHUB_ENV
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }} -m "Version ${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
- name: Set RELEASE_TAG variable
if: github.event.inputs.release_image == 'true'
run: echo "RELEASE_TAG=golden-" >> $GITHUB_ENV
- name: Set IMAGE_NAME_AND_TAGS variable
run: echo "IMAGE_NAME_AND_TAGS=${{ env.CONTAINER_NAME }}:${{ env.RELEASE_TAG }}${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}" >> $GITHUB_ENV
- name: New env vars
run: echo "RELEASE_TAG='$RELEASE_TAG' GIT_TAG='$GIT_TAG' IMAGE_NAME_AND_TAGS='$IMAGE_NAME_AND_TAGS'"
# - name: Remove existing tag if exists
# run: |
# if git rev-parse $${{ env.GIT_TAG }} >/dev/null 2>&1; then
# git push --delete origin $${{ env.GIT_TAG }}
# git tag -d $${{ env.GIT_TAG }}
# fi
# - name: Create tag
# run: |
# git tag -a $${{ env.GIT_TAG }} -m "Version ${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}"
# git push origin $${{ env.GIT_TAG }}
- name: BuildAndPushImageOnHarbor
run: |
docker build --build-arg GIT_REF=${{ github.event.inputs.gateway_probe_git_ref }} -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
docker build --build-arg GIT_REF=${{ github.event.inputs.gateway_probe_git_ref }} -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.IMAGE_NAME_AND_TAGS }}
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+33 -17
View File
@@ -1,7 +1,13 @@
name: Build and upload Node Status API container to harbor.nymte.ch
on:
workflow_dispatch:
inputs:
release_image:
description: 'Tag image as a release'
required: true
default: false
type: boolean
env:
WORKING_DIRECTORY: "nym-node-status-api/nym-node-status-api"
CONTAINER_NAME: "node-status-api"
@@ -31,25 +37,35 @@ jobs:
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Check if tag exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
echo "Tag ${{ steps.get_version.outputs.result }} already exists"
fi
- name: Set GIT_TAG variable
run: echo "GIT_TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}" >> $GITHUB_ENV
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
fi
- name: Set RELEASE_TAG variable
if: github.event.inputs.release_image == 'true'
run: echo "RELEASE_TAG=golden-" >> $GITHUB_ENV
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
- name: Set IMAGE_NAME_AND_TAGS variable
run: echo "IMAGE_NAME_AND_TAGS=${{ env.CONTAINER_NAME }}:${{ env.RELEASE_TAG }}${{ steps.get_version.outputs.result }}" >> $GITHUB_ENV
- name: New env vars
run: echo "RELEASE_TAG='$RELEASE_TAG' GIT_TAG='$GIT_TAG' IMAGE_NAME_AND_TAGS='$IMAGE_NAME_AND_TAGS'"
# - name: Remove existing tag if exists, then create
# run: |
# if git rev-parse "$GIT_TAG" >/dev/null 2>&1; then
# echo "Tag '$GIT_TAG' already exists, deleting"
# git push --delete origin "$GIT_TAG"
# git tag -d "$GIT_TAG"
# echo "Tag '$GIT_TAG' deleted"
# else
# echo "Tag '$GIT_TAG' does not exist, creating it"
# git tag -a $GIT_TAG -m "Version ${{ steps.get_version.outputs.result }}"
# git push origin $GIT_TAG
# echo "Tag '$GIT_TAG' created"
# fi
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.IMAGE_NAME_AND_TAGS }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+51
View File
@@ -0,0 +1,51 @@
name: Build and upload Nym APU container to harbor.nymte.ch
on:
workflow_dispatch:
env:
WORKING_DIRECTORY: "."
CONTAINER_NAME: "nym-api"
jobs:
build-container:
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: harbor.nymte.ch
username: ${{ secrets.HARBOR_ROBOT_USERNAME }}
password: ${{ secrets.HARBOR_ROBOT_SECRET }}
- name: Checkout repo
uses: actions/checkout@v4
- name: Configure git identity
run: |
git config --global user.email "lawrence@nymtech.net"
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-api/Cargo.toml
- name: Remove existing tag if exists
run: |
echo "Checking if tag ${{ env.CONTAINER_NAME }}-${{ steps.get_version.outputs.result }} exists..."
if git rev-parse ${{ env.CONTAINER_NAME }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
echo "Tag ${{ env.CONTAINER_NAME }}-${{ steps.get_version.outputs.result }} already exists"
git push --delete origin ${{ env.CONTAINER_NAME }}-${{ steps.get_version.outputs.result }}
git tag -d ${{ env.CONTAINER_NAME }}-${{ steps.get_version.outputs.result }}
fi
- name: Create tag
run: |
git tag -a ${{ env.CONTAINER_NAME }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.CONTAINER_NAME }}-${{ steps.get_version.outputs.result }}
- name: BuildAndPushImageOnHarbor
run: |
docker build -f nym-api.dockerfile ${{ env.WORKING_DIRECTORY }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
@@ -0,0 +1,42 @@
name: Build and upload Nym Statistics API container to harbor.nymte.ch
on:
workflow_dispatch:
env:
WORKING_DIRECTORY: "nym-statistics-api"
CONTAINER_NAME: "nym-statistics-api"
jobs:
build-container:
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: harbor.nymte.ch
username: ${{ secrets.HARBOR_ROBOT_USERNAME }}
password: ${{ secrets.HARBOR_ROBOT_SECRET }}
- name: Checkout repo
uses: actions/checkout@v4
- name: Configure git identity
run: |
git config --global user.email "lawrence@nymtech.net"
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+2 -1
View File
@@ -35,12 +35,13 @@ validator-api/keypair
contracts/mixnet/code_id
contracts/mixnet/Justfile
contracts/mixnet/Makefile
artifacts
contracts/artifacts
validator-config
*.patch
validator-api-config.toml
dist
storybook-static
envs/qwerty.env
.parcel-cache
**/.DS_Store
cpu-cycles/libcpucycles/build
+92
View File
@@ -4,6 +4,98 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
## [Unreleased]
## [2025.12-dolcelatte] (2025-07-07)
- bugfix: key-rotation + reply SURBs ([#5876])
- Bugfix/backwards compat ([#5865])
- bugfix: allow gateways to permit authentication from v4 clients ([#5862])
- fixed client route for obtaining v2 list of gateways ([#5859])
- Updated browser extension piece removal ([#5849])
- Remove/old env references ([#5848])
- Remove qa env ([#5847])
- remove not used old mock-api ([#5845])
- remove bity dir ([#5844])
- build(deps-dev): bump webpack-dev-server from 4.13.2 to 5.2.1 in /wasm/mix-fetch/internal-dev ([#5843])
- Amended the buy section ([#5841])
- Removing test-net faucet ([#5840])
- Feature/node status dvpn directory ([#5829])
- build(deps-dev): bump webpack-dev-server from 4.15.2 to 5.2.1 in /nym-credential-proxy/vpn-api-lib-wasm/internal-dev ([#5826])
- bugfix: fix swapped total and circulating supplies ([#5822])
- build(deps): bump tar-fs from 3.0.8 to 3.0.9 in /sdk/typescript/tests/integration-tests/mix-fetch ([#5821])
- Url scheme warning log ([#5819])
- chore: adjust heuristic for wireguard peer activity ([#5818])
- Use the same client bandwidth for top up ([#5813])
- Replace chrono with time in NS API ([#5811])
- build(deps-dev): bump http-proxy-middleware from 2.0.4 to 2.0.9 in /clients/native/examples/js-examples/websocket ([#5810])
- build(deps): bump tokio from 1.44.2 to 1.45.1 ([#5798])
- Close sqlite pool before moving or reopening databases ([#5796])
- HTTP Client Retries, Fallbacks, and Redirects ([#5789])
- feat: key rotation ([#5777])
- build(deps): bump next from 14.2.15 to 14.2.26 in /documentation/docs ([#5772])
- build(deps): bump undici from 5.28.5 to 5.29.0 in /.github/actions/nym-hash-releases/src ([#5771])
- build(deps): bump cargo_metadata from 0.18.1 to 0.19.2 ([#5765])
- build(deps): bump tempfile from 3.19.1 to 3.20.0 ([#5764])
- [Feature] Noise XKpsk3 integration (2025 version) ([#5692])
- feature: nympool contract ([#5464])
- chore: fixed typo in API endpoint parameter ([#5449])
[#5876]: https://github.com/nymtech/nym/pull/5876
[#5865]: https://github.com/nymtech/nym/pull/5865
[#5862]: https://github.com/nymtech/nym/pull/5862
[#5859]: https://github.com/nymtech/nym/pull/5859
[#5849]: https://github.com/nymtech/nym/pull/5849
[#5848]: https://github.com/nymtech/nym/pull/5848
[#5847]: https://github.com/nymtech/nym/pull/5847
[#5845]: https://github.com/nymtech/nym/pull/5845
[#5844]: https://github.com/nymtech/nym/pull/5844
[#5843]: https://github.com/nymtech/nym/pull/5843
[#5841]: https://github.com/nymtech/nym/pull/5841
[#5840]: https://github.com/nymtech/nym/pull/5840
[#5829]: https://github.com/nymtech/nym/pull/5829
[#5826]: https://github.com/nymtech/nym/pull/5826
[#5822]: https://github.com/nymtech/nym/pull/5822
[#5821]: https://github.com/nymtech/nym/pull/5821
[#5819]: https://github.com/nymtech/nym/pull/5819
[#5818]: https://github.com/nymtech/nym/pull/5818
[#5813]: https://github.com/nymtech/nym/pull/5813
[#5811]: https://github.com/nymtech/nym/pull/5811
[#5810]: https://github.com/nymtech/nym/pull/5810
[#5798]: https://github.com/nymtech/nym/pull/5798
[#5796]: https://github.com/nymtech/nym/pull/5796
[#5789]: https://github.com/nymtech/nym/pull/5789
[#5777]: https://github.com/nymtech/nym/pull/5777
[#5772]: https://github.com/nymtech/nym/pull/5772
[#5771]: https://github.com/nymtech/nym/pull/5771
[#5765]: https://github.com/nymtech/nym/pull/5765
[#5764]: https://github.com/nymtech/nym/pull/5764
[#5692]: https://github.com/nymtech/nym/pull/5692
[#5464]: https://github.com/nymtech/nym/pull/5464
[#5449]: https://github.com/nymtech/nym/pull/5449
## [2025.11-cheddar] (2025-06-10)
- No autoremoval of peers ([#5831])
- Set cached storage counters to 0 ([#5812])
- hack: temporarily use next.config.js instead of next.config.ts ([#5805])
- chore: resolve 1.87 clippy warnings ([#5802])
- Nym Statistics API ([#5800])
- QoL: RequestPath trait for http-api-client ([#5788])
- Fix contains ticketbook function that always returned true ([#5787])
- swap a decode into a fromrow to please future postgres feature ([#5785])
- Make address cache configurable ([#5784])
- Track wireguard credential retries ([#5783])
[#5831]: https://github.com/nymtech/nym/pull/5831
[#5812]: https://github.com/nymtech/nym/pull/5812
[#5805]: https://github.com/nymtech/nym/pull/5805
[#5802]: https://github.com/nymtech/nym/pull/5802
[#5800]: https://github.com/nymtech/nym/pull/5800
[#5788]: https://github.com/nymtech/nym/pull/5788
[#5787]: https://github.com/nymtech/nym/pull/5787
[#5785]: https://github.com/nymtech/nym/pull/5785
[#5784]: https://github.com/nymtech/nym/pull/5784
[#5783]: https://github.com/nymtech/nym/pull/5783
## [2025.10-brie] (2025-05-27)
- Backport PR 5779 ([#5801])
Generated
+1553 -608
View File
File diff suppressed because it is too large Load Diff
+19 -12
View File
@@ -33,11 +33,14 @@ members = [
"common/commands",
"common/config",
"common/cosmwasm-smart-contracts/coconut-dkg",
"common/cosmwasm-smart-contracts/contracts-common", "common/cosmwasm-smart-contracts/easy_addr",
"common/cosmwasm-smart-contracts/contracts-common",
"common/cosmwasm-smart-contracts/contracts-common-testing",
"common/cosmwasm-smart-contracts/easy_addr",
"common/cosmwasm-smart-contracts/ecash-contract",
"common/cosmwasm-smart-contracts/group-contract",
"common/cosmwasm-smart-contracts/mixnet-contract",
"common/cosmwasm-smart-contracts/multisig-contract",
"common/cosmwasm-smart-contracts/multisig-contract", "common/cosmwasm-smart-contracts/nym-performance-contract",
"common/cosmwasm-smart-contracts/nym-pool-contract",
"common/cosmwasm-smart-contracts/vesting-contract",
"common/credential-storage",
"common/credential-utils",
@@ -64,6 +67,8 @@ members = [
"common/nym-id",
"common/nym-metrics",
"common/nym_offline_compact_ecash",
"common/nymnoise",
"common/nymnoise/keys",
"common/nymsphinx",
"common/nymsphinx/acknowledgements",
"common/nymsphinx/addressing",
@@ -97,7 +102,6 @@ members = [
"common/wireguard-types",
"documentation/autodoc",
"gateway",
"integrations/bity",
"nym-api",
"nym-api/nym-api-requests",
"nym-browser-extension/storage",
@@ -112,6 +116,7 @@ members = [
"nym-node/nym-node-metrics",
"nym-node/nym-node-requests",
"nym-outfox",
"nym-statistics-api",
"nym-validator-rewarder",
"nyx-chain-watcher",
"sdk/ffi/cpp",
@@ -122,6 +127,7 @@ members = [
"service-providers/common",
"service-providers/ip-packet-router",
"service-providers/network-requester",
"sqlx-pool-guard",
"tools/echo-server",
"tools/internal/contract-state-importer/importer-cli",
"tools/internal/contract-state-importer/importer-contract",
@@ -131,7 +137,7 @@ members = [
"tools/internal/testnet-manager",
"tools/internal/testnet-manager",
"tools/internal/testnet-manager/dkg-bypass-contract",
"tools/internal/testnet-manager/dkg-bypass-contract", "tools/internal/validator-status-check",
"tools/internal/validator-status-check",
"tools/nym-cli",
"tools/nym-id-cli",
"tools/nym-nr-query",
@@ -152,6 +158,7 @@ default-members = [
"nym-node",
"nym-node-status-api/nym-node-status-agent",
"nym-node-status-api/nym-node-status-api",
"nym-statistics-api",
"nym-validator-rewarder",
"nyx-chain-watcher",
"service-providers/authenticator",
@@ -198,7 +205,7 @@ bloomfilter = "3.0.1"
bs58 = "0.5.1"
bytecodec = "0.4.15"
bytes = "1.10.1"
cargo_metadata = "0.18.1"
cargo_metadata = "0.19.2"
celes = "2.6.0"
cfg-if = "1.0.0"
chacha20 = "0.9.0"
@@ -281,6 +288,7 @@ petgraph = "0.6.5"
pin-project = "1.1"
pin-project-lite = "0.2.16"
publicsuffix = "2.3.0"
proc_pidinfo = "0.1.3"
quote = "1"
rand = "0.8.5"
rand_chacha = "0.3"
@@ -305,8 +313,9 @@ serde_with = "3.9.0"
serde_yaml = "0.9.25"
sha2 = "0.10.9"
si-scale = "0.2.3"
snow = "0.9.6"
sphinx-packet = "=0.6.0"
sqlx = "0.7.4"
sqlx = "0.8.6"
strum = "0.26"
strum_macros = "0.26"
subtle-encoding = "0.5"
@@ -314,10 +323,10 @@ syn = "1"
sysinfo = "0.33.0"
tap = "1.0.1"
tar = "0.4.44"
tempfile = "3.19"
tempfile = "3.20"
thiserror = "2.0"
time = "0.3.41"
tokio = "1.44"
tokio = "1.45"
tokio-postgres = "0.7"
tokio-stream = "0.1.17"
tokio-test = "0.4.4"
@@ -344,7 +353,6 @@ utoipauto = "0.2"
uuid = "*"
vergen = { version = "=8.3.1", default-features = false }
walkdir = "2"
wasm-bindgen-test = "0.3.49"
x25519-dalek = "2.0.0"
zeroize = "1.7.0"
@@ -362,9 +370,6 @@ subtle = "2.5.0"
# cosmwasm-related
cosmwasm-schema = "=2.2.2"
cosmwasm-std = "=2.2.2"
# use 1.0.1 as that's the version used by cosmwasm-std 2.2.1
# (and ideally we don't want to pull the same dependency twice)
serde-json-wasm = "=1.0.1"
# same version as used by cosmwasm
cw-utils = "=2.0.0"
cw-storage-plus = "=2.0.0"
@@ -372,6 +377,7 @@ cw2 = { version = "=2.0.0" }
cw3 = { version = "=2.0.0" }
cw4 = { version = "=2.0.0" }
cw-controllers = { version = "=2.0.0" }
cw-multi-test = "=2.3.2"
# cosmrs-related
bip32 = { version = "0.5.3", default-features = false }
@@ -392,6 +398,7 @@ serde-wasm-bindgen = "0.6.5"
tsify = "0.4.5"
wasm-bindgen = "0.2.99"
wasm-bindgen-futures = "0.4.49"
wasm-bindgen-test = "0.3.49"
wasmtimer = "0.4.1"
web-sys = "0.3.76"
+58 -10
View File
@@ -12,7 +12,11 @@ help:
@echo " clippy: run clippy for all workspaces"
@echo " test: run clippy, unit tests, and formatting."
@echo " test-all: like test, but also includes the expensive tests"
@echo " deb: build debian packages
@echo " deb: build debian packages"
@echo ""
@echo "Contract building targets:"
@echo " contracts: build contracts for development (includes wasm-opt)"
@echo " publish-contracts: build contracts using Docker optimizer (deterministic)"
# -----------------------------------------------------------------------------
# Meta targets
@@ -130,25 +134,69 @@ cargo-test: sdk-wasm-test
clippy: sdk-wasm-lint
# -----------------------------------------------------------------------------
# Build contracts ready for deploy
# Build CosmWasm contracts (deterministic docker build)
# -----------------------------------------------------------------------------
CONTRACTS=vesting_contract mixnet_contract nym_ecash cw3_flex_multisig cw4_group nym_coconut_dkg
CONTRACTS_WASM=$(addsuffix .wasm, $(CONTRACTS))
CONTRACTS_OUT_DIR=contracts/target/wasm32-unknown-unknown/release
contracts: build-release-contracts wasm-opt-contracts cosmwasm-check-contracts
WASM_CONTRACT_DIR := contracts/target/wasm32-unknown-unknown/release
# Find every direct contract folder that contains a Cargo.toml
CONTRACT_DIRS := $(shell find contracts -type f -name Cargo.toml \( ! -path "contracts/Cargo.toml" \) | grep -v integration-tests | xargs -n1 dirname | sort -u)
CONTRACTS_OUT_DIR = contracts/artifacts
# Build all contracts via the official CosmWasm optimizer image (one invocation per contract)
# See : https://github.com/CosmWasm/optimizer?tab=readme-ov-file#contracts-excluded-from-workspace
# The optimizer ships separate multi-arch images. ARM builds are *not* bit-for-bit identical to the
# canonical x86_64 build (see README notice in CosmWasm/optimizer). For reproducible artefacts we
# therefore always run the amd64 variant by default.
# Override with :
# $ COSMWASM_OPTIMIZER_IMAGE=cosmwasm/optimizer-arm64:0.17.0 make contracts-publish
#
COSMWASM_OPTIMIZER_IMAGE ?= cosmwasm/optimizer:0.17.0
COSMWASM_OPTIMIZER_PLATFORM ?= linux/amd64
# Ensure clean build environment and run the optimizer
optimize-contracts:
@rm -rf artifacts 2>/dev/null || true
@echo "=== Ensuring clean build environment"
docker volume rm nym_contracts_cache 2>/dev/null || true
docker volume rm registry_cache 2>/dev/null || true
@for DIR in $(CONTRACT_DIRS); do \
echo "=== Optimizing $${DIR}"; \
docker run --rm --platform $(COSMWASM_OPTIMIZER_PLATFORM) \
-v $(CURDIR):/code \
--mount type=volume,source=nym_contracts_cache,target=/target \
--mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
-e CARGO_BUILD_INCREMENTAL=false \
-e RUSTFLAGS="-C target-cpu=generic -C debuginfo=0" \
-e SOURCE_DATE_EPOCH=1 \
$(COSMWASM_OPTIMIZER_IMAGE) $${DIR}; \
done
@mkdir -p $(CONTRACTS_OUT_DIR)
@cp artifacts/*.wasm $(CONTRACTS_OUT_DIR)/ 2>/dev/null || true
@cd $(CONTRACTS_OUT_DIR) && sha256sum *.wasm > checksums.txt
# Cleanup temporary artefacts directory
@rm -rf artifacts 2>/dev/null || true
wasm-opt-contracts:
for contract in $(CONTRACTS_WASM); do \
wasm-opt --signext-lowering -Os $(CONTRACTS_OUT_DIR)/$$contract -o $(CONTRACTS_OUT_DIR)/$$contract; \
@for WASM in $(WASM_CONTRACT_DIR)/*.wasm; do \
echo "Running wasm-opt on $$WASM"; \
wasm-opt --signext-lowering -Os $$WASM -o $$WASM ; \
done
cosmwasm-check-contracts:
for contract in $(CONTRACTS_WASM); do \
cosmwasm-check $(CONTRACTS_OUT_DIR)/$$contract; \
@for WASM in $(WASM_CONTRACT_DIR)/*.wasm; do \
echo "Checking $$WASM"; \
cosmwasm-check $$WASM ; \
done
# Default development build
contracts: build-release-contracts wasm-opt-contracts cosmwasm-check-contracts
# Publishing build used by CI deterministic Docker optimiser
publish-contracts: optimize-contracts cosmwasm-check-contracts
# Consider adding 's' to make plural consistent (beware: used in github workflow)
contract-schema:
$(MAKE) -C contracts schema
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-client"
version = "1.1.56"
version = "1.1.58"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
description = "Implementation of the Nym Client"
edition = "2021"
@@ -2048,10 +2048,11 @@
}
},
"node_modules/http-proxy-middleware": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.4.tgz",
"integrity": "sha512-m/4FxX17SUvz4lJ5WPXOHDUuCwIqXLfLHs1s0uZ3oYjhoXlx9csYxaOa0ElDEJ+h8Q4iJ1s+lTMbiCa4EXIJqg==",
"version": "2.0.9",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.9.tgz",
"integrity": "sha512-c1IyJYLYppU574+YI7R4QyX2ystMtVXZwIdzazUIPIJsHuWNd+mho2j+bKoHftndicGj9yh+xjd+l0yj7VeT1Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/http-proxy": "^1.17.8",
"http-proxy": "^1.18.1",
@@ -6095,9 +6096,9 @@
}
},
"http-proxy-middleware": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.4.tgz",
"integrity": "sha512-m/4FxX17SUvz4lJ5WPXOHDUuCwIqXLfLHs1s0uZ3oYjhoXlx9csYxaOa0ElDEJ+h8Q4iJ1s+lTMbiCa4EXIJqg==",
"version": "2.0.9",
"resolved": "https://registry.npmjs.org/http-proxy-middleware/-/http-proxy-middleware-2.0.9.tgz",
"integrity": "sha512-c1IyJYLYppU574+YI7R4QyX2ystMtVXZwIdzazUIPIJsHuWNd+mho2j+bKoHftndicGj9yh+xjd+l0yj7VeT1Q==",
"dev": true,
"requires": {
"@types/http-proxy": "^1.17.8",
+1 -1
View File
@@ -318,7 +318,7 @@ impl Handler {
async fn handle_text_message(&mut self, msg: String) -> Option<WsMessage> {
debug!("Handling text message request");
trace!("Content: {:?}", msg);
trace!("Content: {msg:?}");
self.received_response_type = ReceivedResponseType::Text;
let client_request = ClientRequest::try_from_text(msg);
+2 -2
View File
@@ -68,9 +68,9 @@ impl Listener {
new_conn = tcp_listener.accept() => {
match new_conn {
Ok((mut socket, remote_addr)) => {
debug!("Received connection from {:?}", remote_addr);
debug!("Received connection from {remote_addr:?}");
if self.state.is_connected() {
warn!("Tried to open a duplicate websocket connection. The request came from {}", remote_addr);
warn!("Tried to open a duplicate websocket connection. The request came from {remote_addr}");
// if we've already got a connection, don't allow another one
// while we only ever want to accept a single connection, we don't want
// to leave clients hanging (and also allow for reconnection if it somehow
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-socks5-client"
version = "1.1.56"
version = "1.1.58"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
description = "A SOCKS5 localhost proxy that converts incoming messages to Sphinx and sends them to a Nym address"
edition = "2021"
+1 -1
View File
@@ -137,7 +137,7 @@ impl AsyncFileWatcher {
log::error!("the file watcher receiver has been dropped!");
}
} else {
log::debug!("will not propagate information about {:?}", event);
log::debug!("will not propagate information about {event:?}");
}
}
Err(err) => {
@@ -108,7 +108,7 @@ impl GatewayClient {
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key
let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
let dh = gateway_key.inner().diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
@@ -117,7 +117,7 @@ impl GatewayClient {
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key
let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
let dh = gateway_key.inner().diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
@@ -117,7 +117,7 @@ impl GatewayClient {
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key
let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
let dh = gateway_key.inner().diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
@@ -169,7 +169,7 @@ impl GatewayClient {
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key
let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
let dh = gateway_key.inner().diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
@@ -169,7 +169,7 @@ impl GatewayClient {
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// use gateways key as a ref to an x25519_dalek key
let dh = (gateway_key.as_ref()).diffie_hellman(&self.pub_key);
let dh = gateway_key.inner().diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
+1 -1
View File
@@ -11,7 +11,7 @@ impl std::fmt::Display for BandwidthStatusMessage {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
BandwidthStatusMessage::RemainingBandwidth(b) => {
write!(f, "remaining bandwidth: {}", b)
write!(f, "remaining bandwidth: {b}")
}
BandwidthStatusMessage::NoBandwidth => write!(f, "no bandwidth left"),
}
+5 -6
View File
@@ -15,8 +15,7 @@ bs58 = { workspace = true }
clap = { workspace = true, optional = true }
comfy-table = { workspace = true, optional = true }
futures = { workspace = true }
humantime-serde = { workspace = true }
log = { workspace = true }
humantime = { workspace = true }
rand = { workspace = true }
rand_chacha = { workspace = true }
serde = { workspace = true, features = ["derive"] }
@@ -25,26 +24,23 @@ sha2 = { workspace = true }
si-scale = { workspace = true }
thiserror = { workspace = true }
url = { workspace = true, features = ["serde"] }
tokio = { workspace = true, features = ["macros"] }
time = { workspace = true }
tokio = { workspace = true, features = ["sync", "macros"] }
tracing = { workspace = true }
zeroize = { workspace = true }
# internal
nym-id = { path = "../nym-id" }
nym-bandwidth-controller = { path = "../bandwidth-controller" }
nym-config = { path = "../config" }
nym-crypto = { path = "../crypto" }
nym-gateway-client = { path = "../client-libs/gateway-client" }
nym-gateway-requests = { path = "../gateway-requests" }
nym-http-api-client = { path = "../http-api-client" }
nym-metrics = { path = "../nym-metrics" }
nym-nonexhaustive-delayqueue = { path = "../nonexhaustive-delayqueue" }
nym-sphinx = { path = "../nymsphinx" }
nym-statistics-common = { path = "../statistics" }
nym-pemstore = { path = "../pemstore" }
nym-topology = { path = "../topology", features = ["persistence"] }
nym-mixnet-client = { path = "../client-libs/mixnet-client", default-features = false }
nym-validator-client = { path = "../client-libs/validator-client", default-features = false }
nym-task = { path = "../task" }
nym-credentials-interface = { path = "../credentials-interface" }
@@ -57,6 +53,9 @@ nym-client-core-surb-storage = { path = "./surb-storage" }
nym-client-core-gateways-storage = { path = "./gateways-storage" }
nym-ecash-time = { path = "../ecash-time" }
[target."cfg(not(target_arch = \"wasm32\"))".dependencies]
nym-mixnet-client = { path = "../client-libs/mixnet-client", default-features = false }
### For serving prometheus metrics
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.hyper]
workspace = true
+8 -12
View File
@@ -57,9 +57,7 @@ const DEFAULT_MAXIMUM_ALLOWED_SURB_REQUEST_SIZE: u32 = 500;
const DEFAULT_MAXIMUM_REPLY_SURB_REREQUEST_WAITING_PERIOD: Duration = Duration::from_secs(10);
const DEFAULT_MAXIMUM_REPLY_SURB_DROP_WAITING_PERIOD: Duration = Duration::from_secs(5 * 60);
// 12 hours
const DEFAULT_MAXIMUM_REPLY_SURB_AGE: Duration = Duration::from_secs(12 * 60 * 60);
const DEFAULT_MAXIMUM_REPLY_SURB_REREQUESTS: usize = 5;
// 24 hours
const DEFAULT_MAXIMUM_REPLY_KEY_AGE: Duration = Duration::from_secs(24 * 60 * 60);
@@ -418,6 +416,9 @@ pub struct Traffic {
/// will be routed as usual, to the entry gateway, through three mix nodes, egressing
/// through the exit gateway. If mix hops are disabled, traffic will be routed directly
/// from the entry gateway to the exit gateway, bypassing the mix nodes.
///
/// This overrides the `use_legacy_sphinx_format` setting as reduced mix hops
/// requires use of the updated SURB packet format.
pub disable_mix_hops: bool,
}
@@ -625,10 +626,9 @@ pub struct ReplySurbs {
#[serde(with = "humantime_serde")]
pub maximum_reply_surb_drop_waiting_period: Duration,
/// Defines maximum amount of time given reply surb is going to be valid for.
/// This is going to be superseded by key rotation once implemented.
#[serde(with = "humantime_serde")]
pub maximum_reply_surb_age: Duration,
/// Defines maximum number of times the client is going to re-request reply surbs
/// for clearing pending messages before giving up after making no progress.
pub maximum_reply_surbs_rerequests: usize,
/// Defines maximum amount of time given reply key is going to be valid for.
/// This is going to be superseded by key rotation once implemented.
@@ -638,9 +638,6 @@ pub struct ReplySurbs {
/// Specifies the number of mixnet hops the packet should go through. If not specified, then
/// the default value is used.
pub surb_mix_hops: Option<u8>,
/// Specifies if we should reset all the sender tags on startup
pub fresh_sender_tags: bool,
}
impl Default for ReplySurbs {
@@ -655,10 +652,9 @@ impl Default for ReplySurbs {
maximum_reply_surb_rerequest_waiting_period:
DEFAULT_MAXIMUM_REPLY_SURB_REREQUEST_WAITING_PERIOD,
maximum_reply_surb_drop_waiting_period: DEFAULT_MAXIMUM_REPLY_SURB_DROP_WAITING_PERIOD,
maximum_reply_surb_age: DEFAULT_MAXIMUM_REPLY_SURB_AGE,
maximum_reply_surbs_rerequests: DEFAULT_MAXIMUM_REPLY_SURB_REREQUESTS,
maximum_reply_key_age: DEFAULT_MAXIMUM_REPLY_KEY_AGE,
surb_mix_hops: None,
fresh_sender_tags: false,
}
}
}
@@ -189,14 +189,13 @@ impl From<ConfigV6> for Config {
.debug
.reply_surbs
.maximum_reply_surb_drop_waiting_period,
maximum_reply_surb_age: value.debug.reply_surbs.maximum_reply_surb_age,
maximum_reply_key_age: value.debug.reply_surbs.maximum_reply_key_age,
surb_mix_hops: value.debug.reply_surbs.surb_mix_hops,
minimum_reply_surb_threshold_buffer: value
.debug
.reply_surbs
.minimum_reply_surb_threshold_buffer,
fresh_sender_tags: value.debug.reply_surbs.fresh_sender_tags,
..Default::default()
},
stats_reporting: StatsReporting {
enabled: value.debug.stats_reporting.enabled,
@@ -9,11 +9,11 @@ license.workspace = true
[dependencies]
async-trait.workspace = true
cosmrs.workspace = true
log.workspace = true
serde = { workspace = true, features = ["derive"] }
thiserror.workspace = true
time.workspace = true
tokio = { workspace = true, features = ["sync"] }
tracing.workspace = true
url.workspace = true
zeroize = { workspace = true, features = ["zeroize_derive"] }
@@ -2,8 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use crate::BadGateway;
use std::io;
use std::path::PathBuf;
use std::{io, path::PathBuf};
use thiserror::Error;
#[derive(Debug, Error)]
@@ -19,7 +18,6 @@ pub enum StorageError {
#[error("failed to perform sqlx migration: {source}")]
MigrationError {
#[source]
#[from]
source: sqlx::migrate::MigrateError,
},
@@ -32,7 +30,6 @@ pub enum StorageError {
#[error("failed to run the SQL query: {source}")]
QueryError {
#[source]
#[from]
source: sqlx::error::Error,
},
@@ -7,12 +7,12 @@ use crate::{
RawActiveGateway, RawCustomGatewayDetails, RawRegisteredGateway, RawRemoteGatewayDetails,
},
};
use log::{debug, error};
use sqlx::{
sqlite::{SqliteAutoVacuum, SqliteSynchronous},
ConnectOptions,
};
use std::path::Path;
use tracing::{debug, error};
#[derive(Debug, Clone)]
pub struct StorageManager {
@@ -87,7 +87,7 @@ impl StorageManager {
sqlx::query!("SELECT EXISTS (SELECT 1 FROM registered_gateway WHERE gateway_id_bs58 = ?) AS 'exists'", gateway_id)
.fetch_one(&self.connection_pool)
.await
.map(|result| result.exists == Some(1))
.map(|result| result.exists == 1)
}
pub(crate) async fn maybe_get_registered_gateway(
@@ -12,12 +12,12 @@ use crate::{
error::ClientCoreError,
init::types::{GatewaySelectionSpecification, GatewaySetup},
};
use log::info;
use nym_client_core_gateways_storage::GatewayDetails;
use nym_crypto::asymmetric::ed25519;
use nym_topology::NymTopology;
use nym_validator_client::UserAgent;
use std::path::PathBuf;
use tracing::info;
#[cfg_attr(feature = "cli", derive(clap::Args))]
#[derive(Debug, Clone)]
@@ -81,14 +81,14 @@ where
// Attempt to use a user-provided gateway, if possible
let user_chosen_gateway_id = common_args.gateway_id;
log::debug!("User chosen gateway id: {user_chosen_gateway_id:?}");
tracing::debug!("User chosen gateway id: {user_chosen_gateway_id:?}");
let selection_spec = GatewaySelectionSpecification::new(
user_chosen_gateway_id.map(|id| id.to_base58_string()),
Some(common_args.latency_based_selection),
common_args.force_tls_gateway,
);
log::debug!("Gateway selection specification: {selection_spec:?}");
tracing::debug!("Gateway selection specification: {selection_spec:?}");
let registered_gateways = get_all_registered_identities(&details_store).await?;
@@ -12,7 +12,6 @@ use crate::{
},
init::types::{GatewaySelectionSpecification, GatewaySetup, InitResults},
};
use log::info;
use nym_client_core_gateways_storage::GatewayDetails;
use nym_crypto::asymmetric::ed25519;
use nym_sphinx::addressing::Recipient;
@@ -20,6 +19,7 @@ use nym_topology::NymTopology;
use nym_validator_client::UserAgent;
use rand::rngs::OsRng;
use std::path::PathBuf;
use tracing::info;
// we can suppress this warning (as suggested by linter itself) since we're only using it in our own code
#[allow(async_fn_in_trait)]
@@ -130,23 +130,23 @@ where
// Attempt to use a user-provided gateway, if possible
let user_chosen_gateway_id = common_args.gateway;
log::debug!("User chosen gateway id: {user_chosen_gateway_id:?}");
tracing::debug!("User chosen gateway id: {user_chosen_gateway_id:?}");
let selection_spec = GatewaySelectionSpecification::new(
user_chosen_gateway_id.map(|id| id.to_base58_string()),
Some(common_args.latency_based_selection),
common_args.force_tls_gateway,
);
log::debug!("Gateway selection specification: {selection_spec:?}");
tracing::debug!("Gateway selection specification: {selection_spec:?}");
// Load and potentially override config
log::debug!("Init arguments: {init_args:#?}");
tracing::debug!("Init arguments: {init_args:#?}");
let config = C::construct_config(&init_args);
log::debug!("Constructed config: {config:#?}");
tracing::debug!("Constructed config: {config:#?}");
let paths = config.common_paths();
let core = config.core_config();
log::info!(
tracing::info!(
"Using nym-api: {}",
core.client
.nym_api_urls
@@ -18,6 +18,7 @@ use crate::client::received_buffer::{
ReceivedBufferRequestReceiver, ReceivedBufferRequestSender, ReceivedMessagesBufferController,
};
use crate::client::replies::reply_controller;
use crate::client::replies::reply_controller::key_rotation_helpers::KeyRotationConfig;
use crate::client::replies::reply_controller::{ReplyControllerReceiver, ReplyControllerSender};
use crate::client::replies::reply_storage::{
CombinedReplyStorage, PersistentReplyStorage, ReplyStorageBackend, SentReplyKeys,
@@ -34,7 +35,6 @@ use crate::init::{
};
use crate::{config, spawn_future};
use futures::channel::mpsc;
use log::*;
use nym_bandwidth_controller::BandwidthController;
use nym_client_core_config_types::{ForgetMe, RememberMe};
use nym_client_core_gateways_storage::{GatewayDetails, GatewaysDetailsStore};
@@ -56,13 +56,18 @@ use nym_task::connections::{ConnectionCommandReceiver, ConnectionCommandSender,
use nym_task::{TaskClient, TaskHandle};
use nym_topology::provider_trait::TopologyProvider;
use nym_topology::HardcodedTopologyProvider;
use nym_validator_client::{nyxd::contract_traits::DkgQueryClient, UserAgent};
use nym_validator_client::nym_api::NymApiClientExt;
use nym_validator_client::{nyxd::contract_traits::DkgQueryClient, NymApiClient, UserAgent};
use rand::prelude::SliceRandom;
use rand::rngs::OsRng;
use rand::thread_rng;
use std::fmt::Debug;
use std::os::raw::c_int as RawFd;
use std::path::Path;
use std::sync::Arc;
use time::OffsetDateTime;
use tokio::sync::mpsc::Sender;
use tracing::*;
use url::Url;
#[cfg(all(
@@ -338,6 +343,7 @@ where
#[allow(clippy::too_many_arguments)]
fn start_real_traffic_controller(
controller_config: real_messages_control::Config,
key_rotation_config: KeyRotationConfig,
topology_accessor: TopologyAccessor,
ack_receiver: AcknowledgementReceiver,
input_receiver: InputMessageReceiver,
@@ -355,6 +361,7 @@ where
RealMessagesController::new(
controller_config,
key_rotation_config,
ack_receiver,
input_receiver,
mix_sender,
@@ -453,10 +460,10 @@ where
};
let gateway_failure = |err| {
log::error!("Could not authenticate and start up the gateway connection - {err}");
tracing::error!("Could not authenticate and start up the gateway connection - {err}");
ClientCoreError::GatewayClientError {
gateway_id: details.gateway_id.to_base58_string(),
source: err,
source: Box::new(err),
}
};
@@ -555,14 +562,14 @@ where
custom_provider: Option<Box<dyn TopologyProvider + Send + Sync>>,
config_topology: config::Topology,
nym_api_urls: Vec<Url>,
user_agent: Option<UserAgent>,
nym_api_client: NymApiClient,
) -> Box<dyn TopologyProvider + Send + Sync> {
// if no custom provider was ... provided ..., create one using nym-api
custom_provider.unwrap_or_else(|| {
Box::new(NymApiTopologyProvider::new(
config_topology,
nym_api_urls,
user_agent,
nym_api_client,
))
})
}
@@ -598,7 +605,7 @@ where
topology_refresher.try_refresh().await;
if let Err(err) = topology_refresher.ensure_topology_is_routable().await {
log::error!(
tracing::error!(
"The current network topology seem to be insufficient to route any packets through \
- check if enough nodes and a gateway are online - source: {err}"
);
@@ -674,16 +681,26 @@ where
// TODO: rename it as it implies the data is persistent whilst one can use InMemBackend
async fn setup_persistent_reply_storage(
backend: S::ReplyStore,
key_rotation_config: KeyRotationConfig,
shutdown: TaskClient,
) -> Result<CombinedReplyStorage, ClientCoreError>
where
<S::ReplyStore as ReplyStorageBackend>::StorageError: Sync + Send,
S::ReplyStore: Send + Sync,
{
log::trace!("Setup persistent reply storage");
tracing::trace!("Setup persistent reply storage");
let now = OffsetDateTime::now_utc();
let expected_current_key_rotation_start =
key_rotation_config.expected_current_key_rotation_start(now);
// time of the start of one epoch BEFORE the CURRENT rotation has begun
// this indicates the starting time of when packets with the current keys might have been constructed
// (i.e. any surbs OLDER than that MUST BE invalid)
let prior_epoch_start =
expected_current_key_rotation_start - key_rotation_config.epoch_duration;
let persistent_storage = PersistentReplyStorage::new(backend);
let mem_store = persistent_storage
.load_state_from_backend()
.load_state_from_backend(prior_epoch_start)
.await
.map_err(|err| ClientCoreError::SurbStorageError {
source: Box::new(err),
@@ -725,6 +742,23 @@ where
setup_gateway(setup_method, key_store, details_store).await
}
fn construct_nym_api_client(config: &Config, user_agent: Option<UserAgent>) -> NymApiClient {
let mut nym_api_urls = config.get_nym_api_endpoints();
nym_api_urls.shuffle(&mut thread_rng());
if let Some(user_agent) = user_agent {
NymApiClient::new_with_user_agent(nym_api_urls[0].clone(), user_agent)
} else {
NymApiClient::new(nym_api_urls[0].clone())
}
}
async fn determine_key_rotation_state(
client: &NymApiClient,
) -> Result<KeyRotationConfig, ClientCoreError> {
Ok(client.nym_api.get_key_rotation_info().await?.into())
}
pub async fn start_base(mut self) -> Result<BaseClient, ClientCoreError>
where
S::ReplyStore: Send + Sync,
@@ -789,11 +823,14 @@ where
.dkg_query_client
.map(|client| BandwidthController::new(credential_store, client));
let nym_api_client = Self::construct_nym_api_client(&self.config, self.user_agent.clone());
let key_rotation_config = Self::determine_key_rotation_state(&nym_api_client).await?;
let topology_provider = Self::setup_topology_provider(
self.custom_topology_provider.take(),
self.config.debug.topology,
self.config.get_nym_api_endpoints(),
self.user_agent.clone(),
nym_api_client,
);
let stats_reporter = Self::start_statistics_control(
@@ -838,6 +875,7 @@ where
let reply_storage = Self::setup_persistent_reply_storage(
reply_storage_backend,
key_rotation_config,
shutdown.fork("persistent_reply_storage"),
)
.await?;
@@ -878,6 +916,7 @@ where
Self::start_real_traffic_controller(
controller_config,
key_rotation_config,
shared_topology_accessor.clone(),
ack_receiver,
input_receiver,
@@ -1,32 +1,30 @@
// Copyright 2022-2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::replies::reply_storage::{
fs_backend, CombinedReplyStorage, ReplyStorageBackend,
use crate::{
client::replies::reply_storage::{fs_backend, CombinedReplyStorage, ReplyStorageBackend},
config,
config::Config,
error::ClientCoreError,
};
use crate::config;
use crate::config::Config;
use crate::error::ClientCoreError;
use log::{error, info, trace};
use nym_bandwidth_controller::BandwidthController;
use nym_client_core_gateways_storage::OnDiskGatewaysDetails;
use nym_credential_storage::storage::Storage as CredentialStorage;
use nym_validator_client::nyxd;
use nym_validator_client::QueryHttpRpcNyxdClient;
use std::path::Path;
use std::{fs, io};
use nym_validator_client::{nyxd, QueryHttpRpcNyxdClient};
use std::{io, path::Path};
use time::OffsetDateTime;
use tracing::{error, info, trace};
use url::Url;
async fn setup_fresh_backend<P: AsRef<Path>>(
db_path: P,
surb_config: &config::ReplySurbs,
) -> Result<fs_backend::Backend, ClientCoreError> {
info!("creating fresh surb database");
info!("Creating fresh surb database");
let mut storage_backend = match fs_backend::Backend::init(db_path).await {
Ok(backend) => backend,
Err(err) => {
error!("failed to setup persistent storage backend for our reply needs: {err}");
error!("setup_fresh_backend: Failed to setup persistent storage backend for our reply needs: {err}");
return Err(ClientCoreError::SurbStorageError {
source: Box::new(err),
});
@@ -40,14 +38,15 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
surb_config.minimum_reply_surb_storage_threshold,
surb_config.maximum_reply_surb_storage_threshold,
);
storage_backend
.init_fresh(&mem_store)
.await
.map_err(|err| ClientCoreError::SurbStorageError {
source: Box::new(err),
})?;
Ok(storage_backend)
match storage_backend.init_fresh(&mem_store).await {
Ok(()) => Ok(storage_backend),
Err(err) => {
storage_backend.shutdown().await;
Err(ClientCoreError::SurbStorageError {
source: Box::new(err),
})
}
}
}
// fn setup_inactive_backend(surb_config: &config::ReplySurbs) -> fs_backend::Backend {
@@ -58,12 +57,11 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
// )
// }
fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> {
async fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> {
let db_path = db_path.as_ref();
debug_assert!(db_path.exists());
let now = OffsetDateTime::now_utc().unix_timestamp();
let suffix = format!("_{now}.corrupted");
let new_extension =
@@ -72,11 +70,15 @@ fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> {
} else {
suffix
};
let renamed = db_path.with_extension(new_extension);
let mut renamed = db_path.to_owned();
renamed.set_extension(new_extension);
fs::rename(db_path, renamed)
tokio::fs::rename(db_path, &renamed).await.inspect_err(|_| {
error!(
"Failed to rename corrupt database file: {} to {}",
db_path.display(),
renamed.display()
);
})
}
pub async fn setup_fs_reply_surb_backend<P: AsRef<Path>>(
@@ -87,13 +89,12 @@ pub async fn setup_fs_reply_surb_backend<P: AsRef<Path>>(
// the existing one
let db_path = db_path.as_ref();
if db_path.exists() {
info!("loading existing surb database");
match fs_backend::Backend::try_load(db_path, surb_config.fresh_sender_tags).await {
info!("Loading existing surb database");
match fs_backend::Backend::try_load(db_path).await {
Ok(backend) => Ok(backend),
Err(err) => {
error!("failed to setup persistent storage backend for our reply needs: {err}. We're going to create a fresh database instead. This behaviour might change in the future");
archive_corrupted_database(db_path)?;
error!("setup_fs_reply_surb_backend: Failed to setup persistent storage backend for our reply needs: {err}. We're going to create a fresh database instead. This behaviour might change in the future");
archive_corrupted_database(db_path).await?;
setup_fresh_backend(db_path, surb_config).await
}
}
@@ -6,7 +6,6 @@ use crate::client::topology_control::TopologyAccessor;
use crate::{config, spawn_future};
use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt};
use log::*;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::cover::generate_loop_cover_packet;
@@ -19,6 +18,7 @@ use std::pin::Pin;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::mpsc::error::TrySendError;
use tracing::*;
#[cfg(not(target_arch = "wasm32"))]
use tokio::time::{sleep, Sleep};
@@ -210,10 +210,10 @@ impl LoopCoverTrafficStream<OsRng> {
TrySendError::Full(_) => {
// This isn't a problem, if the channel is full means we're already sending the
// max amount of messages downstream can handle.
log::debug!("Failed to send cover message - channel full");
tracing::debug!("Failed to send cover message - channel full");
}
TrySendError::Closed(_) => {
log::warn!("Failed to send cover message - channel closed");
tracing::warn!("Failed to send cover message - channel closed");
}
}
} else {
@@ -258,20 +258,20 @@ impl LoopCoverTrafficStream<OsRng> {
tokio::select! {
biased;
_ = shutdown.recv() => {
log::trace!("LoopCoverTrafficStream: Received shutdown");
tracing::trace!("LoopCoverTrafficStream: Received shutdown");
}
next = self.next() => {
if next.is_some() {
self.on_new_message().await;
} else {
log::trace!("LoopCoverTrafficStream: Stopping since channel closed");
tracing::trace!("LoopCoverTrafficStream: Stopping since channel closed");
break;
}
}
}
}
shutdown.recv_timeout().await;
log::debug!("LoopCoverTrafficStream: Exiting");
tracing::debug!("LoopCoverTrafficStream: Exiting");
})
}
}
@@ -135,7 +135,9 @@ impl InputMessage {
recipient_tag,
data,
lane,
max_retransmissions: None,
// \/ set it to SOME sane default so that if we run out of surbs and constantly
// fail to request more, we wouldn't be stuck in limbo
max_retransmissions: Some(10),
};
if let Some(packet_type) = packet_type {
InputMessage::new_wrapper(message, packet_type)
@@ -4,10 +4,10 @@
use crate::client::mix_traffic::transceiver::GatewayTransceiver;
use crate::error::ClientCoreError;
use crate::spawn_future;
use log::*;
use nym_gateway_requests::ClientRequest;
use nym_sphinx::forwarding::packet::MixPacket;
use nym_task::TaskClient;
use tracing::*;
use transceiver::ErasedGatewayError;
pub type BatchMixMessageSender = tokio::sync::mpsc::Sender<Vec<MixPacket>>;
@@ -138,7 +138,7 @@ impl MixTrafficController {
}
},
None => {
log::trace!("MixTrafficController: Stopping since channel closed");
tracing::trace!("MixTrafficController: Stopping since channel closed");
break;
}
},
@@ -146,22 +146,22 @@ impl MixTrafficController {
Some(client_request) => {
match self.gateway_transceiver.send_client_request(client_request).await {
Ok(_) => (),
Err(e) => error!("Failed to send client request: {}", e),
Err(e) => error!("Failed to send client request: {e}"),
};
},
None => {
log::trace!("MixTrafficController, client request channel closed");
tracing::trace!("MixTrafficController, client request channel closed");
}
},
_ = self.task_client.recv() => {
log::trace!("MixTrafficController: Received shutdown");
tracing::trace!("MixTrafficController: Received shutdown");
break;
}
}
}
self.task_client.recv_timeout().await;
log::debug!("MixTrafficController: Exiting");
tracing::debug!("MixTrafficController: Exiting");
});
}
}
@@ -2,7 +2,6 @@
// SPDX-License-Identifier: Apache-2.0
use async_trait::async_trait;
use log::{debug, error};
use nym_credential_storage::storage::Storage as CredentialStorage;
use nym_crypto::asymmetric::ed25519;
use nym_gateway_client::error::GatewayClientError;
@@ -14,6 +13,7 @@ use nym_validator_client::nyxd::contract_traits::DkgQueryClient;
use std::fmt::Debug;
use std::os::raw::c_int as RawFd;
use thiserror::Error;
use tracing::{debug, error};
#[cfg(not(target_arch = "wasm32"))]
use futures::channel::oneshot;
@@ -27,7 +27,7 @@ fn erase_err<E: std::error::Error + Send + Sync + 'static>(err: E) -> ErasedGate
ErasedGatewayError(Box::new(err))
}
/// This combines combines the functionalities of being able to send and receive mix packets.
/// This combines the functionalities of being able to send and receive mix packets.
#[async_trait]
pub trait GatewayTransceiver: GatewaySender + GatewayReceiver {
fn gateway_identity(&self) -> ed25519::PublicKey;
@@ -87,7 +87,7 @@ impl<G: GatewayTransceiver + ?Sized + Send> GatewayTransceiver for Box<G> {
message: ClientRequest,
) -> Result<(), GatewayClientError> {
let _ = (**self).send_client_request(message.clone()).await?;
log::debug!("Sent client request: {:?}", message);
tracing::debug!("Sent client request: {:?}", message);
Ok(())
}
}
@@ -5,7 +5,6 @@ use super::action_controller::{AckActionSender, Action};
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use futures::StreamExt;
use log::*;
use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::{
acknowledgements::{identifier::recover_identifier, AckKey},
@@ -13,6 +12,7 @@ use nym_sphinx::{
};
use nym_task::TaskClient;
use std::sync::Arc;
use tracing::*;
/// Module responsible for listening for any data resembling acknowledgements from the network
/// and firing actions to remove them from the 'Pending' state.
@@ -65,7 +65,7 @@ impl AcknowledgementListener {
return;
}
trace!("Received {} from the mix network", frag_id);
trace!("Received {frag_id} from the mix network");
self.stats_tx
.report(PacketStatisticsEvent::RealAckReceived(ack_content.len()).into());
if let Err(err) = self
@@ -93,16 +93,16 @@ impl AcknowledgementListener {
acks = self.ack_receiver.next() => match acks {
Some(acks) => self.handle_ack_receiver_item(acks).await,
None => {
log::trace!("AcknowledgementListener: Stopping since channel closed");
tracing::trace!("AcknowledgementListener: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("AcknowledgementListener: Received shutdown");
tracing::trace!("AcknowledgementListener: Received shutdown");
}
}
}
self.task_client.recv_timeout().await;
log::debug!("AcknowledgementListener: Exiting");
tracing::debug!("AcknowledgementListener: Exiting");
}
}
@@ -5,7 +5,6 @@ use super::PendingAcknowledgement;
use crate::client::real_messages_control::acknowledgement_control::RetransmissionRequestSender;
use futures::channel::mpsc;
use futures::StreamExt;
use log::*;
use nym_nonexhaustive_delayqueue::{Expired, NonExhaustiveDelayQueue, QueueKey};
use nym_sphinx::chunking::fragment::FragmentIdentifier;
use nym_sphinx::Delay as SphinxDelay;
@@ -13,6 +12,7 @@ use nym_task::TaskClient;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use tracing::*;
pub(crate) type AckActionSender = mpsc::UnboundedSender<Action>;
pub(crate) type AckActionReceiver = mpsc::UnboundedReceiver<Action>;
@@ -126,7 +126,7 @@ impl ActionController {
fn handle_insert(&mut self, pending_acks: Vec<PendingAcknowledgement>) {
for pending_ack in pending_acks {
let frag_id = pending_ack.message_chunk.fragment_identifier();
trace!("{} is inserted", frag_id);
trace!("{frag_id} is inserted");
if self
.pending_acks_data
@@ -161,22 +161,16 @@ impl ActionController {
let new_queue_key = self.pending_acks_timers.insert(frag_id, timeout);
*queue_key = Some(new_queue_key)
} else {
debug!(
"Tried to START TIMER on pending ack that is already gone! - {}",
frag_id
);
debug!("Tried to START TIMER on pending ack that is already gone! - {frag_id}");
}
}
fn handle_remove(&mut self, frag_id: FragmentIdentifier) {
trace!("{} is getting removed", frag_id);
trace!("{frag_id} is getting removed");
match self.pending_acks_data.remove(&frag_id) {
None => {
debug!(
"Tried to REMOVE pending ack that is already gone! - {}",
frag_id
);
debug!("Tried to REMOVE pending ack that is already gone! - {frag_id}");
}
Some((_, queue_key)) => {
if let Some(queue_key) = queue_key {
@@ -188,10 +182,7 @@ impl ActionController {
} else {
// I'm not 100% sure if having a `None` key is even possible here
// (REMOVE would have to be called before START TIMER),
debug!(
"Tried to REMOVE pending ack without TIMER active - {}",
frag_id
);
debug!("Tried to REMOVE pending ack without TIMER active - {frag_id}");
}
}
}
@@ -200,7 +191,7 @@ impl ActionController {
// initiated basically as a first step of retransmission. At first data has its delay updated
// (as new sphinx packet was created with new expected delivery time)
fn handle_update_pending_ack(&mut self, frag_id: FragmentIdentifier, delay: SphinxDelay) {
trace!("{} is updating its delay", frag_id);
trace!("{frag_id} is updating its delay");
// TODO: is it possible to solve this without either locking or temporarily removing the value?
if let Some((pending_ack_data, queue_key)) = self.pending_acks_data.remove(&frag_id) {
// this Action is triggered by `RetransmissionRequestListener` (for 'normal' packets)
@@ -213,10 +204,7 @@ impl ActionController {
self.pending_acks_data
.insert(frag_id, (Arc::new(inner_data), queue_key));
} else {
debug!(
"Tried to UPDATE TIMER on pending ack that is already gone! - {}",
frag_id
);
debug!("Tried to UPDATE TIMER on pending ack that is already gone! - {frag_id}");
}
}
@@ -241,7 +229,7 @@ impl ActionController {
.unbounded_send(Arc::downgrade(pending_ack_data))
{
if !self.task_client.is_shutdown_poll() {
log::error!("Failed to send pending ack for retransmission: {err}");
tracing::error!("Failed to send pending ack for retransmission: {err}");
}
}
} else {
@@ -269,7 +257,7 @@ impl ActionController {
action = self.incoming_actions.next() => match action {
Some(action) => self.process_action(action),
None => {
log::trace!(
tracing::trace!(
"ActionController: Stopping since incoming actions channel closed"
);
break;
@@ -278,17 +266,17 @@ impl ActionController {
expired_ack = self.pending_acks_timers.next() => match expired_ack {
Some(expired_ack) => self.handle_expired_ack_timer(expired_ack),
None => {
log::trace!("ActionController: Stopping since ack channel closed");
tracing::trace!("ActionController: Stopping since ack channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("ActionController: Received shutdown");
tracing::trace!("ActionController: Received shutdown");
break;
}
}
}
self.task_client.recv_timeout().await;
log::debug!("ActionController: Exiting");
tracing::debug!("ActionController: Exiting");
}
}
@@ -5,7 +5,6 @@ use crate::client::inbound_messages::{InputMessage, InputMessageReceiver};
use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::real_messages_control::real_traffic_stream::RealMessage;
use crate::client::replies::reply_controller::ReplyControllerSender;
use log::*;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::forwarding::packet::MixPacket;
@@ -13,6 +12,7 @@ use nym_sphinx::params::PacketType;
use nym_task::connections::TransmissionLane;
use nym_task::TaskClient;
use rand::{CryptoRng, Rng};
use tracing::*;
/// Module responsible for dealing with the received messages: splitting them, creating acknowledgements,
/// putting everything into sphinx packets, etc.
@@ -228,16 +228,16 @@ where
self.on_input_message(input_msg).await;
},
None => {
log::trace!("InputMessageListener: Stopping since channel closed");
tracing::trace!("InputMessageListener: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("InputMessageListener: Received shutdown");
tracing::trace!("InputMessageListener: Received shutdown");
}
}
}
self.task_client.recv_timeout().await;
log::debug!("InputMessageListener: Exiting");
tracing::debug!("InputMessageListener: Exiting");
}
}
@@ -13,7 +13,6 @@ use crate::client::replies::reply_controller::ReplyControllerSender;
use crate::spawn_future;
use action_controller::AckActionReceiver;
use futures::channel::mpsc;
use log::*;
use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::params::{PacketSize, PacketType};
@@ -30,6 +29,7 @@ use std::{
sync::{Arc, Weak},
time::Duration,
};
use tracing::*;
pub(crate) use action_controller::{AckActionSender, Action};
@@ -10,13 +10,13 @@ use crate::client::real_messages_control::message_handler::{MessageHandler, Prep
use crate::client::real_messages_control::real_traffic_stream::RealMessage;
use crate::client::replies::reply_controller::ReplyControllerSender;
use futures::StreamExt;
use log::*;
use nym_sphinx::chunking::fragment::Fragment;
use nym_sphinx::preparer::PreparedFragment;
use nym_sphinx::{addressing::clients::Recipient, params::PacketType};
use nym_task::{connections::TransmissionLane, TaskClient};
use rand::{CryptoRng, Rng};
use std::sync::{Arc, Weak};
use tracing::*;
// responsible for packet retransmission upon fired timer
pub(super) struct RetransmissionRequestListener<R> {
@@ -182,16 +182,16 @@ where
timed_out_ack = self.request_receiver.next() => match timed_out_ack {
Some(timed_out_ack) => self.on_retransmission_request(timed_out_ack, packet_type).await,
None => {
log::trace!("RetransmissionRequestListener: Stopping since channel closed");
tracing::trace!("RetransmissionRequestListener: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("RetransmissionRequestListener: Received shutdown");
tracing::trace!("RetransmissionRequestListener: Received shutdown");
}
}
}
self.task_client.recv_timeout().await;
log::debug!("RetransmissionRequestListener: Exiting");
tracing::debug!("RetransmissionRequestListener: Exiting");
}
}
@@ -4,9 +4,9 @@
use super::action_controller::{AckActionSender, Action};
use super::SentPacketNotificationReceiver;
use futures::StreamExt;
use log::*;
use nym_sphinx::chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID};
use nym_task::TaskClient;
use tracing::*;
/// Module responsible for starting up retransmission timers.
/// It is required because when we send our packet to the `real traffic stream` controlled
@@ -56,17 +56,17 @@ impl SentNotificationListener {
self.on_sent_message(frag_id).await;
}
None => {
log::trace!("SentNotificationListener: Stopping since channel closed");
tracing::trace!("SentNotificationListener: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv() => {
log::trace!("SentNotificationListener: Received shutdown");
tracing::trace!("SentNotificationListener: Received shutdown");
break;
}
}
}
assert!(self.task_client.is_shutdown_poll());
log::debug!("SentNotificationListener: Exiting");
tracing::debug!("SentNotificationListener: Exiting");
}
}
@@ -9,10 +9,11 @@ use crate::client::real_messages_control::{AckActionSender, Action};
use crate::client::replies::reply_controller::MaxRetransmissions;
use crate::client::replies::reply_storage::{ReceivedReplySurbsMap, SentReplyKeys, UsedSenderTags};
use crate::client::topology_control::{TopologyAccessor, TopologyReadPermit};
use nym_client_core_surb_storage::RetrievedReplySurb;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::{AnonymousSenderTag, RepliableMessage, ReplyMessage};
use nym_sphinx::anonymous_replies::{ReplySurb, SurbEncryptionKey};
use nym_sphinx::anonymous_replies::ReplySurbWithKeyRotation;
use nym_sphinx::chunking::fragment::{Fragment, FragmentIdentifier};
use nym_sphinx::message::NymMessage;
use nym_sphinx::params::{PacketSize, PacketType};
@@ -44,7 +45,7 @@ pub enum PreparationError {
}
impl PreparationError {
fn return_surbs(self, returned_surbs: Vec<ReplySurb>) -> SurbWrappedPreparationError {
fn return_surbs(self, returned_surbs: Vec<RetrievedReplySurb>) -> SurbWrappedPreparationError {
SurbWrappedPreparationError {
source: self,
returned_surbs: Some(returned_surbs),
@@ -58,7 +59,7 @@ pub struct SurbWrappedPreparationError {
#[source]
source: PreparationError,
returned_surbs: Option<Vec<ReplySurb>>,
returned_surbs: Option<Vec<RetrievedReplySurb>>,
}
impl<T> From<T> for SurbWrappedPreparationError
@@ -80,7 +81,7 @@ impl SurbWrappedPreparationError {
target: &AnonymousSenderTag,
) -> PreparationError {
if let Some(reply_surbs) = self.returned_surbs {
surb_storage.insert_surbs(target, reply_surbs)
surb_storage.re_insert_reply_surbs(target, reply_surbs)
}
self.source
}
@@ -102,6 +103,9 @@ pub(crate) struct Config {
/// will be routed as usual, to the entry gateway, through three mix nodes, egressing
/// through the exit gateway. If mix hops are disabled, traffic will be routed directly
/// from the entry gateway to the exit gateway, bypassing the mix nodes.
///
/// This overrides the `use_legacy_sphinx_format` setting as reduced mix hops
/// requires use of the updated SURB packet format.
disable_mix_hops: bool,
/// Average delay a data packet is going to get delay at a single mixnode.
@@ -156,8 +160,12 @@ impl Config {
}
/// Configure whether messages senders using this config should use mix hops or not when sending messages.
///
/// This overrides the `use_legacy_sphinx_format` setting as disabled mix hops
/// requires use of the updated SURB packet format.
pub fn disable_mix_hops(mut self, disable_mix_hops: bool) -> Self {
self.disable_mix_hops = disable_mix_hops;
self.use_legacy_sphinx_format = false;
self
}
}
@@ -221,6 +229,10 @@ where
}
}
pub(crate) fn topology_access_handle(&self) -> &TopologyAccessor {
&self.topology_access
}
fn get_or_create_sender_tag(&mut self, recipient: &Recipient) -> AnonymousSenderTag {
if let Some(existing) = self.tag_storage.try_get_existing(recipient) {
trace!("we already had sender tag for {recipient}");
@@ -268,10 +280,10 @@ where
}
}
async fn generate_reply_surbs_with_keys(
async fn generate_reply_surbs(
&mut self,
amount: usize,
) -> Result<(Vec<ReplySurb>, Vec<SurbEncryptionKey>), PreparationError> {
) -> Result<Vec<ReplySurbWithKeyRotation>, PreparationError> {
let topology_permit = self.topology_access.get_read_permit().await;
let topology = self.get_topology(&topology_permit)?;
@@ -281,19 +293,14 @@ where
topology,
)?;
let reply_keys = reply_surbs
.iter()
.map(|s| *s.encryption_key())
.collect::<Vec<_>>();
Ok((reply_surbs, reply_keys))
Ok(reply_surbs)
}
pub(crate) async fn try_send_single_surb_message(
&mut self,
target: AnonymousSenderTag,
message: ReplyMessage,
reply_surb: ReplySurb,
reply_surb: RetrievedReplySurb,
is_extra_surb_request: bool,
) -> Result<(), SurbWrappedPreparationError> {
let msg = NymMessage::new_reply(message);
@@ -324,7 +331,10 @@ where
Some(chunk.fragment_identifier()),
);
let delay = prepared_fragment.total_delay;
let max_retransmissions = None;
// we have to set a maximum number of retransmissions in case we fail to retrieve
// surbs for a long period of time; we don't want to be stuck constantly resending the data
let max_retransmissions = Some(10);
let pending_ack = PendingAcknowledgement::new_anonymous(
chunk,
delay,
@@ -347,7 +357,7 @@ where
pub(crate) async fn try_request_additional_reply_surbs(
&mut self,
from: AnonymousSenderTag,
reply_surb: ReplySurb,
reply_surb: RetrievedReplySurb,
amount: u32,
) -> Result<(), SurbWrappedPreparationError> {
debug!("requesting {amount} reply SURBs from {from}");
@@ -387,11 +397,9 @@ where
&mut self,
target: AnonymousSenderTag,
fragments: Vec<FragmentWithMaxRetransmissions>,
reply_surbs: Vec<ReplySurb>,
reply_surbs: impl IntoIterator<Item = RetrievedReplySurb>,
lane: TransmissionLane,
) -> Result<(), SurbWrappedPreparationError> {
// TODO: technically this is performing an unnecessary cloning, but in the grand scheme of things
// is it really that bad?
self.try_send_reply_chunks(
target,
fragments.into_iter().map(|f| (lane, f)).collect(),
@@ -404,7 +412,7 @@ where
&mut self,
target: AnonymousSenderTag,
fragments: Vec<(TransmissionLane, FragmentWithMaxRetransmissions)>,
reply_surbs: Vec<ReplySurb>,
reply_surbs: impl IntoIterator<Item = RetrievedReplySurb>,
) -> Result<(), SurbWrappedPreparationError> {
let prepared_fragments = self
.prepare_reply_chunks_for_sending(
@@ -541,8 +549,12 @@ where
) -> Result<(), PreparationError> {
debug!("Sending additional reply SURBs with packet type {packet_type}");
let sender_tag = self.get_or_create_sender_tag(&recipient);
let (reply_surbs, reply_keys) =
self.generate_reply_surbs_with_keys(amount as usize).await?;
let reply_surbs = self.generate_reply_surbs(amount as usize).await?;
let reply_keys = reply_surbs
.iter()
.map(|s| *s.encryption_key())
.collect::<Vec<_>>();
let message = NymMessage::new_repliable(RepliableMessage::new_additional_surbs(
self.config.use_legacy_sphinx_format,
@@ -562,7 +574,7 @@ where
)
.await?;
log::trace!("storing {} reply keys", reply_keys.len());
tracing::trace!("storing {} reply keys", reply_keys.len());
self.reply_key_storage.insert_multiple(reply_keys);
Ok(())
@@ -579,9 +591,12 @@ where
) -> Result<(), SurbWrappedPreparationError> {
debug!("Sending message with reply SURBs with packet type {packet_type}");
let sender_tag = self.get_or_create_sender_tag(&recipient);
let (reply_surbs, reply_keys) = self
.generate_reply_surbs_with_keys(num_reply_surbs as usize)
.await?;
let reply_surbs = self.generate_reply_surbs(num_reply_surbs as usize).await?;
let reply_keys = reply_surbs
.iter()
.map(|s| *s.encryption_key())
.collect::<Vec<_>>();
let message = NymMessage::new_repliable(RepliableMessage::new_data(
self.config.use_legacy_sphinx_format,
@@ -599,7 +614,7 @@ where
)
.await?;
log::trace!("storing {} reply keys", reply_keys.len());
tracing::trace!("storing {} reply keys", reply_keys.len());
self.reply_key_storage.insert_multiple(reply_keys);
Ok(())
@@ -629,20 +644,12 @@ where
pub(crate) async fn prepare_reply_chunks_for_sending(
&mut self,
fragments: Vec<Fragment>,
reply_surbs: Vec<ReplySurb>,
reply_surbs: impl IntoIterator<Item = RetrievedReplySurb>,
) -> Result<Vec<PreparedFragment>, SurbWrappedPreparationError> {
debug_assert_eq!(
fragments.len(),
reply_surbs.len(),
"attempted to send {} fragments with {} reply surbs",
fragments.len(),
reply_surbs.len()
);
let topology_permit = self.topology_access.get_read_permit().await;
let topology = match self.get_topology(&topology_permit) {
Ok(topology) => topology,
Err(err) => return Err(err.return_surbs(reply_surbs)),
Err(err) => return Err(err.return_surbs(reply_surbs.into_iter().collect())),
};
Ok(fragments
@@ -655,7 +662,7 @@ where
fragment,
topology,
&self.config.ack_key,
reply_surb,
reply_surb.into(),
PacketType::Mix,
)
.unwrap()
@@ -665,7 +672,7 @@ where
pub(crate) async fn try_prepare_single_reply_chunk_for_sending(
&mut self,
reply_surb: ReplySurb,
reply_surb: RetrievedReplySurb,
chunk: Fragment,
) -> Result<PreparedFragment, SurbWrappedPreparationError> {
let topology_permit = self.topology_access.get_read_permit().await;
@@ -678,7 +685,7 @@ where
chunk,
topology,
&self.config.ack_key,
reply_surb,
reply_surb.into(),
PacketType::Mix,
)?;
@@ -24,7 +24,6 @@ use crate::{
spawn_future,
};
use futures::channel::mpsc;
use log::*;
use nym_gateway_client::AcknowledgementReceiver;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
@@ -34,7 +33,9 @@ use nym_task::connections::{ConnectionCommandReceiver, LaneQueueLengths};
use nym_task::TaskClient;
use rand::{rngs::OsRng, CryptoRng, Rng};
use std::sync::Arc;
use tracing::*;
use crate::client::replies::reply_controller::key_rotation_helpers::KeyRotationConfig;
pub(crate) use acknowledgement_control::{AckActionSender, Action};
pub(crate) mod acknowledgement_control;
@@ -85,12 +86,6 @@ impl<'a> From<&'a Config> for real_traffic_stream::Config {
}
}
impl<'a> From<&'a Config> for reply_controller::Config {
fn from(cfg: &'a Config) -> Self {
reply_controller::Config::new(cfg.reply_surbs)
}
}
impl<'a> From<&'a Config> for message_handler::Config {
fn from(cfg: &'a Config) -> Self {
message_handler::Config::new(
@@ -139,6 +134,7 @@ impl RealMessagesController<OsRng> {
#[allow(clippy::too_many_arguments)]
pub(crate) fn new(
config: Config,
key_rotation_config: KeyRotationConfig,
ack_receiver: AcknowledgementReceiver,
input_receiver: InputMessageReceiver,
mix_sender: BatchMixMessageSender,
@@ -169,7 +165,8 @@ impl RealMessagesController<OsRng> {
// create all configs for the components
let ack_control_config = (&config).into();
let out_queue_config = (&config).into();
let reply_controller_config = (&config).into();
let reply_controller_config =
reply_controller::Config::new(config.reply_surbs, key_rotation_config);
let message_handler_config = (&config).into();
// create the actual components
@@ -9,7 +9,6 @@ use crate::client::transmission_buffer::TransmissionBuffer;
use crate::config;
use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt};
use log::*;
use nym_sphinx::acknowledgements::AckKey;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::chunking::fragment::FragmentIdentifier;
@@ -27,6 +26,7 @@ use rand::{CryptoRng, Rng};
use std::pin::Pin;
use std::sync::Arc;
use std::time::Duration;
use tracing::*;
#[cfg(not(target_arch = "wasm32"))]
use tokio::time::{sleep, Sleep};
@@ -202,7 +202,7 @@ where
// well technically the message was not sent just yet, but now it's up to internal
// queues and client load rather than the required delay. So realistically we can treat
// whatever is about to happen as negligible additional delay.
trace!("{} is about to get sent to the mixnet", frag_id);
trace!("{frag_id} is about to get sent to the mixnet");
if let Err(err) = self.sent_notifier.unbounded_send(frag_id) {
error!("Failed to notify about sent message: {err}");
}
@@ -280,7 +280,7 @@ where
if let Err(err) = self.mix_tx.send(vec![next_message]).await {
if !self.task_client.is_shutdown_poll() {
log::error!("Failed to send: {err}");
tracing::error!("Failed to send: {err}");
}
} else {
let event = if fragment_id.is_some() {
@@ -313,7 +313,7 @@ where
}
fn on_close_connection(&mut self, connection_id: ConnectionId) {
log::debug!("Removing lane for connection: {connection_id}");
tracing::debug!("Removing lane for connection: {connection_id}");
self.transmission_buffer
.remove(&TransmissionLane::ConnectionId(connection_id));
}
@@ -325,7 +325,7 @@ where
fn adjust_current_average_message_sending_delay(&mut self) {
let used_slots = self.mix_tx.max_capacity() - self.mix_tx.capacity();
log::trace!(
tracing::trace!(
"used_slots: {used_slots}, current_multiplier: {}",
self.sending_delay_controller.current_multiplier()
);
@@ -334,7 +334,7 @@ where
.sending_delay_controller
.is_backpressure_currently_detected(used_slots)
{
log::trace!("Backpressure detected");
tracing::trace!("Backpressure detected");
self.sending_delay_controller.record_backpressure_detected();
}
@@ -436,7 +436,7 @@ where
Poll::Ready(None) => Poll::Ready(None),
Poll::Ready(Some((real_messages, conn_id))) => {
log::trace!("handling real_messages: size: {}", real_messages.len());
tracing::trace!("handling real_messages: size: {}", real_messages.len());
self.transmission_buffer.store(&conn_id, real_messages);
let real_next = self.pop_next_message().expect("Just stored one");
@@ -483,7 +483,7 @@ where
Poll::Ready(None) => Poll::Ready(None),
Poll::Ready(Some((real_messages, conn_id))) => {
log::trace!("handling real_messages: size: {}", real_messages.len());
tracing::trace!("handling real_messages: size: {}", real_messages.len());
// First store what we got for the given connection id
self.transmission_buffer.store(&conn_id, real_messages);
@@ -538,11 +538,11 @@ where
};
if packets > 1000 {
log::warn!("{status_str}");
tracing::warn!("{status_str}");
} else if packets > 0 {
log::info!("{status_str}");
tracing::info!("{status_str}");
} else {
log::debug!("{status_str}");
tracing::debug!("{status_str}");
}
// Send status message to whoever is listening (possibly UI)
@@ -566,7 +566,7 @@ where
tokio::select! {
biased;
_ = shutdown.recv() => {
log::trace!("OutQueueControl: Received shutdown");
tracing::trace!("OutQueueControl: Received shutdown");
break;
}
_ = status_timer.tick() => {
@@ -575,7 +575,7 @@ where
next_message = self.next() => if let Some(next_message) = next_message {
self.on_message(next_message).await;
} else {
log::trace!("OutQueueControl: Stopping since channel closed");
tracing::trace!("OutQueueControl: Stopping since channel closed");
break;
}
}
@@ -589,18 +589,18 @@ where
tokio::select! {
biased;
_ = shutdown.recv() => {
log::trace!("OutQueueControl: Received shutdown");
tracing::trace!("OutQueueControl: Received shutdown");
}
next_message = self.next() => if let Some(next_message) = next_message {
self.on_message(next_message).await;
} else {
log::trace!("OutQueueControl: Stopping since channel closed");
tracing::trace!("OutQueueControl: Stopping since channel closed");
break;
}
}
}
}
log::debug!("OutQueueControl: Exiting");
tracing::debug!("OutQueueControl: Exiting");
}
}
@@ -98,12 +98,12 @@ impl SendingDelayController {
self.current_multiplier =
(self.current_multiplier + 1).clamp(self.lower_bound, self.upper_bound);
self.time_when_changed = get_time_now();
log::debug!(
tracing::debug!(
"Increasing sending delay multiplier to: {}",
self.current_multiplier
);
} else {
log::warn!("Trying to increase delay multipler higher than allowed");
tracing::warn!("Trying to increase delay multipler higher than allowed");
}
}
@@ -112,7 +112,7 @@ impl SendingDelayController {
self.current_multiplier =
(self.current_multiplier - 1).clamp(self.lower_bound, self.upper_bound);
self.time_when_changed = get_time_now();
log::debug!(
tracing::debug!(
"Decreasing sending delay multiplier to: {}",
self.current_multiplier
);
@@ -164,11 +164,11 @@ impl SendingDelayController {
self.current_multiplier()
);
if self.current_multiplier() > 0 {
log::debug!("{}", status_str);
tracing::debug!("{status_str}");
} else if self.current_multiplier() > 1 {
log::info!("{}", status_str);
tracing::info!("{status_str}");
} else if self.current_multiplier() > 2 {
log::warn!("{}", status_str);
tracing::warn!("{status_str}");
}
self.time_when_logged_about_elevated_multiplier = now;
}
@@ -8,7 +8,6 @@ use crate::spawn_future;
use futures::channel::mpsc;
use futures::lock::Mutex;
use futures::StreamExt;
use log::*;
use nym_crypto::asymmetric::x25519;
use nym_crypto::Digest;
use nym_gateway_client::MixnetMessageReceiver;
@@ -24,6 +23,7 @@ use nym_task::TaskClient;
use std::collections::HashSet;
use std::sync::Arc;
use std::time::{Duration, Instant};
use tracing::*;
// The interval at which we check for stale buffers
const STALE_BUFFER_CHECK_INTERVAL: Duration = Duration::from_secs(10);
@@ -221,10 +221,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
let stored_messages = std::mem::take(&mut guard.messages);
if !stored_messages.is_empty() {
if let Err(err) = sender.unbounded_send(stored_messages) {
error!(
"The sender channel we just received is already invalidated - {:?}",
err
);
error!("The sender channel we just received is already invalidated - {err:?}");
// put the values back to the buffer
// the returned error has two fields: err: SendError and val: T,
// where val is the value that was failed to get sent;
@@ -310,13 +307,15 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
}
};
if let Err(err) = self.reply_controller_sender.send_additional_surbs(
msg.sender_tag,
reply_surbs,
from_surb_request,
) {
if !self.task_client.is_shutdown_poll() {
error!("{err}");
if !reply_surbs.is_empty() {
if let Err(err) = self.reply_controller_sender.send_additional_surbs(
msg.sender_tag,
reply_surbs,
from_surb_request,
) {
if !self.task_client.is_shutdown_poll() {
error!("{err}");
}
}
}
}
@@ -500,20 +499,20 @@ impl<R: MessageReceiver> RequestReceiver<R> {
tokio::select! {
biased;
_ = self.task_client.recv() => {
log::trace!("RequestReceiver: Received shutdown");
tracing::trace!("RequestReceiver: Received shutdown");
}
request = self.query_receiver.next() => {
if let Some(message) = request {
self.handle_message(message).await
} else {
log::trace!("RequestReceiver: Stopping since channel closed");
tracing::trace!("RequestReceiver: Stopping since channel closed");
break;
}
},
}
}
self.task_client.recv().await;
log::debug!("RequestReceiver: Exiting");
tracing::debug!("RequestReceiver: Exiting");
}
}
@@ -544,17 +543,17 @@ impl<R: MessageReceiver> FragmentedMessageReceiver<R> {
if let Some(new_messages) = new_messages {
self.received_buffer.handle_new_received(new_messages).await?;
} else {
log::trace!("FragmentedMessageReceiver: Stopping since channel closed");
tracing::trace!("FragmentedMessageReceiver: Stopping since channel closed");
break;
}
},
_ = self.task_client.recv_with_delay() => {
log::trace!("FragmentedMessageReceiver: Received shutdown");
tracing::trace!("FragmentedMessageReceiver: Received shutdown");
}
}
}
self.task_client.recv_timeout().await;
log::debug!("FragmentedMessageReceiver: Exiting");
tracing::debug!("FragmentedMessageReceiver: Exiting");
Ok(())
}
}
@@ -0,0 +1,169 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use nym_topology::NymTopologyMetadata;
use nym_validator_client::models::{
EpochId, KeyRotationId, KeyRotationInfoResponse, KeyRotationState,
};
use std::time::Duration;
use time::OffsetDateTime;
#[derive(Clone, Copy)]
pub(crate) enum SurbRefreshState {
WaitingForNextRotation { last_known: KeyRotationId },
ScheduledForNextInvocation,
}
#[derive(Clone, Copy)]
pub(crate) struct ReferenceEpoch {
pub(crate) absolute_epoch_id: EpochId,
pub(crate) start_time: OffsetDateTime,
}
#[derive(Clone, Copy)]
pub(crate) struct KeyRotationConfig {
pub(crate) epoch_duration: Duration,
pub(crate) rotation_state: KeyRotationState,
pub(crate) reference_epoch: ReferenceEpoch,
}
impl From<KeyRotationInfoResponse> for KeyRotationConfig {
fn from(value: KeyRotationInfoResponse) -> Self {
KeyRotationConfig {
epoch_duration: value.details.epoch_duration,
rotation_state: value.details.key_rotation_state,
reference_epoch: ReferenceEpoch {
absolute_epoch_id: value.details.current_absolute_epoch_id,
start_time: value.details.current_epoch_start,
},
}
}
}
impl KeyRotationConfig {
pub(crate) fn rotation_lifetime(&self) -> Duration {
(self.rotation_state.validity_epochs + 1) * self.epoch_duration
}
pub(crate) fn key_rotation_id(&self, current_absolute_epoch_id: EpochId) -> KeyRotationId {
self.rotation_state
.key_rotation_id(current_absolute_epoch_id)
}
// this is called with the assumption that now is always > reference epoch start
pub(crate) fn expected_current_epoch_id(&self, now: OffsetDateTime) -> EpochId {
let diff_secs = (now - self.reference_epoch.start_time).as_seconds_f64();
let epochs = (diff_secs / self.epoch_duration.as_secs_f64()).floor() as u32;
self.reference_epoch.absolute_epoch_id + epochs
}
fn initial_rotation_epoch_start(&self) -> OffsetDateTime {
let epochs_diff = self
.reference_epoch
.absolute_epoch_id
.saturating_sub(self.rotation_state.initial_epoch_id);
self.reference_epoch.start_time - epochs_diff * self.epoch_duration
}
pub(crate) fn key_rotation_start(&self, key_rotation_id: KeyRotationId) -> OffsetDateTime {
let rotation_duration = self.rotation_state.validity_epochs * self.epoch_duration;
let initial_start = self.initial_rotation_epoch_start();
// note: key rotation starts from 0
initial_start + rotation_duration * key_rotation_id
}
pub(crate) fn expected_current_key_rotation_id(&self, now: OffsetDateTime) -> KeyRotationId {
let expected_current_epoch = self.expected_current_epoch_id(now);
self.key_rotation_id(expected_current_epoch)
}
pub(crate) fn expected_current_key_rotation_start(
&self,
now: OffsetDateTime,
) -> OffsetDateTime {
let expected_current_key_rotation_id = self.expected_current_key_rotation_id(now);
self.key_rotation_start(expected_current_key_rotation_id)
}
pub(crate) fn epoch_stuck(&self, topology_metadata: NymTopologyMetadata) -> bool {
// add leeway of 2mins each direction since transition is not instantaneous
let lower_bound = topology_metadata.refreshed_at - Duration::from_secs(2);
let upper_bound = topology_metadata.refreshed_at + Duration::from_secs(2);
let expected_epoch_lower = self.expected_current_epoch_id(lower_bound);
let expected_epoch_upper = self.expected_current_epoch_id(upper_bound);
topology_metadata.absolute_epoch_id != expected_epoch_lower
&& topology_metadata.absolute_epoch_id != expected_epoch_upper
}
}
#[cfg(test)]
mod tests {
use super::*;
use time::macros::datetime;
fn mock_config() -> KeyRotationConfig {
KeyRotationConfig {
epoch_duration: Duration::from_secs(60 * 60),
rotation_state: KeyRotationState {
validity_epochs: 10,
initial_epoch_id: 80,
},
reference_epoch: ReferenceEpoch {
absolute_epoch_id: 100,
start_time: datetime!(2025-06-30 12:00:00+00:00),
},
}
}
#[test]
fn expected_current_key_rotation_start() {
// rot0: 80-89
// rot1: 90-99
// rot2: 100-109
// rot3: 110-119
// ... etc
let cfg = mock_config();
assert_eq!(
cfg.initial_rotation_epoch_start(),
datetime!(2025-06-29 16:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 12:00:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 100);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 2);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 12:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 12:30:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 100);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 2);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 12:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 13:01:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 101);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 2);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 12:00:00+00:00)
);
let fake_now = datetime!(2025-06-30 22:02:00+00:00);
assert_eq!(cfg.expected_current_epoch_id(fake_now), 110);
assert_eq!(cfg.expected_current_key_rotation_id(fake_now), 3);
assert_eq!(
cfg.expected_current_key_rotation_start(fake_now),
datetime!(2025-06-30 22:00:00+00:00)
);
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,899 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::real_messages_control::acknowledgement_control::PendingAcknowledgement;
use crate::client::real_messages_control::message_handler::{
FragmentWithMaxRetransmissions, MessageHandler, PreparationError,
};
use crate::client::replies::reply_controller::key_rotation_helpers::SurbRefreshState;
use crate::client::replies::reply_controller::Config;
use crate::client::topology_control::TopologyAccessor;
use crate::client::transmission_buffer::TransmissionBuffer;
use futures::channel::oneshot;
use nym_client_core_surb_storage::{ReceivedReplySurb, ReceivedReplySurbsMap};
use nym_crypto::aes::cipher::crypto_common::rand_core::CryptoRng;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::anonymous_replies::ReplySurbWithKeyRotation;
use nym_sphinx::chunking::fragment::FragmentIdentifier;
use nym_task::connections::{ConnectionId, TransmissionLane};
use nym_topology::NymTopologyMetadata;
use rand::Rng;
use std::cmp::{max, min};
use std::collections::btree_map::Entry;
use std::collections::{BTreeMap, HashMap};
use std::mem;
use std::sync::{Arc, Weak};
use time::OffsetDateTime;
use tracing::{debug, error, info, trace, warn};
struct SenderData {
current_clear_rerequest_counter: usize,
pending_replies: TransmissionBuffer<FragmentWithMaxRetransmissions>,
pending_retransmissions: BTreeMap<FragmentIdentifier, Weak<PendingAcknowledgement>>,
last_request_failure: OffsetDateTime,
}
impl Default for SenderData {
fn default() -> Self {
SenderData {
current_clear_rerequest_counter: 0,
pending_replies: Default::default(),
pending_retransmissions: Default::default(),
last_request_failure: OffsetDateTime::UNIX_EPOCH,
}
}
}
impl SenderData {
fn total_pending(&self) -> usize {
let pending_replies = self.pending_replies.total_size();
let pending_retransmissions = self.pending_retransmissions.len();
let total_pending = pending_retransmissions + pending_replies;
debug!("total queue size: {total_pending} = pending data {pending_replies} + pending retransmission {pending_retransmissions}");
total_pending
}
pub(crate) fn increment_current_clear_rerequest_counter(&mut self) {
self.current_clear_rerequest_counter += 1;
}
pub(crate) fn reset_current_clear_rerequest_counter(&mut self) {
self.current_clear_rerequest_counter = 0;
}
pub(crate) fn reset_last_request_failure(&mut self, now: OffsetDateTime) -> OffsetDateTime {
mem::replace(&mut self.last_request_failure, now)
}
}
/// Reply controller responsible for controlling receiver-related part
/// of replies, such as requesting additional reply SURBs
pub struct ReceiverReplyController<R> {
config: Config,
surb_refresh_state: SurbRefreshState,
topology_access: TopologyAccessor,
surb_senders: HashMap<AnonymousSenderTag, SenderData>,
unavailable: HashMap<AnonymousSenderTag, OffsetDateTime>,
surbs_storage: ReceivedReplySurbsMap,
// TODO: incorporate that field at some point
// and use binomial distribution to determine the expected required number
// of surbs required to send the message through
// expected_reliability: f32,
message_handler: MessageHandler<R>,
}
impl<R> ReceiverReplyController<R>
where
R: CryptoRng + Rng,
{
pub(crate) fn new(
config: Config,
storage: ReceivedReplySurbsMap,
message_handler: MessageHandler<R>,
) -> Self {
let topology_access = message_handler.topology_access_handle().clone();
ReceiverReplyController {
config,
surb_refresh_state: SurbRefreshState::WaitingForNextRotation {
last_known: config
.key_rotation
.expected_current_key_rotation_id(OffsetDateTime::now_utc()),
},
topology_access,
surb_senders: Default::default(),
unavailable: Default::default(),
surbs_storage: storage,
message_handler,
}
}
fn get_or_create_surb_sender(&mut self, tag: &AnonymousSenderTag) -> &mut SenderData {
self.surb_senders.entry(*tag).or_default()
}
async fn current_topology_metadata(&self) -> Option<NymTopologyMetadata> {
self.topology_access.current_metadata().await
}
fn insert_pending_replies<I: IntoIterator<Item = FragmentWithMaxRetransmissions>>(
&mut self,
recipient: &AnonymousSenderTag,
fragments: I,
lane: TransmissionLane,
) {
trace!("buffering pending replies for {recipient}");
self.surb_senders
.entry(*recipient)
.or_default()
.pending_replies
.store(&lane, fragments)
}
fn re_insert_pending_replies(
&mut self,
recipient: &AnonymousSenderTag,
fragments: Vec<(TransmissionLane, FragmentWithMaxRetransmissions)>,
) {
trace!("re-inserting pending replies for {recipient}");
// the buffer should ALWAYS exist at this point, if it doesn't, it's a bug...
self.surb_senders
.entry(*recipient)
.or_default()
.pending_replies
.store_multiple(fragments)
}
fn re_insert_pending_retransmission(
&mut self,
recipient: &AnonymousSenderTag,
data: Vec<Arc<PendingAcknowledgement>>,
) {
trace!("re-inserting pending retransmissions for {recipient}");
// the underlying entry MUST exist as we've just got data from there
// and we hold a mut reference
let map_entry = &mut self
.surb_senders
.get_mut(recipient)
.expect("our pending retransmission entry is somehow gone!")
.pending_retransmissions;
for pending in data {
// if it's 0, we don't need to do anything - we just got that ack!
if Arc::strong_count(&pending) > 1 {
let id = pending.inner_fragment_identifier();
let downgraded = Arc::downgrade(&pending);
map_entry.insert(id, downgraded);
}
}
}
fn should_request_more_surbs(&self, target: &AnonymousSenderTag) -> bool {
trace!("checking if we should request more surbs from {target}");
let total_queue = self
.surb_senders
.get(target)
.map(|pending| pending.total_pending())
.unwrap_or_default();
// only consider 'fresh' surbs
let available_surbs = self.surbs_storage.available_fresh_surbs(target);
let pending_surbs = self.surbs_storage.pending_reception(target) as usize;
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_surbs_threshold = self.surbs_storage.max_surb_threshold();
let min_surbs_threshold_buffer =
self.config.reply_surbs.minimum_reply_surb_threshold_buffer;
// After clearing the queue, we want to have at least `min_surbs_threshold` surbs available
// and reserved for requesting additional surbs, and in addition to that we also want to
// have `min_surbs_threshold_buffer` surbs available proactively.
let target_surbs_after_clearing_queue = min_surbs_threshold + min_surbs_threshold_buffer;
// Check if we have enough surbs to handle the total queue and maintain minimum thresholds
let total_required_surbs = total_queue + target_surbs_after_clearing_queue;
let total_available_surbs = pending_surbs + available_surbs;
debug!("available surbs: {available_surbs} pending surbs: {pending_surbs} threshold range: {min_surbs_threshold}..+{min_surbs_threshold_buffer}..{max_surbs_threshold}");
// We should request more surbs if:
// 1. We haven't hit the maximum surb threshold, and
// 2. We don't have enough surbs to handle the queue plus minimum thresholds
let is_below_max_threshold = total_available_surbs < max_surbs_threshold;
let is_below_required_surbs = total_available_surbs < total_required_surbs;
is_below_max_threshold && is_below_required_surbs
}
pub(crate) async fn handle_send_reply(
&mut self,
recipient_tag: AnonymousSenderTag,
data: Vec<u8>,
lane: TransmissionLane,
max_retransmissions: Option<u32>,
) {
if !self.surbs_storage.contains_surbs_for(&recipient_tag) {
if self
.unavailable
.insert(recipient_tag, OffsetDateTime::now_utc())
.is_none()
{
// don't report it every single time
warn!("received reply request for {recipient_tag} but we don't have any surbs stored for that recipient!");
} else {
trace!("received reply request for {recipient_tag} but we don't have any surbs stored for that recipient!");
}
return;
}
trace!("handling reply to {recipient_tag}");
let mut fragments = self.message_handler.split_reply_message(data);
let total_size = fragments.len();
trace!("This reply requires {total_size} SURBs");
// for the purposes of sending reply, do allow using possibly stale entries
let available_surbs = self.surbs_storage.available_surbs(&recipient_tag);
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_to_send = if available_surbs > min_surbs_threshold {
min(fragments.len(), available_surbs - min_surbs_threshold)
} else {
0
};
if max_to_send > 0 {
let (surbs, surbs_left) = self
.surbs_storage
.get_reply_surbs(&recipient_tag, max_to_send);
debug!(
"retrieved {} reply surbs. {surbs_left} surbs remaining in storage",
surbs.as_ref().map(|s| s.len()).unwrap_or_default()
);
if let Some(reply_surbs) = surbs {
let to_send = fragments
.drain(..reply_surbs.len())
.map(|f| FragmentWithMaxRetransmissions {
fragment: f,
max_retransmissions,
})
.collect::<Vec<_>>();
if let Err(err) = self
.message_handler
.try_send_reply_chunks_on_lane(
recipient_tag,
to_send.clone(),
reply_surbs,
lane,
)
.await
{
let err = err.return_unused_surbs(&self.surbs_storage, &recipient_tag);
warn!("failed to send reply to {recipient_tag}: {err}");
info!(
"buffering {no_fragments} fragments for {recipient_tag}",
no_fragments = to_send.len()
);
self.insert_pending_replies(&recipient_tag, to_send, lane);
}
}
}
// if there's leftover data we didn't send because we didn't have enough (or any) surbs - buffer it
if !fragments.is_empty() {
// Ideally we should have enough surbs above the minimum threshold to handle sending
// new replies without having to first request more surbs. That's why I'd like to log
// these cases as they might indicate a problem with the surb management.
debug!(
"buffering {no_fragments} fragments for {recipient_tag}",
no_fragments = fragments.len()
);
let fragments: Vec<_> = fragments
.into_iter()
.map(|fragment| FragmentWithMaxRetransmissions {
fragment,
max_retransmissions,
})
.collect();
self.insert_pending_replies(&recipient_tag, fragments, lane);
}
if self.should_request_more_surbs(&recipient_tag) {
self.request_reply_surbs_for_queue_clearing(recipient_tag)
.await;
}
}
async fn request_additional_reply_surbs(
&mut self,
target: AnonymousSenderTag,
amount: u32,
) -> Result<(), PreparationError> {
debug!("requesting {amount} additional reply surbs for {target}");
let (reply_surb, _) = self
.surbs_storage
.get_reply_surb_ignoring_threshold(&target);
let reply_surb = reply_surb.ok_or(PreparationError::NotEnoughSurbs {
available: 0,
required: 1,
})?;
if let Err(err) = self
.message_handler
.try_request_additional_reply_surbs(target, reply_surb, amount)
.await
{
let err = err.return_unused_surbs(&self.surbs_storage, &target);
warn!("failed to request additional surbs from {target}: {err}",);
return Err(err);
} else {
self.surbs_storage
.increment_pending_reception(&target, amount);
}
Ok(())
}
async fn try_clear_pending_retransmission(&mut self, target: AnonymousSenderTag) {
trace!("trying to clear pending retransmission queue");
let available_surbs = self.surbs_storage.available_surbs(&target);
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_to_clear = if available_surbs > min_surbs_threshold {
available_surbs - min_surbs_threshold
} else {
trace!("we don't have enough surbs for retransmission queue clearing...");
return;
};
trace!("we can clear up to {max_to_clear} entries");
let Some(pending) = self.surb_senders.get_mut(&target) else {
trace!("no pending entry for {target}!");
return;
};
let mut to_take = Vec::new();
while to_take.len() < max_to_clear {
if let Some((_, data)) = pending.pending_retransmissions.pop_first() {
// no need to do anything if we failed to upgrade the reference,
// it means we got the ack while the data was waiting in the queue
if let Some(upgraded) = data.upgrade() {
to_take.push(upgraded)
}
} else {
// our map is empty!
break;
}
}
if to_take.is_empty() {
// no need to do anything
return;
}
let (surbs_for_reply, _) = self.surbs_storage.get_reply_surbs(&target, to_take.len());
let Some(surbs_for_reply) = surbs_for_reply else {
error!("somehow different task has stolen our reply surbs! - this should have been impossible");
self.re_insert_pending_retransmission(&target, to_take);
return;
};
let to_send_vec = to_take.iter().map(|ack| ack.fragment_data()).collect();
let prepared_fragments = match self
.message_handler
.prepare_reply_chunks_for_sending(to_send_vec, surbs_for_reply)
.await
{
Ok(prepared) => prepared,
Err(err) => {
let err = err.return_unused_surbs(&self.surbs_storage, &target);
self.re_insert_pending_retransmission(&target, to_take);
warn!("failed to clear pending retransmission queue for {target}: {err}",);
return;
}
};
// we can't fail at this point, so drop all references to acks so that timer updates wouldn't blow up
drop(to_take);
self.message_handler
.send_retransmission_reply_chunks(prepared_fragments, TransmissionLane::Retransmission)
.await;
}
fn pop_at_most_pending_replies(
&mut self,
from: &AnonymousSenderTag,
amount: usize,
) -> Option<Vec<(TransmissionLane, FragmentWithMaxRetransmissions)>> {
// if possible, pop all pending replies, if not, pop only entries for which we'd have a reply surb
let pending = self.surb_senders.get_mut(from)?;
let total = pending.pending_replies.total_size();
trace!("pending queue has {total} elements");
if total == 0 {
return None;
}
pending
.pending_replies
.pop_at_most_n_next_messages_at_random(amount)
}
async fn try_clear_pending_queue(&mut self, target: AnonymousSenderTag) {
trace!("trying to clear pending queue");
let available_surbs = self.surbs_storage.available_surbs(&target);
let min_surbs_threshold = self.surbs_storage.min_surb_threshold();
let max_to_clear = if available_surbs > min_surbs_threshold {
available_surbs - min_surbs_threshold
} else {
trace!("we don't have enough surbs for queue clearing...");
return;
};
trace!("we can clear up to {max_to_clear} entries");
// we're guaranteed to not get more entries than we have reply surbs for
if let Some(to_send) = self.pop_at_most_pending_replies(&target, max_to_clear) {
let to_send_clone = to_send.clone();
if to_send_clone.is_empty() {
panic!(
"please let the devs know if you ever see this message (reply_controller.rs)"
);
}
let (surbs_for_reply, _) = self
.surbs_storage
.get_reply_surbs(&target, to_send_clone.len());
let Some(surbs_for_reply) = surbs_for_reply else {
error!("somehow different task has stolen our reply surbs! - this should have been impossible");
self.re_insert_pending_replies(&target, to_send);
return;
};
if let Err(err) = self
.message_handler
.try_send_reply_chunks(target, to_send_clone, surbs_for_reply)
.await
{
let err = err.return_unused_surbs(&self.surbs_storage, &target);
self.re_insert_pending_replies(&target, to_send);
warn!("failed to clear pending queue for {target}: {err}");
}
} else {
trace!("the pending queue is empty");
}
}
fn reset_rerequest_counter(&mut self, from: &AnonymousSenderTag) {
if let Some(pending) = self.surb_senders.get_mut(from) {
pending.reset_current_clear_rerequest_counter()
}
}
pub(crate) async fn handle_received_surbs(
&mut self,
from: AnonymousSenderTag,
reply_surbs: Vec<ReplySurbWithKeyRotation>,
from_surb_request: bool,
) {
trace!("handling received surbs");
// clear the requesting flag since we should have been asking for surbs
if from_surb_request {
self.surbs_storage
.decrement_pending_reception(&from, reply_surbs.len() as u32);
}
// store received surbs
self.surbs_storage.insert_fresh_surbs(&from, reply_surbs);
// reset, if applicable, request counter
self.reset_rerequest_counter(&from);
// use as many as we can for clearing pending retransmission queue
self.try_clear_pending_retransmission(from).await;
// use as many as we can for clearing pending 'normal' queue
self.try_clear_pending_queue(from).await;
// if we have to, request more
if self.should_request_more_surbs(&from) {
self.request_reply_surbs_for_queue_clearing(from).await;
}
}
fn buffer_pending_ack(
&mut self,
recipient: AnonymousSenderTag,
ack_ref: Arc<PendingAcknowledgement>,
weak_ack_ref: Weak<PendingAcknowledgement>,
) {
let frag_id = ack_ref.inner_fragment_identifier();
let pending = self.surb_senders.entry(recipient).or_default();
if let Entry::Vacant(e) = pending.pending_retransmissions.entry(frag_id) {
e.insert(weak_ack_ref);
} else {
warn!(
"we're already trying to retransmit {frag_id}. We must be really behind in surbs!"
);
}
}
pub(crate) async fn handle_reply_retransmission(
&mut self,
recipient_tag: AnonymousSenderTag,
timed_out_ack: Weak<PendingAcknowledgement>,
extra_surbs_request: bool,
) {
// seems we got the ack in the end
let ack_ref = match timed_out_ack.upgrade() {
Some(ack) => ack,
None => {
debug!("we received the ack for one of the reply packets as we were putting it in the retransmission queue");
return;
}
};
// if this is retransmission for obtaining additional reply surbs,
// we can dip below the storage threshold
let (maybe_reply_surb, _) = if extra_surbs_request {
self.surbs_storage
.get_reply_surb_ignoring_threshold(&recipient_tag)
} else {
self.surbs_storage.get_reply_surb(&recipient_tag)
};
if let Some(reply_surb) = maybe_reply_surb {
match self
.message_handler
.try_prepare_single_reply_chunk_for_sending(reply_surb, ack_ref.fragment_data())
.await
{
Ok(prepared) => {
// drop the ack ref so that controller would not panic on `UpdateTimer` if that task
// got to handle the action before this function terminated (which is very much
// possible if `forward_messages` takes a while)
drop(ack_ref);
self.message_handler
.update_ack_delay(prepared.fragment_identifier, prepared.total_delay);
self.message_handler
.forward_messages(vec![prepared.into()], TransmissionLane::Retransmission)
.await;
}
Err(err) => {
let err = err.return_unused_surbs(&self.surbs_storage, &recipient_tag);
warn!("failed to prepare message for retransmission - {err}");
// we buffer that packet and to try another day
self.buffer_pending_ack(recipient_tag, ack_ref, timed_out_ack);
if self.should_request_more_surbs(&recipient_tag) {
self.request_reply_surbs_for_queue_clearing(recipient_tag)
.await;
}
}
};
} else {
self.buffer_pending_ack(recipient_tag, ack_ref, timed_out_ack);
if self.should_request_more_surbs(&recipient_tag) {
self.request_reply_surbs_for_queue_clearing(recipient_tag)
.await;
}
}
}
// to be honest this doesn't make a lot of sense in the context of `connection_id`,
// it should really be asked per tag
pub(crate) fn handle_lane_queue_length(
&self,
connection_id: ConnectionId,
response_channel: oneshot::Sender<usize>,
) {
// TODO: if we ever have duplicate ids for different senders, it means our rng is super weak
// thus I don't think we have to worry about it?
let lane = TransmissionLane::ConnectionId(connection_id);
for buf in self.surb_senders.values().map(|p| &p.pending_replies) {
if let Some(length) = buf.lane_length(&lane) {
if response_channel.send(length).is_err() {
error!("the requester for lane queue length has dropped the response channel!")
}
return;
}
}
// make sure that if we didn't find that lane, we reply with 0
if response_channel.send(0).is_err() {
error!("the requester for lane queue length has dropped the response channel!")
}
}
// TODO: modify this method to more accurately determine the amount of surbs it needs to request
// it should take into consideration the average latency, sending rate and queue size.
// it should request as many surbs as it takes to saturate its sending rate before next batch arrives
async fn request_reply_surbs_for_queue_clearing(&mut self, target: AnonymousSenderTag) {
trace!("requesting surbs for queue clearing");
let total_queue = self
.surb_senders
.get(&target)
.map(|pending| pending.total_pending() as u32)
.unwrap_or_default();
let min_surbs_buffer = self.config.reply_surbs.minimum_reply_surb_threshold_buffer as u32;
// To proactively request additional surbs, we aim to have a buffer of extra surbs in our
// storage.
let total_queue_with_buffer = total_queue + min_surbs_buffer;
let request_size = min(
self.config.reply_surbs.maximum_reply_surb_request_size,
max(
total_queue_with_buffer,
self.config.reply_surbs.minimum_reply_surb_request_size,
),
);
if let Err(err) = self
.request_additional_reply_surbs(target, request_size)
.await
{
let now = OffsetDateTime::now_utc();
let sender_info = self.get_or_create_surb_sender(&target);
let last_failure = sender_info.reset_last_request_failure(now);
// only log at higher level if it's the first time this error has occurred in a while
if now - last_failure > time::Duration::seconds(30) {
warn!("failed to request more surbs to clear pending queue of size {total_queue} (attempted to request: {request_size}): {err}")
} else {
debug!("failed to request more surbs to clear pending queue of size {total_queue} (attempted to request: {request_size}): {err}")
}
}
}
pub(crate) async fn inspect_stale_pending_data(&mut self) {
let mut to_request = Vec::new();
let mut to_remove = Vec::new();
let now = OffsetDateTime::now_utc();
for (pending_reply_target, vals) in self.surb_senders.iter_mut() {
// for now recreate old behaviour
let retransmission_buf = &vals.pending_replies;
if retransmission_buf.is_empty() {
continue;
}
let Some(last_received_time) = self
.surbs_storage
.surbs_last_received_at(pending_reply_target)
else {
error!("we have {} pending replies for {pending_reply_target}, but we somehow never received any reply surbs from them!", retransmission_buf.total_size());
to_remove.push(*pending_reply_target);
continue;
};
let diff = now - last_received_time;
let max_rerequest_wait = self
.config
.reply_surbs
.maximum_reply_surb_rerequest_waiting_period;
let max_drop_wait = self
.config
.reply_surbs
.maximum_reply_surb_drop_waiting_period;
let max_rerequests = self.config.reply_surbs.maximum_reply_surbs_rerequests;
// if we have already requested extra surbs because of the stale entry,
// don't do it again (otherwise we'll get stuck in a constant cycle of requesting more surbs
// if client is offline)
if vals.current_clear_rerequest_counter > max_rerequests {
to_remove.push(*pending_reply_target);
debug!("we have reached the maximum threshold of attempting to request surbs from {pending_reply_target}. dropping the sender");
continue;
}
if diff > max_rerequest_wait {
if diff > max_drop_wait {
to_remove.push(*pending_reply_target)
} else {
debug!("We haven't received any surbs in {} from {pending_reply_target}. Going to explicitly ask for more", humantime::format_duration(diff.unsigned_abs()));
vals.increment_current_clear_rerequest_counter();
to_request.push(*pending_reply_target);
}
}
}
for pending_reply_target in to_request {
self.request_reply_surbs_for_queue_clearing(pending_reply_target)
.await;
self.surbs_storage
.reset_pending_reception(&pending_reply_target)
}
for to_remove in to_remove {
// TODO: in the 'old' version we just removed pending messages,
// not retransmissions, but I think those should follow the same logic.
// if something breaks because of that. I guess here is your explanation, future reader
self.surb_senders.remove(&to_remove);
}
}
pub(crate) async fn check_surb_refresh(&mut self) {
let Some(current_rotation_id) = self.topology_access.current_key_rotation_id().await else {
warn!("failed to retrieve current key rotation id from the network topology");
return;
};
if let SurbRefreshState::WaitingForNextRotation { last_known } = self.surb_refresh_state {
if last_known == current_rotation_id {
trace!("no changes in key rotation id");
} else {
// key rotation actually changed and given the polling rate (1/8th epoch) we should have plenty
// of time to perform the upgrade.
// but wait for one more call before doing this so that the clients could also resync
// their topologies and discover new rotation
self.surb_refresh_state = SurbRefreshState::ScheduledForNextInvocation;
}
return;
}
// here we are in `SurbRefreshState::ScheduledForNextInvocation` state
let mut marked_as_stale = HashMap::new();
// 1. mark all existing surbs we have as possibly stale
for mut map_entry in self.surbs_storage.as_raw_iter_mut() {
let (sender, received) = map_entry.pair_mut();
let num_downgraded = received.downgrade_freshness();
trace!("{sender}: {num_downgraded} downgraded");
if num_downgraded != 0 {
marked_as_stale.insert(*sender, num_downgraded);
}
}
// 2. attempt to re-request the equivalent number of fresh surbs
// TODO PROBLEM: if our request gets lost, we might be in trouble...
// we need some sort of retry mechanism
for (sender, num_to_request) in marked_as_stale {
if self
.request_additional_reply_surbs(sender, num_to_request as u32)
.await
.is_err()
{
warn!("surb refresh request failed")
}
}
self.surb_refresh_state = SurbRefreshState::WaitingForNextRotation {
last_known: current_rotation_id,
};
}
pub(crate) async fn inspect_and_clear_stale_data(&mut self, now: OffsetDateTime) {
// technically we don't know if epoch is stuck, but we're flying in blind here,
// so we have to assume the worst and not purge anything depending on proper epoch progression
let is_epoch_stuck = self
.current_topology_metadata()
.await
.map(|m| self.config.key_rotation.epoch_stuck(m))
.unwrap_or(false);
// expected time of when the CURRENT key rotation has begun
let expected_current_key_rotation_start = self
.config
.key_rotation
.expected_current_key_rotation_start(now);
// expected ID of the CURRENT key rotation
let expected_current_key_rotation = self
.config
.key_rotation
.expected_current_key_rotation_id(now);
// time of the start of one epoch BEFORE the CURRENT rotation has begun
// this indicates the starting time of when packets with the current keys might have been constructed
let prior_epoch_start =
expected_current_key_rotation_start - self.config.key_rotation.epoch_duration;
// time of the start of one epoch AFTER the current rotation has begun
// this indicates the end of transition period and any packets constructed with keys different
// from the current one are definitely invalid
let following_epoch_start =
expected_current_key_rotation_start + self.config.key_rotation.epoch_duration;
// define a closure for validating individual surbs
// (we have to run it twice for different piles)
let basic_surb_retention_logic = |received_surb: &ReceivedReplySurb| {
if is_epoch_stuck {
let diff = now - received_surb.received_at();
return diff < self.config.key_rotation.rotation_lifetime();
}
if received_surb.received_at() < prior_epoch_start {
// it's definitely from previous rotation
return false;
}
let surb_rotation = received_surb.key_rotation();
if surb_rotation.is_unknown() {
// can't do anything, so just retain it
return true;
}
// TODO: will this backfire during transition period where we need surbs to refresh surbs
// and we failed to send a request?
if surb_rotation.is_even() && expected_current_key_rotation % 2 == 1 {
return false;
}
if surb_rotation.is_odd() && expected_current_key_rotation % 2 == 0 {
return false;
}
true
};
// 1. purge full old clients data (this applies to RECEIVER)
self.surbs_storage.retain(|_, received| {
if is_epoch_stuck {
// if epoch is stuck, we can't do much (because we don't know for certain if rotation has advanced)
// apart from the basic check of surbs being received more than maximum lifetime of a rotation
// because at that point we know they must be invalid
let diff = now - received.surbs_last_received_at();
return diff < self.config.key_rotation.rotation_lifetime();
}
// if surbs were received more than 1h before the start of the current rotation,
// they're DEFINITELY invalid.
// if it was up until 1h AFTER the start of the current rotation they MIGHT be valid -
// we don't know for sure, unless the client explicitly attached rotation information
// (which only applies to more recent versions of clients so we can't 100% rely on that)
if received.surbs_last_received_at() < prior_epoch_start {
return false;
}
// 1.1. check individual surbs (same basic logic applies)
received.retain_fresh_surbs(&basic_surb_retention_logic);
// 1.2. check the possibly stale entries
// 1.2.1. check if we're beyond the key rotation transition period,
// if so those surbs are definitely unusable
if now > following_epoch_start {
received.drop_possibly_stale_surbs();
}
// 1.2.2. otherwise continue with the same logic as the fresh ones
received.retain_possibly_stale_surbs(&basic_surb_retention_logic);
// no surbs left, we're not expecting any AND we haven't received anything in a while
// (i.e. sender probably abandoned us)
let max_drop_wait = self
.config
.reply_surbs
.maximum_reply_surb_drop_waiting_period;
let last_received = received.surbs_last_received_at();
let possibly_abandoned = last_received + max_drop_wait < now;
if received.is_empty() && received.pending_reception() == 0 && possibly_abandoned {
return false;
}
true
});
// 1.3 inspect old unavailable receivers to clear any stale data
self.unavailable
.retain(|_, last_reported| now - *last_reported < time::Duration::seconds(30));
}
}
@@ -3,12 +3,12 @@
use crate::client::real_messages_control::acknowledgement_control::PendingAcknowledgement;
use futures::channel::{mpsc, oneshot};
use log::error;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::anonymous_replies::ReplySurb;
use nym_sphinx::anonymous_replies::ReplySurbWithKeyRotation;
use nym_task::connections::{ConnectionId, TransmissionLane};
use std::sync::Weak;
use tracing::error;
pub(crate) fn new_control_channels() -> (ReplyControllerSender, ReplyControllerReceiver) {
let (tx, rx) = mpsc::unbounded();
@@ -81,7 +81,7 @@ impl ReplyControllerSender {
pub(crate) fn send_additional_surbs(
&self,
sender_tag: AnonymousSenderTag,
reply_surbs: Vec<ReplySurb>,
reply_surbs: Vec<ReplySurbWithKeyRotation>,
from_surb_request: bool,
) -> Result<(), ReplyControllerSenderError> {
self.0
@@ -167,7 +167,7 @@ pub enum ReplyControllerMessage {
AdditionalSurbs {
sender_tag: AnonymousSenderTag,
reply_surbs: Vec<ReplySurb>,
reply_surbs: Vec<ReplySurbWithKeyRotation>,
from_surb_request: bool,
},
@@ -0,0 +1,101 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::replies::reply_controller::Config;
use nym_client_core_surb_storage::{CombinedReplyStorage, SentReplyKeys, UsedSenderTags};
use nym_crypto::aes::cipher::crypto_common::rand_core::CryptoRng;
use nym_sphinx::addressing::Recipient;
use rand::Rng;
use std::cmp::min;
use std::time::Duration;
use time::OffsetDateTime;
use tracing::{debug, trace, warn};
/// Reply controller responsible for controlling sender-related part
/// of replies, such as checking if any reply keys are stale
pub struct SenderReplyController<R> {
config: Config,
tags_storage: UsedSenderTags,
sent_reply_keys: SentReplyKeys,
message_handler: MessageHandler<R>,
}
impl<R> SenderReplyController<R>
where
R: CryptoRng + Rng,
{
pub(crate) fn new(
config: Config,
storage: &CombinedReplyStorage,
message_handler: MessageHandler<R>,
) -> Self {
SenderReplyController {
config,
tags_storage: storage.tags_storage(),
sent_reply_keys: storage.key_storage(),
message_handler,
}
}
pub(crate) async fn handle_surb_request(&mut self, recipient: Recipient, mut amount: u32) {
// 1. check whether we sent any surbs in the past to this recipient, otherwise
// they have no business in asking for more
if !self.tags_storage.exists(&recipient) {
warn!("{recipient} asked us for reply SURBs even though we never sent them any anonymous messages before!");
return;
}
// 2. check whether the requested amount is within sane range
if amount
> self
.config
.reply_surbs
.maximum_allowed_reply_surb_request_size
{
warn!("The requested reply surb amount is larger than our maximum allowed ({amount} > {}). Lowering it to a more sane value...", self.config.reply_surbs.maximum_allowed_reply_surb_request_size);
amount = self
.config
.reply_surbs
.maximum_allowed_reply_surb_request_size;
}
// 3. construct and send the surbs away
// (send them in smaller batches to make the experience a bit smoother
let mut remaining = amount;
while remaining > 0 {
let to_send = min(remaining, 100);
if let Err(err) = self
.message_handler
.try_send_additional_reply_surbs(
recipient,
to_send,
nym_sphinx::params::PacketType::Mix,
)
.await
{
warn!("failed to send additional surbs to {recipient} - {err}");
} else {
trace!("sent {to_send} reply SURBs to {recipient}");
}
remaining -= to_send;
}
}
pub(crate) fn inspect_and_clear_stale_data(&self, now: OffsetDateTime) {
// check reply keys (this applies to SENDER)
self.sent_reply_keys.retain(|_, reply_key| {
let diff = now - reply_key.sent_at;
if diff > self.config.reply_surbs.maximum_reply_key_age {
let std_diff = Duration::try_from(diff).unwrap_or_default();
let diff_formatted = humantime::format_duration(std_diff);
debug!("it's been {diff_formatted} since we created this reply key. it's probably never going to get used, so we're going to purge it...");
false
} else {
true
}
});
}
}
@@ -93,14 +93,14 @@ impl StatisticsControl {
None,
);
if let Err(err) = self.report_tx.send(report_message).await {
log::error!("Failed to report client stats: {:?}", err);
tracing::error!("Failed to report client stats: {err:?}");
} else {
self.stats.reset();
}
}
async fn run(&mut self) {
log::debug!("Started StatisticsControl with graceful shutdown support");
tracing::debug!("Started StatisticsControl with graceful shutdown support");
#[cfg(not(target_arch = "wasm32"))]
let mut stats_report_interval = tokio_stream::wrappers::IntervalStream::new(
@@ -133,13 +133,13 @@ impl StatisticsControl {
tokio::select! {
biased;
_ = self.task_client.recv() => {
log::trace!("StatisticsControl: Received shutdown");
tracing::trace!("StatisticsControl: Received shutdown");
break;
},
stats_event = self.stats_rx.recv() => match stats_event {
Some(stats_event) => self.stats.handle_event(stats_event),
None => {
log::trace!("StatisticsControl: shutting down due to closed stats channel");
tracing::trace!("StatisticsControl: shutting down due to closed stats channel");
break;
}
},
@@ -161,7 +161,7 @@ impl StatisticsControl {
}
}
}
log::debug!("StatisticsControl: Exiting");
tracing::debug!("StatisticsControl: Exiting");
}
pub(crate) fn start(mut self) {
@@ -2,7 +2,8 @@
// SPDX-License-Identifier: Apache-2.0
use nym_sphinx::addressing::clients::Recipient;
use nym_topology::{NymRouteProvider, NymTopology, NymTopologyError};
use nym_topology::{NymRouteProvider, NymTopology, NymTopologyError, NymTopologyMetadata};
use nym_validator_client::models::KeyRotationId;
use std::ops::Deref;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
@@ -134,6 +135,21 @@ impl TopologyAccessor {
}
}
pub async fn current_mixnet_epoch_id(&self) -> Option<u32> {
let route_provider = self.current_route_provider().await?;
Some(route_provider.absolute_epoch_id())
}
pub async fn current_key_rotation_id(&self) -> Option<KeyRotationId> {
let route_provider = self.current_route_provider().await?;
Some(route_provider.current_key_rotation())
}
pub async fn current_metadata(&self) -> Option<NymTopologyMetadata> {
let route_provider = self.current_route_provider().await?;
Some(route_provider.metadata())
}
pub async fn manually_change_topology(&self, new_topology: NymTopology) {
self.inner.controlled_manually.store(true, Ordering::SeqCst);
self.inner.update(Some(new_topology)).await;
@@ -4,11 +4,11 @@
use crate::spawn_future;
pub(crate) use accessor::{TopologyAccessor, TopologyReadPermit};
use futures::StreamExt;
use log::*;
use nym_sphinx::addressing::nodes::NodeIdentity;
use nym_task::TaskClient;
use nym_topology::NymTopologyError;
use std::time::Duration;
use tracing::*;
#[cfg(not(target_arch = "wasm32"))]
use tokio::time::sleep;
@@ -20,7 +20,7 @@ mod accessor;
pub mod nym_api_provider;
pub use nym_api_provider::{Config as NymApiTopologyProviderConfig, NymApiTopologyProvider};
pub use nym_topology::provider_trait::TopologyProvider;
pub use nym_topology::provider_trait::{ToTopologyMetadata, TopologyProvider};
// TODO: move it to config later
const MAX_FAILURE_COUNT: usize = 10;
@@ -169,12 +169,12 @@ impl TopologyRefresher {
self.try_refresh().await;
},
_ = self.task_client.recv() => {
log::trace!("TopologyRefresher: Received shutdown");
tracing::trace!("TopologyRefresher: Received shutdown");
},
}
}
self.task_client.recv_timeout().await;
log::debug!("TopologyRefresher: Exiting");
tracing::debug!("TopologyRefresher: Exiting");
})
}
}
@@ -2,13 +2,12 @@
// SPDX-License-Identifier: Apache-2.0
use async_trait::async_trait;
use log::{debug, error, warn};
use nym_topology::provider_trait::TopologyProvider;
use nym_topology::provider_trait::{ToTopologyMetadata, TopologyProvider};
use nym_topology::NymTopology;
use nym_validator_client::UserAgent;
use rand::prelude::SliceRandom;
use rand::thread_rng;
use std::cmp::min;
use tracing::{debug, error, warn};
use url::Url;
#[derive(Debug)]
@@ -49,18 +48,10 @@ impl NymApiTopologyProvider {
pub fn new(
config: impl Into<Config>,
mut nym_api_urls: Vec<Url>,
user_agent: Option<UserAgent>,
mut validator_client: nym_validator_client::client::NymApiClient,
) -> Self {
nym_api_urls.shuffle(&mut thread_rng());
let validator_client = if let Some(user_agent) = user_agent {
nym_validator_client::client::NymApiClient::new_with_user_agent(
nym_api_urls[0].clone(),
user_agent,
)
} else {
nym_validator_client::client::NymApiClient::new(nym_api_urls[0].clone())
};
validator_client.change_nym_api(nym_api_urls[0].clone());
NymApiTopologyProvider {
config: config.into(),
@@ -89,55 +80,76 @@ impl NymApiTopologyProvider {
let rewarded_set_fut = self.validator_client.get_current_rewarded_set();
let topology = if self.config.use_extended_topology {
let all_nodes_fut = self.validator_client.get_all_basic_nodes();
let all_nodes_fut = self.validator_client.get_all_basic_nodes_with_metadata();
// Join rewarded_set_fut and all_nodes_fut concurrently
let (rewarded_set, all_nodes) = futures::try_join!(rewarded_set_fut, all_nodes_fut)
let (rewarded_set, all_nodes_res) = futures::try_join!(rewarded_set_fut, all_nodes_fut)
.inspect_err(|err| error!("failed to get network nodes: {err}"))
.ok()?;
let metadata = all_nodes_res.metadata;
let all_nodes = all_nodes_res.nodes;
debug!(
"there are {} nodes on the network (before filtering)",
all_nodes.len()
);
let mut topology = NymTopology::new_empty(rewarded_set);
topology.add_additional_nodes(all_nodes.iter().filter(|n| {
n.performance.round_to_integer() >= self.config.min_node_performance()
}));
let nodes_filtered = all_nodes
.into_iter()
.filter(|n| n.performance.round_to_integer() >= self.config.min_node_performance())
.collect::<Vec<_>>();
topology
NymTopology::new(metadata.to_topology_metadata(), rewarded_set, Vec::new())
.with_skimmed_nodes(&nodes_filtered)
} else {
// if we're not using extended topology, we're only getting active set mixnodes and gateways
let mixnodes_fut = self
.validator_client
.get_all_basic_active_mixing_assigned_nodes();
.get_all_basic_active_mixing_assigned_nodes_with_metadata();
// TODO: we really should be getting ACTIVE gateways only
let gateways_fut = self.validator_client.get_all_basic_entry_assigned_nodes();
let gateways_fut = self
.validator_client
.get_all_basic_entry_assigned_nodes_with_metadata();
let (rewarded_set, mixnodes, gateways) =
let (rewarded_set, mixnodes_res, gateways_res) =
futures::try_join!(rewarded_set_fut, mixnodes_fut, gateways_fut)
.inspect_err(|err| {
error!("failed to get network nodes: {err}");
})
.ok()?;
let metadata = mixnodes_res.metadata;
let mixnodes = mixnodes_res.nodes;
if !gateways_res.metadata.consistency_check(&metadata) {
warn!("inconsistent nodes metadata between mixnodes and gateways calls! {metadata:?} and {:?}", gateways_res.metadata);
return None;
}
let gateways = gateways_res.nodes;
debug!(
"there are {} mixnodes and {} gateways in total (before performance filtering)",
mixnodes.len(),
gateways.len()
);
let mut topology = NymTopology::new_empty(rewarded_set);
topology.add_additional_nodes(mixnodes.iter().filter(|m| {
m.performance.round_to_integer() >= self.config.min_mixnode_performance
}));
topology.add_additional_nodes(gateways.iter().filter(|m| {
m.performance.round_to_integer() >= self.config.min_gateway_performance
}));
let mut nodes = Vec::new();
for mix in mixnodes {
if mix.performance.round_to_integer() >= self.config.min_mixnode_performance {
nodes.push(mix)
}
}
for gateway in gateways {
if gateway.performance.round_to_integer() >= self.config.min_gateway_performance {
nodes.push(gateway)
}
}
topology
NymTopology::new(metadata.to_topology_metadata(), rewarded_set, Vec::new())
.with_skimmed_nodes(&nodes)
};
if !topology.is_minimally_routable() {
@@ -36,11 +36,18 @@ impl SizedData for Fragment {
}
}
#[derive(Default)]
pub(crate) struct TransmissionBuffer<T> {
buffer: HashMap<TransmissionLane, LaneBufferEntry<T>>,
}
impl<T> Default for TransmissionBuffer<T> {
fn default() -> Self {
TransmissionBuffer {
buffer: HashMap::new(),
}
}
}
impl<T> TransmissionBuffer<T> {
pub(crate) fn new() -> Self {
TransmissionBuffer {
@@ -211,7 +218,7 @@ impl<T> TransmissionBuffer<T> {
};
let msg = self.pop_front_from_lane(&lane)?;
log::trace!("picking to send from lane: {:?}", lane);
tracing::trace!("picking to send from lane: {lane:?}");
Some((lane, msg))
}
+20 -6
View File
@@ -6,6 +6,7 @@ use nym_crypto::asymmetric::ed25519::Ed25519RecoveryError;
use nym_gateway_client::error::GatewayClientError;
use nym_topology::node::RoutingNodeError;
use nym_topology::{NodeId, NymTopologyError};
use nym_validator_client::nym_api::error::NymAPIError;
use nym_validator_client::ValidatorClientError;
use std::error::Error;
use std::path::PathBuf;
@@ -18,7 +19,7 @@ pub enum ClientCoreError {
#[error("gateway client error ({gateway_id}): {source}")]
GatewayClientError {
gateway_id: String,
source: GatewayClientError,
source: Box<GatewayClientError>,
},
#[error("custom gateway client error: {source}")]
@@ -52,7 +53,15 @@ pub enum ClientCoreError {
#[error("list of nym apis is empty")]
ListOfNymApisIsEmpty,
#[error("the current network topology seem to be insufficient to route any packets through")]
#[error("failed to resolve a query to nym API: {source}")]
NymApiQueryFailure {
#[from]
source: NymAPIError,
},
#[error(
"the current network topology seem to be insufficient to route any packets through:\n\t{0}"
)]
InsufficientNetworkTopology(#[from] NymTopologyError),
#[error("experienced a failure with our reply surb persistent storage: {source}")]
@@ -88,10 +97,7 @@ pub enum ClientCoreError {
},
#[error("failed to establish connection to gateway: {source}")]
GatewayConnectionFailure {
#[from]
source: tungstenite::Error,
},
GatewayConnectionFailure { source: Box<tungstenite::Error> },
#[cfg(target_arch = "wasm32")]
#[error("failed to establish gateway connection (wasm)")]
@@ -227,6 +233,14 @@ pub enum ClientCoreError {
HkdfDerivationError {},
}
impl From<tungstenite::Error> for ClientCoreError {
fn from(err: tungstenite::Error) -> ClientCoreError {
ClientCoreError::GatewayConnectionFailure {
source: Box::new(err),
}
}
}
/// Set of messages that the client can send to listeners via the task manager
#[derive(Debug)]
pub enum ClientCoreStatusMessage {
+15 -12
View File
@@ -4,7 +4,6 @@
use crate::error::ClientCoreError;
use crate::init::types::RegistrationResult;
use futures::{SinkExt, StreamExt};
use log::{debug, info, trace, warn};
use nym_crypto::asymmetric::ed25519;
use nym_gateway_client::GatewayClient;
use nym_topology::node::RoutingNode;
@@ -14,6 +13,7 @@ use rand::{seq::SliceRandom, Rng};
#[cfg(unix)]
use std::os::fd::RawFd;
use std::{sync::Arc, time::Duration};
use tracing::{debug, info, trace, warn};
use tungstenite::Message;
use url::Url;
@@ -105,12 +105,15 @@ pub async fn gateways_for_init<R: Rng>(
nym_validator_client::client::NymApiClient::new(nym_api.clone())
};
log::debug!("Fetching list of gateways from: {nym_api}");
tracing::debug!("Fetching list of gateways from: {nym_api}");
let gateways = client.get_all_basic_entry_assigned_nodes().await?;
let gateways = client
.get_all_basic_entry_assigned_nodes_with_metadata()
.await?
.nodes;
info!("nym api reports {} gateways", gateways.len());
log::trace!("Gateways: {:#?}", gateways);
tracing::trace!("Gateways: {gateways:#?}");
// filter out gateways below minimum performance and ones that could operate as a mixnode
// (we don't want instability)
@@ -120,10 +123,10 @@ pub async fn gateways_for_init<R: Rng>(
.filter(|g| g.performance.round_to_integer() >= minimum_performance)
.filter_map(|gateway| gateway.try_into().ok())
.collect::<Vec<_>>();
log::debug!("After checking validity: {}", valid_gateways.len());
log::trace!("Valid gateways: {:#?}", valid_gateways);
tracing::debug!("After checking validity: {}", valid_gateways.len());
tracing::trace!("Valid gateways: {valid_gateways:#?}");
log::info!(
tracing::info!(
"and {} after validity and performance filtering",
valid_gateways.len()
);
@@ -286,7 +289,7 @@ pub(super) fn get_specified_gateway(
gateways: &[RoutingNode],
must_use_tls: bool,
) -> Result<RoutingNode, ClientCoreError> {
log::debug!("Requesting specified gateway: {}", gateway_identity);
tracing::debug!("Requesting specified gateway: {gateway_identity}");
let user_gateway = ed25519::PublicKey::from_base58_string(gateway_identity)
.map_err(ClientCoreError::UnableToCreatePublicKeyFromGatewayId)?;
@@ -326,20 +329,20 @@ pub(super) async fn register_with_gateway(
);
gateway_client.establish_connection().await.map_err(|err| {
log::warn!("Failed to establish connection with gateway!");
tracing::warn!("Failed to establish connection with gateway!");
ClientCoreError::GatewayClientError {
gateway_id: gateway_id.to_base58_string(),
source: err,
source: Box::new(err),
}
})?;
let auth_response = gateway_client
.perform_initial_authentication()
.await
.map_err(|err| {
log::warn!("Failed to register with the gateway {gateway_id}: {err}");
tracing::warn!("Failed to register with the gateway {gateway_id}: {err}");
ClientCoreError::GatewayClientError {
gateway_id: gateway_id.to_base58_string(),
source: err,
source: Box::new(err),
}
})?;
+6 -6
View File
@@ -63,7 +63,7 @@ where
K::StorageError: Send + Sync + 'static,
D::StorageError: Send + Sync + 'static,
{
log::trace!("Setting up new gateway");
tracing::trace!("Setting up new gateway");
// if we're setting up new gateway, we must have had generated long-term client keys before
let client_keys = load_client_keys(key_store).await?;
@@ -202,10 +202,10 @@ where
K::StorageError: Send + Sync + 'static,
D::StorageError: Send + Sync + 'static,
{
log::debug!("Setting up gateway");
tracing::debug!("Setting up gateway");
match setup {
GatewaySetup::MustLoad { gateway_id } => {
log::debug!("GatewaySetup::MustLoad with id: {gateway_id:?}");
tracing::debug!("GatewaySetup::MustLoad with id: {gateway_id:?}");
use_loaded_gateway_details(key_store, details_store, gateway_id).await
}
GatewaySetup::New {
@@ -214,7 +214,7 @@ where
#[cfg(unix)]
connection_fd_callback,
} => {
log::debug!("GatewaySetup::New with spec: {specification:?}");
tracing::debug!("GatewaySetup::New with spec: {specification:?}");
setup_new_gateway(
key_store,
details_store,
@@ -230,9 +230,9 @@ where
gateway_details,
client_keys: managed_keys,
} => {
log::debug!("GatewaySetup::ReuseConnection");
tracing::debug!("GatewaySetup::ReuseConnection");
Ok(reuse_gateway_connection(
authenticated_ephemeral_client,
*authenticated_ephemeral_client,
*gateway_details,
managed_keys,
))
+2 -2
View File
@@ -218,7 +218,7 @@ pub enum GatewaySetup {
ReuseConnection {
/// The authenticated ephemeral client that was created during `init`
authenticated_ephemeral_client: InitGatewayClient,
authenticated_ephemeral_client: Box<InitGatewayClient>,
// Details of this pre-initialised client (i.e. gateway and keys)
gateway_details: Box<GatewayRegistration>,
@@ -261,7 +261,7 @@ impl GatewaySetup {
pub fn try_reuse_connection(init_res: InitialisationResult) -> Result<Self, ClientCoreError> {
if let Some(authenticated_ephemeral_client) = init_res.authenticated_ephemeral_client {
Ok(GatewaySetup::ReuseConnection {
authenticated_ephemeral_client,
authenticated_ephemeral_client: Box::new(authenticated_ephemeral_client),
gateway_details: Box::new(init_res.gateway_registration),
client_keys: init_res.client_keys,
})
+14 -3
View File
@@ -9,7 +9,7 @@ license.workspace = true
[dependencies]
async-trait.workspace = true
dashmap.workspace = true
log.workspace = true
tracing.workspace = true
thiserror.workspace = true
time.workspace = true
@@ -17,15 +17,26 @@ nym-crypto = { path = "../../crypto", optional = true, default-features = false
nym-sphinx = { path = "../../nymsphinx" }
nym-task = { path = "../../task" }
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio]
workspace = true
features = ["fs"]
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.sqlx]
workspace = true
features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate"]
features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate", "time"]
optional = true
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.sqlx-pool-guard]
path = "../../../sqlx-pool-guard"
[build-dependencies]
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
sqlx = { workspace = true, features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate"] }
sqlx = { workspace = true, features = [
"runtime-tokio-rustls",
"sqlite",
"macros",
"migrate",
] }
[features]
fs-surb-storage = ["sqlx", "nym-crypto", "nym-crypto/hashing"]
@@ -0,0 +1,8 @@
/*
* Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
* SPDX-License-Identifier: Apache-2.0
*/
-- default value of 0 implies 'unknown' variant
ALTER TABLE reply_surb
ADD COLUMN encoded_key_rotation TINYINT NOT NULL DEFAULT 0;
@@ -0,0 +1,81 @@
/*
* Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
* SPDX-License-Identifier: Apache-2.0
*/
-- change `previous_flush_timestamp` unix timestamp to `previous_flush` timestamp
CREATE TABLE status_new
(
flush_in_progress INTEGER NOT NULL,
previous_flush TIMESTAMP WITHOUT TIME ZONE NOT NULL,
client_in_use INTEGER NOT NULL
);
INSERT INTO status_new (flush_in_progress, previous_flush, client_in_use)
SELECT flush_in_progress,
datetime(previous_flush_timestamp, 'unixepoch') AS previous_flush,
client_in_use
FROM status;
DROP TABLE status;
ALTER TABLE status_new
RENAME TO status;
-- change `sent_at_timestamp` unix timestamp to `sent_at` timestamp
CREATE TABLE reply_key_new
(
key_digest BLOB NOT NULL UNIQUE,
reply_key BLOB NOT NULL UNIQUE,
sent_at TIMESTAMP WITHOUT TIME ZONE NOT NULL
);
INSERT INTO reply_key_new (key_digest, reply_key, sent_at)
SELECT key_digest,
reply_key,
datetime(sent_at_timestamp, 'unixepoch') AS sent_at
FROM reply_key;
DROP TABLE reply_key;
ALTER TABLE reply_key_new
RENAME TO reply_key;
-- change `last_sent_timestamp` unix timestamp to `sent_at` last_sent
CREATE TABLE reply_surb_sender_new
(
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
last_sent TIMESTAMP WITHOUT TIME ZONE NOT NULL,
tag BLOB NOT NULL UNIQUE
);
INSERT INTO reply_surb_sender_new (id, last_sent, tag)
SELECT id,
datetime(last_sent_timestamp, 'unixepoch') AS last_sent,
tag
FROM reply_surb_sender;
-- recreate `reply_surb` table due to foreign key constraint
CREATE TABLE reply_surb_new
(
reply_surb_sender_id INTEGER NOT NULL,
reply_surb BLOB NOT NULL,
encoded_key_rotation TINYINT NOT NULL,
FOREIGN KEY (reply_surb_sender_id) REFERENCES reply_surb_sender_new (id)
);
INSERT INTO reply_surb_new
SELECT *
FROM reply_surb;
DROP TABLE reply_surb;
ALTER TABLE reply_surb_new
RENAME TO reply_surb;
DROP TABLE reply_surb_sender;
ALTER TABLE reply_surb_sender_new
RENAME TO reply_surb_sender;
@@ -0,0 +1,12 @@
/*
* Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
* SPDX-License-Identifier: Apache-2.0
*/
-- don't persist sender_tag in the DB. instead generate fresh one on each restart
-- this will:
-- A) further help against correlation attacks
-- B) realistically after client restarts, we might be in new key rotation anyway meaning receiver would have to start
-- "from scratch" with surbs
DROP TABLE sender_tag;
@@ -4,6 +4,7 @@
use crate::backend::Empty;
use crate::{CombinedReplyStorage, ReplyStorageBackend};
use async_trait::async_trait;
use time::OffsetDateTime;
// well, right now we don't have the browser storage : (
// so we keep everything in memory
@@ -38,7 +39,10 @@ impl ReplyStorageBackend for Backend {
self.empty.init_fresh(fresh).await
}
async fn load_surb_storage(&self) -> Result<CombinedReplyStorage, Self::StorageError> {
self.empty.load_surb_storage().await
async fn load_surb_storage(
&self,
surb_freshness_cutoff: OffsetDateTime,
) -> Result<CombinedReplyStorage, Self::StorageError> {
self.empty.load_surb_storage(surb_freshness_cutoff).await
}
}
@@ -1,8 +1,7 @@
// Copyright 2022 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use std::io;
use std::path::PathBuf;
use std::{io, path::PathBuf};
use thiserror::Error;
#[derive(Debug, Error)]
@@ -30,7 +29,6 @@ pub enum StorageError {
#[error("failed to perform sqlx migration: {source}")]
MigrationError {
#[source]
#[from]
source: sqlx::migrate::MigrateError,
},
@@ -43,7 +41,6 @@ pub enum StorageError {
#[error("failed to run the SQL query: {source}")]
QueryError {
#[source]
#[from]
source: sqlx::error::Error,
},
@@ -3,21 +3,21 @@
use crate::backend::fs_backend::{
error::StorageError,
models::{
ReplySurbStorageMetadata, StoredReplyKey, StoredReplySurb, StoredSenderTag,
StoredSurbSender,
},
models::{ReplySurbStorageMetadata, StoredReplyKey, StoredReplySurb, StoredSurbSender},
};
use log::{error, info};
use sqlx::{
sqlite::{SqliteAutoVacuum, SqliteSynchronous},
ConnectOptions,
};
use std::path::Path;
use time::OffsetDateTime;
use tracing::{error, info};
use sqlx_pool_guard::SqlitePoolGuard;
#[derive(Debug, Clone)]
pub struct StorageManager {
pub connection_pool: sqlx::SqlitePool,
connection_pool: SqlitePoolGuard,
}
// all SQL goes here
@@ -37,7 +37,7 @@ impl StorageManager {
.journal_mode(sqlx::sqlite::SqliteJournalMode::Wal)
.synchronous(SqliteSynchronous::Normal)
.auto_vacuum(SqliteAutoVacuum::Incremental)
.filename(database_path)
.filename(&database_path)
.create_if_missing(fresh)
.disable_statement_logging();
@@ -49,11 +49,15 @@ impl StorageManager {
}
};
let connection_pool =
SqlitePoolGuard::new(database_path.as_ref().to_path_buf(), connection_pool);
if let Err(err) = sqlx::migrate!("./fs_surbs_migrations")
.run(&connection_pool)
.run(&*connection_pool)
.await
{
error!("Failed to initialize SQLx database: {err}");
connection_pool.close().await;
return Err(err.into());
}
@@ -61,53 +65,60 @@ impl StorageManager {
Ok(StorageManager { connection_pool })
}
/// Close connection pool waiting for all connections to be closed.
pub async fn close_pool(&self) {
self.connection_pool.close().await;
}
#[allow(dead_code)]
pub async fn status_table_exists(&self) -> Result<bool, sqlx::Error> {
sqlx::query!("SELECT name FROM sqlite_master WHERE type='table' AND name='status'")
.fetch_optional(&self.connection_pool)
.fetch_optional(&*self.connection_pool)
.await
.map(|r| r.is_some())
}
pub async fn create_status_table(&self) -> Result<(), sqlx::Error> {
sqlx::query!("INSERT INTO status(flush_in_progress, previous_flush_timestamp, client_in_use) VALUES (0, 0, 1)")
.execute(&self.connection_pool)
.await?;
sqlx::query!(
"INSERT INTO status(flush_in_progress, previous_flush, client_in_use) VALUES (0, 0, 1)"
)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn get_flush_status(&self) -> Result<bool, sqlx::Error> {
sqlx::query!("SELECT flush_in_progress FROM status;")
.fetch_one(&self.connection_pool)
.fetch_one(&*self.connection_pool)
.await
.map(|r| r.flush_in_progress > 0)
}
pub async fn set_previous_flush_timestamp(&self, timestamp: i64) -> Result<(), sqlx::Error> {
sqlx::query!("UPDATE status SET previous_flush_timestamp = ?", timestamp)
.execute(&self.connection_pool)
pub async fn set_previous_flush(&self, timestamp: OffsetDateTime) -> Result<(), sqlx::Error> {
sqlx::query!("UPDATE status SET previous_flush = ?", timestamp)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn get_previous_flush_timestamp(&self) -> Result<i64, sqlx::Error> {
sqlx::query!("SELECT previous_flush_timestamp FROM status;")
.fetch_one(&self.connection_pool)
pub async fn get_previous_flush_time(&self) -> Result<OffsetDateTime, sqlx::Error> {
sqlx::query!(r#"SELECT previous_flush AS "previous_flush: OffsetDateTime" FROM status"#)
.fetch_one(&*self.connection_pool)
.await
.map(|r| r.previous_flush_timestamp)
.map(|r| r.previous_flush)
}
pub async fn set_flush_status(&self, in_progress: bool) -> Result<(), sqlx::Error> {
let in_progress_int = i64::from(in_progress);
sqlx::query!("UPDATE status SET flush_in_progress = ?", in_progress_int)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn get_client_in_use_status(&self) -> Result<bool, sqlx::Error> {
sqlx::query!("SELECT client_in_use FROM status;")
.fetch_one(&self.connection_pool)
.fetch_one(&*self.connection_pool)
.await
.map(|r| r.client_in_use > 0)
}
@@ -115,47 +126,21 @@ impl StorageManager {
pub async fn set_client_in_use_status(&self, in_use: bool) -> Result<(), sqlx::Error> {
let in_use_int = i64::from(in_use);
sqlx::query!("UPDATE status SET client_in_use = ?", in_use_int)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn delete_all_tags(&self) -> Result<(), sqlx::Error> {
sqlx::query!("DELETE FROM sender_tag;")
.execute(&self.connection_pool)
.await?;
Ok(())
}
pub async fn get_tags(&self) -> Result<Vec<StoredSenderTag>, sqlx::Error> {
sqlx::query_as!(StoredSenderTag, "SELECT * FROM sender_tag;",)
.fetch_all(&self.connection_pool)
.await
}
pub async fn insert_tag(&self, stored_tag: StoredSenderTag) -> Result<(), sqlx::Error> {
sqlx::query!(
r#"
INSERT INTO sender_tag(recipient, tag) VALUES (?, ?);
"#,
stored_tag.recipient,
stored_tag.tag
)
.execute(&self.connection_pool)
.await?;
Ok(())
}
pub async fn delete_all_reply_keys(&self) -> Result<(), sqlx::Error> {
sqlx::query!("DELETE FROM reply_key;")
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn get_reply_keys(&self) -> Result<Vec<StoredReplyKey>, sqlx::Error> {
sqlx::query_as!(StoredReplyKey, "SELECT * FROM reply_key;",)
.fetch_all(&self.connection_pool)
sqlx::query_as("SELECT * FROM reply_key;")
.fetch_all(&*self.connection_pool)
.await
}
@@ -165,20 +150,20 @@ impl StorageManager {
) -> Result<(), sqlx::Error> {
sqlx::query!(
r#"
INSERT INTO reply_key(key_digest, reply_key, sent_at_timestamp) VALUES (?, ?, ?);
INSERT INTO reply_key(key_digest, reply_key, sent_at) VALUES (?, ?, ?);
"#,
stored_reply_key.key_digest,
stored_reply_key.reply_key,
stored_reply_key.sent_at_timestamp
stored_reply_key.sent_at
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn get_surb_senders(&self) -> Result<Vec<StoredSurbSender>, sqlx::Error> {
sqlx::query_as!(StoredSurbSender, "SELECT * FROM reply_surb_sender;",)
.fetch_all(&self.connection_pool)
sqlx::query_as("SELECT * FROM reply_surb_sender;")
.fetch_all(&*self.connection_pool)
.await
}
@@ -188,12 +173,12 @@ impl StorageManager {
) -> Result<i64, sqlx::Error> {
let id = sqlx::query!(
r#"
INSERT INTO reply_surb_sender(tag, last_sent_timestamp) VALUES (?, ?);
INSERT INTO reply_surb_sender(tag, last_sent) VALUES (?, ?);
"#,
stored_surb_sender.tag,
stored_surb_sender.last_sent_timestamp
stored_surb_sender.last_sent
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?
.last_insert_rowid();
Ok(id)
@@ -205,20 +190,23 @@ impl StorageManager {
) -> Result<Vec<StoredReplySurb>, sqlx::Error> {
sqlx::query_as!(
StoredReplySurb,
"SELECT * FROM reply_surb WHERE reply_surb_sender_id = ?",
r#"
SELECT reply_surb_sender_id, reply_surb, encoded_key_rotation as "encoded_key_rotation: u8" FROM reply_surb
WHERE reply_surb_sender_id = ?
"#,
sender_id
)
.fetch_all(&self.connection_pool)
.fetch_all(&*self.connection_pool)
.await
}
pub async fn delete_all_reply_surb_data(&self) -> Result<(), sqlx::Error> {
sqlx::query!("DELETE FROM reply_surb;")
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
sqlx::query!("DELETE FROM reply_surb_sender;")
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
@@ -230,12 +218,13 @@ impl StorageManager {
) -> Result<(), sqlx::Error> {
sqlx::query!(
r#"
INSERT INTO reply_surb(reply_surb_sender_id, reply_surb) VALUES (?, ?);
INSERT INTO reply_surb(reply_surb_sender_id, reply_surb, encoded_key_rotation) VALUES (?, ?, ?);
"#,
stored_reply_surb.reply_surb_sender_id,
stored_reply_surb.reply_surb
stored_reply_surb.reply_surb,
stored_reply_surb.encoded_key_rotation
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
@@ -249,7 +238,7 @@ impl StorageManager {
SELECT min_reply_surb_threshold as "min_reply_surb_threshold: u32", max_reply_surb_threshold as "max_reply_surb_threshold: u32" FROM reply_surb_storage_metadata;
"#,
)
.fetch_one(&self.connection_pool)
.fetch_one(&*self.connection_pool)
.await
}
@@ -263,7 +252,7 @@ impl StorageManager {
"#,
metadata.min_reply_surb_threshold,
metadata.max_reply_surb_threshold,
).execute(&self.connection_pool).await?;
).execute(&*self.connection_pool).await?;
Ok(())
}
}
@@ -1,20 +1,19 @@
// Copyright 2022 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::backend::fs_backend::manager::StorageManager;
use crate::backend::fs_backend::models::{
ReplySurbStorageMetadata, StoredReplyKey, StoredReplySurb, StoredSenderTag, StoredSurbSender,
};
use crate::surb_storage::ReceivedReplySurbs;
use crate::{
CombinedReplyStorage, ReceivedReplySurbsMap, ReplyStorageBackend, SentReplyKeys, UsedSenderTags,
backend::fs_backend::{
manager::StorageManager,
models::{ReplySurbStorageMetadata, StoredReplyKey, StoredReplySurb, StoredSurbSender},
},
surb_storage::ReceivedReplySurbs,
CombinedReplyStorage, ReceivedReplySurbsMap, ReplyStorageBackend, SentReplyKeys,
};
use async_trait::async_trait;
use log::{debug, error, info, warn};
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use std::fs;
use std::path::{Path, PathBuf};
use time::OffsetDateTime;
use tracing::{error, info, warn};
pub use self::error::StorageError;
@@ -41,21 +40,20 @@ impl Backend {
}
let manager = StorageManager::init(database_path, true).await?;
manager.create_status_table().await?;
let backend = Backend {
temporary_old_path: None,
database_path: owned_path,
manager,
};
Ok(backend)
match manager.create_status_table().await {
Ok(()) => Ok(Backend {
temporary_old_path: None,
database_path: owned_path,
manager,
}),
Err(err) => {
manager.close_pool().await;
Err(err.into())
}
}
}
pub async fn try_load<P: AsRef<Path>>(
database_path: P,
fresh_sender_tags: bool,
) -> Result<Self, StorageError> {
pub async fn try_load<P: AsRef<Path>>(database_path: P) -> Result<Self, StorageError> {
let owned_path: PathBuf = database_path.as_ref().into();
if owned_path.file_name().is_none() {
return Err(StorageError::DatabasePathWithoutFilename {
@@ -64,15 +62,33 @@ impl Backend {
}
let manager = StorageManager::init(database_path, false).await?;
match Self::try_load_inner(&manager).await {
Ok(()) => Ok(Backend {
temporary_old_path: None,
database_path: owned_path,
manager,
}),
Err(e) => {
manager.close_pool().await;
Err(e)
}
}
}
/// Gracefully close sqlite connection pool and drop backend.
pub async fn shutdown(self) {
self.manager.close_pool().await
}
async fn try_load_inner(manager: &StorageManager) -> Result<(), StorageError> {
// the database flush wasn't fully finished and thus the data is in inconsistent state
// (we don't really know what's properly saved or what's not)
if manager.get_flush_status().await? {
return Err(StorageError::IncompleteDataFlush);
}
let last_flush_timestamp = manager.get_previous_flush_timestamp().await?;
if last_flush_timestamp == 0 {
let last_flush = manager.get_previous_flush_time().await?;
if last_flush == OffsetDateTime::UNIX_EPOCH {
// either this client has been running since 1970 or the flush failed
return Err(StorageError::IncompleteDataFlush);
}
@@ -92,15 +108,6 @@ impl Backend {
return Err(err.into());
}
let last_flush = match OffsetDateTime::from_unix_timestamp(last_flush_timestamp) {
Ok(last_flush) => last_flush,
Err(err) => {
return Err(StorageError::CorruptedData {
details: format!("failed to parse stored timestamp - {err}"),
});
}
};
// in theory clients can use our reply surbs whenever they want, even a year in the future
// (assuming no key rotation has happened)
// but the way it's currently coded, everyone will purge old data
@@ -118,28 +125,11 @@ impl Backend {
manager.delete_all_reply_keys().await?;
}
if days > 2 {
info!("it's been over {days} days and {hours} hours since we last used our data store. our used sender tags are already outdated - we're going to purge them now.");
manager.delete_all_tags().await?;
} else if fresh_sender_tags {
debug!("starting with fresh sender tags");
manager.delete_all_tags().await?;
}
Ok(Backend {
temporary_old_path: None,
database_path: owned_path,
// manager: StorageManagerState::Storage(manager),
manager,
})
}
async fn close_pool(&mut self) {
self.manager.connection_pool.close().await;
Ok(())
}
async fn rotate(&mut self) -> Result<(), StorageError> {
self.close_pool().await;
self.manager.close_pool().await;
let new_extension = if let Some(existing_extension) =
self.database_path.extension().and_then(|ext| ext.to_str())
@@ -152,7 +142,8 @@ impl Backend {
let mut temp_old = self.database_path.clone();
temp_old.set_extension(new_extension);
fs::rename(&self.database_path, &temp_old)
tokio::fs::rename(&self.database_path, &temp_old)
.await
.map_err(|err| StorageError::DatabaseRenameError { source: err })?;
self.manager = StorageManager::init(&self.database_path, true).await?;
self.manager.create_status_table().await?;
@@ -161,9 +152,10 @@ impl Backend {
Ok(())
}
fn remove_old(&mut self) -> Result<(), StorageError> {
async fn remove_old(&mut self) -> Result<(), StorageError> {
if let Some(old_path) = self.temporary_old_path.take() {
fs::remove_file(old_path)
tokio::fs::remove_file(old_path)
.await
.map_err(|err| StorageError::DatabaseOldFileRemoveError { source: err })
} else {
warn!("the old database file doesn't seem to exist!");
@@ -177,7 +169,7 @@ impl Backend {
async fn end_storage_flush(&self) -> Result<(), StorageError> {
self.manager
.set_previous_flush_timestamp(OffsetDateTime::now_utc().unix_timestamp())
.set_previous_flush(OffsetDateTime::now_utc())
.await?;
Ok(self.manager.set_flush_status(false).await?)
}
@@ -190,29 +182,6 @@ impl Backend {
Ok(self.manager.set_client_in_use_status(false).await?)
}
async fn get_stored_tags(&self) -> Result<UsedSenderTags, StorageError> {
let stored = self.manager.get_tags().await?;
// stop at the first instance of corruption. if even a single entry is malformed,
// something weird has happened and we can't trust the rest of the data
let raw = stored
.into_iter()
.map(TryInto::try_into)
.collect::<Result<_, _>>()?;
Ok(UsedSenderTags::from_raw(raw))
}
async fn dump_sender_tags(&self, tags: &UsedSenderTags) -> Result<(), StorageError> {
for map_ref in tags.as_raw_iter() {
let (recipient, tag) = map_ref.pair();
self.manager
.insert_tag(StoredSenderTag::new(*recipient, *tag))
.await?;
}
Ok(())
}
async fn get_stored_reply_keys(&self) -> Result<SentReplyKeys, StorageError> {
let stored = self.manager.get_reply_keys().await?;
@@ -236,14 +205,17 @@ impl Backend {
Ok(())
}
async fn get_stored_reply_surbs(&self) -> Result<ReceivedReplySurbsMap, StorageError> {
async fn get_stored_reply_surbs(
&self,
surb_freshness_cutoff: OffsetDateTime,
) -> Result<ReceivedReplySurbsMap, StorageError> {
let surb_senders = self.manager.get_surb_senders().await?;
let metadata = self.get_reply_surb_storage_metadata().await?;
let mut received_surbs = Vec::with_capacity(surb_senders.len());
for sender in surb_senders {
let sender_id = sender.id;
let (sender_tag, surbs_last_received_at_timestamp): (AnonymousSenderTag, i64) =
let (sender_tag, surbs_last_received_at): (AnonymousSenderTag, OffsetDateTime) =
sender.try_into()?;
let stored_surbs = self
.manager
@@ -255,15 +227,17 @@ impl Backend {
received_surbs.push((
sender_tag,
ReceivedReplySurbs::new_retrieved(stored_surbs, surbs_last_received_at_timestamp),
ReceivedReplySurbs::new_retrieved(stored_surbs, surbs_last_received_at),
))
}
Ok(ReceivedReplySurbsMap::from_raw(
let received_surbs = ReceivedReplySurbsMap::from_raw(
metadata.min_reply_surb_threshold as usize,
metadata.max_reply_surb_threshold as usize,
received_surbs,
))
);
received_surbs.drop_stale_loaded_surbs(surb_freshness_cutoff);
Ok(received_surbs)
}
async fn dump_reply_surbs(
@@ -285,6 +259,14 @@ impl Backend {
.insert_reply_surb(StoredReplySurb::new(sender_id, reply_surb))
.await?
}
// TODO: should we also retain the stale ones?
if received_surbs.possibly_stale_left() != 0 {
warn!(
"dropping {} possibly stale surbs for {tag}",
received_surbs.possibly_stale_left()
);
}
}
Ok(())
}
@@ -328,14 +310,13 @@ impl ReplyStorageBackend for Backend {
self.rotate().await?;
self.start_storage_flush().await?;
self.dump_sender_tags(storage.tags_storage_ref()).await?;
self.dump_sender_reply_keys(storage.key_storage_ref())
.await?;
let surbs_ref = storage.surbs_storage_ref();
self.dump_reply_surb_storage_metadata(surbs_ref).await?;
self.dump_reply_surbs(surbs_ref).await?;
self.remove_old()?;
self.remove_old().await?;
self.end_storage_flush().await
}
@@ -345,12 +326,14 @@ impl ReplyStorageBackend for Backend {
.await
}
async fn load_surb_storage(&self) -> Result<CombinedReplyStorage, Self::StorageError> {
async fn load_surb_storage(
&self,
surb_freshness_cutoff: OffsetDateTime,
) -> Result<CombinedReplyStorage, Self::StorageError> {
let reply_keys = self.get_stored_reply_keys().await?;
let tags = self.get_stored_tags().await?;
let reply_surbs = self.get_stored_reply_surbs().await?;
let reply_surbs = self.get_stored_reply_surbs(surb_freshness_cutoff).await?;
Ok(CombinedReplyStorage::load(reply_keys, reply_surbs, tags))
Ok(CombinedReplyStorage::load(reply_keys, reply_surbs))
}
async fn stop_storage_session(self) -> Result<(), Self::StorageError> {
@@ -3,13 +3,18 @@
use crate::backend::fs_backend::error::StorageError;
use crate::key_storage::UsedReplyKey;
use crate::ReceivedReplySurb;
use nym_crypto::generic_array::typenum::Unsigned;
use nym_crypto::Digest;
use nym_sphinx::addressing::clients::{Recipient, RecipientBytes};
use nym_sphinx::anonymous_replies::encryption_key::EncryptionKeyDigest;
use nym_sphinx::anonymous_replies::requests::{AnonymousSenderTag, SENDER_TAG_SIZE};
use nym_sphinx::anonymous_replies::{ReplySurb, SurbEncryptionKey, SurbEncryptionKeySize};
use nym_sphinx::params::ReplySurbKeyDigestAlgorithm;
use nym_sphinx::anonymous_replies::{
ReplySurb, ReplySurbWithKeyRotation, SurbEncryptionKey, SurbEncryptionKeySize,
};
use nym_sphinx::params::{ReplySurbKeyDigestAlgorithm, SphinxKeyRotation};
use sqlx::FromRow;
use time::OffsetDateTime;
#[derive(Debug, Clone)]
pub struct StoredSenderTag {
@@ -56,11 +61,11 @@ impl TryFrom<StoredSenderTag> for (RecipientBytes, AnonymousSenderTag) {
}
}
#[derive(Debug, Clone)]
#[derive(Debug, Clone, FromRow)]
pub struct StoredReplyKey {
pub key_digest: Vec<u8>,
pub reply_key: Vec<u8>,
pub sent_at_timestamp: i64,
pub sent_at: OffsetDateTime,
}
impl StoredReplyKey {
@@ -68,7 +73,7 @@ impl StoredReplyKey {
StoredReplyKey {
key_digest: key_digest.to_vec(),
reply_key: (*reply_key).to_bytes(),
sent_at_timestamp: reply_key.sent_at_timestamp,
sent_at: reply_key.sent_at,
}
}
}
@@ -98,32 +103,30 @@ impl TryFrom<StoredReplyKey> for (EncryptionKeyDigest, UsedReplyKey) {
});
};
Ok((
digest,
UsedReplyKey::new(reply_key, value.sent_at_timestamp),
))
Ok((digest, UsedReplyKey::new(reply_key, value.sent_at)))
}
}
#[derive(FromRow)]
pub struct StoredSurbSender {
pub id: i64,
pub tag: Vec<u8>,
pub last_sent_timestamp: i64,
pub last_sent: OffsetDateTime,
}
impl StoredSurbSender {
pub fn new(tag: AnonymousSenderTag, last_sent_timestamp: i64) -> Self {
pub fn new(tag: AnonymousSenderTag, last_sent: OffsetDateTime) -> Self {
StoredSurbSender {
// for the purposes of STORING data,
// we ignore that field anyway
id: 0,
tag: tag.to_bytes().to_vec(),
last_sent_timestamp,
last_sent,
}
}
}
impl TryFrom<StoredSurbSender> for (AnonymousSenderTag, i64) {
impl TryFrom<StoredSurbSender> for (AnonymousSenderTag, OffsetDateTime) {
type Error = StorageError;
fn try_from(value: StoredSurbSender) -> Result<Self, Self::Error> {
@@ -138,7 +141,7 @@ impl TryFrom<StoredSurbSender> for (AnonymousSenderTag, i64) {
Ok((
AnonymousSenderTag::from_bytes(sender_tag_bytes),
value.last_sent_timestamp,
value.last_sent,
))
}
}
@@ -146,24 +149,40 @@ impl TryFrom<StoredSurbSender> for (AnonymousSenderTag, i64) {
pub struct StoredReplySurb {
pub reply_surb_sender_id: i64,
pub reply_surb: Vec<u8>,
// encodes only whether it's 'even', 'odd' or 'unknown' (default)
// and not the whole id because that's redundant
pub encoded_key_rotation: u8,
}
impl StoredReplySurb {
pub fn new(reply_surb_sender_id: i64, reply_surb: &ReplySurb) -> Self {
pub fn new(reply_surb_sender_id: i64, reply_surb: &ReceivedReplySurb) -> Self {
StoredReplySurb {
reply_surb_sender_id,
reply_surb: reply_surb.to_bytes(),
reply_surb: reply_surb.surb.inner_reply_surb().to_bytes(),
encoded_key_rotation: reply_surb.key_rotation() as u8,
}
}
}
impl TryFrom<StoredReplySurb> for ReplySurb {
impl TryFrom<StoredReplySurb> for ReplySurbWithKeyRotation {
type Error = StorageError;
fn try_from(value: StoredReplySurb) -> Result<Self, Self::Error> {
ReplySurb::from_bytes(&value.reply_surb).map_err(|err| StorageError::CorruptedData {
details: format!("failed to recover the reply surb: {err}"),
})
let key_rotation =
SphinxKeyRotation::try_from(value.encoded_key_rotation).map_err(|err| {
StorageError::CorruptedData {
details: format!("stored key rotation was malformed: {err}"),
}
})?;
let reply_surb = ReplySurb::from_bytes(&value.reply_surb).map_err(|err| {
StorageError::CorruptedData {
details: format!("failed to recover the reply surb: {err}"),
}
})?;
Ok(reply_surb.with_key_rotation(key_rotation))
}
}
@@ -5,6 +5,7 @@ use crate::CombinedReplyStorage;
use async_trait::async_trait;
use std::error::Error;
use thiserror::Error;
use time::OffsetDateTime;
// TODO: this should now live inside our wasm/client-core
pub mod browser_backend;
@@ -53,7 +54,10 @@ impl ReplyStorageBackend for Empty {
Ok(())
}
async fn load_surb_storage(&self) -> Result<CombinedReplyStorage, Self::StorageError> {
async fn load_surb_storage(
&self,
_: OffsetDateTime,
) -> Result<CombinedReplyStorage, Self::StorageError> {
Ok(CombinedReplyStorage::new(
self.min_surb_threshold,
self.max_surb_threshold,
@@ -80,7 +84,10 @@ pub trait ReplyStorageBackend: Sized {
/// (such as surb thresholds)
async fn init_fresh(&mut self, fresh: &CombinedReplyStorage) -> Result<(), Self::StorageError>;
async fn load_surb_storage(&self) -> Result<CombinedReplyStorage, Self::StorageError>;
async fn load_surb_storage(
&self,
surb_freshness_cutoff: OffsetDateTime,
) -> Result<CombinedReplyStorage, Self::StorageError>;
async fn stop_storage_session(self) -> Result<(), Self::StorageError> {
Ok(())
@@ -25,12 +25,11 @@ impl CombinedReplyStorage {
pub fn load(
sent_reply_keys: SentReplyKeys,
received_reply_surbs: ReceivedReplySurbsMap,
used_tags: UsedSenderTags,
) -> Self {
CombinedReplyStorage {
sent_reply_keys,
received_reply_surbs,
used_tags,
used_tags: UsedSenderTags::new(),
}
}
@@ -47,8 +47,12 @@ impl SentReplyKeys {
self.inner.data.iter()
}
pub fn retain(&self, f: impl FnMut(&EncryptionKeyDigest, &mut UsedReplyKey) -> bool) {
self.inner.data.retain(f);
}
pub fn insert_multiple(&self, keys: Vec<SurbEncryptionKey>) {
let now = OffsetDateTime::now_utc().unix_timestamp();
let now = OffsetDateTime::now_utc();
for key in keys {
self.insert(UsedReplyKey::new(key, now))
}
@@ -71,15 +75,12 @@ impl SentReplyKeys {
pub struct UsedReplyKey {
key: SurbEncryptionKey,
// the purpose of this field is to perform invalidation at relatively very long intervals
pub sent_at_timestamp: i64,
pub sent_at: OffsetDateTime,
}
impl UsedReplyKey {
pub(crate) fn new(key: SurbEncryptionKey, sent_at_timestamp: i64) -> Self {
UsedReplyKey {
key,
sent_at_timestamp,
}
pub(crate) fn new(key: SurbEncryptionKey, sent_at: OffsetDateTime) -> Self {
UsedReplyKey { key, sent_at }
}
}
+8 -6
View File
@@ -4,8 +4,9 @@
pub use backend::*;
pub use combined::CombinedReplyStorage;
pub use key_storage::SentReplyKeys;
pub use surb_storage::ReceivedReplySurbsMap;
pub use surb_storage::{ReceivedReplySurb, ReceivedReplySurbsMap, RetrievedReplySurb};
pub use tag_storage::UsedSenderTags;
use time::OffsetDateTime;
mod backend;
mod combined;
@@ -29,17 +30,19 @@ where
PersistentReplyStorage { backend }
}
pub async fn load_state_from_backend(&self) -> Result<CombinedReplyStorage, T::StorageError> {
self.backend.load_surb_storage().await
pub async fn load_state_from_backend(
&self,
surb_freshness_cutoff: OffsetDateTime,
) -> Result<CombinedReplyStorage, T::StorageError> {
self.backend.load_surb_storage(surb_freshness_cutoff).await
}
// this will have to get enabled after merging develop
pub async fn flush_on_shutdown(
mut self,
mem_state: CombinedReplyStorage,
mut shutdown: nym_task::TaskClient,
) {
use log::{debug, error, info};
use tracing::{debug, error, info};
debug!("Started PersistentReplyStorage");
if let Err(err) = self.backend.start_storage_session().await {
@@ -50,7 +53,6 @@ where
shutdown.recv().await;
info!("PersistentReplyStorage is flushing all reply-related data to underlying storage");
info!("you MUST NOT forcefully shutdown now or you risk data corruption!");
if let Err(err) = self.backend.flush_surb_storage(&mem_state).await {
error!("failed to flush our reply-related data to the persistent storage: {err}")
} else {
@@ -1,15 +1,45 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use dashmap::iter::Iter;
use dashmap::iter::{Iter, IterMut};
use dashmap::DashMap;
use log::trace;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use nym_sphinx::anonymous_replies::ReplySurb;
use nym_sphinx::anonymous_replies::ReplySurbWithKeyRotation;
use nym_sphinx::params::SphinxKeyRotation;
use std::cmp::min;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use time::OffsetDateTime;
use tracing::{error, info, trace};
#[derive(Debug)]
pub struct RetrievedReplySurb {
pub(crate) reply_surb: ReceivedReplySurb,
pub(crate) stale_pile: bool,
}
impl RetrievedReplySurb {
pub(crate) fn new_fresh(reply_surb: ReceivedReplySurb) -> Self {
RetrievedReplySurb {
reply_surb,
stale_pile: false,
}
}
pub(crate) fn new_stale(reply_surb: ReceivedReplySurb) -> Self {
RetrievedReplySurb {
reply_surb,
stale_pile: true,
}
}
}
impl From<RetrievedReplySurb> for ReplySurbWithKeyRotation {
fn from(retrieved: RetrievedReplySurb) -> Self {
retrieved.reply_surb.into()
}
}
#[derive(Debug, Clone)]
pub struct ReceivedReplySurbsMap {
@@ -57,17 +87,40 @@ impl ReceivedReplySurbsMap {
self.inner.data.iter()
}
pub fn remove(&self, target: &AnonymousSenderTag) {
self.inner.data.remove(target);
pub fn as_raw_iter_mut(&self) -> IterMut<'_, AnonymousSenderTag, ReceivedReplySurbs> {
self.inner.data.iter_mut()
}
pub fn reset_surbs_last_received_at(&self, target: &AnonymousSenderTag) {
if let Some(mut entry) = self.inner.data.get_mut(target) {
entry.surbs_last_received_at_timestamp = OffsetDateTime::now_utc().unix_timestamp();
fn total_surbs(&self) -> usize {
self.inner
.data
.iter()
.map(|entry| entry.value().data.len())
.sum()
}
pub fn drop_stale_loaded_surbs(&self, cutoff: OffsetDateTime) {
let before = self.total_surbs();
self.inner.data.retain(|_, v| {
if v.surbs_last_received_at() < cutoff {
return false;
}
v.data.retain(|s| s.received_at > cutoff);
!v.data.is_empty()
});
let after = self.total_surbs();
let diff = before - after;
if diff != 0 {
info!("removed {diff} stale reply SURBs")
}
}
pub fn surbs_last_received_at(&self, target: &AnonymousSenderTag) -> Option<i64> {
pub fn retain(&self, f: impl FnMut(&AnonymousSenderTag, &mut ReceivedReplySurbs) -> bool) {
self.inner.data.retain(f);
}
pub fn surbs_last_received_at(&self, target: &AnonymousSenderTag) -> Option<OffsetDateTime> {
self.inner
.data
.get(target)
@@ -126,15 +179,25 @@ impl ReceivedReplySurbsMap {
.unwrap_or_default()
}
pub fn available_fresh_surbs(&self, target: &AnonymousSenderTag) -> usize {
self.inner
.data
.get(target)
.map(|entry| entry.fresh_left())
.unwrap_or_default()
}
pub fn contains_surbs_for(&self, target: &AnonymousSenderTag) -> bool {
self.inner.data.contains_key(target)
}
/// Attempt to retrieve the specified number of reply SURBs for the target sender
/// and return the number of SURBs remaining in the storage after the call.
pub fn get_reply_surbs(
&self,
target: &AnonymousSenderTag,
amount: usize,
) -> (Option<Vec<ReplySurb>>, usize) {
) -> (Option<Vec<RetrievedReplySurb>>, usize) {
if let Some(mut entry) = self.inner.data.get_mut(target) {
let surbs_left = entry.items_left();
if surbs_left < self.min_surb_threshold() + amount {
@@ -150,34 +213,72 @@ impl ReceivedReplySurbsMap {
pub fn get_reply_surb_ignoring_threshold(
&self,
target: &AnonymousSenderTag,
) -> Option<(Option<ReplySurb>, usize)> {
self.inner
.data
.get_mut(target)
.map(|mut s| s.get_reply_surb())
) -> (Option<RetrievedReplySurb>, usize) {
let Some(mut entry) = self.inner.data.get_mut(target) else {
return (None, 0);
};
entry.get_reply_surb()
}
pub fn get_reply_surb(
&self,
target: &AnonymousSenderTag,
) -> Option<(Option<ReplySurb>, usize)> {
self.inner.data.get_mut(target).map(|mut entry| {
let surbs_left = entry.items_left();
if surbs_left < self.min_surb_threshold() {
(None, surbs_left)
} else {
entry.get_reply_surb()
}
})
) -> (Option<RetrievedReplySurb>, usize) {
let Some(mut entry) = self.inner.data.get_mut(target) else {
return (None, 0);
};
let surbs_left = entry.items_left();
if surbs_left < self.min_surb_threshold() {
(None, surbs_left)
} else {
entry.get_reply_surb()
}
}
pub fn insert_surbs<I: IntoIterator<Item = ReplySurb>>(
pub fn re_insert_reply_surbs(
&self,
target: &AnonymousSenderTag,
surbs: Vec<RetrievedReplySurb>,
) {
error!("re-inserting {} unused surbs", surbs.len());
let mut entry = self.inner.data.entry(*target).or_insert_with(|| {
// this branch should realistically NEVER happen, but software be software, so let's not crash
error!("attempting to return surbs to no longer existing entry {target}");
ReceivedReplySurbs::new(VecDeque::new())
});
let entry = entry.value_mut();
for returned_surb in surbs.into_iter().rev() {
if returned_surb.stale_pile {
entry.possibly_stale.push_front(returned_surb.reply_surb)
} else {
entry.data.push_front(returned_surb.reply_surb)
}
}
}
pub fn insert_fresh_surbs<I: IntoIterator<Item = ReplySurbWithKeyRotation>>(
&self,
target: &AnonymousSenderTag,
surbs: I,
) {
if let Some(mut existing_data) = self.inner.data.get_mut(target) {
existing_data.insert_reply_surbs(surbs)
existing_data.insert_fresh_reply_surbs(surbs);
if existing_data.possibly_stale.is_empty() {
return;
}
// if we're above the minimum threshold, remove stale surbs
let threshold = self.min_surb_threshold();
let diff = existing_data.data.len().saturating_sub(threshold);
trace!("will attempt to remove up to {diff} stale surbs");
if diff > 0 {
existing_data.remove_stale_surbs(diff);
}
} else {
let new_entry = ReceivedReplySurbs::new(surbs.into_iter().collect());
self.inner.data.insert(*target, new_entry);
@@ -185,44 +286,102 @@ impl ReceivedReplySurbsMap {
}
}
#[derive(Debug)]
pub struct ReceivedReplySurb {
pub(crate) surb: ReplySurbWithKeyRotation,
pub(crate) received_at: OffsetDateTime,
}
impl From<ReceivedReplySurb> for ReplySurbWithKeyRotation {
fn from(surb: ReceivedReplySurb) -> Self {
surb.surb
}
}
impl ReceivedReplySurb {
pub fn received_at(&self) -> OffsetDateTime {
self.received_at
}
pub fn key_rotation(&self) -> SphinxKeyRotation {
self.surb.key_rotation()
}
}
#[derive(Debug)]
pub struct ReceivedReplySurbs {
// in the future we'd probably want to put extra data here to indicate when the SURBs got received
// so we could invalidate entries from the previous key rotations
data: VecDeque<ReplySurb>,
data: VecDeque<ReceivedReplySurb>,
possibly_stale: VecDeque<ReceivedReplySurb>,
pending_reception: u32,
surbs_last_received_at_timestamp: i64,
surbs_last_received_at: OffsetDateTime,
}
impl ReceivedReplySurbs {
fn new(initial_surbs: VecDeque<ReplySurb>) -> Self {
ReceivedReplySurbs {
data: initial_surbs,
fn new(initial_surbs: VecDeque<ReplySurbWithKeyRotation>) -> Self {
let mut this = ReceivedReplySurbs {
data: Default::default(),
possibly_stale: Default::default(),
pending_reception: 0,
surbs_last_received_at_timestamp: OffsetDateTime::now_utc().unix_timestamp(),
}
surbs_last_received_at: OffsetDateTime::now_utc(),
};
this.insert_fresh_reply_surbs(initial_surbs);
this
}
#[cfg(all(not(target_arch = "wasm32"), feature = "fs-surb-storage"))]
pub fn new_retrieved(
surbs: Vec<ReplySurb>,
surbs_last_received_at_timestamp: i64,
surbs: Vec<ReplySurbWithKeyRotation>,
surbs_last_received_at: OffsetDateTime,
) -> ReceivedReplySurbs {
ReceivedReplySurbs {
data: surbs.into(),
let mut this = ReceivedReplySurbs {
data: Default::default(),
possibly_stale: Default::default(),
pending_reception: 0,
surbs_last_received_at_timestamp,
}
surbs_last_received_at,
};
this.insert_fresh_reply_surbs(surbs);
this.surbs_last_received_at = surbs_last_received_at;
this
}
pub fn downgrade_freshness(&mut self) -> usize {
debug_assert!(self.possibly_stale.is_empty());
std::mem::swap(&mut self.data, &mut self.possibly_stale);
self.possibly_stale.len()
}
pub fn is_empty(&self) -> bool {
self.data.is_empty() && self.possibly_stale.is_empty()
}
#[cfg(all(not(target_arch = "wasm32"), feature = "fs-surb-storage"))]
pub fn surbs_ref(&self) -> &VecDeque<ReplySurb> {
pub fn surbs_ref(&self) -> &VecDeque<ReceivedReplySurb> {
&self.data
}
pub fn surbs_last_received_at(&self) -> i64 {
self.surbs_last_received_at_timestamp
pub fn retain_fresh_surbs(&mut self, f: impl FnMut(&ReceivedReplySurb) -> bool) {
self.data.retain(f);
}
pub fn retain_possibly_stale_surbs(&mut self, f: impl FnMut(&ReceivedReplySurb) -> bool) {
self.possibly_stale.retain(f);
}
pub fn fresh_left(&self) -> usize {
self.data.len()
}
pub fn possibly_stale_left(&self) -> usize {
self.possibly_stale.len()
}
pub fn drop_possibly_stale_surbs(&mut self) {
self.possibly_stale = VecDeque::new();
}
pub fn surbs_last_received_at(&self) -> OffsetDateTime {
self.surbs_last_received_at
}
pub fn pending_reception(&self) -> u32 {
@@ -243,33 +402,78 @@ impl ReceivedReplySurbs {
self.pending_reception = 0;
}
pub fn get_reply_surbs(&mut self, amount: usize) -> (Option<Vec<ReplySurb>>, usize) {
/// Attempt to retrieve the specified number of reply SURBs (if at least that many are present)
/// and return the number of SURBs remaining in the storage after the call.
pub fn get_reply_surbs(&mut self, amount: usize) -> (Option<Vec<RetrievedReplySurb>>, usize) {
if self.items_left() < amount {
(None, self.items_left())
} else {
let surbs = self.data.drain(..amount).collect();
(Some(surbs), self.items_left())
let available_fresh = self.fresh_left();
// prefer the 'fresh' data if available. otherwise fallback to the possibly stale entries
let mut reply_surbs = Vec::with_capacity(amount);
let fresh_to_retrieve = min(available_fresh, amount);
for surb in self.data.drain(..fresh_to_retrieve) {
reply_surbs.push(RetrievedReplySurb::new_fresh(surb))
}
if available_fresh < amount {
let stale_to_retrieve = amount - fresh_to_retrieve;
for surb in self.possibly_stale.drain(..stale_to_retrieve) {
reply_surbs.push(RetrievedReplySurb::new_stale(surb))
}
}
(Some(reply_surbs), self.items_left())
}
}
pub fn get_reply_surb(&mut self) -> (Option<ReplySurb>, usize) {
pub fn get_reply_surb(&mut self) -> (Option<RetrievedReplySurb>, usize) {
(self.pop_surb(), self.items_left())
}
fn pop_surb(&mut self) -> Option<ReplySurb> {
self.data.pop_front()
fn pop_surb(&mut self) -> Option<RetrievedReplySurb> {
// prefer the 'fresh' data if available. otherwise fallback to the possibly stale entries
if let Some(fresh) = self.data.pop_front() {
return Some(RetrievedReplySurb::new_fresh(fresh));
}
if let Some(stale) = self.possibly_stale.pop_front() {
return Some(RetrievedReplySurb::new_stale(stale));
}
None
}
fn items_left(&self) -> usize {
self.data.len()
self.data.len() + self.possibly_stale.len()
}
pub fn remove_stale_surbs(&mut self, amount: usize) {
// remove up to amount number of possibly stale surbs
let amount = min(amount, self.possibly_stale.len());
self.possibly_stale.drain(..amount);
}
// realistically we're always going to be getting multiple surbs at once
pub fn insert_reply_surbs<I: IntoIterator<Item = ReplySurb>>(&mut self, surbs: I) {
let mut v = surbs.into_iter().collect::<VecDeque<_>>();
pub(crate) fn insert_fresh_reply_surbs<I: IntoIterator<Item = ReplySurbWithKeyRotation>>(
&mut self,
surbs: I,
) {
let received_at = OffsetDateTime::now_utc();
let mut v = surbs
.into_iter()
.map(|surb| ReceivedReplySurb { surb, received_at })
.collect::<VecDeque<_>>();
if v.is_empty() {
return;
}
trace!("storing {} surbs in the storage", v.len());
self.data.append(&mut v);
self.surbs_last_received_at_timestamp = OffsetDateTime::now_utc().unix_timestamp();
self.surbs_last_received_at = received_at;
trace!("we now have {} surbs!", self.data.len());
}
}
@@ -21,8 +21,8 @@ use nym_crypto::asymmetric::ed25519;
use nym_gateway_requests::registration::handshake::client_handshake;
use nym_gateway_requests::{
BinaryRequest, ClientControlRequest, ClientRequest, GatewayProtocolVersionExt,
SensitiveServerResponse, ServerResponse, SharedGatewayKey, SharedSymmetricKey,
CREDENTIAL_UPDATE_V2_PROTOCOL_VERSION, CURRENT_PROTOCOL_VERSION,
GatewayRequestsError, SensitiveServerResponse, ServerResponse, SharedGatewayKey,
SharedSymmetricKey, CREDENTIAL_UPDATE_V2_PROTOCOL_VERSION, CURRENT_PROTOCOL_VERSION,
};
use nym_sphinx::forwarding::packet::MixPacket;
use nym_statistics_common::clients::connection::ConnectionStatsEvent;
@@ -272,7 +272,7 @@ impl<C, St> GatewayClient<C, St> {
) -> Result<(), GatewayClientError> {
if let Some(shared_key) = self.shared_key() {
let encrypted = message.encrypt(&*shared_key)?;
Box::pin(self.send_websocket_message(encrypted)).await?;
Box::pin(self.send_websocket_message_without_response(encrypted)).await?;
Ok(())
} else {
Err(GatewayClientError::ConnectionInInvalidState)
@@ -330,9 +330,80 @@ impl<C, St> GatewayClient<C, St> {
}
}
/// Attempt to send a websocket message to the gateway without waiting for any response
async fn send_websocket_message_without_response(
&mut self,
msg: impl Into<Message>,
) -> Result<(), GatewayClientError> {
match self.connection {
SocketState::Available(ref mut conn) => Ok(conn.send(msg.into()).await?),
SocketState::PartiallyDelegated(ref mut partially_delegated) => {
if let Err(err) = partially_delegated.send_without_response(msg.into()).await {
error!("failed to send message without response - {err}...");
// we must ensure we do not leave the task still active
if let Err(err) = self.recover_socket_connection().await {
error!("... and the delegated stream has also errored out - {err}")
}
Err(err)
} else {
Ok(())
}
}
SocketState::NotConnected => Err(GatewayClientError::ConnectionNotEstablished),
_ => Err(GatewayClientError::ConnectionInInvalidState),
}
}
// A very nasty hack due to lack of id tags on messages - send a non-sphinx packet websocket
// message and wait until first non 'Send' response within timeout
pub async fn send_websocket_message_with_non_send_response(
&mut self,
msg: impl Into<Message>,
) -> Result<ServerResponse, GatewayClientError> {
let should_restart_mixnet_listener = if self.connection.is_partially_delegated() {
self.recover_socket_connection().await?;
true
} else {
false
};
let conn = match self.connection {
SocketState::Available(ref mut conn) => conn,
SocketState::NotConnected => return Err(GatewayClientError::ConnectionNotEstablished),
_ => return Err(GatewayClientError::ConnectionInInvalidState),
};
conn.send(msg.into()).await?;
let timeout = sleep(self.cfg.connection.response_timeout_duration);
tokio::pin!(timeout);
let response = loop {
tokio::select! {
_ = &mut timeout => {
break Err(GatewayClientError::Timeout);
}
// note: the below will also listen for shutdown signals
msg = self.read_control_response() => {
match msg {
Ok(res) => if !res.is_send() {
break Ok(res);
},
Err(err) => break Err(err),
}
}
}
};
if should_restart_mixnet_listener {
self.start_listening_for_mixnet_messages()?;
}
response
}
/// Attempt to send a websocket message to the gateway and wait until we receive a response.
// If we want to send a message (with response), we need to have a full control over the socket,
// as we need to be able to write the request and read the subsequent response
pub async fn send_websocket_message(
pub async fn send_websocket_message_with_response(
&mut self,
msg: impl Into<Message>,
) -> Result<ServerResponse, GatewayClientError> {
@@ -387,29 +458,6 @@ impl<C, St> GatewayClient<C, St> {
}
}
async fn send_websocket_message_without_response(
&mut self,
msg: Message,
) -> Result<(), GatewayClientError> {
match self.connection {
SocketState::Available(ref mut conn) => Ok(conn.send(msg).await?),
SocketState::PartiallyDelegated(ref mut partially_delegated) => {
if let Err(err) = partially_delegated.send_without_response(msg).await {
error!("failed to send message without response - {err}...");
// we must ensure we do not leave the task still active
if let Err(err) = self.recover_socket_connection().await {
error!("... and the delegated stream has also errored out - {err}")
}
Err(err)
} else {
Ok(())
}
}
SocketState::NotConnected => Err(GatewayClientError::ConnectionNotEstablished),
_ => Err(GatewayClientError::ConnectionInInvalidState),
}
}
fn check_gateway_protocol(
&self,
gateway_protocol: Option<u8>,
@@ -535,7 +583,10 @@ impl<C, St> GatewayClient<C, St> {
.encrypt(legacy_key)?;
info!("sending upgrade request and awaiting the acknowledgement back");
let (ciphertext, nonce) = match self.send_websocket_message(upgrade_request).await? {
let (ciphertext, nonce) = match self
.send_websocket_message_with_response(upgrade_request)
.await?
{
ServerResponse::EncryptedResponse { ciphertext, nonce } => (ciphertext, nonce),
ServerResponse::Error { message } => {
return Err(GatewayClientError::GatewayError(message))
@@ -567,7 +618,7 @@ impl<C, St> GatewayClient<C, St> {
&mut self,
msg: ClientControlRequest,
) -> Result<(), GatewayClientError> {
match self.send_websocket_message(msg).await? {
match self.send_websocket_message_with_response(msg).await? {
ServerResponse::Authenticate {
protocol_version,
status,
@@ -662,6 +713,7 @@ impl<C, St> GatewayClient<C, St> {
let supports_aes_gcm_siv = gw_protocol.supports_aes256_gcm_siv();
let supports_auth_v2 = gw_protocol.supports_authenticate_v2();
let supports_key_rotation_info = gw_protocol.supports_key_rotation_packet();
if !supports_aes_gcm_siv {
warn!("this gateway is on an old version that doesn't support AES256-GCM-SIV");
@@ -669,6 +721,9 @@ impl<C, St> GatewayClient<C, St> {
if !supports_auth_v2 {
warn!("this gateway is on an old version that doesn't support authentication v2")
}
if !supports_key_rotation_info {
warn!("this gateway is on an old version that doesn't support key rotation packets")
}
if self.authenticated {
debug!("Already authenticated");
@@ -713,13 +768,16 @@ impl<C, St> GatewayClient<C, St> {
}
}
/// Attempt to retrieve the currently supported gateway protocol version of the remote.
pub async fn get_gateway_protocol(&mut self) -> Result<u8, GatewayClientError> {
if !self.connection.is_established() {
return Err(GatewayClientError::ConnectionNotEstablished);
}
match self
.send_websocket_message(ClientControlRequest::SupportedProtocol {})
.send_websocket_message_with_non_send_response(
ClientControlRequest::SupportedProtocol {},
)
.await?
{
ServerResponse::SupportedProtocol { version } => Ok(version),
@@ -736,7 +794,10 @@ impl<C, St> GatewayClient<C, St> {
credential,
self.shared_key.as_ref().unwrap(),
)?;
let bandwidth_remaining = match self.send_websocket_message(msg).await? {
let bandwidth_remaining = match self
.send_websocket_message_with_non_send_response(msg)
.await?
{
ServerResponse::Bandwidth { available_total } => Ok(available_total),
ServerResponse::Error { message } => Err(GatewayClientError::GatewayError(message)),
ServerResponse::TypedError { error } => {
@@ -754,7 +815,10 @@ impl<C, St> GatewayClient<C, St> {
async fn try_claim_testnet_bandwidth(&mut self) -> Result<(), GatewayClientError> {
let msg = ClientControlRequest::ClaimFreeTestnetBandwidth;
let bandwidth_remaining = match self.send_websocket_message(msg).await? {
let bandwidth_remaining = match self
.send_websocket_message_with_non_send_response(msg)
.await?
{
ServerResponse::Bandwidth { available_total } => Ok(available_total),
ServerResponse::Error { message } => Err(GatewayClientError::GatewayError(message)),
other => Err(GatewayClientError::UnexpectedResponse { name: other.name() }),
@@ -849,6 +913,22 @@ impl<C, St> GatewayClient<C, St> {
}
}
fn mix_packet_to_ws_message(&self, packet: MixPacket) -> Result<Message, GatewayRequestsError> {
// note: into_ws_message encrypts the requests and adds a MAC on it. Perhaps it should
// be more explicit in the naming?
let req = if self.negotiated_protocol.supports_key_rotation_packet() {
BinaryRequest::ForwardSphinxV2 { packet }
} else {
BinaryRequest::ForwardSphinx { packet }
};
req.into_ws_message(
self.shared_key
.as_ref()
.expect("no shared key present even though we're authenticated!"),
)
}
pub async fn batch_send_mix_packets(
&mut self,
packets: Vec<MixPacket>,
@@ -877,13 +957,7 @@ impl<C, St> GatewayClient<C, St> {
let messages: Result<Vec<_>, _> = packets
.into_iter()
.map(|mix_packet| {
BinaryRequest::ForwardSphinx { packet: mix_packet }.into_ws_message(
self.shared_key
.as_ref()
.expect("no shared key present even though we're authenticated!"),
)
})
.map(|mix_packet| self.mix_packet_to_ws_message(mix_packet))
.collect();
if let Err(err) = self
@@ -949,13 +1023,8 @@ impl<C, St> GatewayClient<C, St> {
if !self.connection.is_established() {
return Err(GatewayClientError::ConnectionNotEstablished);
}
// note: into_ws_message encrypts the requests and adds a MAC on it. Perhaps it should
// be more explicit in the naming?
let msg = BinaryRequest::ForwardSphinx { packet: mix_packet }.into_ws_message(
self.shared_key
.as_ref()
.expect("no shared key present even though we're authenticated!"),
)?;
let msg = self.mix_packet_to_ws_message(mix_packet)?;
self.send_with_reconnection_on_failure(msg).await
}
@@ -56,7 +56,7 @@ pub(crate) async fn connect_async(
}
.map_err(|err| GatewayClientError::NetworkConnectionFailed {
address: endpoint.to_owned(),
source: err.into(),
source: Box::new(tungstenite::Error::from(err)),
})?;
#[cfg(unix)]
@@ -72,7 +72,7 @@ pub(crate) async fn connect_async(
Err(err) => {
stream = Err(GatewayClientError::NetworkConnectionFailed {
address: endpoint.to_owned(),
source: err.into(),
source: Box::new(tungstenite::Error::from(err)),
});
continue;
}
@@ -83,6 +83,6 @@ pub(crate) async fn connect_async(
.await
.map_err(|error| GatewayClientError::NetworkConnectionFailed {
address: endpoint.to_owned(),
source: error,
source: Box::new(error),
})
}
+12 -3
View File
@@ -25,7 +25,7 @@ pub enum GatewayClientError {
RequestError(#[from] GatewayRequestsError),
#[error("There was a network error: {0}")]
NetworkError(#[from] WsError),
NetworkError(Box<WsError>),
#[error("failed to upgrade our shared key - the gateway sent malformed response")]
FatalKeyUpgradeFailure,
@@ -41,7 +41,10 @@ pub enum GatewayClientError {
NetworkErrorWasm(#[from] JsError),
#[error("connection failed: {address}: {source}")]
NetworkConnectionFailed { address: String, source: WsError },
NetworkConnectionFailed {
address: String,
source: Box<WsError>,
},
#[error("no socket address for endpoint: {address}")]
NoEndpointForConnection { address: String },
@@ -127,10 +130,16 @@ pub enum GatewayClientError {
ShutdownInProgress,
}
impl From<WsError> for GatewayClientError {
fn from(error: WsError) -> Self {
GatewayClientError::NetworkError(Box::new(error))
}
}
impl GatewayClientError {
pub fn is_closed_connection(&self) -> bool {
match self {
GatewayClientError::NetworkError(ws_err) => match ws_err {
GatewayClientError::NetworkError(ws_err) => match ws_err.as_ref() {
WsError::AlreadyClosed | WsError::ConnectionClosed => true,
WsError::Io(io_err) => matches!(
io_err.kind(),
+2 -2
View File
@@ -28,7 +28,7 @@ pub(crate) fn cleanup_socket_message(
msg: Option<Result<Message, WsError>>,
) -> Result<Message, GatewayClientError> {
match msg {
Some(msg) => msg.map_err(GatewayClientError::NetworkError),
Some(msg) => msg.map_err(GatewayClientError::from),
None => Err(GatewayClientError::ConnectionAbruptlyClosed),
}
}
@@ -39,7 +39,7 @@ pub(crate) fn cleanup_socket_messages(
match msgs {
Some(msgs) => msgs
.into_iter()
.map(|msg| msg.map_err(GatewayClientError::NetworkError))
.map(|msg| msg.map_err(GatewayClientError::from))
.collect(),
None => Err(GatewayClientError::ConnectionAbruptlyClosed),
}
+6 -1
View File
@@ -16,9 +16,14 @@ tokio-util = { workspace = true, features = ["codec"], optional = true }
tokio-stream = { workspace = true }
# internal
nym-noise = { path = "../../nymnoise" }
nym-sphinx = { path = "../../nymsphinx" }
nym-task = { path = "../../task", optional = true }
[features]
default = ["client"]
client = ["tokio-util", "nym-task", "tokio/net", "tokio/rt"]
client = ["tokio-util", "nym-task", "tokio/net", "tokio/rt"]
[dev-dependencies]
nym-crypto = { path = "../../crypto" }
rand = { workspace = true }
+57 -25
View File
@@ -3,11 +3,11 @@
use dashmap::DashMap;
use futures::StreamExt;
use nym_sphinx::addressing::nodes::NymNodeRoutingAddress;
use nym_noise::config::NoiseConfig;
use nym_noise::upgrade_noise_initiator;
use nym_sphinx::forwarding::packet::MixPacket;
use nym_sphinx::framing::codec::NymCodec;
use nym_sphinx::framing::packet::FramedNymPacket;
use nym_sphinx::params::PacketType;
use nym_sphinx::NymPacket;
use std::io;
use std::net::SocketAddr;
use std::ops::Deref;
@@ -28,6 +28,7 @@ pub struct Config {
pub maximum_reconnection_backoff: Duration,
pub initial_connection_timeout: Duration,
pub maximum_connection_buffer_size: usize,
pub use_legacy_packet_encoding: bool,
}
impl Config {
@@ -36,12 +37,14 @@ impl Config {
maximum_reconnection_backoff: Duration,
initial_connection_timeout: Duration,
maximum_connection_buffer_size: usize,
use_legacy_packet_encoding: bool,
) -> Self {
Config {
initial_reconnection_backoff,
maximum_reconnection_backoff,
initial_connection_timeout,
maximum_connection_buffer_size,
use_legacy_packet_encoding,
}
}
}
@@ -49,23 +52,19 @@ impl Config {
pub trait SendWithoutResponse {
// Without response in this context means we will not listen for anything we might get back (not
// that we should get anything), including any possible io errors
fn send_without_response(
&self,
address: NymNodeRoutingAddress,
packet: NymPacket,
packet_type: PacketType,
) -> io::Result<()>;
fn send_without_response(&self, packet: MixPacket) -> io::Result<()>;
}
pub struct Client {
active_connections: ActiveConnections,
noise_config: NoiseConfig,
connections_count: Arc<AtomicUsize>,
config: Config,
}
#[derive(Default, Clone)]
pub struct ActiveConnections {
inner: Arc<DashMap<NymNodeRoutingAddress, ConnectionSender>>,
inner: Arc<DashMap<SocketAddr, ConnectionSender>>,
}
impl ActiveConnections {
@@ -82,7 +81,7 @@ impl ActiveConnections {
}
impl Deref for ActiveConnections {
type Target = DashMap<NymNodeRoutingAddress, ConnectionSender>;
type Target = DashMap<SocketAddr, ConnectionSender>;
fn deref(&self) -> &Self::Target {
&self.inner
}
@@ -104,6 +103,7 @@ impl ConnectionSender {
struct ManagedConnection {
address: SocketAddr,
noise_config: NoiseConfig,
message_receiver: ReceiverStream<FramedNymPacket>,
connection_timeout: Duration,
current_reconnection: Arc<AtomicU32>,
@@ -112,12 +112,14 @@ struct ManagedConnection {
impl ManagedConnection {
fn new(
address: SocketAddr,
noise_config: NoiseConfig,
message_receiver: mpsc::Receiver<FramedNymPacket>,
connection_timeout: Duration,
current_reconnection: Arc<AtomicU32>,
) -> Self {
ManagedConnection {
address,
noise_config,
message_receiver: ReceiverStream::new(message_receiver),
connection_timeout,
current_reconnection,
@@ -132,9 +134,21 @@ impl ManagedConnection {
Ok(stream_res) => match stream_res {
Ok(stream) => {
debug!("Managed to establish connection to {}", self.address);
// if we managed to connect, reset the reconnection count (whatever it might have been)
let noise_stream =
match upgrade_noise_initiator(stream, &self.noise_config).await {
Ok(noise_stream) => noise_stream,
Err(err) => {
error!("Failed to perform Noise handshake with {address} - {err}");
// we failed to finish the noise handshake - increase reconnection attempt
self.current_reconnection.fetch_add(1, Ordering::SeqCst);
return;
}
};
// if we managed to connect AND do the noise handshake, reset the reconnection count (whatever it might have been)
self.current_reconnection.store(0, Ordering::Release);
Framed::new(stream, NymCodec)
debug!("Noise initiator handshake completed for {:?}", address);
Framed::new(noise_stream, NymCodec)
}
Err(err) => {
debug!("failed to establish connection to {address} (err: {err})",);
@@ -167,9 +181,14 @@ impl ManagedConnection {
}
impl Client {
pub fn new(config: Config, connections_count: Arc<AtomicUsize>) -> Client {
pub fn new(
config: Config,
noise_config: NoiseConfig,
connections_count: Arc<AtomicUsize>,
) -> Client {
Client {
active_connections: Default::default(),
noise_config,
connections_count,
config,
}
@@ -196,7 +215,7 @@ impl Client {
}
}
fn make_connection(&self, address: NymNodeRoutingAddress, pending_packet: FramedNymPacket) {
fn make_connection(&self, address: SocketAddr, pending_packet: FramedNymPacket) {
let (sender, receiver) = mpsc::channel(self.config.maximum_connection_buffer_size);
// this CAN'T fail because we just created the channel which has a non-zero capacity
@@ -224,6 +243,7 @@ impl Client {
let initial_connection_timeout = self.config.initial_connection_timeout;
let connections_count = self.connections_count.clone();
let noise_config = self.noise_config.clone();
tokio::spawn(async move {
// before executing the manager, wait for what was specified, if anything
if let Some(backoff) = backoff {
@@ -233,7 +253,8 @@ impl Client {
connections_count.fetch_add(1, Ordering::SeqCst);
ManagedConnection::new(
address.into(),
address,
noise_config,
receiver,
initial_connection_timeout,
current_reconnection_attempt,
@@ -246,18 +267,19 @@ impl Client {
}
impl SendWithoutResponse for Client {
fn send_without_response(
&self,
address: NymNodeRoutingAddress,
packet: NymPacket,
packet_type: PacketType,
) -> io::Result<()> {
trace!("Sending packet to {address:?}");
let framed_packet = FramedNymPacket::new(packet, packet_type);
fn send_without_response(&self, packet: MixPacket) -> io::Result<()> {
let address = packet.next_hop_address();
trace!("Sending packet to {address}");
// TODO: optimisation for the future: rather than constantly using legacy encoding,
// once we're addressing by node_id (and thus have full node info here),
// we could simply infer supported encoding based on their version
let framed_packet =
FramedNymPacket::from_mix_packet(packet, self.config.use_legacy_packet_encoding);
let Some(sender) = self.active_connections.get_mut(&address) else {
// there was never a connection to begin with
debug!("establishing initial connection to {}", address);
debug!("establishing initial connection to {address}");
// it's not a 'big' error, but we did not manage to send the packet, but queue the packet
// for sending for as soon as the connection is created
self.make_connection(address, framed_packet);
@@ -302,15 +324,25 @@ impl SendWithoutResponse for Client {
#[cfg(test)]
mod tests {
use super::*;
use nym_crypto::asymmetric::x25519;
use nym_noise::config::NoiseNetworkView;
use rand::rngs::OsRng;
fn dummy_client() -> Client {
let mut rng = OsRng; //for test only, so we don't care if rng source isn't crypto grade
Client::new(
Config {
initial_reconnection_backoff: Duration::from_millis(10_000),
maximum_reconnection_backoff: Duration::from_millis(300_000),
initial_connection_timeout: Duration::from_millis(1_500),
maximum_connection_buffer_size: 128,
use_legacy_packet_encoding: false,
},
NoiseConfig::new(
Arc::new(x25519::KeyPair::new(&mut rng)),
NoiseNetworkView::new_empty(),
Duration::from_millis(1_500),
),
Default::default(),
)
}
@@ -19,6 +19,7 @@ nym-vesting-contract-common = { path = "../../cosmwasm-smart-contracts/vesting-c
nym-ecash-contract-common = { path = "../../cosmwasm-smart-contracts/ecash-contract" }
nym-multisig-contract-common = { path = "../../cosmwasm-smart-contracts/multisig-contract" }
nym-group-contract-common = { path = "../../cosmwasm-smart-contracts/group-contract" }
nym-performance-contract-common = { path = "../../cosmwasm-smart-contracts/nym-performance-contract" }
nym-serde-helpers = { path = "../../serde-helpers", features = ["hex", "base64"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
@@ -16,8 +16,8 @@ async fn main() {
let prefix = "n";
let denom: Denom = "unym".parse().unwrap();
let signer_mnemonic: bip39::Mnemonic = "<MNEMONIC WITH FUNDS HERE>".parse().unwrap();
let validator = "https://qwerty-validator.qa.nymte.ch";
let to_address: AccountId = "n19kdst4srf76xgwe55jg32mpcpcyf6aqgp6qrdk".parse().unwrap();
let validator = "https://rpc.sandbox.nymtech.net";
let to_address: AccountId = "n1pefc2utwpy5w78p2kqdsfmpjxfwmn9d39k5mqa".parse().unwrap();
let signer = DirectSecp256k1HdWallet::from_mnemonic(prefix, signer_mnemonic);
let signer_address = signer.try_derive_accounts().unwrap()[0].address().clone();
+106 -63
View File
@@ -25,7 +25,9 @@ use nym_api_requests::models::{
NymNodeDescription, RewardEstimationResponse, StakeSaturationResponse,
};
use nym_api_requests::models::{LegacyDescribedGateway, MixNodeBondAnnotated};
use nym_api_requests::nym_nodes::{NodesByAddressesResponse, SkimmedNode};
use nym_api_requests::nym_nodes::{
NodesByAddressesResponse, SemiSkimmedNodesWithMetadata, SkimmedNode, SkimmedNodesWithMetadata,
};
use nym_coconut_dkg_common::types::EpochId;
use nym_http_api_client::UserAgent;
use nym_mixnet_contract_common::EpochRewardedSet;
@@ -46,6 +48,46 @@ use crate::rpc::http_client;
#[cfg(feature = "http-client")]
use crate::{DirectSigningHttpRpcValidatorClient, HttpRpcClient, QueryHttpRpcValidatorClient};
// a simple helper macro to define to repeatedly call a paged query until a full response is constructed
macro_rules! collect_paged_skimmed_v2 {
( $self:ident, $f: ident ) => {{
// unroll first loop iteration in order to obtain the metadata
let mut page = 0;
let res = $self
.nym_api
.$f(false, Some(page), None, $self.use_bincode)
.await?;
let mut nodes = res.nodes.data;
let metadata = res.metadata;
if res.nodes.pagination.total == nodes.len() {
return Ok(SkimmedNodesWithMetadata::new(nodes, metadata));
}
page += 1;
loop {
let mut res = $self
.nym_api
.$f(false, Some(page), None, $self.use_bincode)
.await?;
if !metadata.consistency_check(&res.metadata) {
return Err(ValidatorClientError::InconsistentPagedMetadata);
}
nodes.append(&mut res.nodes.data);
if nodes.len() < res.nodes.pagination.total {
page += 1
} else {
break;
}
}
Ok(SkimmedNodesWithMetadata::new(nodes, metadata))
}};
}
#[must_use]
#[derive(Debug, Clone)]
pub struct Config {
@@ -200,11 +242,11 @@ impl<C, S> Client<C, S> {
#[allow(deprecated)]
impl<C, S> Client<C, S> {
pub fn api_url(&self) -> &Url {
self.nym_api.current_url()
self.nym_api.current_url().as_ref()
}
pub fn change_nym_api(&mut self, new_endpoint: Url) {
self.nym_api.change_base_url(new_endpoint)
self.nym_api.change_base_urls(vec![new_endpoint.into()])
}
#[deprecated]
@@ -402,11 +444,11 @@ impl NymApiClient {
}
pub fn api_url(&self) -> &Url {
self.nym_api.current_url()
self.nym_api.current_url().as_ref()
}
pub fn change_nym_api(&mut self, new_endpoint: Url) {
self.nym_api.change_base_url(new_endpoint);
self.nym_api.change_base_urls(vec![new_endpoint.into()]);
}
#[deprecated(note = "use get_all_basic_active_mixing_assigned_nodes instead")]
@@ -425,92 +467,93 @@ impl NymApiClient {
/// retrieve basic information for nodes are capable of operating as an entry gateway
/// this includes legacy gateways and nym-nodes
#[deprecated(note = "use get_all_basic_entry_assigned_nodes_with_metadata instead")]
pub async fn get_all_basic_entry_assigned_nodes(
&self,
) -> Result<Vec<SkimmedNode>, ValidatorClientError> {
// TODO: deal with paging in macro or some helper function or something, because it's the same pattern everywhere
let mut page = 0;
let mut nodes = Vec::new();
self.get_all_basic_entry_assigned_nodes_with_metadata()
.await
.map(|res| res.nodes)
}
loop {
let mut res = self
.nym_api
.get_basic_entry_assigned_nodes(false, Some(page), None, self.use_bincode)
.await?;
nodes.append(&mut res.nodes.data);
if nodes.len() < res.nodes.pagination.total {
page += 1
} else {
break;
}
}
Ok(nodes)
pub async fn get_all_basic_entry_assigned_nodes_with_metadata(
&self,
) -> Result<SkimmedNodesWithMetadata, ValidatorClientError> {
collect_paged_skimmed_v2!(self, get_basic_entry_assigned_nodes_v2)
}
/// retrieve basic information for nodes that got assigned 'mixing' node in this epoch
/// this includes legacy mixnodes and nym-nodes
#[deprecated(note = "use get_all_basic_active_mixing_assigned_nodes_with_metadata instead")]
pub async fn get_all_basic_active_mixing_assigned_nodes(
&self,
) -> Result<Vec<SkimmedNode>, ValidatorClientError> {
// TODO: deal with paging in macro or some helper function or something, because it's the same pattern everywhere
let mut page = 0;
let mut nodes = Vec::new();
self.get_all_basic_active_mixing_assigned_nodes_with_metadata()
.await
.map(|res| res.nodes)
}
loop {
let mut res = self
.nym_api
.get_basic_active_mixing_assigned_nodes(false, Some(page), None, self.use_bincode)
.await?;
nodes.append(&mut res.nodes.data);
if nodes.len() < res.nodes.pagination.total {
page += 1
} else {
break;
}
}
Ok(nodes)
pub async fn get_all_basic_active_mixing_assigned_nodes_with_metadata(
&self,
) -> Result<SkimmedNodesWithMetadata, ValidatorClientError> {
collect_paged_skimmed_v2!(self, get_basic_active_mixing_assigned_nodes_v2)
}
/// retrieve basic information for nodes are capable of operating as a mixnode
/// this includes legacy mixnodes and nym-nodes
#[deprecated(note = "use get_all_basic_mixing_capable_nodes_with_metadata instead")]
pub async fn get_all_basic_mixing_capable_nodes(
&self,
) -> Result<Vec<SkimmedNode>, ValidatorClientError> {
// TODO: deal with paging in macro or some helper function or something, because it's the same pattern everywhere
let mut page = 0;
let mut nodes = Vec::new();
self.get_all_basic_mixing_capable_nodes_with_metadata()
.await
.map(|res| res.nodes)
}
loop {
let mut res = self
.nym_api
.get_basic_mixing_capable_nodes(false, Some(page), None, self.use_bincode)
.await?;
nodes.append(&mut res.nodes.data);
if nodes.len() < res.nodes.pagination.total {
page += 1
} else {
break;
}
}
Ok(nodes)
pub async fn get_all_basic_mixing_capable_nodes_with_metadata(
&self,
) -> Result<SkimmedNodesWithMetadata, ValidatorClientError> {
collect_paged_skimmed_v2!(self, get_basic_mixing_capable_nodes_v2)
}
/// retrieve basic information for all bonded nodes on the network
#[deprecated(note = "use get_all_basic_nodes_with_metadata instead")]
pub async fn get_all_basic_nodes(&self) -> Result<Vec<SkimmedNode>, ValidatorClientError> {
// TODO: deal with paging in macro or some helper function or something, because it's the same pattern everywhere
self.get_all_basic_nodes_with_metadata()
.await
.map(|res| res.nodes)
}
pub async fn get_all_basic_nodes_with_metadata(
&self,
) -> Result<SkimmedNodesWithMetadata, ValidatorClientError> {
collect_paged_skimmed_v2!(self, get_basic_nodes_v2)
}
/// retrieve expanded information for all bonded nodes on the network
pub async fn get_all_expanded_nodes(
&self,
) -> Result<SemiSkimmedNodesWithMetadata, ValidatorClientError> {
// Unroll the first iteration to get the metadata
let mut page = 0;
let mut nodes = Vec::new();
let res = self
.nym_api
.get_expanded_nodes(false, Some(page), None)
.await?;
let mut nodes = res.nodes.data;
let metadata = res.metadata;
if res.nodes.pagination.total == nodes.len() {
return Ok(SemiSkimmedNodesWithMetadata::new(nodes, metadata));
}
page += 1;
loop {
let mut res = self
.nym_api
.get_basic_nodes(false, Some(page), None, self.use_bincode)
.get_expanded_nodes(false, Some(page), None)
.await?;
nodes.append(&mut res.nodes.data);
@@ -521,7 +564,7 @@ impl NymApiClient {
}
}
Ok(nodes)
Ok(SemiSkimmedNodesWithMetadata::new(nodes, metadata))
}
pub async fn health(&self) -> Result<ApiHealthResponse, ValidatorClientError> {
@@ -22,6 +22,9 @@ pub enum ValidatorClientError {
#[error("nyxd request failed: {0}")]
NyxdError(#[from] crate::nyxd::error::NyxdError),
#[error("the response metadata has changed between pages")]
InconsistentPagedMetadata,
#[error("No validator API url has been provided")]
NoAPIUrlAvailable,
}
@@ -14,11 +14,12 @@ use nym_api_requests::ecash::models::{
use nym_api_requests::ecash::VerificationKeyResponse;
use nym_api_requests::models::{
AnnotationResponse, ApiHealthResponse, BinaryBuildInformationOwned, ChainStatusResponse,
LegacyDescribedMixNode, NodePerformanceResponse, NodeRefreshBody, NymNodeDescription,
PerformanceHistoryResponse, RewardedSetResponse,
KeyRotationInfoResponse, LegacyDescribedMixNode, NodePerformanceResponse, NodeRefreshBody,
NymNodeDescription, PerformanceHistoryResponse, RewardedSetResponse,
};
use nym_api_requests::nym_nodes::{
NodesByAddressesRequestBody, NodesByAddressesResponse, PaginatedCachedNodesResponse,
NodesByAddressesRequestBody, NodesByAddressesResponse, PaginatedCachedNodesResponseV1,
PaginatedCachedNodesResponseV2,
};
use nym_api_requests::pagination::PaginatedResponse;
pub use nym_api_requests::{
@@ -34,7 +35,7 @@ pub use nym_api_requests::{
MixnodeStatusResponse, MixnodeUptimeHistoryResponse, RewardEstimationResponse,
StakeSaturationResponse, UptimeResponse,
},
nym_nodes::{CachedNodesResponse, SkimmedNode},
nym_nodes::{CachedNodesResponse, SemiSkimmedNode, SkimmedNode},
NymNetworkDetailsResponse,
};
use nym_contracts_common::IdentityKey;
@@ -62,7 +63,7 @@ pub trait NymApiClientExt: ApiClient {
async fn health(&self) -> Result<ApiHealthResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::API_STATUS_ROUTES,
routes::HEALTH,
],
@@ -75,7 +76,7 @@ pub trait NymApiClientExt: ApiClient {
async fn build_information(&self) -> Result<BinaryBuildInformationOwned, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::API_STATUS_ROUTES,
routes::BUILD_INFORMATION,
],
@@ -87,7 +88,7 @@ pub trait NymApiClientExt: ApiClient {
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnodes(&self) -> Result<Vec<MixNodeDetails>, NymAPIError> {
self.get_json(&[routes::API_VERSION, routes::MIXNODES], NO_PARAMS)
self.get_json(&[routes::V1_API_VERSION, routes::MIXNODES], NO_PARAMS)
.await
}
@@ -96,7 +97,7 @@ pub trait NymApiClientExt: ApiClient {
async fn get_mixnodes_detailed(&self) -> Result<Vec<MixNodeBondAnnotated>, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS,
routes::MIXNODES,
routes::DETAILED,
@@ -111,7 +112,7 @@ pub trait NymApiClientExt: ApiClient {
async fn get_gateways_detailed(&self) -> Result<Vec<GatewayBondAnnotated>, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS,
routes::GATEWAYS,
routes::DETAILED,
@@ -128,7 +129,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<Vec<GatewayBondAnnotated>, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS,
routes::GATEWAYS,
routes::DETAILED_UNFILTERED,
@@ -145,7 +146,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<Vec<MixNodeBondAnnotated>, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS,
routes::MIXNODES,
routes::DETAILED_UNFILTERED,
@@ -158,7 +159,7 @@ pub trait NymApiClientExt: ApiClient {
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_gateways(&self) -> Result<Vec<GatewayBond>, NymAPIError> {
self.get_json(&[routes::API_VERSION, routes::GATEWAYS], NO_PARAMS)
self.get_json(&[routes::V1_API_VERSION, routes::GATEWAYS], NO_PARAMS)
.await
}
@@ -166,7 +167,7 @@ pub trait NymApiClientExt: ApiClient {
#[instrument(level = "debug", skip(self))]
async fn get_gateways_described(&self) -> Result<Vec<LegacyDescribedGateway>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::GATEWAYS, routes::DESCRIBED],
&[routes::V1_API_VERSION, routes::GATEWAYS, routes::DESCRIBED],
NO_PARAMS,
)
.await
@@ -176,7 +177,7 @@ pub trait NymApiClientExt: ApiClient {
#[instrument(level = "debug", skip(self))]
async fn get_mixnodes_described(&self) -> Result<Vec<LegacyDescribedMixNode>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::MIXNODES, routes::DESCRIBED],
&[routes::V1_API_VERSION, routes::MIXNODES, routes::DESCRIBED],
NO_PARAMS,
)
.await
@@ -201,7 +202,7 @@ pub trait NymApiClientExt: ApiClient {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::NYM_NODES_ROUTES,
routes::NYM_NODES_PERFORMANCE_HISTORY,
&*node_id.to_string(),
@@ -229,7 +230,7 @@ pub trait NymApiClientExt: ApiClient {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::NYM_NODES_ROUTES,
routes::NYM_NODES_DESCRIBED,
],
@@ -256,7 +257,7 @@ pub trait NymApiClientExt: ApiClient {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::NYM_NODES_ROUTES,
routes::NYM_NODES_BONDED,
],
@@ -270,7 +271,7 @@ pub trait NymApiClientExt: ApiClient {
async fn get_basic_mixnodes(&self) -> Result<CachedNodesResponse<SkimmedNode>, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"mixnodes",
@@ -286,7 +287,7 @@ pub trait NymApiClientExt: ApiClient {
async fn get_basic_gateways(&self) -> Result<CachedNodesResponse<SkimmedNode>, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"gateways",
@@ -301,7 +302,7 @@ pub trait NymApiClientExt: ApiClient {
async fn get_rewarded_set(&self) -> Result<RewardedSetResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::NYM_NODES_ROUTES,
routes::NYM_NODES_REWARDED_SET,
],
@@ -312,6 +313,7 @@ pub trait NymApiClientExt: ApiClient {
/// retrieve basic information for nodes are capable of operating as an entry gateway
/// this includes legacy gateways and nym-nodes
#[deprecated(note = "use get_basic_entry_assigned_nodes_v2")]
#[instrument(level = "debug", skip(self))]
async fn get_basic_entry_assigned_nodes(
&self,
@@ -319,7 +321,7 @@ pub trait NymApiClientExt: ApiClient {
page: Option<u32>,
per_page: Option<u32>,
use_bincode: bool,
) -> Result<PaginatedCachedNodesResponse<SkimmedNode>, NymAPIError> {
) -> Result<PaginatedCachedNodesResponseV1<SkimmedNode>, NymAPIError> {
let mut params = Vec::new();
if no_legacy {
@@ -340,7 +342,7 @@ pub trait NymApiClientExt: ApiClient {
self.get_response(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"skimmed",
@@ -352,16 +354,16 @@ pub trait NymApiClientExt: ApiClient {
.await
}
/// retrieve basic information for nodes that got assigned 'mixing' node in this epoch
/// this includes legacy mixnodes and nym-nodes
/// retrieve basic information for nodes are capable of operating as an entry gateway
/// this includes legacy gateways and nym-nodes
#[instrument(level = "debug", skip(self))]
async fn get_basic_active_mixing_assigned_nodes(
async fn get_basic_entry_assigned_nodes_v2(
&self,
no_legacy: bool,
page: Option<u32>,
per_page: Option<u32>,
use_bincode: bool,
) -> Result<PaginatedCachedNodesResponse<SkimmedNode>, NymAPIError> {
) -> Result<PaginatedCachedNodesResponseV2<SkimmedNode>, NymAPIError> {
let mut params = Vec::new();
if no_legacy {
@@ -382,7 +384,49 @@ pub trait NymApiClientExt: ApiClient {
self.get_response(
&[
routes::API_VERSION,
routes::V2_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"skimmed",
"entry-gateways",
],
&params,
)
.await
}
/// retrieve basic information for nodes that got assigned 'mixing' node in this epoch
/// this includes legacy mixnodes and nym-nodes
#[deprecated(note = "use get_basic_active_mixing_assigned_nodes_v2")]
#[instrument(level = "debug", skip(self))]
async fn get_basic_active_mixing_assigned_nodes(
&self,
no_legacy: bool,
page: Option<u32>,
per_page: Option<u32>,
use_bincode: bool,
) -> Result<PaginatedCachedNodesResponseV1<SkimmedNode>, NymAPIError> {
let mut params = Vec::new();
if no_legacy {
params.push(("no_legacy", "true".to_string()))
}
if let Some(page) = page {
params.push(("page", page.to_string()))
}
if let Some(per_page) = per_page {
params.push(("per_page", per_page.to_string()))
}
if use_bincode {
params.push(("output", "bincode".to_string()))
}
self.get_response(
&[
routes::V1_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"skimmed",
@@ -397,13 +441,13 @@ pub trait NymApiClientExt: ApiClient {
/// retrieve basic information for nodes that got assigned 'mixing' node in this epoch
/// this includes legacy mixnodes and nym-nodes
#[instrument(level = "debug", skip(self))]
async fn get_basic_mixing_capable_nodes(
async fn get_basic_active_mixing_assigned_nodes_v2(
&self,
no_legacy: bool,
page: Option<u32>,
per_page: Option<u32>,
use_bincode: bool,
) -> Result<PaginatedCachedNodesResponse<SkimmedNode>, NymAPIError> {
) -> Result<PaginatedCachedNodesResponseV2<SkimmedNode>, NymAPIError> {
let mut params = Vec::new();
if no_legacy {
@@ -424,7 +468,50 @@ pub trait NymApiClientExt: ApiClient {
self.get_response(
&[
routes::API_VERSION,
routes::V2_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"skimmed",
"mixnodes",
"active",
],
&params,
)
.await
}
/// retrieve basic information for nodes that got assigned 'mixing' node in this epoch
/// this includes legacy mixnodes and nym-nodes
#[deprecated(note = "use get_basic_mixing_capable_nodes_v2")]
#[instrument(level = "debug", skip(self))]
async fn get_basic_mixing_capable_nodes(
&self,
no_legacy: bool,
page: Option<u32>,
per_page: Option<u32>,
use_bincode: bool,
) -> Result<PaginatedCachedNodesResponseV1<SkimmedNode>, NymAPIError> {
let mut params = Vec::new();
if no_legacy {
params.push(("no_legacy", "true".to_string()))
}
if let Some(page) = page {
params.push(("page", page.to_string()))
}
if let Some(per_page) = per_page {
params.push(("per_page", per_page.to_string()))
}
if use_bincode {
params.push(("output", "bincode".to_string()))
}
self.get_response(
&[
routes::V1_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"skimmed",
@@ -436,14 +523,16 @@ pub trait NymApiClientExt: ApiClient {
.await
}
/// retrieve basic information for nodes that got assigned 'mixing' node in this epoch
/// this includes legacy mixnodes and nym-nodes
#[instrument(level = "debug", skip(self))]
async fn get_basic_nodes(
async fn get_basic_mixing_capable_nodes_v2(
&self,
no_legacy: bool,
page: Option<u32>,
per_page: Option<u32>,
use_bincode: bool,
) -> Result<PaginatedCachedNodesResponse<SkimmedNode>, NymAPIError> {
) -> Result<PaginatedCachedNodesResponseV2<SkimmedNode>, NymAPIError> {
let mut params = Vec::new();
if no_legacy {
@@ -464,10 +553,122 @@ pub trait NymApiClientExt: ApiClient {
self.get_response(
&[
routes::API_VERSION,
routes::V2_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"skimmed",
"mixnodes",
"all",
],
&params,
)
.await
}
#[deprecated(note = "use get_basic_nodes_v2")]
#[instrument(level = "debug", skip(self))]
async fn get_basic_nodes(
&self,
no_legacy: bool,
page: Option<u32>,
per_page: Option<u32>,
use_bincode: bool,
) -> Result<PaginatedCachedNodesResponseV1<SkimmedNode>, NymAPIError> {
let mut params = Vec::new();
if no_legacy {
params.push(("no_legacy", "true".to_string()))
}
if let Some(page) = page {
params.push(("page", page.to_string()))
}
if let Some(per_page) = per_page {
params.push(("per_page", per_page.to_string()))
}
if use_bincode {
params.push(("output", "bincode".to_string()))
}
self.get_response(
&[
routes::V1_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"skimmed",
],
&params,
)
.await
}
#[instrument(level = "debug", skip(self))]
async fn get_basic_nodes_v2(
&self,
no_legacy: bool,
page: Option<u32>,
per_page: Option<u32>,
use_bincode: bool,
) -> Result<PaginatedCachedNodesResponseV2<SkimmedNode>, NymAPIError> {
let mut params = Vec::new();
if no_legacy {
params.push(("no_legacy", "true".to_string()))
}
if let Some(page) = page {
params.push(("page", page.to_string()))
}
if let Some(per_page) = per_page {
params.push(("per_page", per_page.to_string()))
}
if use_bincode {
params.push(("output", "bincode".to_string()))
}
self.get_response(
&[
routes::V2_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"skimmed",
],
&params,
)
.await
}
#[instrument(level = "debug", skip(self))]
async fn get_expanded_nodes(
&self,
no_legacy: bool,
page: Option<u32>,
per_page: Option<u32>,
) -> Result<PaginatedCachedNodesResponseV2<SemiSkimmedNode>, NymAPIError> {
let mut params = Vec::new();
if no_legacy {
params.push(("no_legacy", "true".to_string()))
}
if let Some(page) = page {
params.push(("page", page.to_string()))
}
if let Some(per_page) = per_page {
params.push(("per_page", per_page.to_string()))
}
self.get_json(
&[
routes::V2_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
"semi-skimmed",
],
&params,
)
@@ -478,7 +679,7 @@ pub trait NymApiClientExt: ApiClient {
#[instrument(level = "debug", skip(self))]
async fn get_active_mixnodes(&self) -> Result<Vec<MixNodeDetails>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::MIXNODES, routes::ACTIVE],
&[routes::V1_API_VERSION, routes::MIXNODES, routes::ACTIVE],
NO_PARAMS,
)
.await
@@ -489,7 +690,7 @@ pub trait NymApiClientExt: ApiClient {
async fn get_active_mixnodes_detailed(&self) -> Result<Vec<MixNodeBondAnnotated>, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS,
routes::MIXNODES,
routes::ACTIVE,
@@ -504,7 +705,7 @@ pub trait NymApiClientExt: ApiClient {
#[instrument(level = "debug", skip(self))]
async fn get_rewarded_mixnodes(&self) -> Result<Vec<MixNodeDetails>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::MIXNODES, routes::REWARDED],
&[routes::V1_API_VERSION, routes::MIXNODES, routes::REWARDED],
NO_PARAMS,
)
.await
@@ -518,7 +719,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<MixnodeStatusReportResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS,
routes::MIXNODE,
&mix_id.to_string(),
@@ -537,7 +738,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<GatewayStatusReportResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS,
routes::GATEWAY,
identity,
@@ -556,7 +757,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<MixnodeUptimeHistoryResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS,
routes::MIXNODE,
&mix_id.to_string(),
@@ -575,7 +776,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<GatewayUptimeHistoryResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS,
routes::GATEWAY,
identity,
@@ -593,7 +794,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<Vec<MixNodeBondAnnotated>, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS,
routes::MIXNODES,
routes::REWARDED,
@@ -614,7 +815,7 @@ pub trait NymApiClientExt: ApiClient {
if let Some(since) = since {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS_ROUTES,
routes::GATEWAY,
identity,
@@ -626,7 +827,7 @@ pub trait NymApiClientExt: ApiClient {
} else {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS_ROUTES,
routes::GATEWAY,
identity,
@@ -647,7 +848,7 @@ pub trait NymApiClientExt: ApiClient {
if let Some(since) = since {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS_ROUTES,
routes::MIXNODE,
&mix_id.to_string(),
@@ -659,7 +860,7 @@ pub trait NymApiClientExt: ApiClient {
} else {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS_ROUTES,
routes::MIXNODE,
&mix_id.to_string(),
@@ -679,7 +880,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<MixnodeStatusResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS_ROUTES,
routes::MIXNODE,
&mix_id.to_string(),
@@ -698,7 +899,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<RewardEstimationResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS_ROUTES,
routes::MIXNODE,
&mix_id.to_string(),
@@ -718,7 +919,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<RewardEstimationResponse, NymAPIError> {
self.post_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS_ROUTES,
routes::MIXNODE,
&mix_id.to_string(),
@@ -738,7 +939,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<StakeSaturationResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS_ROUTES,
routes::MIXNODE,
&mix_id.to_string(),
@@ -758,7 +959,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<nym_api_requests::models::InclusionProbabilityResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS_ROUTES,
routes::MIXNODE,
&mix_id.to_string(),
@@ -776,7 +977,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<NodePerformanceResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::NYM_NODES_ROUTES,
routes::NYM_NODES_PERFORMANCE,
&node_id.to_string(),
@@ -792,7 +993,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<AnnotationResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::NYM_NODES_ROUTES,
routes::NYM_NODES_ANNOTATION,
&node_id.to_string(),
@@ -806,7 +1007,7 @@ pub trait NymApiClientExt: ApiClient {
async fn get_mixnode_avg_uptime(&self, mix_id: NodeId) -> Result<UptimeResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::STATUS_ROUTES,
routes::MIXNODE,
&mix_id.to_string(),
@@ -821,7 +1022,11 @@ pub trait NymApiClientExt: ApiClient {
#[instrument(level = "debug", skip(self))]
async fn get_mixnodes_blacklisted(&self) -> Result<Vec<NodeId>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::MIXNODES, routes::BLACKLISTED],
&[
routes::V1_API_VERSION,
routes::MIXNODES,
routes::BLACKLISTED,
],
NO_PARAMS,
)
.await
@@ -831,7 +1036,11 @@ pub trait NymApiClientExt: ApiClient {
#[instrument(level = "debug", skip(self))]
async fn get_gateways_blacklisted(&self) -> Result<Vec<IdentityKey>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::GATEWAYS, routes::BLACKLISTED],
&[
routes::V1_API_VERSION,
routes::GATEWAYS,
routes::BLACKLISTED,
],
NO_PARAMS,
)
.await
@@ -844,7 +1053,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<BlindedSignatureResponse, NymAPIError> {
self.post_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::ECASH_BLIND_SIGN,
],
@@ -861,7 +1070,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<EcashTicketVerificationResponse, NymAPIError> {
self.post_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::VERIFY_ECASH_TICKET,
],
@@ -878,7 +1087,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<EcashBatchTicketRedemptionResponse, NymAPIError> {
self.post_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::BATCH_REDEEM_ECASH_TICKETS,
],
@@ -903,7 +1112,7 @@ pub trait NymApiClientExt: ApiClient {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::PARTIAL_EXPIRATION_DATE_SIGNATURES,
],
@@ -924,7 +1133,7 @@ pub trait NymApiClientExt: ApiClient {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::PARTIAL_COIN_INDICES_SIGNATURES,
],
@@ -948,7 +1157,7 @@ pub trait NymApiClientExt: ApiClient {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::GLOBAL_EXPIRATION_DATE_SIGNATURES,
],
@@ -969,7 +1178,7 @@ pub trait NymApiClientExt: ApiClient {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::GLOBAL_COIN_INDICES_SIGNATURES,
],
@@ -989,7 +1198,7 @@ pub trait NymApiClientExt: ApiClient {
};
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
ecash::MASTER_VERIFICATION_KEY,
],
@@ -1005,7 +1214,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<(), NymAPIError> {
self.post_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::NYM_NODES_ROUTES,
routes::NYM_NODES_REFRESH_DESCRIBED,
],
@@ -1022,7 +1231,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<IssuedTicketbooksForResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::ECASH_ISSUED_TICKETBOOKS_FOR,
&expiration_date.to_string(),
@@ -1039,7 +1248,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<IssuedTicketbooksForCountResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::ECASH_ISSUED_TICKETBOOKS_FOR_COUNT,
&expiration_date.to_string(),
@@ -1056,7 +1265,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<IssuedTicketbooksChallengeCommitmentResponse, NymAPIError> {
self.post_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::ECASH_ISSUED_TICKETBOOKS_CHALLENGE_COMMITMENT,
],
@@ -1073,7 +1282,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<IssuedTicketbooksDataResponse, NymAPIError> {
self.post_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
routes::ECASH_ROUTES,
routes::ECASH_ISSUED_TICKETBOOKS_DATA,
],
@@ -1089,7 +1298,7 @@ pub trait NymApiClientExt: ApiClient {
) -> Result<NodesByAddressesResponse, NymAPIError> {
self.post_json(
&[
routes::API_VERSION,
routes::V1_API_VERSION,
"unstable",
routes::NYM_NODES_ROUTES,
routes::nym_nodes::BY_ADDRESSES,
@@ -1103,7 +1312,7 @@ pub trait NymApiClientExt: ApiClient {
#[instrument(level = "debug", skip(self))]
async fn get_network_details(&self) -> Result<NymNetworkDetailsResponse, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::NETWORK, routes::DETAILS],
&[routes::V1_API_VERSION, routes::NETWORK, routes::DETAILS],
NO_PARAMS,
)
.await
@@ -1112,7 +1321,24 @@ pub trait NymApiClientExt: ApiClient {
#[instrument(level = "debug", skip(self))]
async fn get_chain_status(&self) -> Result<ChainStatusResponse, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::NETWORK, routes::CHAIN_STATUS],
&[
routes::V1_API_VERSION,
routes::NETWORK,
routes::CHAIN_STATUS,
],
NO_PARAMS,
)
.await
}
#[instrument(level = "debug", skip(self))]
async fn get_key_rotation_info(&self) -> Result<KeyRotationInfoResponse, NymAPIError> {
self.get_json(
&[
routes::V1_API_VERSION,
routes::EPOCH,
routes::KEY_ROTATION_INFO,
],
NO_PARAMS,
)
.await
@@ -1,9 +1,8 @@
// Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use nym_network_defaults::NYM_API_VERSION;
pub const API_VERSION: &str = NYM_API_VERSION;
pub const V1_API_VERSION: &str = "v1";
pub const V2_API_VERSION: &str = "v2";
pub const MIXNODES: &str = "mixnodes";
pub const GATEWAYS: &str = "gateways";
pub const DESCRIBED: &str = "described";
@@ -79,3 +78,11 @@ pub const SERVICE_PROVIDERS: &str = "services";
pub const DETAILS: &str = "details";
pub const CHAIN_STATUS: &str = "chain-status";
pub const NETWORK: &str = "network";
pub const EPOCH: &str = "epoch";
pub use epoch_routes::*;
pub mod epoch_routes {
pub const CURRENT: &str = "current";
pub const KEY_ROTATION_INFO: &str = "key-rotation-info";
}
@@ -12,8 +12,8 @@ use nym_mixnet_contract_common::gateway::{PreassignedGatewayIdsResponse, Preassi
use nym_mixnet_contract_common::nym_node::{
EpochAssignmentResponse, NodeDetailsByIdentityResponse, NodeDetailsResponse,
NodeOwnershipResponse, NodeRewardingDetailsResponse, PagedNymNodeBondsResponse,
PagedNymNodeDetailsResponse, PagedUnbondedNymNodesResponse, Role, RolesMetadataResponse,
StakeSaturationResponse, UnbondedNodeResponse, UnbondedNymNode,
PagedNymNodeDetailsResponse, PagedUnbondedNymNodesResponse, RewardedSetMetadata, Role,
RolesMetadataResponse, StakeSaturationResponse, UnbondedNodeResponse, UnbondedNymNode,
};
use nym_mixnet_contract_common::reward_params::WorkFactor;
use nym_mixnet_contract_common::{
@@ -28,12 +28,12 @@ use nym_mixnet_contract_common::{
ContractBuildInformation, ContractState, ContractStateParams, CurrentIntervalResponse,
CurrentNymNodeVersionResponse, Delegation, EpochEventId, EpochRewardedSet, EpochStatus,
GatewayBond, GatewayBondResponse, GatewayOwnershipResponse, HistoricalNymNodeVersionEntry,
IdentityKey, IdentityKeyRef, IntervalEventId, MixNodeBond, MixNodeDetails,
MixOwnershipResponse, MixnodeDetailsByIdentityResponse, MixnodeDetailsResponse, NodeId,
NumberOfPendingEventsResponse, NymNodeBond, NymNodeDetails, NymNodeVersionHistoryResponse,
PagedAllDelegationsResponse, PagedDelegatorDelegationsResponse, PagedGatewayResponse,
PagedMixnodeBondsResponse, PagedNodeDelegationsResponse, PendingEpochEvent,
PendingEpochEventResponse, PendingEpochEventsResponse, PendingIntervalEvent,
IdentityKey, IdentityKeyRef, IntervalEventId, KeyRotationIdResponse, KeyRotationState,
MixNodeBond, MixNodeDetails, MixOwnershipResponse, MixnodeDetailsByIdentityResponse,
MixnodeDetailsResponse, NodeId, NumberOfPendingEventsResponse, NymNodeBond, NymNodeDetails,
NymNodeVersionHistoryResponse, PagedAllDelegationsResponse, PagedDelegatorDelegationsResponse,
PagedGatewayResponse, PagedMixnodeBondsResponse, PagedNodeDelegationsResponse,
PendingEpochEvent, PendingEpochEventResponse, PendingEpochEventsResponse, PendingIntervalEvent,
PendingIntervalEventResponse, PendingIntervalEventsResponse, QueryMsg as MixnetQueryMsg,
RewardedSet, UnbondedMixnode,
};
@@ -546,6 +546,16 @@ pub trait MixnetQueryClient {
})
.await
}
async fn get_key_rotation_state(&self) -> Result<KeyRotationState, NyxdError> {
self.query_mixnet_contract(MixnetQueryMsg::GetKeyRotationState {})
.await
}
async fn get_key_rotation_id(&self) -> Result<KeyRotationIdResponse, NyxdError> {
self.query_mixnet_contract(MixnetQueryMsg::GetKeyRotationId {})
.await
}
}
// extension trait to the query client to deal with the paged queries
@@ -673,12 +683,20 @@ pub trait MixnetQueryClientExt: MixnetQueryClient {
async fn get_rewarded_set(&self) -> Result<EpochRewardedSet, NyxdError> {
let error_response = |message| Err(NyxdError::extension_query_failure("mixnet", message));
// bypass for catch 22 for fresh contracts. we can't refresh cache because there's no rewarded set,
// but we can't set the rewarded set because we didn't refresh the cache
let metadata = self.get_rewarded_set_metadata().await?;
if !metadata.metadata.fully_assigned {
let is_default = metadata.metadata == RewardedSetMetadata::default();
if !metadata.metadata.fully_assigned && !is_default {
return error_response("the rewarded set hasn't been fully assigned for this epoch");
}
let expected_epoch_id = metadata.metadata.epoch_id;
if is_default {
return Ok(Default::default());
}
// if we have to query those things more frequently, we could do it concurrently,
// but as it stands now, it happens so infrequently it might as well be sequential
let entry = self.get_role_assignment(Role::EntryGateway).await?;
@@ -955,6 +973,8 @@ mod tests {
QueryMsg::GetNymNodeVersionHistory { limit, start_after } => client
.get_nym_node_version_history_paged(start_after, limit)
.ignore(),
QueryMsg::GetKeyRotationState {} => client.get_key_rotation_state().ignore(),
QueryMsg::GetKeyRotationId {} => client.get_key_rotation_id().ignore(),
}
}
}
@@ -13,6 +13,7 @@ pub mod ecash_query_client;
pub mod group_query_client;
pub mod mixnet_query_client;
pub mod multisig_query_client;
pub mod performance_query_client;
pub mod vesting_query_client;
// signing clients
@@ -21,6 +22,7 @@ pub mod ecash_signing_client;
pub mod group_signing_client;
pub mod mixnet_signing_client;
pub mod multisig_signing_client;
pub mod performance_signing_client;
pub mod vesting_signing_client;
// re-export query traits
@@ -29,6 +31,7 @@ pub use ecash_query_client::{EcashQueryClient, PagedEcashQueryClient};
pub use group_query_client::{GroupQueryClient, PagedGroupQueryClient};
pub use mixnet_query_client::{MixnetQueryClient, PagedMixnetQueryClient};
pub use multisig_query_client::{MultisigQueryClient, PagedMultisigQueryClient};
pub use performance_query_client::{PagedPerformanceQueryClient, PerformanceQueryClient};
pub use vesting_query_client::{PagedVestingQueryClient, VestingQueryClient};
// re-export signing traits
@@ -37,6 +40,7 @@ pub use ecash_signing_client::EcashSigningClient;
pub use group_signing_client::GroupSigningClient;
pub use mixnet_signing_client::MixnetSigningClient;
pub use multisig_signing_client::MultisigSigningClient;
pub use performance_signing_client::PerformanceSigningClient;
pub use vesting_signing_client::VestingSigningClient;
// helper for providing blanket implementation for query clients
@@ -44,6 +48,7 @@ pub trait NymContractsProvider {
// main
fn mixnet_contract_address(&self) -> Option<&AccountId>;
fn vesting_contract_address(&self) -> Option<&AccountId>;
fn performance_contract_address(&self) -> Option<&AccountId>;
// coconut-related
fn ecash_contract_address(&self) -> Option<&AccountId>;
@@ -56,6 +61,7 @@ pub trait NymContractsProvider {
pub struct TypedNymContracts {
pub mixnet_contract_address: Option<AccountId>,
pub vesting_contract_address: Option<AccountId>,
pub performance_contract_address: Option<AccountId>,
pub ecash_contract_address: Option<AccountId>,
pub group_contract_address: Option<AccountId>,
@@ -76,6 +82,10 @@ impl TryFrom<NymContracts> for TypedNymContracts {
.vesting_contract_address
.map(|addr| addr.parse())
.transpose()?,
performance_contract_address: value
.performance_contract_address
.map(|addr| addr.parse())
.transpose()?,
ecash_contract_address: value
.ecash_contract_address
.map(|addr| addr.parse())
@@ -0,0 +1,271 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::collect_paged;
use crate::nyxd::contract_traits::NymContractsProvider;
use crate::nyxd::error::NyxdError;
use crate::nyxd::CosmWasmClient;
use async_trait::async_trait;
use cosmrs::AccountId;
use serde::Deserialize;
pub use nym_performance_contract_common::{
msg::QueryMsg as PerformanceQueryMsg, types::NetworkMonitorResponse, EpochId,
EpochMeasurementsPagedResponse, EpochNodePerformance, EpochPerformancePagedResponse,
FullHistoricalPerformancePagedResponse, HistoricalPerformance, LastSubmission,
NetworkMonitorInformation, NetworkMonitorsPagedResponse, NodeId, NodeMeasurement,
NodeMeasurementsResponse, NodePerformance, NodePerformancePagedResponse,
NodePerformanceResponse, RetiredNetworkMonitor, RetiredNetworkMonitorsPagedResponse,
};
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait PerformanceQueryClient {
async fn query_performance_contract<T>(
&self,
query: PerformanceQueryMsg,
) -> Result<T, NyxdError>
where
for<'a> T: Deserialize<'a>;
async fn admin(&self) -> Result<cw_controllers::AdminResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::Admin {})
.await
}
async fn get_node_performance(
&self,
epoch_id: EpochId,
node_id: NodeId,
) -> Result<NodePerformanceResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::NodePerformance { epoch_id, node_id })
.await
}
async fn get_node_performance_paged(
&self,
node_id: NodeId,
start_after: Option<EpochId>,
limit: Option<u32>,
) -> Result<NodePerformancePagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::NodePerformancePaged {
node_id,
start_after,
limit,
})
.await
}
async fn get_node_measurements(
&self,
epoch_id: EpochId,
node_id: NodeId,
) -> Result<NodeMeasurementsResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::NodeMeasurements { epoch_id, node_id })
.await
}
async fn get_epoch_measurements_paged(
&self,
epoch_id: EpochId,
start_after: Option<NodeId>,
limit: Option<u32>,
) -> Result<EpochMeasurementsPagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::EpochMeasurementsPaged {
epoch_id,
start_after,
limit,
})
.await
}
async fn get_epoch_performance_paged(
&self,
epoch_id: EpochId,
start_after: Option<NodeId>,
limit: Option<u32>,
) -> Result<EpochPerformancePagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::EpochPerformancePaged {
epoch_id,
start_after,
limit,
})
.await
}
async fn get_full_historical_performance_paged(
&self,
start_after: Option<(EpochId, NodeId)>,
limit: Option<u32>,
) -> Result<FullHistoricalPerformancePagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::FullHistoricalPerformancePaged {
start_after,
limit,
})
.await
}
async fn get_network_monitor(
&self,
address: &AccountId,
) -> Result<NetworkMonitorResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::NetworkMonitor {
address: address.to_string(),
})
.await
}
async fn get_network_monitors_paged(
&self,
start_after: Option<String>,
limit: Option<u32>,
) -> Result<NetworkMonitorsPagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::NetworkMonitorsPaged {
start_after,
limit,
})
.await
}
async fn get_retired_network_monitors_paged(
&self,
start_after: Option<String>,
limit: Option<u32>,
) -> Result<RetiredNetworkMonitorsPagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::RetiredNetworkMonitorsPaged {
start_after,
limit,
})
.await
}
async fn get_last_submission(&self) -> Result<LastSubmission, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::LastSubmittedMeasurement {})
.await
}
}
// extension trait to the query client to deal with the paged queries
// (it didn't feel appropriate to combine it with the existing trait
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait PagedPerformanceQueryClient: PerformanceQueryClient {
async fn get_all_node_performance(
&self,
node_id: NodeId,
) -> Result<Vec<EpochNodePerformance>, NyxdError> {
collect_paged!(self, get_node_performance_paged, performance, node_id)
}
async fn get_all_epoch_measurements(
&self,
node_id: NodeId,
) -> Result<Vec<NodeMeasurement>, NyxdError> {
collect_paged!(self, get_epoch_measurements_paged, measurements, node_id)
}
async fn get_all_epoch_performance(
&self,
epoch_id: EpochId,
) -> Result<Vec<NodePerformance>, NyxdError> {
collect_paged!(self, get_epoch_performance_paged, performance, epoch_id)
}
async fn get_all_full_historical_performance(
&self,
) -> Result<Vec<HistoricalPerformance>, NyxdError> {
collect_paged!(self, get_full_historical_performance_paged, performance)
}
async fn get_all_network_monitors(&self) -> Result<Vec<NetworkMonitorInformation>, NyxdError> {
collect_paged!(self, get_network_monitors_paged, info)
}
async fn get_all_retired_network_monitors(
&self,
) -> Result<Vec<RetiredNetworkMonitor>, NyxdError> {
collect_paged!(self, get_retired_network_monitors_paged, info)
}
}
#[async_trait]
impl<T> PagedPerformanceQueryClient for T where T: PerformanceQueryClient {}
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
impl<C> PerformanceQueryClient for C
where
C: CosmWasmClient + NymContractsProvider + Send + Sync,
{
async fn query_performance_contract<T>(
&self,
query: PerformanceQueryMsg,
) -> Result<T, NyxdError>
where
for<'a> T: Deserialize<'a>,
{
let performance_contract_address = &self
.performance_contract_address()
.ok_or_else(|| NyxdError::unavailable_contract_address("performance contract"))?;
self.query_contract_smart(performance_contract_address, &query)
.await
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::nyxd::contract_traits::tests::IgnoreValue;
use nym_performance_contract_common::QueryMsg;
// it's enough that this compiles and clippy is happy about it
#[allow(dead_code)]
fn all_query_variants_are_covered<C: PerformanceQueryClient + Send + Sync>(
client: C,
msg: PerformanceQueryMsg,
) {
match msg {
PerformanceQueryMsg::Admin {} => client.admin().ignore(),
PerformanceQueryMsg::NodePerformance { epoch_id, node_id } => {
client.get_node_performance(epoch_id, node_id).ignore()
}
PerformanceQueryMsg::NodePerformancePaged {
node_id,
start_after,
limit,
} => client
.get_node_performance_paged(node_id, start_after, limit)
.ignore(),
PerformanceQueryMsg::NodeMeasurements { epoch_id, node_id } => {
client.get_node_measurements(epoch_id, node_id).ignore()
}
PerformanceQueryMsg::EpochMeasurementsPaged {
epoch_id,
start_after,
limit,
} => client
.get_epoch_measurements_paged(epoch_id, start_after, limit)
.ignore(),
PerformanceQueryMsg::EpochPerformancePaged {
epoch_id,
start_after,
limit,
} => client
.get_epoch_performance_paged(epoch_id, start_after, limit)
.ignore(),
PerformanceQueryMsg::FullHistoricalPerformancePaged { start_after, limit } => client
.get_full_historical_performance_paged(start_after, limit)
.ignore(),
PerformanceQueryMsg::NetworkMonitor { address } => client
.get_network_monitor(&address.parse().unwrap())
.ignore(),
PerformanceQueryMsg::NetworkMonitorsPaged { start_after, limit } => client
.get_network_monitors_paged(start_after, limit)
.ignore(),
PerformanceQueryMsg::RetiredNetworkMonitorsPaged { start_after, limit } => client
.get_retired_network_monitors_paged(start_after, limit)
.ignore(),
QueryMsg::LastSubmittedMeasurement {} => client.get_last_submission().ignore(),
};
}
}
@@ -0,0 +1,217 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::nyxd::coin::Coin;
use crate::nyxd::contract_traits::NymContractsProvider;
use crate::nyxd::cosmwasm_client::types::ExecuteResult;
use crate::nyxd::cosmwasm_client::ContractResponseData;
use crate::nyxd::error::NyxdError;
use crate::nyxd::{Fee, SigningCosmWasmClient};
use crate::signing::signer::OfflineSigner;
use async_trait::async_trait;
use nym_performance_contract_common::{
EpochId, ExecuteMsg as PerformanceExecuteMsg, NodeId, NodePerformance,
RemoveEpochMeasurementsResponse,
};
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait PerformanceSigningClient {
async fn execute_performance_contract(
&self,
fee: Option<Fee>,
msg: PerformanceExecuteMsg,
memo: String,
funds: Vec<Coin>,
) -> Result<ExecuteResult, NyxdError>;
async fn update_admin(
&self,
admin: String,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::UpdateAdmin { admin },
"PerformanceContract::UpdateAdmin".to_string(),
vec![],
)
.await
}
async fn submit_performance(
&self,
epoch: EpochId,
data: NodePerformance,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::Submit { epoch, data },
"PerformanceContract::Submit".to_string(),
vec![],
)
.await
}
async fn batch_submit_performance(
&self,
epoch: EpochId,
data: Vec<NodePerformance>,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::BatchSubmit { epoch, data },
"PerformanceContract::BatchSubmit".to_string(),
vec![],
)
.await
}
async fn authorise_network_monitor(
&self,
address: String,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::AuthoriseNetworkMonitor { address },
"PerformanceContract::AuthoriseNetworkMonitor".to_string(),
vec![],
)
.await
}
async fn retire_network_monitor(
&self,
address: String,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::RetireNetworkMonitor { address },
"PerformanceContract::RetireNetworkMonitor".to_string(),
vec![],
)
.await
}
async fn remove_node_measurements(
&self,
epoch_id: EpochId,
node_id: NodeId,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::RemoveNodeMeasurements { epoch_id, node_id },
"PerformanceContract::RemoveNodeMeasurements".to_string(),
vec![],
)
.await
}
async fn partial_remove_epoch_measurements(
&self,
epoch_id: EpochId,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::RemoveEpochMeasurements { epoch_id },
"PerformanceContract::RemoveEpochMeasurements".to_string(),
vec![],
)
.await
}
async fn remove_epoch_measurements(
&self,
epoch_id: EpochId,
fee: Option<Fee>,
) -> Result<(), NyxdError> {
loop {
let execute_res = self
.partial_remove_epoch_measurements(epoch_id, fee.clone())
.await?;
let response = execute_res
.parse_singleton_json_contract_response::<RemoveEpochMeasurementsResponse>()?;
if !response.additional_entries_to_remove_remaining {
break;
}
}
Ok(())
}
}
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
impl<C> PerformanceSigningClient for C
where
C: SigningCosmWasmClient + NymContractsProvider + Sync,
NyxdError: From<<Self as OfflineSigner>::Error>,
{
async fn execute_performance_contract(
&self,
fee: Option<Fee>,
msg: PerformanceExecuteMsg,
memo: String,
funds: Vec<Coin>,
) -> Result<ExecuteResult, NyxdError> {
let performance_contract_address = &self
.performance_contract_address()
.ok_or_else(|| NyxdError::unavailable_contract_address("performance contract"))?;
let fee = fee.unwrap_or(Fee::Auto(Some(self.simulated_gas_multiplier())));
let signer_address = &self.signer_addresses()?[0];
self.execute(
signer_address,
performance_contract_address,
&msg,
fee,
memo,
funds,
)
.await
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::nyxd::contract_traits::tests::IgnoreValue;
use nym_performance_contract_common::ExecuteMsg;
// it's enough that this compiles and clippy is happy about it
#[allow(dead_code)]
fn all_execute_variants_are_covered<C: PerformanceSigningClient + Send + Sync>(
client: C,
msg: PerformanceExecuteMsg,
) {
match msg {
PerformanceExecuteMsg::UpdateAdmin { admin } => {
client.update_admin(admin, None).ignore()
}
PerformanceExecuteMsg::Submit { epoch, data } => {
client.submit_performance(epoch, data, None).ignore()
}
PerformanceExecuteMsg::BatchSubmit { epoch, data } => {
client.batch_submit_performance(epoch, data, None).ignore()
}
PerformanceExecuteMsg::AuthoriseNetworkMonitor { address } => {
client.authorise_network_monitor(address, None).ignore()
}
PerformanceExecuteMsg::RetireNetworkMonitor { address } => {
client.retire_network_monitor(address, None).ignore()
}
ExecuteMsg::RemoveNodeMeasurements { epoch_id, node_id } => client
.remove_node_measurements(epoch_id, node_id, None)
.ignore(),
ExecuteMsg::RemoveEpochMeasurements { epoch_id } => client
.partial_remove_epoch_measurements(epoch_id, None)
.ignore(),
};
}
}

Some files were not shown because too many files have changed in this diff Show More