Compare commits

...

135 Commits

Author SHA1 Message Date
benedetta davico 87a188ca06 Bump cred-proxy version 2025-09-11 14:28:52 +02:00
benedettadavico 4d0898c633 bump versions 2025-09-08 09:30:59 +02:00
Bogdan-Ștefan Neacşu d984d085a7 Backport metadata endpoint (#6010)
* Wireguard private metadata (#5915)

* Wireguard metadata client library (#5943)
2025-09-05 11:14:37 +03:00
Jędrzej Stuczyński 2e22cad074 bugfix: make sure tables are removed in correct order to not trigger FK constraint issue (#5987) 2025-08-29 09:02:58 +01:00
Jędrzej Stuczyński 17708cdf92 bugfix: manually calculate per node work on rewarded set changes (#5972) 2025-08-27 12:33:24 +01:00
Andrej Mihajlov a9c56ef9ac Merge pull request #5976 from nymtech/am/update-sysinfo
Update sysinfo to the latest
2025-08-27 10:58:06 +02:00
Jędrzej Stuczyński 724420f97c chore: move authenticator into gateway crate (#5982)
* removed unused bits of authenticator config

* moved authenticator into gateway

* cleaned up imports

* clippy
2025-08-27 09:05:02 +01:00
dynco-nym 0a48fa6172 Remove freshness check on testrun submit (#5977)
* Remove freshness check on testrun submit
- freshness is enforced by a background task
  that marks testruns as stale after a
  configured amount of time

* Move code around

* Add humantime

* Update launch script

* Fix typo

* Adjust agent run script

* Configure user agent

* Bump version
2025-08-26 12:26:13 +02:00
Andrej Mihajlov 5c8749a2e1 Update sysinfo to the latest
Shakes out windows@0.57 from the tree
2025-08-23 09:29:47 +02:00
p17o 18d9d807f2 Added VPS provider hostraha.com (#5959) 2025-08-22 12:31:31 +00:00
import this 3a7393d316 [DOCs/operators]: Roll back from v2025.15-gruyere to v2025.14-feta (#5973)
* release notes and version bump up

* update stats

* roll version back and comment new notes
2025-08-22 11:23:16 +00:00
import this 6ce5f707c6 [DOCs/operators]: Release notes for v2025.15 gruyere (#5969)
* release notes and version bump up

* update stats
2025-08-21 11:49:52 +00:00
benedetta davico 766a1d4497 Merge pull request #5965 from nymtech/benny/fix-ns-agent-ci
fixing the ci for ns agent
2025-08-21 12:22:05 +02:00
benedetta davico 35c83f0a31 Merge pull request #5967 from nymtech/release/2025.15-gruyere
merge gruyere to develop
2025-08-21 12:20:43 +02:00
benedetta davico 01dd4a7972 Merge pull request #5958 from nymtech/bugfix/linux-build-ci
bugfix: fix ci-build for linux (and use updated runner)
2025-08-21 10:32:51 +02:00
Jędrzej Stuczyński c2e335557e Feature/testing utils (#5963)
* helper wrapper for stream-sink channel

* similar helper for async read/write

* example tests and clippy
2025-08-20 16:17:09 +01:00
benedettadavico 40e1cbc7a9 update changelog 2025-08-20 12:34:04 +02:00
Jędrzej Stuczyński c133e0e88b chore: updated refs to cheddar rev of nym repo (#5955)
* chore: updated refs to cheddar rev of nym repo

* update statistics-api version
2025-08-19 09:28:42 +01:00
Andrej Mihajlov 5b716633de Merge pull request #5960 from nymtech/am/update-strum
Migrate strum to 0.27.2
2025-08-18 18:28:22 +02:00
Andrej Mihajlov 834538300d Migrate to strum 0.27.2 2025-08-18 18:02:41 +02:00
Jędrzej Stuczyński bd0d70f7cd bugfix: fix ci-build for linux (and use updated runner) 2025-08-15 09:37:41 +01:00
Jędrzej Stuczyński 979485c582 http api client adjustment (#5953)
* missing feature lock for attempting to clone client

* added helper macro to generate user agent without additional imports
2025-08-13 12:52:16 +01:00
Bogdan-Ștefan Neacşu d95f66bd90 Move credential verifier in peer controller (#5938)
* Move credential verifier in peer controller

* Send back errors of peer controller
2025-08-13 13:09:44 +03:00
Jędrzej Stuczyński 906dfb2fb0 change PK/FK on expiration date signatures tables (#5934)
* update nym-credential-proxy

* update credential-storage

* update nym-api

* clippy
2025-08-12 09:03:53 +01:00
Jędrzej Stuczyński 7daa726626 feat: introduce additional checks when attempting to send to bounded channels (#5941)
* feat: introduce additional checks when attempting to send to bounded channels

or to a fallible gateway

* return error rather than panic when merging socket during shutdown
2025-08-11 09:15:12 +01:00
Jędrzej Stuczyński 067f492ad6 chore: fix rust 1.89 clippy issues (#5944) 2025-08-08 13:03:05 +01:00
Jędrzej Stuczyński ed73ec9ce6 chore: remove unused import (#5942) 2025-08-07 09:56:08 +01:00
import this 61606630bd [DOCs/operators]: Release notes v2025.14-feta (#5935)
* update release

* fix typos
2025-08-06 08:24:38 +00:00
benedettadavico 2d3deeb424 bump versions 2025-08-06 09:56:08 +02:00
benedetta davico 3827dc357d Merge pull request #5936 from nymtech/release/2025.14-feta
Merge release/2025.14-feta to develop
2025-08-06 09:54:29 +02:00
benedetta davico a70e9e23d3 Merge branch 'develop' into release/2025.14-feta 2025-08-06 09:48:22 +02:00
Jędrzej Stuczyński dc59149a5d squashed feature/ecash-liveness-check (#5890) (#5890)
delay to gruyere

chore: delay to Feta

added threshold information to the response

nym api test clippy

bugfixes and endpoint improvements

expose results on api endpoints

wip: making nym api monitor network signers

added fallback legacy queries to get basic support idea

refactored the code to expose bool-only methods for status

ecash-signer-check lib for obtaining basic ecash signer information
2025-08-05 12:28:42 +01:00
benedetta davico e418c7587a Merge pull request #5914 from nymtech/feature/nym-node-gw-reset
nym-node debug command to reset providers db
2025-08-05 12:05:44 +02:00
import this 33339c085d [DOCs/operators]: Update ISP list (#5918)
* update ISP list

* remove typo
2025-07-31 13:47:27 +00:00
Sachin Kamath 863f329106 docs: update validator instructions and waitlist callout (#5922) 2025-07-30 15:03:39 +00:00
import this 314a37cabe WG exit policy scripts update (#5921)
* add NIP-3 ports to WG manager script

* add monero ports to local testing script

* console output snippet update
2025-07-30 09:43:39 +00:00
Jack Wampler 917f391948 Make DNS Resolver fallback optional (#5920)
default to no dns system fallback, but keep support
2025-07-29 11:00:24 -06:00
Jędrzej Stuczyński 0b4deda621 nym-node debug command to reset providers db 2025-07-25 13:33:12 +01:00
benedetta davico d01867ca8d save version 2025-07-25 11:27:42 +02:00
benedetta davico 502c63b291 Fix broken CI 2025-07-25 11:19:27 +02:00
Jędrzej Stuczyński a4e674c98b basic zulip client for sending messages (#5913) 2025-07-24 16:22:35 +01:00
Jędrzej Stuczyński 7f97f13799 chore: nym node tokio console (#5909)
* conditionally enable console-subscriber within nym-node

* Update ci-build-upload-binaries.yml

* Update ci-build-upload-binaries.yml

add features console

* updated feature name

* fixed filtering on tracing layers

* add track_caller when spawning futures for better tokio-console support

* allow [client] tasks to specify their names when used within tokio console

* clippy

* pre-emptively fix wasm clippy

---------

Co-authored-by: Tommy Verrall <60836166+tommyv1987@users.noreply.github.com>
2025-07-24 11:00:58 +01:00
Bogdan-Ștefan Neacşu b975d08342 Remove old free credential handle (#5864)
* Set cached storage counters to 0 (#5812)

* Set cached storage counters to 0

* u64 to i64 log possible error

* Check addition too

Debug commit

Remove more data from wg storage peer

Put actual ticket type in storage

Simplify add peer

Finish rebase

Pass defguard Peer

Cache less data for consumption

GatewayStorage traits

Wg API trait

Mock test structures

Unit test for peer controller

EcashManager trait

Init test of Authenticator

Remove peer test

* Fix windows different API

* Use make_bincode_serializer like in other places

* Add log_slow_statements to gateway storage

* Use correct LevelFilter

* Fix clippy

* More win fix

* Win clippy

* Use two error variants more

* Use only one Arc<RwLock<T>> instead of many more

* Remove commented test

* Specific trait import
2025-07-23 17:07:12 +03:00
Jędrzej Stuczyński 8e44f9f07f chore: allow compatibility with 'CDLA-Permissive-2.0' (#5910) 2025-07-23 14:48:40 +01:00
benedettadavico 85604e8305 bump versions 2025-07-23 10:18:45 +02:00
benedetta davico 8461d085a5 Merge pull request #5906 from nymtech/release/2025.13-emmental
merge release/2025.13-emmental to develop
2025-07-22 16:23:29 +02:00
Drazen Urch af9f6e5ca0 Allow PG database backend (#5880)
* feat(db): add SQL query wrapper for PostgreSQL placeholder conversion

- Created query_wrapper module with functions to automatically convert
  SQLite ? placeholders to PostgreSQL $1, $2, ... format
- Updated build.rs to handle mutually exclusive feature flags
- Modified one query in mixnodes.rs as proof of concept
- Added type conversions for PostgreSQL compatibility (u32->i64, u16->i32)

This is a checkpoint commit before converting all queries to use the wrapper.

* feat(nym-node-status-api): add PostgreSQL database support via feature flags

Implement dual database support for SQLite and PostgreSQL through Cargo feature flags.
The implementation uses a query wrapper that automatically converts SQLite-style ?
placeholders to PostgreSQL-style $1, $2, ... placeholders at runtime.

Key changes:
- Add query wrapper functions that handle placeholder conversion
- Convert all sqlx::query\! macros to use wrapper functions
- Handle type conversions between databases (i64 vs i32)
- Add feature-gated implementations for database-specific SQL syntax
- Update Makefile with clippy targets for both database features
- Document database support in README

* feat(nym-node-status-agent): add multi-API support with random selection

Agents can now connect to multiple APIs and randomly select one for each testrun:
- Accept multiple --server arguments in format "address:port:auth_key"
- Randomly shuffle server list before attempting connections
- Try each server until a testrun is obtained
- Submit results back only to the API that provided the testrun
- Continue to next server if one is down or has no testruns available

* feat(nym-node-status): implement primary/secondary server architecture

- Agent now requests testruns only from primary server (first in list)
- Results are submitted to all configured servers in parallel
- Secondary servers accept external testruns via new v2 endpoint
- Added auto-creation of gateway and testrun records on secondary servers
- New database queries: get_or_create_gateway, insert_external_testrun
- Client library enhanced with submit_results_with_context method

* Bump Node status API version

* Fix build workdir

* Bump to 3.1.4

* Fix types and queries

* 3.1.6

* Fix gateway perf, bump 3.1.7

* NodeId -> i32, 3.1.8

* Bump agent version

* i64 -> i32

* Use image yq

* Migration and more types

* Update remaining JSONB columns

* Simplify server config

* Update build path

* Change delimiter

* bump agent

* Split up pg and sqlite builds

* More typing fixes, build-and-push script

* Fix Dockerfile-pg

* Bump node-status-api

* TYping

* Agent build script

* More logging around testruns

* Fail loudly on read errors

* Cleanup

* Debug get gateways query

* Fix get_gateways query

* Use pg cert, 3.1.16

* Submit regular results to primary server

* Bump freshenss cutoff

* Update Cargo.lock

* fix: resolve rebase conflicts and compilation errors

After rebasing onto develop, fixed several issues:
- Fixed borrowed data escapes error by using sqlx::query directly in transaction functions
- Removed unused imports and cleaned up code
- Maintained database-specific implementations for transaction functions

* fmt

* Make PG default to make lives easier

* Performance improvements for Explorer v2

* Fix sqlite build

* Fix PG migration

* Tests round 1

* DB tests

* More tests

* And some more tests

* And some more, more tests

* cargo fmt

* Fix some failing lints

* Fix lioness version problems

* Clippy in tests

---------

Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
2025-07-22 15:25:43 +02:00
import this a9ae2017f5 [DOCs/operators]: Release notes/v2025.13 emmental & NIP-3 announcement (#5908)
* initialise PR, add dev notes and bump node version

* add operators tool and update api stats
2025-07-22 12:10:43 +00:00
Bogdan-Ștefan Neacşu 09ebe7f9e9 Support mnemonic in the NS agent (#5883)
Co-authored-by: benedettadavico <benedetta.davico@gmail.com>
2025-07-22 14:21:12 +03:00
Andrej Mihajlov b72915c224 Merge pull request #5905 from nymtech/am/sqlx-guard-obtain-db-path-from-pool
sqlx-pool-guard: obtain filename from connect options
2025-07-22 11:57:55 +02:00
Andrej Mihajlov add3e864e3 sqlx-pool-guard: obtain filename from connect options 2025-07-22 11:09:39 +02:00
benedettadavico 578c9b0567 update changelog 2025-07-22 11:09:35 +02:00
Andrej Mihajlov 8f6f696f36 Merge pull request #5896 from nymtech/am/handle-table-allocate-more-memory 2025-07-22 11:09:11 +02:00
Jędrzej Stuczyński e9165763b6 Feature/dkg snapshot epoch (#5900)
* define storage item for holding historical DKG state

* make all epoch storage operations go through proxy functions

* make each saving action also apply to the historical item

* removed usage of update_epoch function

* test correct save heights

* exposed query for the epoch state at specified height

* regenerated contract schema

* restored default cw-plus behaviour as in hindsight it makes more sense
2025-07-21 17:32:57 +01:00
mfahampshire 6c1149708b GW Probe docs: Go dep. + new required mnemonic (#5897)
* add note on go dep

* updated -h and useage doc
2025-07-18 12:36:30 +00:00
Mark Sinclair aaf6931d78 nym-node-status-ui placeholder (#5902)
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
2025-07-17 20:04:45 +01:00
Jędrzej Stuczyński 97804f2fe5 Feature/dkg epoch dealers query (#5899)
* feat: add GetEpochDealers and GetEpochDealersAddresses queries to the DKG contract

* extended DkgQueryClient with new queries

* updated contract schema

* unit tests
2025-07-17 12:26:01 +01:00
Jędrzej Stuczyński 802d9b69ca fix: don't allow mixnode running in exit mode (#5898)
* fix: don't allow mixnode running in exit mode

* fixed error message
2025-07-17 10:57:16 +01:00
Andrej Mihajlov 7313857bc8 Allocate more memory to account for a drift in handle table size in between calls 2025-07-16 13:29:45 +02:00
benedettadavico 779174ada5 update wallet changelog 2025-07-15 14:57:49 +02:00
benedettadavico 8771c1dfa6 bump wallet version 2025-07-15 14:47:49 +02:00
benedettadavico 329ad83fc0 bump versions 2025-07-15 10:04:51 +02:00
Jack Wampler aea5872ad0 bump h2 dependency to fix DoH connection close logging (#5893) 2025-07-14 12:56:56 -06:00
Mark Sinclair 9e9abd74d7 Update ci-sonar.yml
[skip ci]
2025-07-14 17:34:26 +01:00
Mark Sinclair 3832508af7 Update sonar-project.properties 2025-07-14 17:33:10 +01:00
Mark Sinclair 69a4e33b17 Create sonar-project.properties 2025-07-14 17:25:30 +01:00
Mark Sinclair 83385421ff Create ci-sonar.yml 2025-07-14 17:24:42 +01:00
Jędrzej Stuczyński ec53b570dc listen for shutdown signals during nym-node startup (#5879)
this is to avoid situation where the process can't be killed without 'kill -9' because the logic to listen to shutdown signals hasn't been hit yet
2025-07-14 12:13:40 +01:00
Jędrzej Stuczyński ebcc658f98 chain scraper: ignore precommits from missing validators (#5867) 2025-07-14 08:46:19 +01:00
Mark Sinclair 6a155721c6 Update push-node-status-agent.yaml 2025-07-11 13:51:10 +01:00
Mark Sinclair 1bb8b3a3ec Update push-node-status-api.yaml 2025-07-11 13:50:07 +01:00
Mark Sinclair 8d1a16eb02 Update push-node-status-api.yaml 2025-07-11 11:46:21 +01:00
Mark Sinclair 8d10cf70e9 Update push-node-status-api.yaml 2025-07-11 11:36:16 +01:00
Mark Sinclair e32df10b4d Update push-node-status-api.yaml 2025-07-11 11:30:26 +01:00
Mark Sinclair d1660c01e6 Update push-node-status-api.yaml 2025-07-11 11:12:09 +01:00
Sachin Kamath 14378b1db9 hotfix: fix contract build in Makefile (#5892) 2025-07-11 15:32:49 +05:30
dynco-nym 35bbf5fd84 Batch SQL writes for packet stats (#5874)
* Move stuff around

* Batch SQL operations

* Clippy

* Bump version

* Remove shared queue which was always re-initialized

* Make max_concurrent_tasks configurable

* fixed typo

* clippy

---------

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
2025-07-11 10:53:19 +01:00
Mark Sinclair c374a4935a Update push-node-status-agent.yaml (#5882)
* Update push-node-status-agent.yaml

* Update push-node-status-api.yaml

* Update push-node-status-api.yaml

Fix up typo

* Update push-node-status-agent.yaml

* Update push-node-status-api.yaml
2025-07-11 10:29:05 +01:00
Jędrzej Stuczyński 513f4f652d Merge pull request #5887 from nymtech/merge/release/2025.12-dolcelatte
merge: release/2025.12-dolcelatte into develop
2025-07-10 09:16:58 +01:00
Sachin Kamath 82b9425ca6 chore: build contracts with cw optimizer (#5888) 2025-07-09 21:45:10 +05:30
Jędrzej Stuczyński 615e98b166 Merge branch 'develop' into merge/release/2025.12-dolcelatte 2025-07-09 15:37:41 +01:00
import this b11f6c6c70 [DOCs/operators]: Release notes v2025.12-dolcelatte (#5881)
* initialise release update

* add dev features and bugfixes

* add version

---------

Co-authored-by: mfahampshire <maxhampshire@pm.me>
2025-07-09 13:32:46 +00:00
Jędrzej Stuczyński 2f5e8e0bcd feat: forbid running mixnode + entry on the same node (#5878) 2025-07-09 08:59:55 +01:00
Jędrzej Stuczyński 812a8782b4 ignore 'Send' responses when claiming bandwidth (#5884) 2025-07-08 09:09:18 +01:00
Jack Wampler a7b57d7e58 Make Mix hops optional for Mixnet Client SURBs (#5861)
* allow SURBs to be configured without mix hops

* gateways require consistency in surb format so if disabling mixnhops - use updated format
2025-07-03 09:21:50 -06:00
benedettadavico 84e10a654c Revert "Bump ns-api version"
This reverts commit d724f94319.
2025-07-01 15:26:55 +02:00
benedetta davico d724f94319 Bump ns-api version 2025-07-01 15:19:56 +02:00
Jędrzej Stuczyński d0692a567a feat: basic performance contract integration [within Nym API] (#5871)
* renamed nym-api config fields

* decouple rewarder startup from network monitor

* additional sections in nym-api config

* removed vesting queries in circulating supply calculator

* added memoized field for last submitted performance measurement

* wip: performance contract refresher

* cleaned up various contract caches

* modified cache refresher to allow passing update fn

* implement performance cache refreshing

* updated lefthook.yml to run cargo fmt

* impl NodePerformanceProvider trait

* dynamically using specific performance provider

* pre warm up performance contract cache and forbid the mode if its empty

* clippy

* introduce fallback setting for performance contract if value for given epoch is not available

* move some functions around
2025-07-01 11:29:50 +01:00
Jędrzej Stuczyński 2ae38b9e49 chore: 1.88 clippy (#5877)
* 1.88 clippy

* wasm clippy

* wallet clippy
2025-07-01 10:28:57 +01:00
benedetta davico ef5990658a Merge pull request #5873 from nymtech/wallet/fix-link 2025-06-26 13:26:36 +02:00
benedettadavico 658dec8299 fix the broken link 2025-06-26 12:44:47 +02:00
dynco-nym 447352b8d6 Set busy_timeout in sqlx (#5872)
* Set busy_timeout

* Bump version
2025-06-26 10:44:06 +02:00
Simon Wicky eb59615c56 StatsAPI qol : disable swagger try it out and remove debug level from nym_http_api_client (#5868) 2025-06-23 14:58:29 +02:00
Bogdan-Ștefan Neacşu 07c908c497 Return true remaining (#5866) 2025-06-23 11:53:39 +03:00
Jędrzej Stuczyński 6de0c4ce92 feat: initial performance contract (#5833)
* initialised basic structure for the performance contract

* shared code for contract testing

* unified common testing methods between performance and nym pool contracts

* impl of ExecuteMsg for the contract

* impl of QueryMsg for the contract

* setting initial authorised NMs during instantiation

* additional tests and fixes

* ibid

* scaffolding for client traits

* completed client traits

* clippy

* naive add performance contract to testnet manager

* placeholder values for the performance contract address

* introduced admin messages to purge old measurements from the storage

* introduced check ensuring performance data is only added to bonded nodes
2025-06-20 09:06:56 +01:00
benedettadavico 05d8b31e51 Merge branch 'remove/old-explorer' into develop 2025-06-18 15:34:40 +02:00
Georgio Nicolas 692fbf1392 Merge pull request #5828 from nymtech/georgio/dkg-crypsen-fixes
Security patches for the `dkg` crate
2025-06-18 10:48:37 +02:00
Andrej Mihajlov 0de4aea77b Merge pull request #5796 from nymtech/am/close-sqlite-pool
Close sqlite pool before moving or reopening databases
2025-06-17 19:01:25 +02:00
Georgio Nicolas a7cd8efc04 dkg: fix clippy suggestions 2025-06-17 16:37:50 +02:00
Georgio Nicolas 56aad75220 dkg: verify integrity of ciphertexts during decryption 2025-06-17 16:30:11 +02:00
Georgio Nicolas e2f2ab89ec dkg: add CryptoRng trait requirement 2025-06-17 16:30:11 +02:00
Georgio Nicolas 4d09b6f2e5 bte/proof_chunking.rs: Check for potential arithmentic overflows 2025-06-17 16:30:11 +02:00
dynco-nym b9339b8f0c Add /status endpoints (#5857)
* Add /status endpoints

* Bump package version

* pub use instead of import
2025-06-16 13:19:35 +02:00
Andrej Mihajlov 43a7360399 Merge pull request #5856 from nymtech/am/remove-surb-screaming-logs
Clear out screaming logs
2025-06-16 11:39:27 +02:00
Andrej Mihajlov 5f9f7f0fac Clear out screaming logs 2025-06-13 11:00:48 +02:00
Andrej Mihajlov df0e2fe489 Merge pull request #5853 from nymtech/am/path-display
Use display when printing paths
2025-06-13 10:54:12 +02:00
benedetta davico bc33cc4c8d Merge pull request #5855 from nymtech/fix-qa-removal 2025-06-13 09:40:56 +02:00
Simon Wicky a31597aca9 fix removal of qa env 2025-06-13 09:30:00 +02:00
Jack Wampler 378229b04e HTTP Discovery objects & network defaults (#5814)
add extended (optional) fields to the NetworkDiscovery and configure fallback hosts
2025-06-12 11:15:36 -06:00
Andrej Mihajlov fec196c097 Use display when printing paths 2025-06-12 17:17:00 +02:00
Andrej Mihajlov 1d7ffc1bb6 test: remove file after closing for a test 2025-06-12 15:39:26 +02:00
Andrej Mihajlov 0caa627960 Fix missing await on self.close_pool_inner() 2025-06-12 15:12:46 +02:00
import this d6b3d7fc0a [DOCs/operators]: Release notes for v2025.11 cheddar (#5852)
* bump up version

* add dev features

* add operator updates

* add updated stats

* update prebuild
2025-06-12 11:19:00 +00:00
dynco-nym ac273480f8 Fix CI version check (#5851)
* Fix version

* Test .rc version

* Undo cargo.toml version

* Remove comment

* Apply to statistics service
2025-06-12 11:17:56 +02:00
benedettadavico 3ac58e0c49 Clean up
remove old explorer references
2025-06-11 16:02:19 +02:00
Andrej Mihajlov e52bd918fb Hide tokio behind feature 2025-06-06 15:00:40 +02:00
Andrej Mihajlov 9d82d6d111 Hide tokio and sqlx behind not(wasm32) 2025-06-06 13:34:56 +02:00
Andrej Mihajlov 3593631e4a Exclude sqlx-pool-guard from wasm builds 2025-06-06 13:24:04 +02:00
Andrej Mihajlov f5846d5bc2 Log all tracing output just in case 2025-06-04 11:40:56 +02:00
Andrej Mihajlov d7779df1b7 Include proc_pidinfo on iOS 2025-06-04 11:00:15 +02:00
Andrej Mihajlov 7fcc188041 Switch to tracing 2025-06-03 17:19:42 +02:00
Andrej Mihajlov b8c8d33c94 Use log here 2025-06-03 15:13:21 +02:00
Andrej Mihajlov 02909c03dd Expose database path 2025-06-03 14:49:49 +02:00
Andrej Mihajlov 11262836d2 Clean up 2025-06-03 09:43:36 +02:00
Andrej Mihajlov f26fd5384d Improve windows 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 085103b333 Cleanup 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 574f7f1abd Revert 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 31e161604a Use sqlite pool guard 2025-06-03 09:43:36 +02:00
Andrej Mihajlov e4e349bea8 Remove logs 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 6391b7ed3a Document 2025-06-03 09:43:36 +02:00
Andrej Mihajlov c225511f95 Add Windows impl 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 4eedbb235a Add Windows implementation 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 548b8717b2 Update Linux impl 2025-06-03 09:43:36 +02:00
Andrej Mihajlov a215b3d0bf Open file watch 2025-06-03 09:43:36 +02:00
Andrej Mihajlov 03d5a133eb Close sqlite pool before erroring 2025-06-03 09:43:36 +02:00
939 changed files with 41092 additions and 111193 deletions
@@ -38,15 +38,14 @@ jobs:
rm -rf ci-builds || true
mkdir -p $OUTPUT_DIR
echo $OUTPUT_DIR
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install libudev-dev
- name: Sets env vars for tokio if set in manual dispatch inputs
run: |
echo 'RUSTFLAGS="--cfg tokio_unstable"' >> $GITHUB_ENV
if: github.event_name == 'workflow_dispatch' && inputs.add_tokio_unstable == true
run: |
echo "RUSTFLAGS=--cfg tokio_unstable" >> $GITHUB_ENV
echo "CARGO_FEATURES=--features tokio-console" >> $GITHUB_ENV
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
@@ -103,7 +102,6 @@ jobs:
if [ ${{ github.event_name == 'workflow_dispatch' && inputs.enable_deb == true }} = true ]; then
cp target/debian/*.deb $OUTPUT_DIR
fi
- name: Deploy branch to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
+7 -7
View File
@@ -38,7 +38,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ arc-ubuntu-22.04, custom-windows-11, custom-macos-15 ]
os: [ arc-linux-latest, custom-windows-11, custom-macos-15 ]
runs-on: ${{ matrix.os }}
env:
CARGO_TERM_COLOR: always
@@ -46,9 +46,9 @@ jobs:
RUSTUP_PERMIT_COPY_RENAME: 1
steps:
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools protobuf-compiler
run: sudo apt-get update && sudo apt-get -y install libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools protobuf-compiler cmake
continue-on-error: true
if: contains(matrix.os, 'ubuntu')
if: contains(matrix.os, 'linux')
- name: Check out repository code
uses: actions/checkout@v4
@@ -63,7 +63,7 @@ jobs:
# To avoid running out of disk space, skip generating debug symbols
- name: Set debug to false (unix)
if: contains(matrix.os, 'ubuntu') || contains(matrix.os, 'mac')
if: contains(matrix.os, 'linux') || contains(matrix.os, 'mac')
run: |
sed -i.bak 's/\[profile.dev\]/\[profile.dev\]\ndebug = false/' Cargo.toml
git diff
@@ -93,14 +93,14 @@ jobs:
command: build
- name: Build all examples
if: contains(matrix.os, 'ubuntu')
if: contains(matrix.os, 'linux')
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --examples
- name: Run all tests
if: contains(matrix.os, 'ubuntu')
if: contains(matrix.os, 'linux')
uses: actions-rs/cargo@v1
env:
NYM_API: https://sandbox-nym-api1.nymtech.net/api
@@ -109,7 +109,7 @@ jobs:
args: --workspace
- name: Run expensive tests
if: (github.ref == 'refs/heads/develop' || github.event.pull_request.base.ref == 'develop' || github.event.pull_request.base.ref == 'master') && contains(matrix.os, 'ubuntu')
if: (github.ref == 'refs/heads/develop' || github.event.pull_request.base.ref == 'develop' || github.event.pull_request.base.ref == 'master') && contains(matrix.os, 'linux')
uses: actions-rs/cargo@v1
with:
command: test
@@ -44,8 +44,10 @@ jobs:
echo "Tag is empty"
exit 1
fi
# first, list all tags for logging purposes
curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq --arg tag $TAG '.tags | contains([$tag])' )
# check if there's a matching tag
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq -r --arg tag "$TAG" 'any(.tags[]; . == $tag)' )
if [[ $exists = "true" ]]; then
echo "Version '$TAG' defined in Cargo.toml ALREADY EXISTS as tag in harbor repo"
exit 1
@@ -53,5 +55,5 @@ jobs:
echo "Version '$TAG' doesn't exist on the remote"
else
echo "Unknown output '$exists'"
exit 1
exit 2
fi
@@ -44,8 +44,10 @@ jobs:
echo "Tag is empty"
exit 1
fi
# first, list all tags for logging purposes
curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq --arg tag $TAG '.tags | contains([$tag])' )
# check if there's a matching tag
exists=$(curl -su ${{ secrets.HARBOR_ROBOT_USERNAME }}:${{ secrets.HARBOR_ROBOT_SECRET }} "$registry/v2/$repo_name/tags/list" | jq -r --arg tag "$TAG" 'any(.tags[]; . == $tag)' )
if [[ $exists = "true" ]]; then
echo "Version '$TAG' defined in Cargo.toml ALREADY EXISTS as tag in harbor repo"
exit 1
@@ -53,5 +55,5 @@ jobs:
echo "Version '$TAG' doesn't exist on the remote"
else
echo "Unknown output '$exists'"
exit 1
exit 2
fi
@@ -31,32 +31,26 @@ jobs:
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
target: wasm32-unknown-unknown
override: true
- name: Install wasm-opt
uses: ./.github/actions/install-wasm-opt
with:
version: '114'
- name: Install cosmwasm-check
run: cargo install cosmwasm-check
- name: Build release contracts
run: make contracts
run: make publish-contracts
- name: Prepare build output
shell: bash
env:
OUTPUT_DIR: ci-contract-builds/${{ github.ref_name }}
run: |
cp contracts/target/wasm32-unknown-unknown/release/mixnet_contract.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/vesting_contract.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_coconut_dkg.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw3_flex_multisig.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw4_group.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_ecash.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_pool_contract.wasm $OUTPUT_DIR
find contracts/artifacts -maxdepth 1 -type f -name '*.wasm' -exec cp {} $OUTPUT_DIR \;
# Also include the optimizer-generated checksums if present
if [ -f contracts/artifacts/checksums.txt ]; then
cp contracts/artifacts/checksums.txt $OUTPUT_DIR
fi
- name: Deploy branch to CI www
continue-on-error: true
+19
View File
@@ -0,0 +1,19 @@
name: Run SonarQube Scan
on:
push:
branches:
- develop
# pull_request:
# types: [opened, synchronize, reopened]
jobs:
sonarqube:
name: SonarQube
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: SonarQube Scan
uses: SonarSource/sonarqube-scan-action@v5
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
+5 -1
View File
@@ -19,7 +19,11 @@ jobs:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-binaries-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
strategy:
fail-fast: false
runs-on: arc-ubuntu-22.04
matrix:
include:
- os: arc-ubuntu-22.04
target: x86_64-unknown-linux-gnu
runs-on: ${{ matrix.os }}
outputs:
release_id: ${{ steps.create-release.outputs.id }}
+38 -15
View File
@@ -5,8 +5,15 @@ on:
inputs:
gateway_probe_git_ref:
type: string
default: nym-vpn-core-v1.4.0
required: true
description: Which gateway probe git ref to build the image with
release_image:
description: 'Tag image as a release'
required: true
default: false
type: boolean
env:
WORKING_DIRECTORY: "nym-node-status-api/nym-node-status-agent"
CONTAINER_NAME: "node-status-agent"
@@ -31,10 +38,10 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
run: |
VERSION=$(yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml)
echo "result=$VERSION" >> $GITHUB_OUTPUT
- name: cleanup-gateway-probe-ref
id: cleanup_gateway_probe_ref
@@ -43,19 +50,35 @@ jobs:
GIT_REF_SLUG="${GATEWAY_PROBE_GIT_REF//\//-}"
echo "git_ref=${GIT_REF_SLUG}" >> $GITHUB_OUTPUT
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
fi
- name: Set GIT_TAG variable
run: echo "GIT_TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}" >> $GITHUB_ENV
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }} -m "Version ${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
- name: Initialize RELEASE_TAG
run: echo "RELEASE_TAG=" >> $GITHUB_ENV
- name: Set RELEASE_TAG for release
if: github.event.inputs.release_image == 'true'
run: echo "RELEASE_TAG=golden-" >> $GITHUB_ENV
- name: Set IMAGE_NAME_AND_TAGS variable
run: echo "IMAGE_NAME_AND_TAGS=${{ env.CONTAINER_NAME }}:${{ env.RELEASE_TAG }}${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}" >> $GITHUB_ENV
- name: New env vars
run: echo "RELEASE_TAG='$RELEASE_TAG' GIT_TAG='$GIT_TAG' IMAGE_NAME_AND_TAGS='$IMAGE_NAME_AND_TAGS'"
# - name: Remove existing tag if exists
# run: |
# if git rev-parse $${{ env.GIT_TAG }} >/dev/null 2>&1; then
# git push --delete origin $${{ env.GIT_TAG }}
# git tag -d $${{ env.GIT_TAG }}
# fi
# - name: Create tag
# run: |
# git tag -a $${{ env.GIT_TAG }} -m "Version ${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}"
# git push origin $${{ env.GIT_TAG }}
- name: BuildAndPushImageOnHarbor
run: |
docker build --build-arg GIT_REF=${{ github.event.inputs.gateway_probe_git_ref }} -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
docker build --build-arg GIT_REF=${{ github.event.inputs.gateway_probe_git_ref }} -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.IMAGE_NAME_AND_TAGS }}
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+35 -20
View File
@@ -1,7 +1,13 @@
name: Build and upload Node Status API container to harbor.nymte.ch
on:
workflow_dispatch:
inputs:
release_image:
description: 'Tag image as a release'
required: true
default: false
type: boolean
env:
WORKING_DIRECTORY: "nym-node-status-api/nym-node-status-api"
CONTAINER_NAME: "node-status-api"
@@ -26,30 +32,39 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Check if tag exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
echo "Tag ${{ steps.get_version.outputs.result }} already exists"
fi
yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
fi
- name: Set GIT_TAG variable
run: echo "GIT_TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}" >> $GITHUB_ENV
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
- name: Set RELEASE_TAG variable
if: github.event.inputs.release_image == 'true'
run: echo "RELEASE_TAG=golden-" >> $GITHUB_ENV
- name: Set IMAGE_NAME_AND_TAGS variable
run: echo "IMAGE_NAME_AND_TAGS=${{ env.CONTAINER_NAME }}:${{ env.RELEASE_TAG }}${{ steps.get_version.outputs.result }}" >> $GITHUB_ENV
- name: New env vars
run: echo "RELEASE_TAG='$RELEASE_TAG' GIT_TAG='$GIT_TAG' IMAGE_NAME_AND_TAGS='$IMAGE_NAME_AND_TAGS'"
# - name: Remove existing tag if exists, then create
# run: |
# if git rev-parse "$GIT_TAG" >/dev/null 2>&1; then
# echo "Tag '$GIT_TAG' already exists, deleting"
# git push --delete origin "$GIT_TAG"
# git tag -d "$GIT_TAG"
# echo "Tag '$GIT_TAG' deleted"
# else
# echo "Tag '$GIT_TAG' does not exist, creating it"
# git tag -a $GIT_TAG -m "Version ${{ steps.get_version.outputs.result }}"
# git push origin $GIT_TAG
# echo "Tag '$GIT_TAG' created"
# fi
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.IMAGE_NAME_AND_TAGS }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+2
View File
@@ -35,6 +35,8 @@ validator-api/keypair
contracts/mixnet/code_id
contracts/mixnet/Justfile
contracts/mixnet/Makefile
artifacts
contracts/artifacts
validator-config
*.patch
validator-api-config.toml
+84
View File
@@ -4,6 +4,90 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
## [Unreleased]
## [2025.15-gruyere] (2025-08-20)
- Migrate strum to 0.27.2 ([#5960])
- WG exit policy scripts update ([#5921])
- Make DNS Resolver fallback optional ([#5920])
- nym-node debug command to reset providers db ([#5914])
- basic zulip client for sending messages ([#5913])
- chore: allow compatibility with 'CDLA-Permissive-2.0' ([#5910])
- feat: ecash liveness check ([#5890])
- Remove old free credential handle ([#5864])
[#5960]: https://github.com/nymtech/nym/pull/5960
[#5921]: https://github.com/nymtech/nym/pull/5921
[#5920]: https://github.com/nymtech/nym/pull/5920
[#5914]: https://github.com/nymtech/nym/pull/5914
[#5913]: https://github.com/nymtech/nym/pull/5913
[#5910]: https://github.com/nymtech/nym/pull/5910
[#5890]: https://github.com/nymtech/nym/pull/5890
[#5864]: https://github.com/nymtech/nym/pull/5864
## [2025.14-feta] (2025-08-05)
- chore: nym node tokio console ([#5909])
- Feature/dkg snapshot epoch ([#5900])
- Feature/dkg epoch dealers query ([#5899])
- sqlx-pool-guard: allocate more memory on windows ([#5896])
- Support mnemonic in the NS agent ([#5883])
- Allow PG database backend ([#5880])
[#5909]: https://github.com/nymtech/nym/pull/5909
[#5900]: https://github.com/nymtech/nym/pull/5900
[#5899]: https://github.com/nymtech/nym/pull/5899
[#5896]: https://github.com/nymtech/nym/pull/5896
[#5883]: https://github.com/nymtech/nym/pull/5883
[#5880]: https://github.com/nymtech/nym/pull/5880
## [2025.13-emmental] (2025-07-22)
- fix: don't allow mixnode running in exit mode ([#5898])
- fix contract build process in Makefile ([#5892])
- bugfix: ignore 'Send' responses when claiming bandwidth ([#5884])
- Update push-node-status-agent.yaml ([#5882])
- listen for shutdown signals during nym-node startup ([#5879])
- feat: forbid running mixnode + entry on the same node ([#5878])
- chore: 1.88 clippy ([#5877])
- Batch SQL writes for packet stats ([#5874])
- fix the broken link ([#5873])
- Set busy_timeout in sqlx ([#5872])
- feat: basic performance contract integration [within Nym API] ([#5871])
- scraper bugfix: ignore precommits from missing validators ([#5867])
- Return true remaining ([#5866])
- Make Mix hops optional for Mixnet Client SURBs ([#5861])
- Check gateway supported versions ([#5860])
- Add build info endpoints ([#5857])
- Clear out screaming logs ([#5856])
- fix removal of qa env ([#5855])
- Use display when printing paths ([#5853])
- feat: initial performance contract ([#5833])
- Security patches for the `dkg` crate ([#5828])
- HTTP Discovery objects & network defaults ([#5814])
[#5898]: https://github.com/nymtech/nym/pull/5898
[#5892]: https://github.com/nymtech/nym/pull/5892
[#5884]: https://github.com/nymtech/nym/pull/5884
[#5882]: https://github.com/nymtech/nym/pull/5882
[#5879]: https://github.com/nymtech/nym/pull/5879
[#5878]: https://github.com/nymtech/nym/pull/5878
[#5877]: https://github.com/nymtech/nym/pull/5877
[#5874]: https://github.com/nymtech/nym/pull/5874
[#5873]: https://github.com/nymtech/nym/pull/5873
[#5872]: https://github.com/nymtech/nym/pull/5872
[#5871]: https://github.com/nymtech/nym/pull/5871
[#5867]: https://github.com/nymtech/nym/pull/5867
[#5866]: https://github.com/nymtech/nym/pull/5866
[#5861]: https://github.com/nymtech/nym/pull/5861
[#5860]: https://github.com/nymtech/nym/pull/5860
[#5857]: https://github.com/nymtech/nym/pull/5857
[#5856]: https://github.com/nymtech/nym/pull/5856
[#5855]: https://github.com/nymtech/nym/pull/5855
[#5853]: https://github.com/nymtech/nym/pull/5853
[#5833]: https://github.com/nymtech/nym/pull/5833
[#5828]: https://github.com/nymtech/nym/pull/5828
[#5814]: https://github.com/nymtech/nym/pull/5814
## [2025.12-dolcelatte] (2025-07-07)
- bugfix: key-rotation + reply SURBs ([#5876])
+686
View File
@@ -0,0 +1,686 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Nym is a privacy platform that uses mixnet technology to protect against metadata surveillance. The platform consists of several key components:
- Mixnet nodes (mixnodes) for packet mixing
- Gateways (entry/exit points for the network)
- Clients for interacting with the network
- Network monitoring tools
- Validators for network consensus
- Various service providers and integrations
## Build Commands
### Rust Components
```bash
# Default build (debug)
cargo build
# Release build
cargo build --release
# Build a specific package
cargo build -p <package-name>
# Build main components
make build
# Build release versions of main binaries and contracts
make build-release
# Build specific binaries
make build-nym-cli
cargo build -p nym-node --release
cargo build -p nym-api --release
```
### Testing
```bash
# Run clippy, unit tests, and formatting
make test
# Run all tests including slow tests
make test-all
# Run clippy on all workspaces
make clippy
# Run unit tests for a specific package
cargo test -p <package-name>
# Run only expensive/ignored tests
cargo test --workspace -- --ignored
# Run API tests
dotenv -f envs/sandbox.env -- cargo test --test public-api-tests
# Run tests with specific log level
RUST_LOG=debug cargo test -p <package-name>
# Run specific test scripts
./nym-node/tests/test_apis.sh
./scripts/wireguard-exit-policy/exit-policy-tests.sh
```
### Linting and Formatting
```bash
# Run rustfmt on all code
make fmt
# Check formatting without modifying
cargo fmt --all -- --check
# Run clippy with all targets
cargo clippy --workspace --all-targets -- -D warnings
# TypeScript linting
yarn lint
yarn lint:fix
yarn types:lint:fix
# Check dependencies for security/licensing issues
cargo deny check
```
### WASM Components
```bash
# Build all WASM components
make sdk-wasm-build
# Build TypeScript SDK
yarn build:sdk
npx lerna run --scope @nymproject/sdk build --stream
# Build and test WASM components
make sdk-wasm
# Build specific WASM packages
cd wasm/client && make
cd wasm/mix-fetch && make
cd wasm/node-tester && make
```
### Contract Development
```bash
# Build all contracts
make contracts
# Build contracts in release mode
make build-release-contracts
# Generate contract schemas
make contract-schema
# Run wasm-opt on contracts
make wasm-opt-contracts
# Check contracts with cosmwasm-check
make cosmwasm-check-contracts
```
### Running Components
```bash
# Run nym-node as a mixnode
cargo run -p nym-node -- run --mode mixnode
# Run nym-node as a gateway
cargo run -p nym-node -- run --mode gateway
# Run the network monitor
cargo run -p nym-network-monitor
# Run the API server
cargo run -p nym-api
# Run with specific environment
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Start a local network
./scripts/localnet_start.sh
```
## Architecture
The Nym platform consists of various components organized as a monorepo:
1. **Core Mixnet Infrastructure**:
- `nym-node`: Core binary supporting mixnode and gateway modes
- `common/nymsphinx`: Implementation of the Sphinx packet format
- `common/topology`: Network topology management
- `common/types`: Shared data types across components
2. **Network Monitoring**:
- `nym-network-monitor`: Monitors the network's reliability and performance
- `nym-api`: API server for network stats and monitoring data
- Metrics tracking for nodes, routes, and overall network health
3. **Client Implementations**:
- `clients/native`: Native Rust client implementation
- `clients/socks5`: SOCKS5 proxy client for standard applications
- `wasm`: WebAssembly client implementations (for browsers)
- `nym-connect`: Desktop and mobile clients
4. **Blockchain & Smart Contracts**:
- `common/cosmwasm-smart-contracts`: Smart contract implementations
- `contracts`: CosmWasm contracts for the Nym network
- `common/ledger`: Blockchain integration
5. **Utilities & Tools**:
- `tools`: Various CLI tools and utilities
- `sdk`: SDKs for different languages and platforms
- `documentation`: Documentation generation and management
## Packet System
Nym uses a modified Sphinx packet format for its mixnet:
1. **Message Chunking**:
- Messages are divided into "sets" and "fragments"
- Each fragment fits in a single Sphinx packet
- The `common/nymsphinx/chunking` module handles message fragmentation
2. **Routing**:
- Packets traverse through 3 layers of mixnodes
- Routing information is encrypted in layers (onion routing)
- The final gateway receives and processes the messages
3. **Monitoring**:
- Monitoring system tracks packet delivery through the network
- Routes are analyzed for reliability statistics
- Node performance metrics are collected
## Network Protocol
Nym implements the Loopix mixnet design with several key privacy features:
1. **Continuous-time Mixing**:
- Each mixnode delays messages independently with an exponential distribution
- This creates random reordering of packets, destroying timing correlations
- Offers better anonymity properties than batch mixing approaches
2. **Cover Traffic**:
- Clients and nodes generate dummy "loop" packets that circulate through the network
- These packets are indistinguishable from real traffic
- Creates a baseline level of traffic that hides actual communication patterns
- Provides unobservability (hiding when and how much real traffic is being sent)
3. **Stratified Network Architecture**:
- Traffic flows through Entry Gateway → 3 Mixnode Layers → Exit Gateway
- Path selection is independent per-message (unlike Tor)
- Each node connects only to adjacent layers
4. **Anonymous Replies**:
- Single-Use Reply Blocks (SURBs) allow receiving messages without revealing identity
- Enables bidirectional communication while maintaining privacy
## Network Monitoring Architecture
The network monitoring system is a core component that measures mixnet reliability:
1. The `nym-network-monitor` sends test packets through the network
2. These packets follow predefined routes through multiple mixnodes
3. Metrics are collected about:
- Successful and failed packet deliveries
- Node reliability (percentage of successful packet handling)
- Route reliability (which specific route combinations work best)
4. Results are stored in the database and used by `nym-api` to:
- Present node performance statistics
- Determine network rewards
- Provide route selection guidance to clients
In the current branch, metrics collection is being enhanced with a fanout approach to submit to multiple API endpoints.
## Development Environment
### Required Dependencies
- Rust toolchain (stable, 1.80+)
- Node.js (v20+) and yarn for TypeScript components
- SQLite for local database development
- PostgreSQL for API database (optional, for full API functionality)
- CosmWasm tools for contract development
- For building contracts: `wasm-opt` tool from `binaryen`
- Python 3.8+ for some scripts
- Docker (optional, for containerized development)
- protoc (Protocol Buffers compiler) for some components
### Environment Configurations
The `envs/` directory contains pre-configured environments:
#### Available Environments
- **`local.env`**: Local development environment
- Points to local services (localhost)
- Uses test mnemonics and keys
- Ideal for testing without external dependencies
- **`sandbox.env`**: Sandbox test network
- Public test network with real nodes
- Test tokens available from faucet
- Contract addresses for sandbox deployment
- API: https://sandbox-nym-api1.nymtech.net
- **`mainnet.env`**: Production mainnet
- Real network with real tokens
- Production contract addresses
- API: https://validator.nymtech.net
- Use with caution!
- **`canary.env`**: Canary deployment
- Pre-release testing environment
- Tests new features before mainnet
- **`mainnet-local-api.env`**: Hybrid environment
- Uses mainnet contracts but local API
- Useful for API development against mainnet data
#### Key Environment Variables
```bash
# Network configuration
NETWORK_NAME=sandbox # Network identifier
BECH32_PREFIX=n # Address prefix (n for sandbox, n for mainnet)
NYM_API=https://sandbox-nym-api1.nymtech.net/api
NYXD=https://rpc.sandbox.nymtech.net
NYM_API_NETWORK=sandbox
# Contract addresses (network-specific)
MIXNET_CONTRACT_ADDRESS=n1xr3rq8yvd7qplsw5yx90ftsr2zdhg4e9z60h5duusgxpv72hud3sjkxkav
VESTING_CONTRACT_ADDRESS=n1unyuj8qnmygvzuex3dwmg9yzt9alhvyeat0uu0jedg2wj33efl5qackslz
# ... other contract addresses
# Mnemonic for testing (NEVER use in production)
MNEMONIC="clutch captain shoe salt awake harvest setup primary inmate ugly among become"
# API Keys and tokens
IPINFO_API_TOKEN=your_token_here
AUTHENTICATOR_PASSWORD=password_here
# Logging
RUST_LOG=info # Options: error, warn, info, debug, trace
RUST_BACKTRACE=1 # Enable backtraces
# Database
DATABASE_URL=postgresql://user:pass@localhost/nym_api
```
#### Using Environment Files
```bash
# Load environment and run command
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Export to shell
source envs/sandbox.env
# Use with make targets
dotenv -f envs/sandbox.env -- make run-api-tests
```
## Initial Setup
### First Time Setup
1. **Install Prerequisites**
```bash
# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install Node.js and yarn
# Via nvm (recommended):
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
nvm install 20
npm install -g yarn
# Install build tools
# Ubuntu/Debian:
sudo apt-get install build-essential pkg-config libssl-dev protobuf-compiler libpq-dev
# macOS:
brew install protobuf postgresql
# Install wasm-opt for contract builds
npm install -g wasm-opt
# Add wasm target for Rust
rustup target add wasm32-unknown-unknown
```
2. **Clone and Setup Repository**
```bash
git clone https://github.com/nymtech/nym.git
cd nym/nym
# Install JavaScript dependencies
yarn install
# Build the project
make build
```
3. **Database Setup (Optional, for API development)**
```bash
# Install PostgreSQL
# Create database
createdb nym_api
# Run migrations (from nym-api directory)
cd nym-api
sqlx migrate run
```
### Quick Start
```bash
# Run a mixnode locally
dotenv -f envs/sandbox.env -- cargo run -p nym-node -- run --mode mixnode --id my-mixnode
# Run a gateway locally
dotenv -f envs/sandbox.env -- cargo run -p nym-node -- run --mode gateway --id my-gateway
# Run the API server
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Run a client
cargo run -p nym-client -- init --id my-client
cargo run -p nym-client -- run --id my-client
```
## CI/CD Pipeline
The project uses GitHub Actions for CI/CD with several key workflows:
1. **Build and Test**:
- `ci-build.yml`: Main build workflow for Rust components
- Tests are run on multiple platforms (Linux, Windows, macOS)
- Includes formatting check (rustfmt) and linting (clippy)
2. **Release Process**:
- Binary artifacts are published on release tags
- Multiple platform builds are created
3. **Documentation**:
- Documentation is automatically built and deployed
## Database Structure
The system uses SQLite databases with tables like:
- `mixnode_status`: Status information about mixnodes
- `gateway_status`: Status information about gateways
- `routes`: Route performance information (success/failure of specific paths)
- `monitor_run`: Information about monitoring test runs
## Development Workflows
### Running a Node
To run the mixnode or gateway:
```bash
# Run nym-node as a mixnode with specified identity
cargo run -p nym-node -- run --mode mixnode --id my-mixnode
# Run nym-node as a gateway
cargo run -p nym-node -- run --mode gateway --id my-gateway
```
### Configuration
Nodes can be configured with files in various locations:
- Command-line arguments
- Environment variables
- `.env` files specified with `--config-env-file`
### Monitoring
To monitor the health of your node:
- View logs for real-time information
- Use the node's HTTP API for status information
- Check the explorer for public node statistics
## Common Libraries
- `common/types`: Shared data types across all components
- `common/crypto`: Cryptographic primitives and wrappers
- `common/client-core`: Core client functionality
- `common/gateway-client`: Client-gateway communication
- `common/task`: Task management and concurrency utilities
- `common/nymsphinx`: Sphinx packet implementation for mixnet
- `common/topology`: Network topology management
- `common/credentials`: Credential system for privacy-preserving authentication
- `common/bandwidth-controller`: Bandwidth management and accounting
## Code Conventions
- Error handling: Use anyhow/thiserror for structured error handling
- Logging: Use the tracing framework for logging and diagnostics
- State management: Generally use Tokio/futures for async code
- Configuration: Use the config crate and env vars with defaults
- Database: Use sqlx for type-safe database queries
- Follow clippy recommendations and rustfmt formatting
- Use semantic commit messages: feat, fix, docs, refactor, test, chore
## When Making Changes
- Run `make test` before submitting PRs
- Follow Rust naming conventions
- Use `clippy` to check for common issues
- Update SQLx query caches when modifying DB queries: `cargo sqlx prepare`
- Consider backward compatibility for protocol changes
- Use lefthook pre-commit hooks for TypeScript formatting
- Run `cargo deny check` to verify dependency compliance
- Test against both sandbox and local environments when possible
- Update relevant documentation and CHANGELOG.md
## Development Tools
### Useful Cargo Commands
```bash
# Check for outdated dependencies
cargo outdated
# Analyze binary size
cargo bloat --release -p nym-node
# Generate dependency graph
cargo tree -p nym-api
# Run with instrumentation
cargo run --features profiling -p nym-node
# Check for security advisories
cargo audit
```
### Database Tools
```bash
# SQLx CLI for migrations
cargo install sqlx-cli
# Create new migration
cd nym-api && sqlx migrate add <migration_name>
# Prepare query metadata for offline compilation
cargo sqlx prepare --workspace
# View database schema
./nym-api/enter_db.sh
```
### Development Scripts
- `scripts/build_topology.py`: Generate network topology files
- `scripts/node_api_check.py`: Verify node API endpoints
- `scripts/network_tunnel_manager.sh`: Manage network tunnels
- `scripts/localnet_start.sh`: Start a local test network
- Various deployment scripts in `deployment/` for different environments
## Debugging
- Enable more verbose logging with the RUST_LOG environment variable:
```
RUST_LOG=debug,nym_node=trace cargo run -p nym-node -- run --mode mixnode
```
- Use the HTTP API endpoints for status information
- Check monitoring data in the database for network performance metrics
- For complex issues, use tracing tools to follow packet flow
- Enable backtraces: `RUST_BACKTRACE=full`
- For WASM debugging: Use browser developer tools with source maps
## Deployment and Advanced Configurations
### Deployment Structure
The `deployment/` directory contains Ansible playbooks and configurations for various deployment scenarios:
- **`aws/`**: AWS-specific deployment configurations
- **`mixnode/`**: Mixnode deployment playbooks
- **`gateway/`**: Gateway deployment playbooks
- **`validator/`**: Validator node deployment
- **`sandbox-v2/`**: Complete sandbox environment setup
- **`big-dipper-2/`**: Block explorer deployment
### Sandbox V2 Deployment
The sandbox-v2 deployment (`deployment/sandbox-v2/`) provides a complete test environment:
```bash
# Key playbooks:
- deploy.yaml # Main deployment orchestrator
- deploy-mixnodes.yaml # Deploy mixnodes
- deploy-gateways.yaml # Deploy gateways
- deploy-validators.yaml # Deploy validator nodes
- deploy-nym-api.yaml # Deploy API services
```
### Custom Environment Setup
To create a custom environment:
1. Copy an existing env file: `cp envs/sandbox.env envs/custom.env`
2. Modify the network endpoints and contract addresses
3. Update the `NETWORK_NAME` to your identifier
4. Set appropriate mnemonics and keys (use fresh ones for production!)
### Contract Addresses
Contract addresses are network-specific and defined in environment files:
- Mixnet contract: Manages mixnode/gateway registry
- Vesting contract: Handles token vesting schedules
- Coconut contracts: Privacy-preserving credentials
- Name service: Human-readable address mapping
- Ecash contract: Electronic cash functionality
### Local Network Setup
For a completely local network:
```bash
# Start local chain
./scripts/localnet_start.sh
# Deploy contracts
cd contracts
make deploy-local
# Start nodes with local config
dotenv -f envs/local.env -- cargo run -p nym-node -- run --mode mixnode
```
## Common Issues and Troubleshooting
### Database Issues
- When modifying database queries, you must update SQLx query caches:
```bash
cargo sqlx prepare
```
- If you see SQLx errors about missing query files, this is likely the cause
- For "database is locked" errors with SQLite, ensure only one process accesses the DB
- For PostgreSQL connection issues, verify DATABASE_URL and that the server is running
### API Connection Issues
- Check the environment variables pointing to the APIs (NYM_API, NYXD)
- Verify network connectivity and API health endpoints
- For authentication issues, check node keys and credentials
- Common endpoints to verify:
- API health: `$NYM_API/health`
- Chain status: `$NYXD/status`
- Contract info: `$NYXD/cosmwasm/wasm/v1/contract/$CONTRACT_ADDRESS`
### Build Problems
- Clean dependencies with `cargo clean` for a fresh build
- Check for compatible Rust version (1.80+ recommended)
- For smart contract builds, ensure wasm-opt is installed: `npm install -g wasm-opt`
- For cross-compilation issues, check target-specific dependencies
- WASM build issues: Ensure wasm32-unknown-unknown target is installed:
```bash
rustup target add wasm32-unknown-unknown
```
- For "cannot find -lpq" errors, install PostgreSQL development files:
```bash
# Ubuntu/Debian
sudo apt-get install libpq-dev
# macOS
brew install postgresql
```
### Environment Issues
- Contract address mismatches: Ensure you're using the correct environment file
- "Account sequence mismatch": The account nonce is out of sync, wait and retry
- Token decimal issues: Sandbox uses different decimal places than mainnet
- API version mismatches: Ensure your local API version matches the network
- "Insufficient funds": Get test tokens from faucet (sandbox) or check balance
- Gateway/mixnode bonding issues: Verify minimum stake requirements
## Working with Routes and Monitoring
1. Route monitoring metrics are stored in a `routes` table with:
- Layer node IDs (layer1, layer2, layer3, gw)
- Success flag (boolean)
- Timestamp
2. To analyze routes:
- Check `NetworkAccount` and `AccountingRoute` in `nym-network-monitor/src/accounting.rs`
- View monitoring logic in `common/nymsphinx/chunking/monitoring.rs`
- Observe how routes are submitted to the database in the `submit_accounting_routes_to_db` function
## Performance Optimization
### Profiling and Benchmarking
```bash
# Run benchmarks
cargo bench -p nym-node
# Profile with perf (Linux)
cargo build --release --features profiling
perf record --call-graph=dwarf ./target/release/nym-node run --mode mixnode
perf report
# Generate flamegraph
cargo install flamegraph
cargo flamegraph --bin nym-node -- run --mode mixnode
```
### Common Performance Considerations
- Use bounded channels for backpressure
- Batch database operations where possible
- Monitor memory usage with `RUST_LOG=nym_node::metrics=debug`
- Use connection pooling for database connections
- Consider using `jemalloc` for better memory allocation performance
Generated
+1755 -2328
View File
File diff suppressed because it is too large Load Diff
+21 -11
View File
@@ -34,11 +34,13 @@ members = [
"common/config",
"common/cosmwasm-smart-contracts/coconut-dkg",
"common/cosmwasm-smart-contracts/contracts-common",
"common/cosmwasm-smart-contracts/contracts-common-testing",
"common/cosmwasm-smart-contracts/easy_addr",
"common/cosmwasm-smart-contracts/ecash-contract",
"common/cosmwasm-smart-contracts/group-contract",
"common/cosmwasm-smart-contracts/mixnet-contract",
"common/cosmwasm-smart-contracts/multisig-contract",
"common/cosmwasm-smart-contracts/nym-performance-contract",
"common/cosmwasm-smart-contracts/nym-pool-contract",
"common/cosmwasm-smart-contracts/vesting-contract",
"common/credential-storage",
@@ -48,6 +50,8 @@ members = [
"common/credentials-interface",
"common/crypto",
"common/dkg",
"common/ecash-signer-check",
"common/ecash-signer-check-types",
"common/ecash-time",
"common/execute",
"common/exit-policy",
@@ -88,7 +92,7 @@ members = [
"common/socks5/requests",
"common/statistics",
"common/store-cipher",
"common/task",
"common/task", "common/test-utils",
"common/ticketbooks-merkle",
"common/topology",
"common/tun",
@@ -98,7 +102,12 @@ members = [
"common/wasm/storage",
"common/wasm/utils",
"common/wireguard",
"common/wireguard-private-metadata/client",
"common/wireguard-private-metadata/server",
"common/wireguard-private-metadata/shared",
"common/wireguard-private-metadata/tests",
"common/wireguard-types",
"common/zulip-client",
"documentation/autodoc",
"gateway",
"nym-api",
@@ -122,10 +131,10 @@ members = [
"sdk/ffi/go",
"sdk/ffi/shared",
"sdk/rust/nym-sdk",
"service-providers/authenticator",
"service-providers/common",
"service-providers/ip-packet-router",
"service-providers/network-requester",
"sqlx-pool-guard",
"tools/echo-server",
"tools/internal/contract-state-importer/importer-cli",
"tools/internal/contract-state-importer/importer-contract",
@@ -135,7 +144,6 @@ members = [
"tools/internal/testnet-manager",
"tools/internal/testnet-manager",
"tools/internal/testnet-manager/dkg-bypass-contract",
"tools/internal/testnet-manager/dkg-bypass-contract",
"tools/internal/validator-status-check",
"tools/nym-cli",
"tools/nym-id-cli",
@@ -160,7 +168,6 @@ default-members = [
"nym-statistics-api",
"nym-validator-rewarder",
"nyx-chain-watcher",
"service-providers/authenticator",
"service-providers/ip-packet-router",
"service-providers/network-requester",
"tools/nymvisor",
@@ -217,7 +224,7 @@ clap_complete_fig = "4.5"
colored = "2.2"
comfy-table = "7.1.4"
console = "0.15.11"
console-subscriber = "0.1.1"
console-subscriber = "0.4.1"
console_error_panic_hook = "0.1"
const-str = "0.5.6"
const_format = "0.2.34"
@@ -233,6 +240,7 @@ digest = "0.10.7"
dirs = "5.0"
doc-comment = "0.3"
dotenvy = "0.15.6"
dyn-clone = "1.0.19"
ecdsa = "0.16"
ed25519-dalek = "2.1"
encoding_rs = "0.8.35"
@@ -287,6 +295,7 @@ petgraph = "0.6.5"
pin-project = "1.1"
pin-project-lite = "0.2.16"
publicsuffix = "2.3.0"
proc_pidinfo = "0.1.3"
quote = "1"
rand = "0.8.5"
rand_chacha = "0.3"
@@ -314,11 +323,11 @@ si-scale = "0.2.3"
snow = "0.9.6"
sphinx-packet = "=0.6.0"
sqlx = "0.8.6"
strum = "0.26"
strum_macros = "0.26"
strum = "0.27.2"
strum_macros = "0.27.2"
subtle-encoding = "0.5"
syn = "1"
sysinfo = "0.33.0"
sysinfo = "0.37.0"
tap = "1.0.1"
tar = "0.4.44"
tempfile = "3.20"
@@ -368,9 +377,6 @@ subtle = "2.5.0"
# cosmwasm-related
cosmwasm-schema = "=2.2.2"
cosmwasm-std = "=2.2.2"
# use 1.0.1 as that's the version used by cosmwasm-std 2.2.1
# (and ideally we don't want to pull the same dependency twice)
serde-json-wasm = "=1.0.1"
# same version as used by cosmwasm
cw-utils = "=2.0.0"
cw-storage-plus = "=2.0.0"
@@ -378,6 +384,7 @@ cw2 = { version = "=2.0.0" }
cw3 = { version = "=2.0.0" }
cw4 = { version = "=2.0.0" }
cw-controllers = { version = "=2.0.0" }
cw-multi-test = "=2.3.2"
# cosmrs-related
bip32 = { version = "0.5.3", default-features = false }
@@ -434,6 +441,9 @@ opt-level = 'z'
# lto = true
opt-level = 'z'
[workspace.lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tokio_unstable)'] }
[workspace.lints.clippy]
unwrap_used = "deny"
expect_used = "deny"
+58 -10
View File
@@ -12,7 +12,11 @@ help:
@echo " clippy: run clippy for all workspaces"
@echo " test: run clippy, unit tests, and formatting."
@echo " test-all: like test, but also includes the expensive tests"
@echo " deb: build debian packages
@echo " deb: build debian packages"
@echo ""
@echo "Contract building targets:"
@echo " contracts: build contracts for development (includes wasm-opt)"
@echo " publish-contracts: build contracts using Docker optimizer (deterministic)"
# -----------------------------------------------------------------------------
# Meta targets
@@ -130,25 +134,69 @@ cargo-test: sdk-wasm-test
clippy: sdk-wasm-lint
# -----------------------------------------------------------------------------
# Build contracts ready for deploy
# Build CosmWasm contracts (deterministic docker build)
# -----------------------------------------------------------------------------
CONTRACTS=vesting_contract mixnet_contract nym_ecash cw3_flex_multisig cw4_group nym_coconut_dkg nym_pool_contract
CONTRACTS_WASM=$(addsuffix .wasm, $(CONTRACTS))
CONTRACTS_OUT_DIR=contracts/target/wasm32-unknown-unknown/release
contracts: build-release-contracts wasm-opt-contracts cosmwasm-check-contracts
WASM_CONTRACT_DIR := contracts/target/wasm32-unknown-unknown/release
# Find every direct contract folder that contains a Cargo.toml
CONTRACT_DIRS := $(shell find contracts -type f -name Cargo.toml \( ! -path "contracts/Cargo.toml" \) | grep -v integration-tests | xargs -n1 dirname | sort -u)
CONTRACTS_OUT_DIR = contracts/artifacts
# Build all contracts via the official CosmWasm optimizer image (one invocation per contract)
# See : https://github.com/CosmWasm/optimizer?tab=readme-ov-file#contracts-excluded-from-workspace
# The optimizer ships separate multi-arch images. ARM builds are *not* bit-for-bit identical to the
# canonical x86_64 build (see README notice in CosmWasm/optimizer). For reproducible artefacts we
# therefore always run the amd64 variant by default.
# Override with :
# $ COSMWASM_OPTIMIZER_IMAGE=cosmwasm/optimizer-arm64:0.17.0 make contracts-publish
#
COSMWASM_OPTIMIZER_IMAGE ?= cosmwasm/optimizer:0.17.0
COSMWASM_OPTIMIZER_PLATFORM ?= linux/amd64
# Ensure clean build environment and run the optimizer
optimize-contracts:
@rm -rf artifacts 2>/dev/null || true
@echo "=== Ensuring clean build environment"
docker volume rm nym_contracts_cache 2>/dev/null || true
docker volume rm registry_cache 2>/dev/null || true
@for DIR in $(CONTRACT_DIRS); do \
echo "=== Optimizing $${DIR}"; \
docker run --rm --platform $(COSMWASM_OPTIMIZER_PLATFORM) \
-v $(CURDIR):/code \
--mount type=volume,source=nym_contracts_cache,target=/target \
--mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
-e CARGO_BUILD_INCREMENTAL=false \
-e RUSTFLAGS="-C target-cpu=generic -C debuginfo=0" \
-e SOURCE_DATE_EPOCH=1 \
$(COSMWASM_OPTIMIZER_IMAGE) $${DIR}; \
done
@mkdir -p $(CONTRACTS_OUT_DIR)
@cp artifacts/*.wasm $(CONTRACTS_OUT_DIR)/ 2>/dev/null || true
@cd $(CONTRACTS_OUT_DIR) && sha256sum *.wasm > checksums.txt
# Cleanup temporary artefacts directory
@rm -rf artifacts 2>/dev/null || true
wasm-opt-contracts:
for contract in $(CONTRACTS_WASM); do \
wasm-opt --signext-lowering -Os $(CONTRACTS_OUT_DIR)/$$contract -o $(CONTRACTS_OUT_DIR)/$$contract; \
@for WASM in $(WASM_CONTRACT_DIR)/*.wasm; do \
echo "Running wasm-opt on $$WASM"; \
wasm-opt --signext-lowering -Os $$WASM -o $$WASM ; \
done
cosmwasm-check-contracts:
for contract in $(CONTRACTS_WASM); do \
cosmwasm-check $(CONTRACTS_OUT_DIR)/$$contract; \
@for WASM in $(WASM_CONTRACT_DIR)/*.wasm; do \
echo "Checking $$WASM"; \
cosmwasm-check $$WASM ; \
done
# Default development build
contracts: build-release-contracts wasm-opt-contracts cosmwasm-check-contracts
# Publishing build used by CI deterministic Docker optimiser
publish-contracts: optimize-contracts cosmwasm-check-contracts
# Consider adding 's' to make plural consistent (beware: used in github workflow)
contract-schema:
$(MAKE) -C contracts schema
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-client"
version = "1.1.58"
version = "1.1.62"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
description = "Implementation of the Nym Client"
edition = "2021"
+1 -1
View File
@@ -111,7 +111,7 @@ impl SocketClient {
let dkg_query_client = if self.config.base.client.disabled_credentials_mode {
None
} else {
Some(default_query_dkg_client_from_config(&self.config.base))
Some(default_query_dkg_client_from_config(&self.config.base)?)
};
let storage = self.initialise_storage().await?;
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-socks5-client"
version = "1.1.58"
version = "1.1.62"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
description = "A SOCKS5 localhost proxy that converts incoming messages to Sphinx and sends them to a Nym address"
edition = "2021"
@@ -28,8 +28,6 @@ pub type HmacSha256 = Hmac<Sha256>;
pub type Nonce = u64;
pub type Taken = Option<SystemTime>;
pub const BANDWIDTH_CAP_PER_DAY: u64 = 250 * 1024 * 1024 * 1024; // 250 GB
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct IpPair {
pub ipv4: Ipv4Addr,
+2 -2
View File
@@ -207,7 +207,7 @@ where
<St as Storage>::StorageError: Send + Sync + 'static,
{
if let Some(stored) = storage
.get_expiration_date_signatures(expiration_date)
.get_expiration_date_signatures(expiration_date, epoch_id)
.await
.map_err(BandwidthControllerError::credential_storage_error)?
{
@@ -220,7 +220,7 @@ where
ecash_apis,
|api| async move {
api.api_client
.global_expiration_date_signatures(Some(expiration_date))
.global_expiration_date_signatures(Some(expiration_date), Some(epoch_id))
.await
},
format!("aggregated coin index signatures for date {expiration_date}"),
+4
View File
@@ -13,6 +13,7 @@ async-trait = { workspace = true }
base64 = { workspace = true }
bs58 = { workspace = true }
clap = { workspace = true, optional = true }
cfg-if = { workspace = true }
comfy-table = { workspace = true, optional = true }
futures = { workspace = true }
humantime = { workspace = true }
@@ -123,3 +124,6 @@ fs-surb-storage = ["nym-client-core-surb-storage/fs-surb-storage"]
fs-gateways-storage = ["nym-client-core-gateways-storage/fs-gateways-storage"]
wasm = ["nym-gateway-client/wasm"]
metrics-server = []
[lints]
workspace = true
@@ -416,6 +416,9 @@ pub struct Traffic {
/// will be routed as usual, to the entry gateway, through three mix nodes, egressing
/// through the exit gateway. If mix hops are disabled, traffic will be routed directly
/// from the entry gateway to the exit gateway, bypassing the mix nodes.
///
/// This overrides the `use_legacy_sphinx_format` setting as reduced mix hops
/// requires use of the updated SURB packet format.
pub disable_mix_hops: bool,
}
@@ -2,8 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use crate::BadGateway;
use std::io;
use std::path::PathBuf;
use std::{io, path::PathBuf};
use thiserror::Error;
#[derive(Debug, Error)]
@@ -19,7 +18,6 @@ pub enum StorageError {
#[error("failed to perform sqlx migration: {source}")]
MigrationError {
#[source]
#[from]
source: sqlx::migrate::MigrateError,
},
@@ -32,7 +30,6 @@ pub enum StorageError {
#[error("failed to run the SQL query: {source}")]
QueryError {
#[source]
#[from]
source: sqlx::error::Error,
},
@@ -58,6 +58,7 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.signatures_path.unwrap())?
}
};
@@ -64,6 +64,7 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.credential_path.unwrap())?
}
};
@@ -58,6 +58,7 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.signatures_path.unwrap())?
}
};
@@ -58,6 +58,7 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.key_path.unwrap())?
}
};
@@ -135,9 +135,11 @@ pub enum ClientInputStatus {
}
impl ClientInputStatus {
#[allow(clippy::panic)]
pub fn register_producer(&mut self) -> ClientInput {
match std::mem::replace(self, ClientInputStatus::Connected) {
ClientInputStatus::AwaitingProducer { client_input } => client_input,
// critical failure implying misuse of software
ClientInputStatus::Connected => panic!("producer was already registered before"),
}
}
@@ -149,9 +151,11 @@ pub enum ClientOutputStatus {
}
impl ClientOutputStatus {
#[allow(clippy::panic)]
pub fn register_consumer(&mut self) -> ClientOutput {
match std::mem::replace(self, ClientOutputStatus::Connected) {
ClientOutputStatus::AwaitingConsumer { client_output } => client_output,
// critical failure implying misuse of software
ClientOutputStatus::Connected => panic!("consumer was already registered before"),
}
}
@@ -707,11 +711,14 @@ where
})?;
let store_clone = mem_store.clone();
spawn_future(async move {
persistent_storage
.flush_on_shutdown(store_clone, shutdown)
.await
});
spawn_future!(
async move {
persistent_storage
.flush_on_shutdown(store_clone, shutdown)
.await
},
"PersistentReplyStorage::flush_on_shutdown"
);
Ok(mem_store)
}
@@ -732,7 +739,7 @@ where
let mut rng = OsRng;
let keys = if let Some(derivation_material) = derivation_material {
ClientKeys::from_master_key(&mut rng, &derivation_material)
.map_err(|_| ClientCoreError::HkdfDerivationError {})?
.map_err(|_| ClientCoreError::HkdfDerivationError)?
} else {
ClientKeys::generate_new(&mut rng)
};
@@ -1,19 +1,17 @@
// Copyright 2022-2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::replies::reply_storage::{
fs_backend, CombinedReplyStorage, ReplyStorageBackend,
use crate::{
client::replies::reply_storage::{fs_backend, CombinedReplyStorage, ReplyStorageBackend},
config,
config::Config,
error::ClientCoreError,
};
use crate::config;
use crate::config::Config;
use crate::error::ClientCoreError;
use nym_bandwidth_controller::BandwidthController;
use nym_client_core_gateways_storage::OnDiskGatewaysDetails;
use nym_credential_storage::storage::Storage as CredentialStorage;
use nym_validator_client::nyxd;
use nym_validator_client::QueryHttpRpcNyxdClient;
use std::path::Path;
use std::{fs, io};
use nym_validator_client::{nyxd, QueryHttpRpcNyxdClient};
use std::{io, path::Path};
use time::OffsetDateTime;
use tracing::{error, info, trace};
use url::Url;
@@ -22,11 +20,11 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
db_path: P,
surb_config: &config::ReplySurbs,
) -> Result<fs_backend::Backend, ClientCoreError> {
info!("creating fresh surb database");
info!("Creating fresh surb database");
let mut storage_backend = match fs_backend::Backend::init(db_path).await {
Ok(backend) => backend,
Err(err) => {
error!("failed to setup persistent storage backend for our reply needs: {err}");
error!("setup_fresh_backend: Failed to setup persistent storage backend for our reply needs: {err}");
return Err(ClientCoreError::SurbStorageError {
source: Box::new(err),
});
@@ -40,14 +38,15 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
surb_config.minimum_reply_surb_storage_threshold,
surb_config.maximum_reply_surb_storage_threshold,
);
storage_backend
.init_fresh(&mem_store)
.await
.map_err(|err| ClientCoreError::SurbStorageError {
source: Box::new(err),
})?;
Ok(storage_backend)
match storage_backend.init_fresh(&mem_store).await {
Ok(()) => Ok(storage_backend),
Err(err) => {
storage_backend.shutdown().await;
Err(ClientCoreError::SurbStorageError {
source: Box::new(err),
})
}
}
}
// fn setup_inactive_backend(surb_config: &config::ReplySurbs) -> fs_backend::Backend {
@@ -58,12 +57,11 @@ async fn setup_fresh_backend<P: AsRef<Path>>(
// )
// }
fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> {
async fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> {
let db_path = db_path.as_ref();
debug_assert!(db_path.exists());
let now = OffsetDateTime::now_utc().unix_timestamp();
let suffix = format!("_{now}.corrupted");
let new_extension =
@@ -72,11 +70,15 @@ fn archive_corrupted_database<P: AsRef<Path>>(db_path: P) -> io::Result<()> {
} else {
suffix
};
let renamed = db_path.with_extension(new_extension);
let mut renamed = db_path.to_owned();
renamed.set_extension(new_extension);
fs::rename(db_path, renamed)
tokio::fs::rename(db_path, &renamed).await.inspect_err(|_| {
error!(
"Failed to rename corrupt database file: {} to {}",
db_path.display(),
renamed.display()
);
})
}
pub async fn setup_fs_reply_surb_backend<P: AsRef<Path>>(
@@ -87,13 +89,12 @@ pub async fn setup_fs_reply_surb_backend<P: AsRef<Path>>(
// the existing one
let db_path = db_path.as_ref();
if db_path.exists() {
info!("loading existing surb database");
info!("Loading existing surb database");
match fs_backend::Backend::try_load(db_path).await {
Ok(backend) => Ok(backend),
Err(err) => {
error!("failed to setup persistent storage backend for our reply needs: {err}. We're going to create a fresh database instead. This behaviour might change in the future");
archive_corrupted_database(db_path)?;
error!("setup_fs_reply_surb_backend: Failed to setup persistent storage backend for our reply needs: {err}. We're going to create a fresh database instead. This behaviour might change in the future");
archive_corrupted_database(db_path).await?;
setup_fresh_backend(db_path, surb_config).await
}
}
@@ -113,41 +114,32 @@ pub async fn setup_fs_gateways_storage<P: AsRef<Path>>(
})
}
pub fn create_bandwidth_controller<St: CredentialStorage>(
config: &Config,
storage: St,
) -> BandwidthController<QueryHttpRpcNyxdClient, St> {
let nyxd_url = config
.get_validator_endpoints()
.pop()
.expect("No nyxd validator endpoint provided");
create_bandwidth_controller_with_urls(nyxd_url, storage)
}
pub fn create_bandwidth_controller_with_urls<St: CredentialStorage>(
nyxd_url: Url,
storage: St,
) -> BandwidthController<QueryHttpRpcNyxdClient, St> {
let client = default_query_dkg_client(nyxd_url);
) -> Result<BandwidthController<QueryHttpRpcNyxdClient, St>, ClientCoreError> {
let client = default_query_dkg_client(nyxd_url)?;
BandwidthController::new(storage, client)
Ok(BandwidthController::new(storage, client))
}
pub fn default_query_dkg_client_from_config(config: &Config) -> QueryHttpRpcNyxdClient {
pub fn default_query_dkg_client_from_config(
config: &Config,
) -> Result<QueryHttpRpcNyxdClient, ClientCoreError> {
let nyxd_url = config
.get_validator_endpoints()
.pop()
.expect("No nyxd validator endpoint provided");
.ok_or(ClientCoreError::RpcClientMissingUrl)?;
default_query_dkg_client(nyxd_url)
}
pub fn default_query_dkg_client(nyxd_url: Url) -> QueryHttpRpcNyxdClient {
pub fn default_query_dkg_client(nyxd_url: Url) -> Result<QueryHttpRpcNyxdClient, ClientCoreError> {
let details = nym_network_defaults::NymNetworkDetails::new_from_env();
let client_config = nyxd::Config::try_from_nym_network_details(&details)
.expect("failed to construct validator client config");
.map_err(|source| ClientCoreError::InvalidNetworkDetails { source })?;
// overwrite env configuration with config URLs
QueryHttpRpcNyxdClient::connect(client_config, nyxd_url.as_str())
.expect("Could not construct query client")
.map_err(|source| ClientCoreError::RpcClientCreationFailure { source })
}
@@ -235,6 +235,7 @@ impl LoopCoverTrafficStream<OsRng> {
tokio::task::yield_now().await;
}
#[allow(clippy::panic)]
pub fn start(mut self) {
if self.cover_traffic.disable_loop_cover_traffic_stream {
// we should have never got here in the first place - the task should have never been created to begin with
@@ -251,27 +252,30 @@ impl LoopCoverTrafficStream<OsRng> {
let mut shutdown = self.task_client.fork("select");
spawn_future(async move {
debug!("Started LoopCoverTrafficStream with graceful shutdown support");
spawn_future!(
async move {
debug!("Started LoopCoverTrafficStream with graceful shutdown support");
while !shutdown.is_shutdown() {
tokio::select! {
biased;
_ = shutdown.recv() => {
tracing::trace!("LoopCoverTrafficStream: Received shutdown");
}
next = self.next() => {
if next.is_some() {
self.on_new_message().await;
} else {
tracing::trace!("LoopCoverTrafficStream: Stopping since channel closed");
break;
while !shutdown.is_shutdown() {
tokio::select! {
biased;
_ = shutdown.recv() => {
tracing::trace!("LoopCoverTrafficStream: Received shutdown");
}
next = self.next() => {
if next.is_some() {
self.on_new_message().await;
} else {
tracing::trace!("LoopCoverTrafficStream: Stopping since channel closed");
break;
}
}
}
}
}
shutdown.recv_timeout().await;
tracing::debug!("LoopCoverTrafficStream: Exiting");
})
shutdown.recv_timeout().await;
tracing::debug!("LoopCoverTrafficStream: Exiting");
},
"LoopCoverTrafficStream"
)
}
}
@@ -96,72 +96,93 @@ impl MixTrafficController {
mut mix_packets: Vec<MixPacket>,
) -> Result<(), ErasedGatewayError> {
debug_assert!(!mix_packets.is_empty());
let result = if mix_packets.len() == 1 {
let send_future = if mix_packets.len() == 1 {
// SAFETY: we just checked we have one packet
#[allow(clippy::unwrap_used)]
let mix_packet = mix_packets.pop().unwrap();
self.gateway_transceiver.send_mix_packet(mix_packet).await
self.gateway_transceiver.send_mix_packet(mix_packet)
} else {
self.gateway_transceiver
.batch_send_mix_packets(mix_packets)
.await
self.gateway_transceiver.batch_send_mix_packets(mix_packets)
};
if result.is_err() {
self.consecutive_gateway_failure_count += 1;
} else {
trace!("We *might* have managed to forward sphinx packet(s) to the gateway!");
self.consecutive_gateway_failure_count = 0;
}
tokio::select! {
biased;
_ = self.task_client.recv() => {
trace!("received shutdown while handling messages");
Ok(())
}
result = send_future => {
if result.is_err() {
self.consecutive_gateway_failure_count += 1;
} else {
trace!("We *might* have managed to forward sphinx packet(s) to the gateway!");
self.consecutive_gateway_failure_count = 0;
}
result
result
}
}
}
async fn on_client_request(&mut self, client_request: ClientRequest) {
tokio::select! {
biased;
_ = self.task_client.recv() => {
trace!("received shutdown while handling client request");
}
result = self.gateway_transceiver.send_client_request(client_request) => {
if let Err(err) = result {
error!("Failed to send client request: {err}")
}
}
}
}
pub fn start(mut self) {
spawn_future(async move {
debug!("Started MixTrafficController with graceful shutdown support");
while !self.task_client.is_shutdown() {
tokio::select! {
mix_packets = self.mix_rx.recv() => match mix_packets {
Some(mix_packets) => {
if let Err(err) = self.on_messages(mix_packets).await {
error!("Failed to send sphinx packet(s) to the gateway: {err}");
if self.consecutive_gateway_failure_count == MAX_FAILURE_COUNT {
// Disconnect from the gateway. If we should try to re-connect
// is handled at a higher layer.
error!("Failed to send sphinx packet to the gateway {MAX_FAILURE_COUNT} times in a row - assuming the gateway is dead");
// Do we need to handle the embedded mixnet client case
// separately?
self.task_client.send_we_stopped(Box::new(ClientCoreError::GatewayFailedToForwardMessages));
break;
}
}
},
None => {
tracing::trace!("MixTrafficController: Stopping since channel closed");
spawn_future!(
async move {
debug!("Started MixTrafficController with graceful shutdown support");
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = self.task_client.recv() => {
tracing::trace!("MixTrafficController: Received shutdown");
break;
}
},
client_request = self.client_rx.recv() => match client_request {
Some(client_request) => {
match self.gateway_transceiver.send_client_request(client_request).await {
Ok(_) => (),
Err(e) => error!("Failed to send client request: {}", e),
};
mix_packets = self.mix_rx.recv() => match mix_packets {
Some(mix_packets) => {
if let Err(err) = self.on_messages(mix_packets).await {
error!("Failed to send sphinx packet(s) to the gateway: {err}");
if self.consecutive_gateway_failure_count == MAX_FAILURE_COUNT {
// Disconnect from the gateway. If we should try to re-connect
// is handled at a higher layer.
error!("Failed to send sphinx packet to the gateway {MAX_FAILURE_COUNT} times in a row - assuming the gateway is dead");
// Do we need to handle the embedded mixnet client case
// separately?
self.task_client.send_we_stopped(Box::new(ClientCoreError::GatewayFailedToForwardMessages));
break;
}
}
},
None => {
tracing::trace!("MixTrafficController: Stopping since channel closed");
break;
}
},
client_request = self.client_rx.recv() => match client_request {
Some(client_request) => {
self.on_client_request(client_request).await;
},
None => {
tracing::trace!("MixTrafficController, client request channel closed");
}
},
None => {
tracing::trace!("MixTrafficController, client request channel closed");
}
},
_ = self.task_client.recv() => {
tracing::trace!("MixTrafficController: Received shutdown");
break;
}
}
}
self.task_client.recv_timeout().await;
tracing::debug!("MixTrafficController: Exiting");
});
self.task_client.recv_timeout().await;
tracing::debug!("MixTrafficController: Exiting");
},
"MixTrafficController"
);
}
}
@@ -269,6 +269,8 @@ pub struct MockGateway {
}
impl Default for MockGateway {
// test code
#[allow(clippy::unwrap_used)]
fn default() -> Self {
MockGateway {
dummy_identity: "3ebjp1Fb9hdcS1AR6AZihgeJiMHkB5jjJUsvqNnfQwU7"
@@ -65,7 +65,7 @@ impl AcknowledgementListener {
return;
}
trace!("Received {} from the mix network", frag_id);
trace!("Received {frag_id} from the mix network");
self.stats_tx
.report(PacketStatisticsEvent::RealAckReceived(ack_content.len()).into());
if let Err(err) = self
@@ -126,7 +126,7 @@ impl ActionController {
fn handle_insert(&mut self, pending_acks: Vec<PendingAcknowledgement>) {
for pending_ack in pending_acks {
let frag_id = pending_ack.message_chunk.fragment_identifier();
trace!("{} is inserted", frag_id);
trace!("{frag_id} is inserted");
if self
.pending_acks_data
@@ -161,22 +161,16 @@ impl ActionController {
let new_queue_key = self.pending_acks_timers.insert(frag_id, timeout);
*queue_key = Some(new_queue_key)
} else {
debug!(
"Tried to START TIMER on pending ack that is already gone! - {}",
frag_id
);
debug!("Tried to START TIMER on pending ack that is already gone! - {frag_id}");
}
}
fn handle_remove(&mut self, frag_id: FragmentIdentifier) {
trace!("{} is getting removed", frag_id);
trace!("{frag_id} is getting removed");
match self.pending_acks_data.remove(&frag_id) {
None => {
debug!(
"Tried to REMOVE pending ack that is already gone! - {}",
frag_id
);
debug!("Tried to REMOVE pending ack that is already gone! - {frag_id}");
}
Some((_, queue_key)) => {
if let Some(queue_key) = queue_key {
@@ -188,10 +182,7 @@ impl ActionController {
} else {
// I'm not 100% sure if having a `None` key is even possible here
// (REMOVE would have to be called before START TIMER),
debug!(
"Tried to REMOVE pending ack without TIMER active - {}",
frag_id
);
debug!("Tried to REMOVE pending ack without TIMER active - {frag_id}");
}
}
}
@@ -200,27 +191,26 @@ impl ActionController {
// initiated basically as a first step of retransmission. At first data has its delay updated
// (as new sphinx packet was created with new expected delivery time)
fn handle_update_pending_ack(&mut self, frag_id: FragmentIdentifier, delay: SphinxDelay) {
trace!("{} is updating its delay", frag_id);
trace!("{frag_id} is updating its delay");
// TODO: is it possible to solve this without either locking or temporarily removing the value?
if let Some((pending_ack_data, queue_key)) = self.pending_acks_data.remove(&frag_id) {
// this Action is triggered by `RetransmissionRequestListener` (for 'normal' packets)
// SAFETY: this Action is triggered by `RetransmissionRequestListener` (for 'normal' packets)
// or `ReplyController` (for 'reply' packets) which held the other potential
// reference to this Arc. HOWEVER, before the Action was pushed onto the queue, the reference
// was dropped hence this unwrap is safe.
#[allow(clippy::unwrap_used)]
let mut inner_data = Arc::try_unwrap(pending_ack_data).unwrap();
inner_data.update_retransmitted(delay);
self.pending_acks_data
.insert(frag_id, (Arc::new(inner_data), queue_key));
} else {
debug!(
"Tried to UPDATE TIMER on pending ack that is already gone! - {}",
frag_id
);
debug!("Tried to UPDATE TIMER on pending ack that is already gone! - {frag_id}");
}
}
// note: when the entry expires it's automatically removed from pending_acks_timers
#[allow(clippy::panic)]
fn handle_expired_ack_timer(&mut self, expired_ack: Expired<FragmentIdentifier>) {
let frag_id = expired_ack.into_inner();
@@ -120,6 +120,7 @@ where
}
}
#[allow(clippy::panic)]
async fn on_input_message(&mut self, msg: InputMessage) {
match msg {
InputMessage::Regular {
@@ -213,7 +214,9 @@ where
self.handle_premade_packets(msgs, lane).await
}
// MessageWrappers can't be nested
InputMessage::MessageWrapper { .. } => unimplemented!(),
InputMessage::MessageWrapper { .. } => {
panic!("attempted to use nested MessageWrapper")
}
},
};
}
@@ -223,6 +226,11 @@ where
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = self.task_client.recv() => {
tracing::trace!("InputMessageListener: Received shutdown");
break;
}
input_msg = self.input_receiver.recv() => match input_msg {
Some(input_msg) => {
self.on_input_message(input_msg).await;
@@ -232,9 +240,7 @@ where
break;
}
},
_ = self.task_client.recv() => {
tracing::trace!("InputMessageListener: Received shutdown");
}
}
}
self.task_client.recv_timeout().await;
@@ -298,29 +298,44 @@ where
let mut sent_notification_listener = self.sent_notification_listener;
let mut action_controller = self.action_controller;
spawn_future(async move {
acknowledgement_listener.run().await;
debug!("The acknowledgement listener has finished execution!");
});
spawn_future!(
async move {
acknowledgement_listener.run().await;
debug!("The acknowledgement listener has finished execution!");
},
"AcknowledgementController::AcknowledgementListener"
);
spawn_future(async move {
input_message_listener.run().await;
debug!("The input listener has finished execution!");
});
spawn_future!(
async move {
input_message_listener.run().await;
debug!("The input listener has finished execution!");
},
"AcknowledgementController::InputMessageListener"
);
spawn_future(async move {
retransmission_request_listener.run(packet_type).await;
debug!("The retransmission request listener has finished execution!");
});
spawn_future!(
async move {
retransmission_request_listener.run(packet_type).await;
debug!("The retransmission request listener has finished execution!");
},
"AcknowledgementController::RetransmissionRequestListener"
);
spawn_future(async move {
sent_notification_listener.run().await;
debug!("The sent notification listener has finished execution!");
});
spawn_future!(
async move {
sent_notification_listener.run().await;
debug!("The sent notification listener has finished execution!");
},
"AcknowledgementController::SentNotificationListener"
);
spawn_future(async move {
action_controller.run().await;
debug!("The controller has finished execution!");
});
spawn_future!(
async move {
action_controller.run().await;
debug!("The controller has finished execution!");
},
"AcknowledgementController::ActionController"
);
}
}
@@ -179,6 +179,11 @@ where
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = self.task_client.recv() => {
tracing::trace!("RetransmissionRequestListener: Received shutdown");
break;
}
timed_out_ack = self.request_receiver.next() => match timed_out_ack {
Some(timed_out_ack) => self.on_retransmission_request(timed_out_ack, packet_type).await,
None => {
@@ -186,9 +191,7 @@ where
break;
}
},
_ = self.task_client.recv() => {
tracing::trace!("RetransmissionRequestListener: Received shutdown");
}
}
}
self.task_client.recv_timeout().await;
@@ -35,6 +35,9 @@ pub enum PreparationError {
#[error(transparent)]
NymTopologyError(#[from] NymTopologyError),
#[error("message wasn't split into any fragments!")]
EmptyFragments,
#[error("message too long for a single SURB, splitting into {fragments} fragments.")]
MessageTooLongForSingleSurb { fragments: usize },
@@ -103,6 +106,9 @@ pub(crate) struct Config {
/// will be routed as usual, to the entry gateway, through three mix nodes, egressing
/// through the exit gateway. If mix hops are disabled, traffic will be routed directly
/// from the entry gateway to the exit gateway, bypassing the mix nodes.
///
/// This overrides the `use_legacy_sphinx_format` setting as reduced mix hops
/// requires use of the updated SURB packet format.
disable_mix_hops: bool,
/// Average delay a data packet is going to get delay at a single mixnode.
@@ -157,8 +163,12 @@ impl Config {
}
/// Configure whether messages senders using this config should use mix hops or not when sending messages.
///
/// This overrides the `use_legacy_sphinx_format` setting as disabled mix hops
/// requires use of the updated SURB packet format.
pub fn disable_mix_hops(mut self, disable_mix_hops: bool) -> Self {
self.disable_mix_hops = disable_mix_hops;
self.use_legacy_sphinx_format = false;
self
}
}
@@ -313,6 +323,16 @@ where
});
}
if fragment.is_empty() {
error!("CRITICAL FAILURE: our split message didn't result in any sendable fragments");
return Err(SurbWrappedPreparationError {
source: PreparationError::EmptyFragments,
returned_surbs: Some(vec![reply_surb]),
});
}
// SAFETY: we just checked we have one fragment
#[allow(clippy::unwrap_used)]
let chunk = fragment.pop().unwrap();
let chunk_clone = chunk.clone();
let prepared_fragment = self
@@ -528,6 +548,7 @@ where
pending_acks.push(pending_ack);
}
drop(topology_permit);
self.insert_pending_acks(pending_acks);
self.forward_messages(real_messages, lane).await;
@@ -650,6 +671,7 @@ where
.zip(reply_surbs.into_iter())
.map(|(fragment, reply_surb)| {
// unwrap here is fine as we know we have a valid topology
#[allow(clippy::unwrap_used)]
self.message_preparer
.prepare_reply_chunk_for_sending(
fragment,
@@ -709,17 +731,21 @@ where
// tells real message sender (with the poisson timer) to send this to the mix network
pub(crate) async fn forward_messages(
&self,
&mut self,
messages: Vec<RealMessage>,
transmission_lane: TransmissionLane,
) {
if let Err(err) = self
.real_message_sender
.send((messages, transmission_lane))
.await
{
if !self.task_client.is_shutdown_poll() {
error!("Failed to forward messages to the real message sender: {err}");
tokio::select! {
biased;
_ = self.task_client.recv() => {
trace!("received shutdown while attempting to forward mixnet messages");
}
sending_res = self.real_message_sender.send((messages, transmission_lane)) => {
if sending_res.is_err() {
error!(
"failed to forward mixnet messages due to closed channel (outside of shutdown!)"
);
}
}
}
}
@@ -224,14 +224,20 @@ impl RealMessagesController<OsRng> {
let ack_control = self.ack_control;
let mut reply_control = self.reply_control;
spawn_future(async move {
out_queue_control.run().await;
debug!("The out queue controller has finished execution!");
});
spawn_future(async move {
reply_control.run().await;
debug!("The reply controller has finished execution!");
});
spawn_future!(
async move {
out_queue_control.run().await;
debug!("The out queue controller has finished execution!");
},
"RealMessagesController::OutQueueControl)"
);
spawn_future!(
async move {
reply_control.run().await;
debug!("The reply controller has finished execution!");
},
"RealMessagesController::ReplyController"
);
ack_control.start(packet_type);
}
@@ -202,7 +202,7 @@ where
// well technically the message was not sent just yet, but now it's up to internal
// queues and client load rather than the required delay. So realistically we can treat
// whatever is about to happen as negligible additional delay.
trace!("{} is about to get sent to the mixnet", frag_id);
trace!("{frag_id} is about to get sent to the mixnet");
if let Err(err) = self.sent_notifier.unbounded_send(frag_id) {
error!("Failed to notify about sent message: {err}");
}
@@ -249,6 +249,8 @@ where
}
};
// SAFETY: our topology must be valid at this point
#[allow(clippy::expect_used)]
(
generate_loop_cover_packet(
&mut self.rng,
@@ -278,17 +280,33 @@ where
}
};
if let Err(err) = self.mix_tx.send(vec![next_message]).await {
if !self.task_client.is_shutdown_poll() {
tracing::error!("Failed to send: {err}");
let sending_res = tokio::select! {
biased;
_ = self.task_client.recv() => {
trace!("received shutdown signal while attempting to send mix message");
return
}
sending_res = self.mix_tx.send(vec![next_message]) => {
sending_res
}
};
match sending_res {
Err(_) => {
if !self.task_client.is_shutdown_poll() {
tracing::error!(
"failed to send mixnet packet due to closed channel (outside of shutdown!)"
);
}
}
Ok(_) => {
let event = if fragment_id.is_some() {
PacketStatisticsEvent::RealPacketSent(packet_size)
} else {
PacketStatisticsEvent::CoverPacketSent(packet_size)
};
self.stats_tx.report(event.into());
}
} else {
let event = if fragment_id.is_some() {
PacketStatisticsEvent::RealPacketSent(packet_size)
} else {
PacketStatisticsEvent::CoverPacketSent(packet_size)
};
self.stats_tx.report(event.into());
}
// notify ack controller about sending our message only after we actually managed to push it
@@ -439,6 +457,8 @@ where
tracing::trace!("handling real_messages: size: {}", real_messages.len());
self.transmission_buffer.store(&conn_id, real_messages);
// SAFETY: we just stored the message
#[allow(clippy::expect_used)]
let real_next = self.pop_next_message().expect("Just stored one");
Poll::Ready(Some(StreamMessage::Real(Box::new(real_next))))
@@ -487,6 +507,8 @@ where
// First store what we got for the given connection id
self.transmission_buffer.store(&conn_id, real_messages);
// SAFETY: we just stored the message
#[allow(clippy::expect_used)]
let real_next = self.pop_next_message().expect("we just added one");
Poll::Ready(Some(StreamMessage::Real(Box::new(real_next))))
@@ -164,11 +164,11 @@ impl SendingDelayController {
self.current_multiplier()
);
if self.current_multiplier() > 0 {
tracing::debug!("{}", status_str);
tracing::debug!("{status_str}");
} else if self.current_multiplier() > 1 {
tracing::info!("{}", status_str);
tracing::info!("{status_str}");
} else if self.current_multiplier() > 2 {
tracing::warn!("{}", status_str);
tracing::warn!("{status_str}");
}
self.time_when_logged_about_elevated_multiplier = now;
}
@@ -198,6 +198,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
}
}
#[allow(clippy::panic)]
async fn disconnect_sender(&mut self) {
let mut guard = self.inner.lock().await;
if guard.message_sender.is_none() {
@@ -208,6 +209,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
guard.message_sender = None;
}
#[allow(clippy::panic)]
async fn connect_sender(&mut self, sender: ReconstructedMessagesSender) {
let mut guard = self.inner.lock().await;
if guard.message_sender.is_some() {
@@ -221,10 +223,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
let stored_messages = std::mem::take(&mut guard.messages);
if !stored_messages.is_empty() {
if let Err(err) = sender.unbounded_send(stored_messages) {
error!(
"The sender channel we just received is already invalidated - {:?}",
err
);
error!("The sender channel we just received is already invalidated - {err:?}");
// put the values back to the buffer
// the returned error has two fields: err: SendError and val: T,
// where val is the value that was failed to get sent;
@@ -602,14 +601,20 @@ impl<R: MessageReceiver + Clone + Send + 'static> ReceivedMessagesBufferControll
let mut fragmented_message_receiver = self.fragmented_message_receiver;
let mut request_receiver = self.request_receiver;
spawn_future(async move {
match fragmented_message_receiver.run().await {
Ok(_) => {}
Err(e) => error!("{e}"),
}
});
spawn_future(async move {
request_receiver.run().await;
});
spawn_future!(
async move {
match fragmented_message_receiver.run().await {
Ok(_) => {}
Err(e) => error!("{e}"),
}
},
"ReceivedMessagesBufferController::FragmentedMessageReceiver"
);
spawn_future!(
async move {
request_receiver.run().await;
},
"ReceivedMessagesBufferController::RequestReceiver"
);
}
}
@@ -155,8 +155,9 @@ where
data: Vec<Arc<PendingAcknowledgement>>,
) {
trace!("re-inserting pending retransmissions for {recipient}");
// the underlying entry MUST exist as we've just got data from there
// SAFETY: the underlying entry MUST exist as we've just got data from there
// and we hold a mut reference
#[allow(clippy::expect_used)]
let map_entry = &mut self
.surb_senders
.get_mut(recipient)
@@ -429,6 +430,7 @@ where
.pop_at_most_n_next_messages_at_random(amount)
}
#[allow(clippy::panic)]
async fn try_clear_pending_queue(&mut self, target: AnonymousSenderTag) {
trace!("trying to clear pending queue");
let available_surbs = self.surbs_storage.available_surbs(&target);
@@ -93,7 +93,7 @@ impl StatisticsControl {
None,
);
if let Err(err) = self.report_tx.send(report_message).await {
tracing::error!("Failed to report client stats: {:?}", err);
tracing::error!("Failed to report client stats: {err:?}");
} else {
self.stats.reset();
}
@@ -165,9 +165,12 @@ impl StatisticsControl {
}
pub(crate) fn start(mut self) {
spawn_future(async move {
self.run().await;
})
spawn_future!(
async move {
self.run().await;
},
"StatisticsControl"
)
}
pub(crate) fn create_and_start(
@@ -126,7 +126,7 @@ impl TopologyAccessor {
.map(|p| p.topology.clone())
}
pub async fn current_route_provider(&self) -> Option<RwLockReadGuard<NymRouteProvider>> {
pub async fn current_route_provider(&self) -> Option<RwLockReadGuard<'_, NymRouteProvider>> {
let provider = self.inner.topology.read().await;
if provider.topology.is_empty() {
None
@@ -145,36 +145,39 @@ impl TopologyRefresher {
}
pub fn start(mut self) {
spawn_future(async move {
debug!("Started TopologyRefresher with graceful shutdown support");
spawn_future!(
async move {
debug!("Started TopologyRefresher with graceful shutdown support");
#[cfg(not(target_arch = "wasm32"))]
let mut interval = tokio_stream::wrappers::IntervalStream::new(tokio::time::interval(
self.refresh_rate,
));
#[cfg(not(target_arch = "wasm32"))]
let mut interval = tokio_stream::wrappers::IntervalStream::new(
tokio::time::interval(self.refresh_rate),
);
#[cfg(target_arch = "wasm32")]
let mut interval =
gloo_timers::future::IntervalStream::new(self.refresh_rate.as_millis() as u32);
#[cfg(target_arch = "wasm32")]
let mut interval =
gloo_timers::future::IntervalStream::new(self.refresh_rate.as_millis() as u32);
// We already have an initial topology, so no need to refresh it immediately.
// My understanding is that js setInterval does not fire immediately, so it's not
// needed there.
#[cfg(not(target_arch = "wasm32"))]
interval.next().await;
// We already have an initial topology, so no need to refresh it immediately.
// My understanding is that js setInterval does not fire immediately, so it's not
// needed there.
#[cfg(not(target_arch = "wasm32"))]
interval.next().await;
while !self.task_client.is_shutdown() {
tokio::select! {
_ = interval.next() => {
self.try_refresh().await;
},
_ = self.task_client.recv() => {
tracing::trace!("TopologyRefresher: Received shutdown");
},
while !self.task_client.is_shutdown() {
tokio::select! {
_ = interval.next() => {
self.try_refresh().await;
},
_ = self.task_client.recv() => {
tracing::trace!("TopologyRefresher: Received shutdown");
},
}
}
}
self.task_client.recv_timeout().await;
tracing::debug!("TopologyRefresher: Exiting");
})
self.task_client.recv_timeout().await;
tracing::debug!("TopologyRefresher: Exiting");
},
"TopologyRefresher"
)
}
}
@@ -218,7 +218,7 @@ impl<T> TransmissionBuffer<T> {
};
let msg = self.pop_front_from_lane(&lane)?;
tracing::trace!("picking to send from lane: {:?}", lane);
tracing::trace!("picking to send from lane: {lane:?}");
Some((lane, msg))
}
+15 -1
View File
@@ -7,7 +7,9 @@ use nym_gateway_client::error::GatewayClientError;
use nym_topology::node::RoutingNodeError;
use nym_topology::{NodeId, NymTopologyError};
use nym_validator_client::nym_api::error::NymAPIError;
use nym_validator_client::nyxd::error::NyxdError;
use nym_validator_client::ValidatorClientError;
use rand::distributions::WeightedError;
use std::error::Error;
use std::path::PathBuf;
@@ -230,7 +232,19 @@ pub enum ClientCoreError {
UnexpectedKeyUpgrade { gateway_id: String },
#[error("failed to derive keys from master key")]
HkdfDerivationError {},
HkdfDerivationError,
#[error("missing url for constructing RPC client")]
RpcClientMissingUrl,
#[error("provided nym network details were malformed: {source}")]
InvalidNetworkDetails { source: NyxdError },
#[error("failed to construct RPC client: {source}")]
RpcClientCreationFailure { source: NyxdError },
#[error("failed to select valid gateway due to incomputable latency")]
GatewaySelectionFailure { source: WeightedError },
}
impl From<tungstenite::Error> for ClientCoreError {
+5 -5
View File
@@ -113,7 +113,7 @@ pub async fn gateways_for_init<R: Rng>(
.nodes;
info!("nym api reports {} gateways", gateways.len());
tracing::trace!("Gateways: {:#?}", gateways);
tracing::trace!("Gateways: {gateways:#?}");
// filter out gateways below minimum performance and ones that could operate as a mixnode
// (we don't want instability)
@@ -124,7 +124,7 @@ pub async fn gateways_for_init<R: Rng>(
.filter_map(|gateway| gateway.try_into().ok())
.collect::<Vec<_>>();
tracing::debug!("After checking validity: {}", valid_gateways.len());
tracing::trace!("Valid gateways: {:#?}", valid_gateways);
tracing::trace!("Valid gateways: {valid_gateways:#?}");
tracing::info!(
"and {} after validity and performance filtering",
@@ -148,7 +148,7 @@ async fn connect(endpoint: &str) -> Result<WsConn, ClientCoreError> {
JSWebsocket::new(endpoint).map_err(|_| ClientCoreError::GatewayJsConnectionFailure)
}
async fn measure_latency<G>(gateway: &G) -> Result<GatewayWithLatency<G>, ClientCoreError>
async fn measure_latency<G>(gateway: &G) -> Result<GatewayWithLatency<'_, G>, ClientCoreError>
where
G: ConnectableGateway,
{
@@ -245,7 +245,7 @@ pub async fn choose_gateway_by_latency<R: Rng, G: ConnectableGateway + Clone>(
let gateways_with_latency = gateways_with_latency.lock().await;
let chosen = gateways_with_latency
.choose_weighted(rng, |item| 1. / item.latency.as_secs_f32())
.expect("invalid selection weight!");
.map_err(|source| ClientCoreError::GatewaySelectionFailure { source })?;
info!(
"chose gateway {} with average latency of {:?}",
@@ -289,7 +289,7 @@ pub(super) fn get_specified_gateway(
gateways: &[RoutingNode],
must_use_tls: bool,
) -> Result<RoutingNode, ClientCoreError> {
tracing::debug!("Requesting specified gateway: {}", gateway_identity);
tracing::debug!("Requesting specified gateway: {gateway_identity}");
let user_gateway = ed25519::PublicKey::from_base58_string(gateway_identity)
.map_err(ClientCoreError::UnableToCreatePublicKeyFromGatewayId)?;
+38 -2
View File
@@ -18,18 +18,54 @@ pub use nym_topology::{
};
#[cfg(target_arch = "wasm32")]
pub(crate) fn spawn_future<F>(future: F)
pub fn spawn_future<F>(future: F)
where
F: Future<Output = ()> + 'static,
{
wasm_bindgen_futures::spawn_local(future);
}
// TODO: expose similar API to the rest of the codebase,
// perhaps with some simple trait for a task to define its name
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn spawn_future<F>(future: F)
#[track_caller]
pub fn spawn_future<F>(future: F)
where
F: Future + Send + 'static,
F::Output: Send + 'static,
{
tokio::spawn(future);
}
#[cfg(not(target_arch = "wasm32"))]
#[track_caller]
pub fn spawn_named_future<F>(future: F, name: &str)
where
F: Future + Send + 'static,
F::Output: Send + 'static,
{
cfg_if::cfg_if! {if #[cfg(tokio_unstable)] {
#[allow(clippy::expect_used)]
tokio::task::Builder::new().name(name).spawn(future).expect("failed to spawn future");
} else {
let _ = name;
tracing::debug!(r#"the underlying binary hasn't been built with `RUSTFLAGS="--cfg tokio_unstable"` - the future naming won't do anything"#);
spawn_future(future);
}}
}
#[macro_export]
macro_rules! spawn_future {
($future:expr) => {{
$crate::spawn_future($future)
}};
($future:expr, $name:expr) => {{
cfg_if::cfg_if! {if #[cfg(not(target_arch = "wasm32"))] {
$crate::spawn_named_future($future, $name)
} else {
let _ = $name;
$crate::spawn_future($future)
}}
}};
}
+12 -1
View File
@@ -17,15 +17,26 @@ nym-crypto = { path = "../../crypto", optional = true, default-features = false
nym-sphinx = { path = "../../nymsphinx" }
nym-task = { path = "../../task" }
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio]
workspace = true
features = ["fs"]
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.sqlx]
workspace = true
features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate", "time"]
optional = true
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.sqlx-pool-guard]
path = "../../../sqlx-pool-guard"
[build-dependencies]
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
sqlx = { workspace = true, features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate"] }
sqlx = { workspace = true, features = [
"runtime-tokio-rustls",
"sqlite",
"macros",
"migrate",
] }
[features]
fs-surb-storage = ["sqlx", "nym-crypto", "nym-crypto/hashing"]
@@ -1,8 +1,7 @@
// Copyright 2022 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use std::io;
use std::path::PathBuf;
use std::{io, path::PathBuf};
use thiserror::Error;
#[derive(Debug, Error)]
@@ -30,7 +29,6 @@ pub enum StorageError {
#[error("failed to perform sqlx migration: {source}")]
MigrationError {
#[source]
#[from]
source: sqlx::migrate::MigrateError,
},
@@ -43,7 +41,6 @@ pub enum StorageError {
#[error("failed to run the SQL query: {source}")]
QueryError {
#[source]
#[from]
source: sqlx::error::Error,
},
@@ -13,9 +13,11 @@ use std::path::Path;
use time::OffsetDateTime;
use tracing::{error, info};
use sqlx_pool_guard::SqlitePoolGuard;
#[derive(Debug, Clone)]
pub struct StorageManager {
pub connection_pool: sqlx::SqlitePool,
connection_pool: SqlitePoolGuard,
}
// all SQL goes here
@@ -47,11 +49,14 @@ impl StorageManager {
}
};
let connection_pool = SqlitePoolGuard::new(connection_pool);
if let Err(err) = sqlx::migrate!("./fs_surbs_migrations")
.run(&connection_pool)
.run(&*connection_pool)
.await
{
error!("Failed to initialize SQLx database: {err}");
connection_pool.close().await;
return Err(err.into());
}
@@ -59,10 +64,15 @@ impl StorageManager {
Ok(StorageManager { connection_pool })
}
/// Close connection pool waiting for all connections to be closed.
pub async fn close_pool(&self) {
self.connection_pool.close().await;
}
#[allow(dead_code)]
pub async fn status_table_exists(&self) -> Result<bool, sqlx::Error> {
sqlx::query!("SELECT name FROM sqlite_master WHERE type='table' AND name='status'")
.fetch_optional(&self.connection_pool)
.fetch_optional(&*self.connection_pool)
.await
.map(|r| r.is_some())
}
@@ -71,28 +81,28 @@ impl StorageManager {
sqlx::query!(
"INSERT INTO status(flush_in_progress, previous_flush, client_in_use) VALUES (0, 0, 1)"
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn get_flush_status(&self) -> Result<bool, sqlx::Error> {
sqlx::query!("SELECT flush_in_progress FROM status;")
.fetch_one(&self.connection_pool)
.fetch_one(&*self.connection_pool)
.await
.map(|r| r.flush_in_progress > 0)
}
pub async fn set_previous_flush(&self, timestamp: OffsetDateTime) -> Result<(), sqlx::Error> {
sqlx::query!("UPDATE status SET previous_flush = ?", timestamp)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn get_previous_flush_time(&self) -> Result<OffsetDateTime, sqlx::Error> {
sqlx::query!(r#"SELECT previous_flush AS "previous_flush: OffsetDateTime" FROM status"#)
.fetch_one(&self.connection_pool)
.fetch_one(&*self.connection_pool)
.await
.map(|r| r.previous_flush)
}
@@ -100,14 +110,14 @@ impl StorageManager {
pub async fn set_flush_status(&self, in_progress: bool) -> Result<(), sqlx::Error> {
let in_progress_int = i64::from(in_progress);
sqlx::query!("UPDATE status SET flush_in_progress = ?", in_progress_int)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn get_client_in_use_status(&self) -> Result<bool, sqlx::Error> {
sqlx::query!("SELECT client_in_use FROM status;")
.fetch_one(&self.connection_pool)
.fetch_one(&*self.connection_pool)
.await
.map(|r| r.client_in_use > 0)
}
@@ -115,21 +125,21 @@ impl StorageManager {
pub async fn set_client_in_use_status(&self, in_use: bool) -> Result<(), sqlx::Error> {
let in_use_int = i64::from(in_use);
sqlx::query!("UPDATE status SET client_in_use = ?", in_use_int)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn delete_all_reply_keys(&self) -> Result<(), sqlx::Error> {
sqlx::query!("DELETE FROM reply_key;")
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn get_reply_keys(&self) -> Result<Vec<StoredReplyKey>, sqlx::Error> {
sqlx::query_as("SELECT * FROM reply_key;")
.fetch_all(&self.connection_pool)
.fetch_all(&*self.connection_pool)
.await
}
@@ -145,14 +155,14 @@ impl StorageManager {
stored_reply_key.reply_key,
stored_reply_key.sent_at
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub async fn get_surb_senders(&self) -> Result<Vec<StoredSurbSender>, sqlx::Error> {
sqlx::query_as("SELECT * FROM reply_surb_sender;")
.fetch_all(&self.connection_pool)
.fetch_all(&*self.connection_pool)
.await
}
@@ -167,7 +177,7 @@ impl StorageManager {
stored_surb_sender.tag,
stored_surb_sender.last_sent
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?
.last_insert_rowid();
Ok(id)
@@ -185,17 +195,17 @@ impl StorageManager {
"#,
sender_id
)
.fetch_all(&self.connection_pool)
.fetch_all(&*self.connection_pool)
.await
}
pub async fn delete_all_reply_surb_data(&self) -> Result<(), sqlx::Error> {
sqlx::query!("DELETE FROM reply_surb;")
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
sqlx::query!("DELETE FROM reply_surb_sender;")
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
@@ -213,7 +223,7 @@ impl StorageManager {
stored_reply_surb.reply_surb,
stored_reply_surb.encoded_key_rotation
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
@@ -227,7 +237,7 @@ impl StorageManager {
SELECT min_reply_surb_threshold as "min_reply_surb_threshold: u32", max_reply_surb_threshold as "max_reply_surb_threshold: u32" FROM reply_surb_storage_metadata;
"#,
)
.fetch_one(&self.connection_pool)
.fetch_one(&*self.connection_pool)
.await
}
@@ -241,7 +251,7 @@ impl StorageManager {
"#,
metadata.min_reply_surb_threshold,
metadata.max_reply_surb_threshold,
).execute(&self.connection_pool).await?;
).execute(&*self.connection_pool).await?;
Ok(())
}
}
@@ -1,15 +1,16 @@
// Copyright 2022 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::backend::fs_backend::manager::StorageManager;
use crate::backend::fs_backend::models::{
ReplySurbStorageMetadata, StoredReplyKey, StoredReplySurb, StoredSurbSender,
use crate::{
backend::fs_backend::{
manager::StorageManager,
models::{ReplySurbStorageMetadata, StoredReplyKey, StoredReplySurb, StoredSurbSender},
},
surb_storage::ReceivedReplySurbs,
CombinedReplyStorage, ReceivedReplySurbsMap, ReplyStorageBackend, SentReplyKeys,
};
use crate::surb_storage::ReceivedReplySurbs;
use crate::{CombinedReplyStorage, ReceivedReplySurbsMap, ReplyStorageBackend, SentReplyKeys};
use async_trait::async_trait;
use nym_sphinx::anonymous_replies::requests::AnonymousSenderTag;
use std::fs;
use std::path::{Path, PathBuf};
use time::OffsetDateTime;
use tracing::{error, info, warn};
@@ -39,15 +40,17 @@ impl Backend {
}
let manager = StorageManager::init(database_path, true).await?;
manager.create_status_table().await?;
let backend = Backend {
temporary_old_path: None,
database_path: owned_path,
manager,
};
Ok(backend)
match manager.create_status_table().await {
Ok(()) => Ok(Backend {
temporary_old_path: None,
database_path: owned_path,
manager,
}),
Err(err) => {
manager.close_pool().await;
Err(err.into())
}
}
}
pub async fn try_load<P: AsRef<Path>>(database_path: P) -> Result<Self, StorageError> {
@@ -59,7 +62,25 @@ impl Backend {
}
let manager = StorageManager::init(database_path, false).await?;
match Self::try_load_inner(&manager).await {
Ok(()) => Ok(Backend {
temporary_old_path: None,
database_path: owned_path,
manager,
}),
Err(e) => {
manager.close_pool().await;
Err(e)
}
}
}
/// Gracefully close sqlite connection pool and drop backend.
pub async fn shutdown(self) {
self.manager.close_pool().await
}
async fn try_load_inner(manager: &StorageManager) -> Result<(), StorageError> {
// the database flush wasn't fully finished and thus the data is in inconsistent state
// (we don't really know what's properly saved or what's not)
if manager.get_flush_status().await? {
@@ -104,20 +125,11 @@ impl Backend {
manager.delete_all_reply_keys().await?;
}
Ok(Backend {
temporary_old_path: None,
database_path: owned_path,
// manager: StorageManagerState::Storage(manager),
manager,
})
}
async fn close_pool(&mut self) {
self.manager.connection_pool.close().await;
Ok(())
}
async fn rotate(&mut self) -> Result<(), StorageError> {
self.close_pool().await;
self.manager.close_pool().await;
let new_extension = if let Some(existing_extension) =
self.database_path.extension().and_then(|ext| ext.to_str())
@@ -130,7 +142,8 @@ impl Backend {
let mut temp_old = self.database_path.clone();
temp_old.set_extension(new_extension);
fs::rename(&self.database_path, &temp_old)
tokio::fs::rename(&self.database_path, &temp_old)
.await
.map_err(|err| StorageError::DatabaseRenameError { source: err })?;
self.manager = StorageManager::init(&self.database_path, true).await?;
self.manager.create_status_table().await?;
@@ -139,9 +152,10 @@ impl Backend {
Ok(())
}
fn remove_old(&mut self) -> Result<(), StorageError> {
async fn remove_old(&mut self) -> Result<(), StorageError> {
if let Some(old_path) = self.temporary_old_path.take() {
fs::remove_file(old_path)
tokio::fs::remove_file(old_path)
.await
.map_err(|err| StorageError::DatabaseOldFileRemoveError { source: err })
} else {
warn!("the old database file doesn't seem to exist!");
@@ -302,7 +316,7 @@ impl ReplyStorageBackend for Backend {
self.dump_reply_surb_storage_metadata(surbs_ref).await?;
self.dump_reply_surbs(surbs_ref).await?;
self.remove_old()?;
self.remove_old().await?;
self.end_storage_flush().await
}
@@ -37,7 +37,6 @@ where
self.backend.load_surb_storage(surb_freshness_cutoff).await
}
// this will have to get enabled after merging develop
pub async fn flush_on_shutdown(
mut self,
mem_state: CombinedReplyStorage,
@@ -54,7 +53,6 @@ where
shutdown.recv().await;
info!("PersistentReplyStorage is flushing all reply-related data to underlying storage");
info!("you MUST NOT forcefully shutdown now or you risk data corruption!");
if let Err(err) = self.backend.flush_surb_storage(&mem_state).await {
error!("failed to flush our reply-related data to the persistent storage: {err}")
} else {
@@ -201,7 +201,7 @@ impl<C, St> GatewayClient<C, St> {
#[cfg(not(target_arch = "wasm32"))]
pub async fn establish_connection(&mut self) -> Result<(), GatewayClientError> {
debug!(
"Attemting to establish connection to gateway at: {}",
"Attempting to establish connection to gateway at: {}",
self.gateway_address
);
let (ws_stream, _) = connect_async(
@@ -272,7 +272,7 @@ impl<C, St> GatewayClient<C, St> {
) -> Result<(), GatewayClientError> {
if let Some(shared_key) = self.shared_key() {
let encrypted = message.encrypt(&*shared_key)?;
Box::pin(self.send_websocket_message(encrypted)).await?;
Box::pin(self.send_websocket_message_without_response(encrypted)).await?;
Ok(())
} else {
Err(GatewayClientError::ConnectionInInvalidState)
@@ -330,9 +330,80 @@ impl<C, St> GatewayClient<C, St> {
}
}
/// Attempt to send a websocket message to the gateway without waiting for any response
async fn send_websocket_message_without_response(
&mut self,
msg: impl Into<Message>,
) -> Result<(), GatewayClientError> {
match self.connection {
SocketState::Available(ref mut conn) => Ok(conn.send(msg.into()).await?),
SocketState::PartiallyDelegated(ref mut partially_delegated) => {
if let Err(err) = partially_delegated.send_without_response(msg.into()).await {
error!("failed to send message without response - {err}...");
// we must ensure we do not leave the task still active
if let Err(err) = self.recover_socket_connection().await {
error!("... and the delegated stream has also errored out - {err}")
}
Err(err)
} else {
Ok(())
}
}
SocketState::NotConnected => Err(GatewayClientError::ConnectionNotEstablished),
_ => Err(GatewayClientError::ConnectionInInvalidState),
}
}
// A very nasty hack due to lack of id tags on messages - send a non-sphinx packet websocket
// message and wait until first non 'Send' response within timeout
pub async fn send_websocket_message_with_non_send_response(
&mut self,
msg: impl Into<Message>,
) -> Result<ServerResponse, GatewayClientError> {
let should_restart_mixnet_listener = if self.connection.is_partially_delegated() {
self.recover_socket_connection().await?;
true
} else {
false
};
let conn = match self.connection {
SocketState::Available(ref mut conn) => conn,
SocketState::NotConnected => return Err(GatewayClientError::ConnectionNotEstablished),
_ => return Err(GatewayClientError::ConnectionInInvalidState),
};
conn.send(msg.into()).await?;
let timeout = sleep(self.cfg.connection.response_timeout_duration);
tokio::pin!(timeout);
let response = loop {
tokio::select! {
_ = &mut timeout => {
break Err(GatewayClientError::Timeout);
}
// note: the below will also listen for shutdown signals
msg = self.read_control_response() => {
match msg {
Ok(res) => if !res.is_send() {
break Ok(res);
},
Err(err) => break Err(err),
}
}
}
};
if should_restart_mixnet_listener {
self.start_listening_for_mixnet_messages()?;
}
response
}
/// Attempt to send a websocket message to the gateway and wait until we receive a response.
// If we want to send a message (with response), we need to have a full control over the socket,
// as we need to be able to write the request and read the subsequent response
pub async fn send_websocket_message(
pub async fn send_websocket_message_with_response(
&mut self,
msg: impl Into<Message>,
) -> Result<ServerResponse, GatewayClientError> {
@@ -387,29 +458,6 @@ impl<C, St> GatewayClient<C, St> {
}
}
async fn send_websocket_message_without_response(
&mut self,
msg: Message,
) -> Result<(), GatewayClientError> {
match self.connection {
SocketState::Available(ref mut conn) => Ok(conn.send(msg).await?),
SocketState::PartiallyDelegated(ref mut partially_delegated) => {
if let Err(err) = partially_delegated.send_without_response(msg).await {
error!("failed to send message without response - {err}...");
// we must ensure we do not leave the task still active
if let Err(err) = self.recover_socket_connection().await {
error!("... and the delegated stream has also errored out - {err}")
}
Err(err)
} else {
Ok(())
}
}
SocketState::NotConnected => Err(GatewayClientError::ConnectionNotEstablished),
_ => Err(GatewayClientError::ConnectionInInvalidState),
}
}
fn check_gateway_protocol(
&self,
gateway_protocol: Option<u8>,
@@ -535,7 +583,10 @@ impl<C, St> GatewayClient<C, St> {
.encrypt(legacy_key)?;
info!("sending upgrade request and awaiting the acknowledgement back");
let (ciphertext, nonce) = match self.send_websocket_message(upgrade_request).await? {
let (ciphertext, nonce) = match self
.send_websocket_message_with_response(upgrade_request)
.await?
{
ServerResponse::EncryptedResponse { ciphertext, nonce } => (ciphertext, nonce),
ServerResponse::Error { message } => {
return Err(GatewayClientError::GatewayError(message))
@@ -567,7 +618,7 @@ impl<C, St> GatewayClient<C, St> {
&mut self,
msg: ClientControlRequest,
) -> Result<(), GatewayClientError> {
match self.send_websocket_message(msg).await? {
match self.send_websocket_message_with_response(msg).await? {
ServerResponse::Authenticate {
protocol_version,
status,
@@ -717,13 +768,16 @@ impl<C, St> GatewayClient<C, St> {
}
}
/// Attempt to retrieve the currently supported gateway protocol version of the remote.
pub async fn get_gateway_protocol(&mut self) -> Result<u8, GatewayClientError> {
if !self.connection.is_established() {
return Err(GatewayClientError::ConnectionNotEstablished);
}
match self
.send_websocket_message(ClientControlRequest::SupportedProtocol {})
.send_websocket_message_with_non_send_response(
ClientControlRequest::SupportedProtocol {},
)
.await?
{
ServerResponse::SupportedProtocol { version } => Ok(version),
@@ -740,7 +794,10 @@ impl<C, St> GatewayClient<C, St> {
credential,
self.shared_key.as_ref().unwrap(),
)?;
let bandwidth_remaining = match self.send_websocket_message(msg).await? {
let bandwidth_remaining = match self
.send_websocket_message_with_non_send_response(msg)
.await?
{
ServerResponse::Bandwidth { available_total } => Ok(available_total),
ServerResponse::Error { message } => Err(GatewayClientError::GatewayError(message)),
ServerResponse::TypedError { error } => {
@@ -758,7 +815,10 @@ impl<C, St> GatewayClient<C, St> {
async fn try_claim_testnet_bandwidth(&mut self) -> Result<(), GatewayClientError> {
let msg = ClientControlRequest::ClaimFreeTestnetBandwidth;
let bandwidth_remaining = match self.send_websocket_message(msg).await? {
let bandwidth_remaining = match self
.send_websocket_message_with_non_send_response(msg)
.await?
{
ServerResponse::Bandwidth { available_total } => Ok(available_total),
ServerResponse::Error { message } => Err(GatewayClientError::GatewayError(message)),
other => Err(GatewayClientError::UnexpectedResponse { name: other.name() }),
@@ -337,7 +337,7 @@ impl PartiallyDelegatedHandle {
// check if the split stream didn't error out
let receive_res = stream_receiver
.try_recv()
.expect("stream sender was somehow dropped without sending anything!");
.map_err(|_| GatewayClientError::ConnectionAbruptlyClosed)?;
if let Some(res) = receive_res {
let _res = res?;
@@ -19,6 +19,7 @@ nym-vesting-contract-common = { path = "../../cosmwasm-smart-contracts/vesting-c
nym-ecash-contract-common = { path = "../../cosmwasm-smart-contracts/ecash-contract" }
nym-multisig-contract-common = { path = "../../cosmwasm-smart-contracts/multisig-contract" }
nym-group-contract-common = { path = "../../cosmwasm-smart-contracts/group-contract" }
nym-performance-contract-common = { path = "../../cosmwasm-smart-contracts/nym-performance-contract" }
nym-serde-helpers = { path = "../../serde-helpers", features = ["hex", "base64"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
@@ -719,10 +719,11 @@ impl NymApiClient {
pub async fn partial_expiration_date_signatures(
&self,
expiration_date: Option<Date>,
epoch_id: Option<EpochId>,
) -> Result<PartialExpirationDateSignatureResponse, ValidatorClientError> {
Ok(self
.nym_api
.partial_expiration_date_signatures(expiration_date)
.partial_expiration_date_signatures(expiration_date, epoch_id)
.await?)
}
@@ -739,10 +740,11 @@ impl NymApiClient {
pub async fn global_expiration_date_signatures(
&self,
expiration_date: Option<Date>,
epoch_id: Option<EpochId>,
) -> Result<AggregatedExpirationDateSignatureResponse, ValidatorClientError> {
Ok(self
.nym_api
.global_expiration_date_signatures(expiration_date)
.global_expiration_date_signatures(expiration_date, epoch_id)
.await?)
}
@@ -6,16 +6,18 @@ use crate::nym_api::routes::{ecash, CORE_STATUS_COUNT, SINCE_ARG};
use async_trait::async_trait;
use nym_api_requests::ecash::models::{
AggregatedCoinIndicesSignatureResponse, AggregatedExpirationDateSignatureResponse,
BatchRedeemTicketsBody, EcashBatchTicketRedemptionResponse, EcashTicketVerificationResponse,
IssuedTicketbooksChallengeCommitmentRequest, IssuedTicketbooksChallengeCommitmentResponse,
IssuedTicketbooksDataRequest, IssuedTicketbooksDataResponse, IssuedTicketbooksForCountResponse,
IssuedTicketbooksForResponse, VerifyEcashTicketBody,
BatchRedeemTicketsBody, EcashBatchTicketRedemptionResponse, EcashSignerStatusResponse,
EcashTicketVerificationResponse, IssuedTicketbooksChallengeCommitmentRequest,
IssuedTicketbooksChallengeCommitmentResponse, IssuedTicketbooksDataRequest,
IssuedTicketbooksDataResponse, IssuedTicketbooksForCountResponse, IssuedTicketbooksForResponse,
VerifyEcashTicketBody,
};
use nym_api_requests::ecash::VerificationKeyResponse;
use nym_api_requests::models::{
AnnotationResponse, ApiHealthResponse, BinaryBuildInformationOwned, ChainStatusResponse,
KeyRotationInfoResponse, LegacyDescribedMixNode, NodePerformanceResponse, NodeRefreshBody,
NymNodeDescription, PerformanceHistoryResponse, RewardedSetResponse,
AnnotationResponse, ApiHealthResponse, BinaryBuildInformationOwned, ChainBlocksStatusResponse,
ChainStatusResponse, KeyRotationInfoResponse, LegacyDescribedMixNode, NodePerformanceResponse,
NodeRefreshBody, NymNodeDescription, PerformanceHistoryResponse, RewardedSetResponse,
SignerInformationResponse,
};
use nym_api_requests::nym_nodes::{
NodesByAddressesRequestBody, NodesByAddressesResponse, PaginatedCachedNodesResponseV1,
@@ -1101,8 +1103,9 @@ pub trait NymApiClientExt: ApiClient {
async fn partial_expiration_date_signatures(
&self,
expiration_date: Option<Date>,
epoch_id: Option<EpochId>,
) -> Result<PartialExpirationDateSignatureResponse, NymAPIError> {
let params = match expiration_date {
let mut params = match expiration_date {
None => Vec::new(),
Some(exp) => vec![(
ecash::EXPIRATION_DATE_PARAM,
@@ -1110,6 +1113,10 @@ pub trait NymApiClientExt: ApiClient {
)],
};
if let Some(epoch_id) = epoch_id {
params.push((ecash::EPOCH_ID_PARAM, epoch_id.to_string()));
}
self.get_json(
&[
routes::V1_API_VERSION,
@@ -1146,8 +1153,9 @@ pub trait NymApiClientExt: ApiClient {
async fn global_expiration_date_signatures(
&self,
expiration_date: Option<Date>,
epoch_id: Option<EpochId>,
) -> Result<AggregatedExpirationDateSignatureResponse, NymAPIError> {
let params = match expiration_date {
let mut params = match expiration_date {
None => Vec::new(),
Some(exp) => vec![(
ecash::EXPIRATION_DATE_PARAM,
@@ -1155,6 +1163,10 @@ pub trait NymApiClientExt: ApiClient {
)],
};
if let Some(epoch_id) = epoch_id {
params.push((ecash::EPOCH_ID_PARAM, epoch_id.to_string()));
}
self.get_json(
&[
routes::V1_API_VERSION,
@@ -1331,6 +1343,22 @@ pub trait NymApiClientExt: ApiClient {
.await
}
async fn get_chain_blocks_status(&self) -> Result<ChainBlocksStatusResponse, NymAPIError> {
self.get_json("/v1/network/chain-blocks-status", NO_PARAMS)
.await
}
#[instrument(level = "debug", skip(self))]
async fn get_signer_status(&self) -> Result<EcashSignerStatusResponse, NymAPIError> {
self.get_json("/v1/ecash/signer-status", NO_PARAMS).await
}
#[instrument(level = "debug", skip(self))]
async fn get_signer_information(&self) -> Result<SignerInformationResponse, NymAPIError> {
self.get_json("/v1/api-status/signer-information", NO_PARAMS)
.await
}
#[instrument(level = "debug", skip(self))]
async fn get_key_rotation_info(&self) -> Result<KeyRotationInfoResponse, NymAPIError> {
self.get_json(
@@ -8,11 +8,11 @@ use crate::nyxd::CosmWasmClient;
use async_trait::async_trait;
use cosmrs::AccountId;
use cosmwasm_std::Addr;
use nym_coconut_dkg_common::dealer::RegisteredDealerDetails;
use nym_coconut_dkg_common::types::{ChunkIndex, NodeIndex, StateAdvanceResponse};
use serde::Deserialize;
use tracing::trace;
use nym_coconut_dkg_common::dealer::RegisteredDealerDetails;
pub use nym_coconut_dkg_common::{
dealer::{DealerDetailsResponse, PagedDealerIndexResponse, PagedDealerResponse},
dealing::{
@@ -21,7 +21,9 @@ pub use nym_coconut_dkg_common::{
},
msg::QueryMsg as DkgQueryMsg,
types::{DealerDetails, DealingIndex, Epoch, EpochId, EpochState, State},
verification_key::{ContractVKShare, PagedVKSharesResponse, VkShareResponse},
verification_key::{
ContractVKShare, PagedVKSharesResponse, VerificationKeyShare, VkShareResponse,
},
};
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
@@ -41,6 +43,11 @@ pub trait DkgQueryClient {
self.query_dkg_contract(request).await
}
async fn get_epoch_at_height(&self, height: u64) -> Result<Option<Epoch>, NyxdError> {
let request = DkgQueryMsg::GetEpochStateAtHeight { height };
self.query_dkg_contract(request).await
}
async fn can_advance_state(&self) -> Result<StateAdvanceResponse, NyxdError> {
let request = DkgQueryMsg::CanAdvanceState {};
self.query_dkg_contract(request).await
@@ -87,6 +94,34 @@ pub trait DkgQueryClient {
self.query_dkg_contract(request).await
}
async fn get_epoch_dealers_paged(
&self,
epoch_id: EpochId,
start_after: Option<String>,
limit: Option<u32>,
) -> Result<PagedDealerResponse, NyxdError> {
let request = DkgQueryMsg::GetEpochDealers {
epoch_id,
start_after,
limit,
};
self.query_dkg_contract(request).await
}
async fn get_epoch_dealers_addresses_paged(
&self,
epoch_id: EpochId,
start_after: Option<String>,
limit: Option<u32>,
) -> Result<PagedDealerResponse, NyxdError> {
let request = DkgQueryMsg::GetEpochDealersAddresses {
epoch_id,
start_after,
limit,
};
self.query_dkg_contract(request).await
}
async fn get_dealer_indices_paged(
&self,
start_after: Option<String>,
@@ -208,6 +243,20 @@ pub trait PagedDkgQueryClient: DkgQueryClient {
collect_paged!(self, get_current_dealers_paged, dealers)
}
async fn get_all_epoch_dealers(
&self,
epoch_id: EpochId,
) -> Result<Vec<DealerDetails>, NyxdError> {
collect_paged!(self, get_epoch_dealers_paged, dealers, epoch_id)
}
async fn get_all_epoch_dealers_addresses(
&self,
epoch_id: EpochId,
) -> Result<Vec<DealerDetails>, NyxdError> {
collect_paged!(self, get_epoch_dealers_addresses_paged, dealers, epoch_id)
}
async fn get_all_dealer_indices(&self) -> Result<Vec<(Addr, NodeIndex)>, NyxdError> {
collect_paged!(self, get_dealer_indices_paged, indices)
}
@@ -257,6 +306,9 @@ mod tests {
match msg {
DkgQueryMsg::GetState {} => client.get_state().ignore(),
DkgQueryMsg::GetCurrentEpochState {} => client.get_current_epoch().ignore(),
DkgQueryMsg::GetEpochStateAtHeight { height } => {
client.get_epoch_at_height(height).ignore()
}
DkgQueryMsg::CanAdvanceState {} => client.can_advance_state().ignore(),
DkgQueryMsg::GetCurrentEpochThreshold {} => {
client.get_current_epoch_threshold().ignore()
@@ -276,6 +328,20 @@ mod tests {
DkgQueryMsg::GetCurrentDealers { limit, start_after } => client
.get_current_dealers_paged(start_after, limit)
.ignore(),
QueryMsg::GetEpochDealers {
epoch_id,
limit,
start_after,
} => client
.get_epoch_dealers_paged(epoch_id, start_after, limit)
.ignore(),
QueryMsg::GetEpochDealersAddresses {
epoch_id,
limit,
start_after,
} => client
.get_epoch_dealers_addresses_paged(epoch_id, start_after, limit)
.ignore(),
DkgQueryMsg::GetDealerIndices { limit, start_after } => {
client.get_dealer_indices_paged(start_after, limit).ignore()
}
@@ -13,6 +13,7 @@ pub mod ecash_query_client;
pub mod group_query_client;
pub mod mixnet_query_client;
pub mod multisig_query_client;
pub mod performance_query_client;
pub mod vesting_query_client;
// signing clients
@@ -21,6 +22,7 @@ pub mod ecash_signing_client;
pub mod group_signing_client;
pub mod mixnet_signing_client;
pub mod multisig_signing_client;
pub mod performance_signing_client;
pub mod vesting_signing_client;
// re-export query traits
@@ -29,6 +31,7 @@ pub use ecash_query_client::{EcashQueryClient, PagedEcashQueryClient};
pub use group_query_client::{GroupQueryClient, PagedGroupQueryClient};
pub use mixnet_query_client::{MixnetQueryClient, PagedMixnetQueryClient};
pub use multisig_query_client::{MultisigQueryClient, PagedMultisigQueryClient};
pub use performance_query_client::{PagedPerformanceQueryClient, PerformanceQueryClient};
pub use vesting_query_client::{PagedVestingQueryClient, VestingQueryClient};
// re-export signing traits
@@ -37,6 +40,7 @@ pub use ecash_signing_client::EcashSigningClient;
pub use group_signing_client::GroupSigningClient;
pub use mixnet_signing_client::MixnetSigningClient;
pub use multisig_signing_client::MultisigSigningClient;
pub use performance_signing_client::PerformanceSigningClient;
pub use vesting_signing_client::VestingSigningClient;
// helper for providing blanket implementation for query clients
@@ -44,6 +48,7 @@ pub trait NymContractsProvider {
// main
fn mixnet_contract_address(&self) -> Option<&AccountId>;
fn vesting_contract_address(&self) -> Option<&AccountId>;
fn performance_contract_address(&self) -> Option<&AccountId>;
// coconut-related
fn ecash_contract_address(&self) -> Option<&AccountId>;
@@ -56,6 +61,7 @@ pub trait NymContractsProvider {
pub struct TypedNymContracts {
pub mixnet_contract_address: Option<AccountId>,
pub vesting_contract_address: Option<AccountId>,
pub performance_contract_address: Option<AccountId>,
pub ecash_contract_address: Option<AccountId>,
pub group_contract_address: Option<AccountId>,
@@ -76,6 +82,10 @@ impl TryFrom<NymContracts> for TypedNymContracts {
.vesting_contract_address
.map(|addr| addr.parse())
.transpose()?,
performance_contract_address: value
.performance_contract_address
.map(|addr| addr.parse())
.transpose()?,
ecash_contract_address: value
.ecash_contract_address
.map(|addr| addr.parse())
@@ -0,0 +1,271 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::collect_paged;
use crate::nyxd::contract_traits::NymContractsProvider;
use crate::nyxd::error::NyxdError;
use crate::nyxd::CosmWasmClient;
use async_trait::async_trait;
use cosmrs::AccountId;
use serde::Deserialize;
pub use nym_performance_contract_common::{
msg::QueryMsg as PerformanceQueryMsg, types::NetworkMonitorResponse, EpochId,
EpochMeasurementsPagedResponse, EpochNodePerformance, EpochPerformancePagedResponse,
FullHistoricalPerformancePagedResponse, HistoricalPerformance, LastSubmission,
NetworkMonitorInformation, NetworkMonitorsPagedResponse, NodeId, NodeMeasurement,
NodeMeasurementsResponse, NodePerformance, NodePerformancePagedResponse,
NodePerformanceResponse, RetiredNetworkMonitor, RetiredNetworkMonitorsPagedResponse,
};
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait PerformanceQueryClient {
async fn query_performance_contract<T>(
&self,
query: PerformanceQueryMsg,
) -> Result<T, NyxdError>
where
for<'a> T: Deserialize<'a>;
async fn admin(&self) -> Result<cw_controllers::AdminResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::Admin {})
.await
}
async fn get_node_performance(
&self,
epoch_id: EpochId,
node_id: NodeId,
) -> Result<NodePerformanceResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::NodePerformance { epoch_id, node_id })
.await
}
async fn get_node_performance_paged(
&self,
node_id: NodeId,
start_after: Option<EpochId>,
limit: Option<u32>,
) -> Result<NodePerformancePagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::NodePerformancePaged {
node_id,
start_after,
limit,
})
.await
}
async fn get_node_measurements(
&self,
epoch_id: EpochId,
node_id: NodeId,
) -> Result<NodeMeasurementsResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::NodeMeasurements { epoch_id, node_id })
.await
}
async fn get_epoch_measurements_paged(
&self,
epoch_id: EpochId,
start_after: Option<NodeId>,
limit: Option<u32>,
) -> Result<EpochMeasurementsPagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::EpochMeasurementsPaged {
epoch_id,
start_after,
limit,
})
.await
}
async fn get_epoch_performance_paged(
&self,
epoch_id: EpochId,
start_after: Option<NodeId>,
limit: Option<u32>,
) -> Result<EpochPerformancePagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::EpochPerformancePaged {
epoch_id,
start_after,
limit,
})
.await
}
async fn get_full_historical_performance_paged(
&self,
start_after: Option<(EpochId, NodeId)>,
limit: Option<u32>,
) -> Result<FullHistoricalPerformancePagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::FullHistoricalPerformancePaged {
start_after,
limit,
})
.await
}
async fn get_network_monitor(
&self,
address: &AccountId,
) -> Result<NetworkMonitorResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::NetworkMonitor {
address: address.to_string(),
})
.await
}
async fn get_network_monitors_paged(
&self,
start_after: Option<String>,
limit: Option<u32>,
) -> Result<NetworkMonitorsPagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::NetworkMonitorsPaged {
start_after,
limit,
})
.await
}
async fn get_retired_network_monitors_paged(
&self,
start_after: Option<String>,
limit: Option<u32>,
) -> Result<RetiredNetworkMonitorsPagedResponse, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::RetiredNetworkMonitorsPaged {
start_after,
limit,
})
.await
}
async fn get_last_submission(&self) -> Result<LastSubmission, NyxdError> {
self.query_performance_contract(PerformanceQueryMsg::LastSubmittedMeasurement {})
.await
}
}
// extension trait to the query client to deal with the paged queries
// (it didn't feel appropriate to combine it with the existing trait
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait PagedPerformanceQueryClient: PerformanceQueryClient {
async fn get_all_node_performance(
&self,
node_id: NodeId,
) -> Result<Vec<EpochNodePerformance>, NyxdError> {
collect_paged!(self, get_node_performance_paged, performance, node_id)
}
async fn get_all_epoch_measurements(
&self,
node_id: NodeId,
) -> Result<Vec<NodeMeasurement>, NyxdError> {
collect_paged!(self, get_epoch_measurements_paged, measurements, node_id)
}
async fn get_all_epoch_performance(
&self,
epoch_id: EpochId,
) -> Result<Vec<NodePerformance>, NyxdError> {
collect_paged!(self, get_epoch_performance_paged, performance, epoch_id)
}
async fn get_all_full_historical_performance(
&self,
) -> Result<Vec<HistoricalPerformance>, NyxdError> {
collect_paged!(self, get_full_historical_performance_paged, performance)
}
async fn get_all_network_monitors(&self) -> Result<Vec<NetworkMonitorInformation>, NyxdError> {
collect_paged!(self, get_network_monitors_paged, info)
}
async fn get_all_retired_network_monitors(
&self,
) -> Result<Vec<RetiredNetworkMonitor>, NyxdError> {
collect_paged!(self, get_retired_network_monitors_paged, info)
}
}
#[async_trait]
impl<T> PagedPerformanceQueryClient for T where T: PerformanceQueryClient {}
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
impl<C> PerformanceQueryClient for C
where
C: CosmWasmClient + NymContractsProvider + Send + Sync,
{
async fn query_performance_contract<T>(
&self,
query: PerformanceQueryMsg,
) -> Result<T, NyxdError>
where
for<'a> T: Deserialize<'a>,
{
let performance_contract_address = &self
.performance_contract_address()
.ok_or_else(|| NyxdError::unavailable_contract_address("performance contract"))?;
self.query_contract_smart(performance_contract_address, &query)
.await
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::nyxd::contract_traits::tests::IgnoreValue;
use nym_performance_contract_common::QueryMsg;
// it's enough that this compiles and clippy is happy about it
#[allow(dead_code)]
fn all_query_variants_are_covered<C: PerformanceQueryClient + Send + Sync>(
client: C,
msg: PerformanceQueryMsg,
) {
match msg {
PerformanceQueryMsg::Admin {} => client.admin().ignore(),
PerformanceQueryMsg::NodePerformance { epoch_id, node_id } => {
client.get_node_performance(epoch_id, node_id).ignore()
}
PerformanceQueryMsg::NodePerformancePaged {
node_id,
start_after,
limit,
} => client
.get_node_performance_paged(node_id, start_after, limit)
.ignore(),
PerformanceQueryMsg::NodeMeasurements { epoch_id, node_id } => {
client.get_node_measurements(epoch_id, node_id).ignore()
}
PerformanceQueryMsg::EpochMeasurementsPaged {
epoch_id,
start_after,
limit,
} => client
.get_epoch_measurements_paged(epoch_id, start_after, limit)
.ignore(),
PerformanceQueryMsg::EpochPerformancePaged {
epoch_id,
start_after,
limit,
} => client
.get_epoch_performance_paged(epoch_id, start_after, limit)
.ignore(),
PerformanceQueryMsg::FullHistoricalPerformancePaged { start_after, limit } => client
.get_full_historical_performance_paged(start_after, limit)
.ignore(),
PerformanceQueryMsg::NetworkMonitor { address } => client
.get_network_monitor(&address.parse().unwrap())
.ignore(),
PerformanceQueryMsg::NetworkMonitorsPaged { start_after, limit } => client
.get_network_monitors_paged(start_after, limit)
.ignore(),
PerformanceQueryMsg::RetiredNetworkMonitorsPaged { start_after, limit } => client
.get_retired_network_monitors_paged(start_after, limit)
.ignore(),
QueryMsg::LastSubmittedMeasurement {} => client.get_last_submission().ignore(),
};
}
}
@@ -0,0 +1,217 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::nyxd::coin::Coin;
use crate::nyxd::contract_traits::NymContractsProvider;
use crate::nyxd::cosmwasm_client::types::ExecuteResult;
use crate::nyxd::cosmwasm_client::ContractResponseData;
use crate::nyxd::error::NyxdError;
use crate::nyxd::{Fee, SigningCosmWasmClient};
use crate::signing::signer::OfflineSigner;
use async_trait::async_trait;
use nym_performance_contract_common::{
EpochId, ExecuteMsg as PerformanceExecuteMsg, NodeId, NodePerformance,
RemoveEpochMeasurementsResponse,
};
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait PerformanceSigningClient {
async fn execute_performance_contract(
&self,
fee: Option<Fee>,
msg: PerformanceExecuteMsg,
memo: String,
funds: Vec<Coin>,
) -> Result<ExecuteResult, NyxdError>;
async fn update_admin(
&self,
admin: String,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::UpdateAdmin { admin },
"PerformanceContract::UpdateAdmin".to_string(),
vec![],
)
.await
}
async fn submit_performance(
&self,
epoch: EpochId,
data: NodePerformance,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::Submit { epoch, data },
"PerformanceContract::Submit".to_string(),
vec![],
)
.await
}
async fn batch_submit_performance(
&self,
epoch: EpochId,
data: Vec<NodePerformance>,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::BatchSubmit { epoch, data },
"PerformanceContract::BatchSubmit".to_string(),
vec![],
)
.await
}
async fn authorise_network_monitor(
&self,
address: String,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::AuthoriseNetworkMonitor { address },
"PerformanceContract::AuthoriseNetworkMonitor".to_string(),
vec![],
)
.await
}
async fn retire_network_monitor(
&self,
address: String,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::RetireNetworkMonitor { address },
"PerformanceContract::RetireNetworkMonitor".to_string(),
vec![],
)
.await
}
async fn remove_node_measurements(
&self,
epoch_id: EpochId,
node_id: NodeId,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::RemoveNodeMeasurements { epoch_id, node_id },
"PerformanceContract::RemoveNodeMeasurements".to_string(),
vec![],
)
.await
}
async fn partial_remove_epoch_measurements(
&self,
epoch_id: EpochId,
fee: Option<Fee>,
) -> Result<ExecuteResult, NyxdError> {
self.execute_performance_contract(
fee,
PerformanceExecuteMsg::RemoveEpochMeasurements { epoch_id },
"PerformanceContract::RemoveEpochMeasurements".to_string(),
vec![],
)
.await
}
async fn remove_epoch_measurements(
&self,
epoch_id: EpochId,
fee: Option<Fee>,
) -> Result<(), NyxdError> {
loop {
let execute_res = self
.partial_remove_epoch_measurements(epoch_id, fee.clone())
.await?;
let response = execute_res
.parse_singleton_json_contract_response::<RemoveEpochMeasurementsResponse>()?;
if !response.additional_entries_to_remove_remaining {
break;
}
}
Ok(())
}
}
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
impl<C> PerformanceSigningClient for C
where
C: SigningCosmWasmClient + NymContractsProvider + Sync,
NyxdError: From<<Self as OfflineSigner>::Error>,
{
async fn execute_performance_contract(
&self,
fee: Option<Fee>,
msg: PerformanceExecuteMsg,
memo: String,
funds: Vec<Coin>,
) -> Result<ExecuteResult, NyxdError> {
let performance_contract_address = &self
.performance_contract_address()
.ok_or_else(|| NyxdError::unavailable_contract_address("performance contract"))?;
let fee = fee.unwrap_or(Fee::Auto(Some(self.simulated_gas_multiplier())));
let signer_address = &self.signer_addresses()?[0];
self.execute(
signer_address,
performance_contract_address,
&msg,
fee,
memo,
funds,
)
.await
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::nyxd::contract_traits::tests::IgnoreValue;
use nym_performance_contract_common::ExecuteMsg;
// it's enough that this compiles and clippy is happy about it
#[allow(dead_code)]
fn all_execute_variants_are_covered<C: PerformanceSigningClient + Send + Sync>(
client: C,
msg: PerformanceExecuteMsg,
) {
match msg {
PerformanceExecuteMsg::UpdateAdmin { admin } => {
client.update_admin(admin, None).ignore()
}
PerformanceExecuteMsg::Submit { epoch, data } => {
client.submit_performance(epoch, data, None).ignore()
}
PerformanceExecuteMsg::BatchSubmit { epoch, data } => {
client.batch_submit_performance(epoch, data, None).ignore()
}
PerformanceExecuteMsg::AuthoriseNetworkMonitor { address } => {
client.authorise_network_monitor(address, None).ignore()
}
PerformanceExecuteMsg::RetireNetworkMonitor { address } => {
client.retire_network_monitor(address, None).ignore()
}
ExecuteMsg::RemoveNodeMeasurements { epoch_id, node_id } => client
.remove_node_measurements(epoch_id, node_id, None)
.ignore(),
ExecuteMsg::RemoveEpochMeasurements { epoch_id } => client
.partial_remove_epoch_measurements(epoch_id, None)
.ignore(),
};
}
}
@@ -12,6 +12,8 @@ use tendermint_rpc::endpoint::broadcast;
use tracing::error;
pub use cosmrs::abci::MsgResponse;
use cosmwasm_std::from_json;
use serde::de::DeserializeOwned;
pub fn parse_singleton_u32_from_contract_response(b: Vec<u8>) -> Result<u32, NyxdError> {
if b.len() != 4 {
@@ -73,6 +75,11 @@ pub fn parse_msg_responses(data: Bytes) -> Vec<MsgResponse> {
// requires there's a single response message
pub trait ContractResponseData: Sized {
fn parse_singleton_json_contract_response<T: DeserializeOwned>(&self) -> Result<T, NyxdError> {
let b = self.to_singleton_contract_data()?;
from_json(&b).map_err(|err| err.into())
}
fn parse_singleton_u32_contract_data(&self) -> Result<u32, NyxdError> {
let b = self.to_singleton_contract_data()?;
parse_singleton_u32_from_contract_response(b)
@@ -139,12 +139,22 @@ impl NyxdClient<HttpClient> {
})
}
pub fn connect_with_network_details<U>(
endpoint: U,
network_details: NymNetworkDetails,
) -> Result<QueryHttpRpcNyxdClient, NyxdError>
where
U: TryInto<HttpClientUrl, Error = TendermintRpcError>,
{
let config = Config::try_from_nym_network_details(&network_details)?;
Self::connect(config, endpoint)
}
pub fn connect_to_default_env<U>(endpoint: U) -> Result<QueryHttpRpcNyxdClient, NyxdError>
where
U: TryInto<HttpClientUrl, Error = TendermintRpcError>,
{
let config = Config::try_from_nym_network_details(&NymNetworkDetails::new_from_env())?;
Self::connect(config, endpoint)
Self::connect_with_network_details(endpoint, NymNetworkDetails::new_from_env())
}
}
@@ -276,6 +286,10 @@ impl<C, S> NymContractsProvider for NyxdClient<C, S> {
self.config.contracts.vesting_contract_address.as_ref()
}
fn performance_contract_address(&self) -> Option<&AccountId> {
self.config.contracts.performance_contract_address.as_ref()
}
fn ecash_contract_address(&self) -> Option<&AccountId> {
self.config.contracts.ecash_contract_address.as_ref()
}
+1 -1
View File
@@ -86,7 +86,7 @@ pub async fn execute(args: Args) -> anyhow::Result<()> {
anyhow!("ticketbook got incorrectly imported - the master verification key is missing")
})?;
let expiration_signatures = persistent_storage
.get_expiration_date_signatures(expiration_date)
.get_expiration_date_signatures(expiration_date, epoch_id)
.await?
.ok_or_else(|| {
anyhow!(
@@ -120,7 +120,7 @@ async fn issue_to_file(args: Args, client: SigningClient) -> anyhow::Result<()>
if args.include_expiration_date_signatures {
let signatures = credentials_store
.get_expiration_date_signatures(expiration_date)
.get_expiration_date_signatures(expiration_date, epoch_id)
.await?
.ok_or(anyhow!("missing expiration date signatures!"))?;
@@ -241,7 +241,7 @@ pub async fn delegate_to_multiple_mixnodes(args: Args, client: SigningClient) {
let node_id = row.node_id.clone().parse::<u32>().unwrap();
let coins: Vec<Coin> = vec![];
undelegation_msgs.push((ExecuteMsg::Undelegate { node_id }, coins));
undelegation_table.add_row(&[row.node_id.clone()]);
undelegation_table.add_row(std::slice::from_ref(&row.node_id));
if row.amount.amount > 0 {
delegation_msgs
@@ -55,6 +55,14 @@ impl DealerDetailsResponse {
}
}
#[cw_serde]
pub struct PagedDealerAddressesResponse {
pub dealers: Vec<Addr>,
/// Field indicating paging information for the following queries if the caller wishes to get further entries.
pub start_next_after: Option<Addr>,
}
#[cw_serde]
pub struct PagedDealerResponse {
pub dealers: Vec<DealerDetails>,
@@ -12,8 +12,8 @@ use cosmwasm_schema::cw_serde;
#[cfg(feature = "schema")]
use crate::{
dealer::{
DealerDetailsResponse, PagedDealerIndexResponse, PagedDealerResponse,
RegisteredDealerDetails,
DealerDetailsResponse, PagedDealerAddressesResponse, PagedDealerIndexResponse,
PagedDealerResponse, RegisteredDealerDetails,
},
dealing::{
DealerDealingsStatusResponse, DealingChunkResponse, DealingChunkStatusResponse,
@@ -84,6 +84,9 @@ pub enum QueryMsg {
#[cfg_attr(feature = "schema", returns(Epoch))]
GetCurrentEpochState {},
#[cfg_attr(feature = "schema", returns(Option<Epoch>))]
GetEpochStateAtHeight { height: u64 },
#[cfg_attr(feature = "schema", returns(u64))]
GetCurrentEpochThreshold {},
@@ -102,6 +105,20 @@ pub enum QueryMsg {
#[cfg_attr(feature = "schema", returns(DealerDetailsResponse))]
GetDealerDetails { dealer_address: String },
#[cfg_attr(feature = "schema", returns(PagedDealerAddressesResponse))]
GetEpochDealersAddresses {
epoch_id: EpochId,
limit: Option<u32>,
start_after: Option<String>,
},
#[cfg_attr(feature = "schema", returns(PagedDealerResponse))]
GetEpochDealers {
epoch_id: EpochId,
limit: Option<u32>,
start_after: Option<String>,
},
#[cfg_attr(feature = "schema", returns(PagedDealerResponse))]
GetCurrentDealers {
limit: Option<u32>,
@@ -0,0 +1,24 @@
[package]
name = "nym-contracts-common-testing"
version = "0.1.0"
authors.workspace = true
repository.workspace = true
homepage.workspace = true
documentation.workspace = true
edition.workspace = true
license.workspace = true
rust-version.workspace = true
readme.workspace = true
[dependencies]
anyhow = { workspace = true }
cosmwasm-std = { workspace = true }
cw-storage-plus = { workspace = true }
serde = { workspace = true }
rand_chacha = { workspace = true }
rand = { workspace = true }
cw-multi-test = { workspace = true }
[lints]
workspace = true
@@ -0,0 +1,127 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use cosmwasm_std::testing::{message_info, MockApi, MockQuerier, MockStorage};
use cosmwasm_std::{
coins, Addr, BankMsg, CosmosMsg, Empty, Env, MemoryStorage, MessageInfo, Order, OwnedDeps,
Response, StdResult, Storage,
};
use cw_storage_plus::{KeyDeserialize, Map, Prefix, PrimaryKey};
use rand::{RngCore, SeedableRng};
use rand_chacha::ChaCha20Rng;
use serde::de::DeserializeOwned;
use serde::Serialize;
pub const TEST_DENOM: &str = "unym";
pub const TEST_PREFIX: &str = "n";
pub fn mock_api() -> MockApi {
MockApi::default().with_prefix(TEST_PREFIX)
}
pub fn mock_dependencies() -> OwnedDeps<MemoryStorage, MockApi, MockQuerier<Empty>> {
OwnedDeps {
storage: MockStorage::default(),
api: mock_api(),
querier: MockQuerier::default(),
custom_query_type: Default::default(),
}
}
pub fn test_rng() -> ChaCha20Rng {
let dummy_seed = [42u8; 32];
rand_chacha::ChaCha20Rng::from_seed(dummy_seed)
}
pub fn deps_with_balance(env: &Env) -> OwnedDeps<MemoryStorage, MockApi, MockQuerier<Empty>> {
let mut deps = mock_dependencies();
deps.querier = MockQuerier::<Empty>::new(&[(
env.contract.address.as_str(),
coins(100000000000, TEST_DENOM).as_slice(),
)]);
deps
}
pub fn generate_sorted_addresses(n: usize) -> Vec<Addr> {
let mut rng = test_rng();
let mut addrs = Vec::with_capacity(n);
for i in 0..n {
addrs.push(mock_api().addr_make(&format!("addr{i}{}", rng.next_u64())));
}
addrs.sort();
addrs
}
pub fn addr<S: AsRef<str>>(raw: S) -> Addr {
mock_api().addr_make(raw.as_ref())
}
pub fn sender<S: AsRef<str>>(raw: S) -> MessageInfo {
message_info(&addr(raw), &[])
}
pub trait ExtractBankMsg {
fn unwrap_bank_msg(self) -> Option<BankMsg>;
}
impl ExtractBankMsg for Response {
fn unwrap_bank_msg(self) -> Option<BankMsg> {
for msg in self.messages {
match msg.msg {
CosmosMsg::Bank(bank_msg) => return Some(bank_msg),
_ => continue,
}
}
None
}
}
pub trait FullReader<'a> {
type Key;
type Value: Serialize + DeserializeOwned;
fn all_values(&self, store: &dyn Storage) -> StdResult<Vec<Self::Value>>;
fn all_key_values(&self, store: &dyn Storage) -> StdResult<Vec<(Self::Key, Self::Value)>>;
}
impl<'a, K, T> FullReader<'a> for Map<K, T>
where
T: Serialize + DeserializeOwned,
K: PrimaryKey<'a> + KeyDeserialize,
K::Output: 'static,
{
type Key = K::Output;
type Value = T;
fn all_values(&self, store: &dyn Storage) -> StdResult<Vec<Self::Value>> {
self.range(store, None, None, Order::Ascending)
.map(|record| record.map(|r| r.1))
.collect()
}
fn all_key_values(&self, store: &dyn Storage) -> StdResult<Vec<(Self::Key, Self::Value)>> {
self.range(store, None, None, Order::Ascending).collect()
}
}
impl<'a, K, T, B> FullReader<'a> for Prefix<K, T, B>
where
K: KeyDeserialize + 'static,
T: Serialize + DeserializeOwned,
B: PrimaryKey<'a>,
{
type Key = K::Output;
type Value = T;
fn all_values(&self, store: &dyn Storage) -> StdResult<Vec<Self::Value>> {
self.range(store, None, None, Order::Ascending)
.map(|record| record.map(|r| r.1))
.collect()
}
fn all_key_values(&self, store: &dyn Storage) -> StdResult<Vec<(Self::Key, Self::Value)>> {
self.range(store, None, None, Order::Ascending).collect()
}
}
@@ -0,0 +1,13 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
// those are all used exclusively for testing thus unwraps, et al. are allowed
#![allow(clippy::unwrap_used)]
#![allow(clippy::expect_used)]
#![allow(clippy::panic)]
pub mod helpers;
pub mod tester;
pub use helpers::*;
pub use tester::*;
@@ -0,0 +1,239 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::{ContractTester, TestableNymContract};
use cosmwasm_std::testing::{message_info, mock_env};
use cosmwasm_std::{
from_json, Addr, Coin, ContractInfo, Deps, DepsMut, Env, MessageInfo, Response, StdResult,
Storage, Timestamp,
};
use cw_multi_test::{next_block, AppResponse, Executor};
use serde::de::DeserializeOwned;
use serde::Serialize;
use std::any::type_name;
use std::fmt::Debug;
pub trait ContractOpts {
type ExecuteMsg;
type QueryMsg;
type ContractError;
fn deps(&self) -> Deps<'_>;
fn deps_mut(&mut self) -> DepsMut<'_>;
fn env(&self) -> Env;
fn addr_make(&self, input: &str) -> Addr;
fn deps_mut_env(&mut self) -> (DepsMut<'_>, Env) {
let env = self.env().clone();
(self.deps_mut(), env)
}
fn storage(&self) -> &dyn Storage;
fn storage_mut(&mut self) -> &mut dyn Storage;
fn read_from_contract_storage<T: DeserializeOwned>(&self, key: impl AsRef<[u8]>) -> Option<T>;
fn set_contract_storage(&mut self, key: impl AsRef<[u8]>, value: impl AsRef<[u8]>);
fn unchecked_read_from_contract_storage<T: DeserializeOwned>(
&self,
key: impl AsRef<[u8]>,
) -> T {
let typ = type_name::<T>();
self.read_from_contract_storage(key)
.unwrap_or_else(|| panic!("value of type {typ} not present in the storage"))
}
fn execute_raw(
&mut self,
sender: Addr,
message: Self::ExecuteMsg,
) -> Result<Response, Self::ContractError> {
self.execute_raw_with_balance(sender, &[], message)
}
fn execute_raw_with_balance(
&mut self,
sender: Addr,
coins: &[Coin],
message: Self::ExecuteMsg,
) -> Result<Response, Self::ContractError>;
}
impl<C> ContractOpts for ContractTester<C>
where
C: TestableNymContract,
{
type ExecuteMsg = C::ExecuteMsg;
type QueryMsg = C::QueryMsg;
type ContractError = C::ContractError;
fn deps(&self) -> Deps<'_> {
Deps {
storage: &self.storage,
api: self.app.api(),
querier: self.app.wrap(),
}
}
fn deps_mut(&mut self) -> DepsMut<'_> {
DepsMut {
storage: &mut self.storage,
api: self.app.api(),
querier: self.app.wrap(),
}
}
fn env(&self) -> Env {
Env {
block: self.app.block_info(),
contract: ContractInfo {
address: self.contract_address.clone(),
},
..mock_env()
}
}
fn addr_make(&self, input: &str) -> Addr {
self.app.api().addr_make(input)
}
fn storage(&self) -> &dyn Storage {
&self.storage
}
fn storage_mut(&mut self) -> &mut dyn Storage {
&mut self.storage
}
fn read_from_contract_storage<T: DeserializeOwned>(&self, key: impl AsRef<[u8]>) -> Option<T> {
let raw = self.deps().storage.get(key.as_ref())?;
from_json(&raw).ok()
}
fn set_contract_storage(&mut self, key: impl AsRef<[u8]>, value: impl AsRef<[u8]>) {
self.deps_mut().storage.set(key.as_ref(), value.as_ref());
}
fn execute_raw_with_balance(
&mut self,
sender: Addr,
coins: &[Coin],
message: C::ExecuteMsg,
) -> Result<Response, C::ContractError> {
let env = self.env();
let info = message_info(&sender, coins);
C::execute()(self.deps_mut(), env, info, message)
}
}
pub trait ChainOpts: ContractOpts {
fn set_contract_balance(&mut self, balance: Coin);
fn next_block(&mut self);
fn set_block_time(&mut self, time: Timestamp);
fn execute_msg(
&mut self,
sender: Addr,
message: &Self::ExecuteMsg,
) -> anyhow::Result<AppResponse> {
self.execute_msg_with_balance(sender, &[], message)
}
fn execute_msg_with_balance(
&mut self,
sender: Addr,
coins: &[Coin],
message: &Self::ExecuteMsg,
) -> anyhow::Result<AppResponse>;
fn execute_arbitrary_contract<T: Serialize + Debug>(
&mut self,
contract: Addr,
sender: MessageInfo,
message: &T,
) -> anyhow::Result<AppResponse>;
fn query_arbitrary_contract<Q: Serialize + Debug, T: DeserializeOwned>(
&self,
contract: Addr,
message: &Q,
) -> StdResult<T>;
fn query<T: DeserializeOwned>(&self, message: &Self::QueryMsg) -> StdResult<T>;
}
impl<C> ChainOpts for ContractTester<C>
where
C: TestableNymContract,
{
fn set_contract_balance(&mut self, balance: Coin) {
let contract_address = &self.contract_address;
self.app
.router()
.bank
.init_balance(
&mut self.storage.inner_storage(),
contract_address,
vec![balance],
)
.unwrap();
}
fn next_block(&mut self) {
self.app.update_block(next_block)
}
fn set_block_time(&mut self, time: Timestamp) {
self.app.update_block(|b| b.time = time)
}
fn execute_msg(
&mut self,
sender: Addr,
message: &C::ExecuteMsg,
) -> anyhow::Result<AppResponse> {
self.execute_msg_with_balance(sender, &[], message)
}
fn execute_msg_with_balance(
&mut self,
sender: Addr,
coins: &[Coin],
message: &C::ExecuteMsg,
) -> anyhow::Result<AppResponse> {
self.app
.execute_contract(sender, self.contract_address.clone(), message, coins)
}
fn execute_arbitrary_contract<T: Serialize + Debug>(
&mut self,
contract: Addr,
sender: MessageInfo,
message: &T,
) -> anyhow::Result<AppResponse> {
let coins = &sender.funds;
let sender = sender.sender;
self.app.execute_contract(sender, contract, message, coins)
}
fn query_arbitrary_contract<Q: Serialize + Debug, T: DeserializeOwned>(
&self,
contract: Addr,
message: &Q,
) -> StdResult<T> {
self.app.wrap().query_wasm_smart(contract, message)
}
fn query<T: DeserializeOwned>(&self, message: &C::QueryMsg) -> StdResult<T> {
self.app
.wrap()
.query_wasm_smart(self.contract_address.as_str(), message)
}
}
@@ -0,0 +1,305 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::{
CommonStorageKeys, ContractOpts, ContractTester, StorageWrapper, TestableNymContract,
TEST_DENOM,
};
use cosmwasm_std::testing::message_info;
use cosmwasm_std::{
coin, coins, from_json, to_json_vec, Addr, Coin, MessageInfo, StdError, StdResult, Storage,
};
use cw_multi_test::Executor;
use cw_storage_plus::{Key, Path, PrimaryKey};
use rand::RngCore;
use rand_chacha::ChaCha20Rng;
use serde::de::DeserializeOwned;
use serde::Serialize;
use std::any::type_name;
use std::ops::Deref;
pub trait StorageReader {
fn common_key(&self, key: CommonStorageKeys) -> Option<&[u8]>;
fn read_common_value<T: DeserializeOwned>(&self, key: CommonStorageKeys) -> Option<T> {
self.read_from_contract_storage(self.common_key(key)?)
}
fn unchecked_read_common_value<T: DeserializeOwned>(&self, key: CommonStorageKeys) -> T {
self.unchecked_read_from_contract_storage(
self.common_key(key)
.unwrap_or_else(|| panic!("no key set for {key:?}")),
)
}
fn read_from_contract_storage<T: DeserializeOwned>(&self, key: impl AsRef<[u8]>) -> Option<T>;
fn unchecked_read_from_contract_storage<T: DeserializeOwned>(
&self,
key: impl AsRef<[u8]>,
) -> T {
let typ = type_name::<T>();
self.read_from_contract_storage(key)
.unwrap_or_else(|| panic!("value of type {typ} not present in the storage"))
}
}
// technically it shouldn't rely on `StorageReader` and `common_key` should be extracted
// but this makes it a tad easier and it's only testing code so it's fine
pub trait StorageWriter: StorageReader {
fn set_common_value<T: Serialize>(
&mut self,
key: CommonStorageKeys,
value: &T,
) -> StdResult<()> {
let key = self
.common_key(key)
.ok_or(StdError::not_found("key not found"))?
.to_vec();
self.set_storage_value(key, value)
}
fn set_storage(&mut self, key: impl AsRef<[u8]>, value: impl AsRef<[u8]>);
fn set_storage_value<T: Serialize>(
&mut self,
key: impl AsRef<[u8]>,
value: &T,
) -> StdResult<()> {
self.set_storage(key, &to_json_vec(value)?);
Ok(())
}
}
pub trait ArbitraryContractStorageReader {
fn may_read_from_contract_storage(
&self,
address: impl Into<String>,
key: impl AsRef<[u8]>,
) -> Option<Vec<u8>>;
fn must_read_from_contract_storage(
&self,
address: impl Into<String>,
key: impl AsRef<[u8]>,
) -> StdResult<Vec<u8>> {
let key = key.as_ref();
self.may_read_from_contract_storage(address, key)
.ok_or(StdError::not_found(format!("no data under {key:?}")))
}
fn may_read_value_from_contract_storage<T: DeserializeOwned>(
&self,
address: impl Into<String>,
key: impl AsRef<[u8]>,
) -> StdResult<Option<T>> {
let Some(bytes) = self.may_read_from_contract_storage(address, key) else {
return Ok(None);
};
from_json(&bytes).map(Some)
}
fn must_read_value_from_contract_storage<T: DeserializeOwned>(
&self,
address: impl Into<String>,
key: impl AsRef<[u8]>,
) -> StdResult<T> {
let bytes = self.must_read_from_contract_storage(address, key)?;
from_json(&bytes)
}
}
pub trait ArbitraryContractStorageWriter {
fn set_contract_storage(
&mut self,
address: impl Into<String>,
key: impl AsRef<[u8]>,
value: impl AsRef<[u8]>,
);
fn set_contract_storage_value<T: Serialize>(
&mut self,
address: impl Into<String>,
key: impl AsRef<[u8]>,
value: &T,
) -> StdResult<()> {
self.set_contract_storage(address, key, &to_json_vec(value)?);
Ok(())
}
// attempts to write to an arbitrary contract `cw_storage_plus::Map`
fn set_contract_map_value<'a, K, T>(
&mut self,
address: impl Into<String>,
namespace: impl AsRef<[u8]>,
key: K,
value: &T,
) -> StdResult<()>
where
K: PrimaryKey<'a>,
T: Serialize + DeserializeOwned,
{
let key_path: Path<T> = Path::new(
namespace.as_ref(),
&key.key().iter().map(Key::as_ref).collect::<Vec<_>>(),
);
let storage_key = key_path.deref();
self.set_contract_storage_value(address, storage_key, value)
}
}
// contract that has an admin
pub trait AdminExt: StorageReader + StorageWriter {
fn admin(&self) -> Option<Addr> {
self.read_common_value(CommonStorageKeys::Admin)
}
fn update_admin(&mut self, admin: &Option<Addr>) -> StdResult<()> {
self.set_common_value(CommonStorageKeys::Admin, admin)
}
fn admin_unchecked(&self) -> Addr {
self.admin().expect("no admin set")
}
fn admin_msg(&self) -> MessageInfo {
message_info(&self.admin_unchecked(), &[])
}
}
// contract that operates on some specific coin denom
pub trait DenomExt: StorageReader {
fn denom(&self) -> String {
self.unchecked_read_common_value(CommonStorageKeys::Denom)
}
fn coin(&self, amount: u128) -> Coin {
coin(amount, self.denom())
}
fn coins(&self, amount: u128) -> Vec<Coin> {
coins(amount, self.denom())
}
}
pub trait RandExt {
fn raw_rng(&mut self) -> &mut ChaCha20Rng;
fn generate_account(&mut self) -> Addr;
fn generate_account_with_balance(&mut self) -> Addr
where
Self: BankExt;
}
pub trait BankExt {
fn send_tokens(&mut self, to: Addr, amount: Coin) -> anyhow::Result<()>;
}
impl<T> AdminExt for T where T: StorageReader + StorageWriter {}
impl<T> DenomExt for T where T: StorageReader {}
impl<C: TestableNymContract> StorageReader for ContractTester<C> {
fn common_key(&self, key: CommonStorageKeys) -> Option<&[u8]> {
self.common_storage_keys.get(&key).map(|v| &**v)
}
fn read_from_contract_storage<T: DeserializeOwned>(&self, key: impl AsRef<[u8]>) -> Option<T> {
<Self as ContractOpts>::read_from_contract_storage(self, key)
}
}
impl<C: TestableNymContract> StorageWriter for ContractTester<C> {
fn set_storage(&mut self, key: impl AsRef<[u8]>, value: impl AsRef<[u8]>) {
<Self as ContractOpts>::set_contract_storage(self, key, value)
}
}
impl<C: TestableNymContract> BankExt for ContractTester<C> {
fn send_tokens(&mut self, to: Addr, amount: Coin) -> anyhow::Result<()> {
self.app
.send_tokens(self.master_address.clone(), to, &[amount])?;
Ok(())
}
}
impl<C: TestableNymContract> RandExt for ContractTester<C> {
fn raw_rng(&mut self) -> &mut ChaCha20Rng {
&mut self.rng
}
fn generate_account(&mut self) -> Addr {
self.app
.api()
.addr_make(&format!("foomp{}", self.rng.next_u64()))
}
fn generate_account_with_balance(&mut self) -> Addr
where
Self: BankExt,
{
let addr = self.generate_account();
let million = 1_000_000_000_000;
self.send_tokens(addr.clone(), coin(million, TEST_DENOM))
.unwrap();
addr
}
}
impl ArbitraryContractStorageReader for StorageWrapper {
fn may_read_from_contract_storage(
&self,
address: impl Into<String>,
key: impl AsRef<[u8]>,
) -> Option<Vec<u8>> {
self.contract_storage_wrapper(&Addr::unchecked(address))
.get(key.as_ref())
}
}
impl ArbitraryContractStorageWriter for StorageWrapper {
fn set_contract_storage(
&mut self,
address: impl Into<String>,
key: impl AsRef<[u8]>,
value: impl AsRef<[u8]>,
) {
// yeah, we're unnecessarily cloning a Rc pointer, but this is a test code, so this inefficiency is fine
let mut wrapped_storage = self
.clone()
.contract_storage_wrapper(&Addr::unchecked(address));
wrapped_storage.set(key.as_ref(), value.as_ref());
}
}
impl<C> ArbitraryContractStorageReader for ContractTester<C>
where
C: TestableNymContract,
{
fn may_read_from_contract_storage(
&self,
address: impl Into<String>,
key: impl AsRef<[u8]>,
) -> Option<Vec<u8>> {
self.storage
.as_inner_storage()
.may_read_from_contract_storage(address, key)
}
}
impl<C> ArbitraryContractStorageWriter for ContractTester<C>
where
C: TestableNymContract,
{
fn set_contract_storage(
&mut self,
address: impl Into<String>,
key: impl AsRef<[u8]>,
value: impl AsRef<[u8]>,
) {
self.storage
.as_inner_storage_mut()
.set_contract_storage(address, key, value);
}
}
@@ -0,0 +1,276 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::{mock_api, test_rng, TEST_DENOM};
use cosmwasm_std::testing::MockApi;
use cosmwasm_std::{
coin, coins, Addr, Binary, Deps, DepsMut, Empty, Env, MessageInfo, Order, QuerierWrapper,
Record, Response, Storage,
};
use cw_multi_test::{App, AppBuilder, BankKeeper, Contract, ContractWrapper, Executor};
use rand_chacha::ChaCha20Rng;
use serde::de::DeserializeOwned;
use serde::Serialize;
use std::collections::HashMap;
use std::fmt::{Debug, Display};
use std::marker::PhantomData;
pub use basic_traits::*;
pub use extensions::*;
pub use crate::tester::storage_wrapper::{ContractStorageWrapper, StorageWrapper};
mod basic_traits;
mod extensions;
mod storage_wrapper;
// copied from cw-multi-test (but removed generics for custom messages and querier for we don't need them for now)
pub type ContractFn<T, E> =
fn(deps: DepsMut, env: Env, info: MessageInfo, msg: T) -> Result<Response, E>;
pub type QueryFn<T, E> = fn(deps: Deps, env: Env, msg: T) -> Result<Binary, E>;
pub type PermissionedFn<T, E> = fn(deps: DepsMut, env: Env, msg: T) -> Result<Response, E>;
pub type ContractClosure<T, E> = Box<dyn Fn(DepsMut, Env, MessageInfo, T) -> Result<Response, E>>;
pub type QueryClosure<T, E> = Box<dyn Fn(Deps, Env, T) -> Result<Binary, E>>;
pub trait TestableNymContract {
const NAME: &'static str;
type InitMsg: DeserializeOwned + Serialize + Debug + 'static;
type ExecuteMsg: DeserializeOwned + Serialize + Debug + 'static;
type QueryMsg: DeserializeOwned + Serialize + Debug + 'static;
type MigrateMsg: DeserializeOwned + Serialize + Debug + 'static;
type ContractError: Display + Debug + Send + Sync + 'static;
fn instantiate() -> ContractFn<Self::InitMsg, Self::ContractError>;
fn execute() -> ContractFn<Self::ExecuteMsg, Self::ContractError>;
fn query() -> QueryFn<Self::QueryMsg, Self::ContractError>;
fn migrate() -> PermissionedFn<Self::MigrateMsg, Self::ContractError>;
fn base_init_msg() -> Self::InitMsg;
// // for now we don't care about custom queriers
// fn contract_wrapper() -> ContractWrapper<
// Self::ExecuteMsg,
// Self::InitMsg,
// Self::QueryMsg,
// Self::ContractError,
// anyhow::Error,
// anyhow::Error,
// Empty,
// Empty,
// Empty,
// Self::ContractError,
// Self::ContractError,
// Self::MigrateMsg,
// Self::ContractError,
// > {
// ContractWrapper::new(Self::execute(), Self::instantiate(), Self::query())
// .with_migrate(Self::migrate())
// }
fn dyn_contract() -> Box<dyn Contract<Empty>> {
Box::new(
ContractWrapper::new(Self::execute(), Self::instantiate(), Self::query())
.with_migrate(Self::migrate()),
)
}
fn init() -> ContractTester<Self>
where
Self: Sized,
{
ContractTesterBuilder::new()
.instantiate::<Self>(None)
.build()
}
}
pub struct ContractTesterBuilder<C> {
contract: PhantomData<C>,
master_address: Addr,
app: App<BankKeeper, MockApi, StorageWrapper>,
storage: StorageWrapper,
pub well_known_contracts: HashMap<&'static str, Addr>,
}
impl<C> ContractTesterBuilder<C> {
#[allow(clippy::new_without_default)]
pub fn new() -> Self
where
C: TestableNymContract,
{
let storage = StorageWrapper::new();
let api = mock_api();
let master_address = api.addr_make("master-owner");
let app = AppBuilder::new()
.with_api(api)
.with_storage(storage.clone())
.build(|router, _api, storage| {
router
.bank
.init_balance(
storage,
&master_address,
coins(1000000000000000, TEST_DENOM),
)
.unwrap()
});
ContractTesterBuilder {
contract: Default::default(),
master_address,
app,
storage,
well_known_contracts: Default::default(),
}
}
pub fn instantiate<D: TestableNymContract>(
mut self,
custom_init_msg: Option<D::InitMsg>,
) -> ContractTesterBuilder<C> {
let code_id = self.app.store_code(D::dyn_contract());
let contract_address = self
.app
.instantiate_contract(
code_id,
self.master_address.clone(),
&custom_init_msg.unwrap_or(D::base_init_msg()),
&[],
D::NAME,
Some(self.master_address.to_string()),
)
.unwrap();
// send some tokens to the contract
self.app
.send_tokens(
self.master_address.clone(),
contract_address.clone(),
&[coin(100000000, TEST_DENOM)],
)
.unwrap();
self.well_known_contracts.insert(D::NAME, contract_address);
self
}
pub fn build(self) -> ContractTester<C>
where
C: TestableNymContract,
{
if !self.well_known_contracts.contains_key(C::NAME) {
panic!("{} contract has not been instantiated", C::NAME);
}
let contract_address = self.well_known_contracts[C::NAME].clone();
ContractTester {
contract: self.contract,
app: self.app,
rng: test_rng(),
master_address: self.master_address,
storage: self.storage.contract_storage_wrapper(&contract_address),
contract_address,
common_storage_keys: Default::default(),
well_known_contracts: self.well_known_contracts,
}
}
pub fn contract_storage_wrapper(&self, contract: &Addr) -> ContractStorageWrapper {
self.storage.contract_storage_wrapper(contract)
}
pub fn api(&self) -> MockApi {
*self.app.api()
}
pub fn querier(&self) -> QuerierWrapper<'_> {
self.app.wrap()
}
}
#[derive(Debug, Copy, Clone, Hash, Eq, PartialEq)]
pub enum CommonStorageKeys {
Admin,
Denom,
}
pub struct ContractTester<C: TestableNymContract> {
contract: PhantomData<C>,
pub app: App<BankKeeper, MockApi, StorageWrapper>,
pub rng: ChaCha20Rng,
pub contract_address: Addr,
pub master_address: Addr,
pub(crate) storage: ContractStorageWrapper,
pub common_storage_keys: HashMap<CommonStorageKeys, Vec<u8>>,
// TODO: limitation: doesn't allow multiple contracts of the same type (but that's fine for the time being)
pub well_known_contracts: HashMap<&'static str, Addr>,
}
impl<C> ContractTester<C>
where
C: TestableNymContract,
{
pub fn insert_common_storage_key(&mut self, key: CommonStorageKeys, value: impl AsRef<[u8]>) {
self.common_storage_keys
.insert(key, value.as_ref().to_vec());
}
pub fn with_common_storage_key(
mut self,
key: CommonStorageKeys,
value: impl AsRef<[u8]>,
) -> Self {
self.insert_common_storage_key(key, value);
self
}
}
impl<C> Storage for ContractTester<C>
where
C: TestableNymContract,
{
fn get(&self, key: &[u8]) -> Option<Vec<u8>> {
self.storage.get(key)
}
fn range<'a>(
&'a self,
start: Option<&[u8]>,
end: Option<&[u8]>,
order: Order,
) -> Box<dyn Iterator<Item = Record> + 'a> {
self.storage.range(start, end, order)
}
fn range_keys<'a>(
&'a self,
start: Option<&[u8]>,
end: Option<&[u8]>,
order: Order,
) -> Box<dyn Iterator<Item = Vec<u8>> + 'a> {
self.storage.range_keys(start, end, order)
}
fn range_values<'a>(
&'a self,
start: Option<&[u8]>,
end: Option<&[u8]>,
order: Order,
) -> Box<dyn Iterator<Item = Vec<u8>> + 'a> {
self.storage.range_values(start, end, order)
}
fn set(&mut self, key: &[u8], value: &[u8]) {
self.storage.set(key, value)
}
fn remove(&mut self, key: &[u8]) {
self.storage.remove(key)
}
}
@@ -11,7 +11,7 @@ use std::rc::Rc;
pub struct StorageWrapper(Rc<RefCell<MemoryStorage>>);
impl StorageWrapper {
pub(super) fn contract_storage_wrapper(&self, contract: &Addr) -> ContractStorageWrapper {
pub fn contract_storage_wrapper(&self, contract: &Addr) -> ContractStorageWrapper {
ContractStorageWrapper {
address: contract.clone(),
inner: self.clone(),
@@ -24,7 +24,7 @@ impl StorageWrapper {
}
#[derive(Debug, Clone)]
pub(crate) struct ContractStorageWrapper {
pub struct ContractStorageWrapper {
address: Addr,
inner: StorageWrapper,
}
@@ -33,6 +33,22 @@ impl ContractStorageWrapper {
pub fn inner_storage(&self) -> StorageWrapper {
self.inner.clone()
}
pub fn as_inner_storage(&self) -> &StorageWrapper {
&self.inner
}
pub fn as_inner_storage_mut(&mut self) -> &mut StorageWrapper {
&mut self.inner
}
#[must_use = "this returns the result of the operation, without modifying the original"]
pub fn change_contract(&self, contract: &Addr) -> Self {
ContractStorageWrapper {
address: contract.clone(),
inner: self.inner.clone(),
}
}
}
impl Storage for StorageWrapper {
@@ -18,6 +18,7 @@ serde = { workspace = true, features = ["derive"] }
thiserror = { workspace = true }
[dev-dependencies]
anyhow = { workspace = true }
serde_json = { workspace = true }
[build-dependencies]
@@ -35,7 +35,7 @@ pub enum ContractsCommonError {
/// Percent represents a value between 0 and 100%
/// (i.e. between 0.0 and 1.0)
#[cw_serde]
#[derive(Copy, Default, PartialOrd)]
#[derive(Copy, Default, PartialOrd, Ord, Eq)]
pub struct Percent(#[serde(deserialize_with = "de_decimal_percent")] Decimal);
impl Percent {
@@ -80,6 +80,44 @@ impl Percent {
pub fn checked_pow(&self, exp: u32) -> Result<Self, OverflowError> {
self.0.checked_pow(exp).map(Percent)
}
// truncate provided percent to only have 2 decimal places,
// e.g. convert "0.1234567" into "0.12"
// the purpose of it is to reduce storage space, in particular for the performance contract
// since that extra precision gains us nothing
#[must_use = "this returns the result of the operation, without modifying the original"]
pub fn round_to_two_decimal_places(&self) -> Self {
let raw = self.0;
const DECIMAL_FRACTIONAL: Uint128 = Uint128::new(1_000_000_000_000_000_000u128); // 1*10**18
const THRESHOLD: Decimal = Decimal::permille(5); // 0.005
// in case it ever changes since it's not exposed in the public API
debug_assert_eq!(
DECIMAL_FRACTIONAL,
Uint128::new(10).pow(Decimal::DECIMAL_PLACES)
);
let int = (raw.atomics() * Uint128::new(100)) / DECIMAL_FRACTIONAL;
#[allow(clippy::unwrap_used)]
let floored = Decimal::from_atomics(int, 2).unwrap();
let diff = raw - floored;
let rounded = if diff >= THRESHOLD {
// ceil
floored + Decimal::percent(1)
} else {
floored
};
Percent(rounded)
}
#[must_use = "this returns the result of the operation, without modifying the original"]
pub fn average(&self, other: &Self) -> Self {
let sum = self.0 + other.0;
let inner = Decimal::from_ratio(sum.numerator(), sum.denominator() * Uint128::new(2));
Percent(inner)
}
}
impl Display for Percent {
@@ -334,6 +372,7 @@ mod tests {
}
#[test]
#[cfg(feature = "naive_float")]
fn naive_float_conversion() {
// around 15 decimal places is the maximum precision we can handle
// which is still way more than enough for what we use it for
@@ -347,4 +386,41 @@ mod tests {
assert!(converted.0 - converted.0 < epsilon);
}
#[test]
fn rounding_percent() {
let test_cases = vec![
("0", "0"),
("0.1", "0.1"),
("0.12", "0.12"),
("0.12", "0.123"),
("0.12", "0.123456789"),
("0.13", "0.125"),
("0.13", "0.126"),
("0.13", "0.126436545676"),
("0.99", "0.99"),
("0.99", "0.994"),
("1", "0.999"),
("1", "0.995"),
];
for (expected, input) in test_cases {
let expected: Percent = expected.parse().unwrap();
let pre_truncated: Percent = input.parse().unwrap();
assert_eq!(expected, pre_truncated.round_to_two_decimal_places())
}
}
#[test]
fn calculating_average() -> anyhow::Result<()> {
fn p(raw: &str) -> Percent {
raw.parse().unwrap()
}
assert_eq!(p("0.1").average(&p("0.1")), p("0.1"));
assert_eq!(p("0.1").average(&p("0.2")), p("0.15"));
assert_eq!(p("1").average(&p("0")), p("0.5"));
assert_eq!(p("0.123").average(&p("0.456")), p("0.2895"));
Ok(())
}
}
@@ -23,7 +23,6 @@ semver = { workspace = true, features = ["serde"] }
schemars = { workspace = true }
thiserror = { workspace = true }
contracts-common = { path = "../contracts-common", package = "nym-contracts-common", version = "0.5.0" }
serde-json-wasm = { workspace = true }
humantime-serde = { workspace = true }
utoipa = { workspace = true, optional = true }
@@ -3,7 +3,7 @@
use crate::{IdentityKey, NodeId, SphinxKey};
use cosmwasm_schema::cw_serde;
use cosmwasm_std::{Addr, Coin};
use cosmwasm_std::{to_json_string, Addr, Coin};
use std::cmp::Ordering;
use std::fmt::Display;
@@ -154,7 +154,7 @@ pub struct GatewayConfigUpdate {
impl GatewayConfigUpdate {
pub fn to_inline_json(&self) -> String {
serde_json_wasm::to_string(self).unwrap_or_else(|_| "serialisation failure".into())
to_json_string(self).unwrap_or_else(|_| "serialisation failure".into())
}
}
@@ -57,7 +57,7 @@ pub trait NodeBond {
fn is_unbonding(&self) -> bool;
fn identity(&self) -> IdentityKeyRef;
fn identity(&self) -> IdentityKeyRef<'_>;
fn original_pledge(&self) -> &Coin;
@@ -125,7 +125,7 @@ impl NodeBond for MixNodeBond {
self.is_unbonding
}
fn identity(&self) -> IdentityKeyRef {
fn identity(&self) -> IdentityKeyRef<'_> {
self.identity()
}
@@ -178,7 +178,7 @@ impl NodeBond for NymNodeBond {
self.is_unbonding
}
fn identity(&self) -> IdentityKeyRef {
fn identity(&self) -> IdentityKeyRef<'_> {
self.identity()
}
@@ -16,7 +16,7 @@ use crate::{
Percent, ProfitMarginRange, SphinxKey,
};
use cosmwasm_schema::cw_serde;
use cosmwasm_std::{Addr, Coin, Decimal, StdResult, Uint128};
use cosmwasm_std::{to_json_string, Addr, Coin, Decimal, StdResult, Uint128};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
use serde_repr::{Deserialize_repr, Serialize_repr};
@@ -604,7 +604,7 @@ pub struct NodeCostParams {
impl NodeCostParams {
pub fn to_inline_json(&self) -> String {
serde_json_wasm::to_string(self).unwrap_or_else(|_| "serialisation failure".into())
to_json_string(self).unwrap_or_else(|_| "serialisation failure".into())
}
}
@@ -773,7 +773,7 @@ pub struct MixNodeConfigUpdate {
impl MixNodeConfigUpdate {
pub fn to_inline_json(&self) -> String {
serde_json_wasm::to_string(self).unwrap_or_else(|_| "serialisation failure".into())
to_json_string(self).unwrap_or_else(|_| "serialisation failure".into())
}
}
@@ -58,7 +58,7 @@ impl<'a> PrimaryKey<'a> for Role {
type Suffix = <u8 as PrimaryKey<'a>>::Suffix;
type SuperSuffix = <u8 as PrimaryKey<'a>>::SuperSuffix;
fn key(&self) -> Vec<Key> {
fn key(&self) -> Vec<Key<'_>> {
// I'm not sure why it wasn't possible to delegate the call to
// `(*self as u8).key()` directly...
// I guess because of the `Key::Ref(&'a [u8])` variant?
@@ -5,7 +5,7 @@ use crate::helpers::IntoBaseDecimal;
use crate::nym_node::Role;
use crate::{error::MixnetContractError, Percent};
use cosmwasm_schema::cw_serde;
use cosmwasm_std::Decimal;
use cosmwasm_std::{to_json_string, Decimal};
pub type Performance = Percent;
pub type WorkFactor = Decimal;
@@ -84,7 +84,26 @@ pub struct IntervalRewardParams {
impl IntervalRewardParams {
pub fn to_inline_json(&self) -> String {
serde_json_wasm::to_string(self).unwrap_or_else(|_| "serialisation failure".into())
to_json_string(self).unwrap_or_else(|_| "serialisation failure".into())
}
pub fn active_node_work(&self, standby_node_work: Decimal) -> WorkFactor {
self.active_set_work_factor * standby_node_work
}
pub fn standby_node_work(
&self,
rewarded_set_size: Decimal,
standby_set_size: Decimal,
) -> WorkFactor {
let f = self.active_set_work_factor;
let k = rewarded_set_size;
let one = Decimal::one();
// nodes in reserve
let k_r = standby_set_size;
one / (f * k - (f - one) * k_r)
}
}
@@ -109,18 +128,15 @@ pub struct RewardingParams {
impl RewardingParams {
pub fn active_node_work(&self) -> WorkFactor {
self.interval.active_set_work_factor * self.standby_node_work()
let standby_work = self.standby_node_work();
self.interval.active_node_work(standby_work)
}
pub fn standby_node_work(&self) -> WorkFactor {
let f = self.interval.active_set_work_factor;
let k = self.dec_rewarded_set_size();
let one = Decimal::one();
// nodes in reserve
let k_r = self.dec_standby_set_size();
one / (f * k - (f - one) * k_r)
let rewarded_set_size = self.dec_rewarded_set_size();
let standby_set_size = self.dec_standby_set_size();
self.interval
.standby_node_work(rewarded_set_size, standby_set_size)
}
pub fn rewarded_set_size(&self) -> u32 {
@@ -410,7 +426,7 @@ impl IntervalRewardingParamsUpdate {
}
pub fn to_inline_json(&self) -> String {
serde_json_wasm::to_string(self).unwrap_or_else(|_| "serialisation failure".into())
to_json_string(self).unwrap_or_else(|_| "serialisation failure".into())
}
}
@@ -44,6 +44,17 @@ pub struct RewardEstimate {
pub operating_cost: Decimal,
}
impl RewardEstimate {
pub const fn zero() -> RewardEstimate {
RewardEstimate {
total_node_reward: Decimal::zero(),
operator: Decimal::zero(),
delegates: Decimal::zero(),
operating_cost: Decimal::zero(),
}
}
}
#[cw_serde]
#[derive(Copy, Default)]
pub struct RewardDistribution {
@@ -3,6 +3,7 @@
use crate::config_score::{ConfigScoreParams, OutdatedVersionWeights, VersionScoreFormulaParams};
use crate::nym_node::Role;
use crate::reward_params::RewardedSetParams;
use crate::EpochId;
use contracts_common::Percent;
use cosmwasm_schema::cw_serde;
@@ -85,7 +86,11 @@ impl RewardedSet {
}
pub fn rewarded_set_size(&self) -> usize {
self.active_set_size() + self.standby.len()
self.active_set_size() + self.standby_set_size()
}
pub fn standby_set_size(&self) -> usize {
self.standby.len()
}
pub fn get_role(&self, node_id: NodeId) -> Option<Role> {
@@ -110,6 +115,13 @@ impl RewardedSet {
}
None
}
pub fn matches_parameters(&self, params: RewardedSetParams) -> bool {
self.entry_gateways.len() <= params.entry_gateways as usize
&& self.exit_gateways.len() <= params.exit_gateways as usize
&& self.layer1.len() + self.layer2.len() + self.layer3.len() <= params.mixnodes as usize
&& self.standby.len() <= params.standby as usize
}
}
#[cw_serde]
@@ -0,0 +1,29 @@
[package]
name = "nym-performance-contract-common"
version = "0.1.0"
authors.workspace = true
repository.workspace = true
homepage.workspace = true
documentation.workspace = true
edition.workspace = true
license.workspace = true
rust-version.workspace = true
readme.workspace = true
[dependencies]
thiserror = { workspace = true }
serde = { workspace = true }
schemars = { workspace = true }
cosmwasm-std = { workspace = true }
cosmwasm-schema = { workspace = true }
cw-controllers = { workspace = true }
nym-contracts-common = { path = "../contracts-common" }
[features]
schema = []
[lints]
workspace = true
@@ -0,0 +1,14 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
pub mod storage_keys {
pub const CONTRACT_ADMIN: &str = "contract-admin";
pub const INITIAL_EPOCH_ID: &str = "initial-epoch-id";
pub const LAST_SUBMISSION: &str = "last-submission";
pub const MIXNET_CONTRACT: &str = "mixnet-contract";
pub const AUTHORISED_COUNT: &str = "authorised-count";
pub const AUTHORISED: &str = "authorised";
pub const RETIRED: &str = "retired";
pub const PERFORMANCE_RESULTS: &str = "performance-results";
pub const SUBMISSION_METADATA: &str = "submission-metadata";
}
@@ -0,0 +1,39 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::{EpochId, NodeId};
use cosmwasm_std::Addr;
use cw_controllers::AdminError;
use thiserror::Error;
#[derive(Error, Debug, PartialEq)]
pub enum NymPerformanceContractError {
#[error("could not perform contract migration: {comment}")]
FailedMigration { comment: String },
#[error(transparent)]
Admin(#[from] AdminError),
#[error(transparent)]
StdErr(#[from] cosmwasm_std::StdError),
#[error("{address} is already an authorised network monitor")]
AlreadyAuthorised { address: Addr },
#[error("{address} is not an authorised network monitor")]
NotAuthorised { address: Addr },
#[error("attempted to submit performance data for epoch {epoch_id} and node {node_id} whilst last submitted was {last_epoch_id} for node {last_node_id}")]
StalePerformanceSubmission {
epoch_id: EpochId,
node_id: NodeId,
last_epoch_id: EpochId,
last_node_id: NodeId,
},
#[error("the batch performance data has not been sorted")]
UnsortedBatchSubmission,
#[error("node {node_id} does not appear to be bonded")]
NodeNotBonded { node_id: NodeId },
}
@@ -0,0 +1,2 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
@@ -0,0 +1,12 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
pub mod constants;
pub mod error;
pub mod helpers;
pub mod msg;
pub mod types;
pub use error::*;
pub use msg::{ExecuteMsg, InstantiateMsg, MigrateMsg, QueryMsg};
pub use types::*;
@@ -0,0 +1,125 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::{EpochId, NodeId, NodePerformance};
use cosmwasm_schema::cw_serde;
#[cfg(feature = "schema")]
use crate::types::{
EpochMeasurementsPagedResponse, EpochPerformancePagedResponse,
FullHistoricalPerformancePagedResponse, LastSubmission, NetworkMonitorResponse,
NetworkMonitorsPagedResponse, NodeMeasurementsResponse, NodePerformancePagedResponse,
NodePerformanceResponse, RetiredNetworkMonitorsPagedResponse,
};
#[cw_serde]
pub struct InstantiateMsg {
pub mixnet_contract_address: String,
pub authorised_network_monitors: Vec<String>,
}
#[cw_serde]
pub enum ExecuteMsg {
/// Change the admin
UpdateAdmin { admin: String },
/// Attempt to submit performance data of a particular node for given epoch
Submit {
epoch: EpochId,
data: NodePerformance,
},
/// Attempt to submit performance data of a batch of nodes for given epoch
BatchSubmit {
epoch: EpochId,
data: Vec<NodePerformance>,
},
/// Attempt to authorise new network monitor for submitting performance data
AuthoriseNetworkMonitor { address: String },
/// Attempt to retire an existing network monitor and forbid it from submitting any future performance data
RetireNetworkMonitor { address: String },
/// An admin method to remove submitted node measurements. Used as an escape hatch should
/// the data stored get too unwieldy.
RemoveNodeMeasurements { epoch_id: EpochId, node_id: NodeId },
/// An admin method to remove submitted nodes measurements. Used as an escape hatch should
/// the data stored get too unwieldy. Note: it is expected to get called multiple times
/// until the response indicates all the epoch data has been removed.
RemoveEpochMeasurements { epoch_id: EpochId },
}
#[cw_serde]
#[cfg_attr(feature = "schema", derive(cosmwasm_schema::QueryResponses))]
pub enum QueryMsg {
#[cfg_attr(feature = "schema", returns(cw_controllers::AdminResponse))]
Admin {},
/// Returns performance of particular node for the provided epoch
#[cfg_attr(feature = "schema", returns(NodePerformanceResponse))]
NodePerformance { epoch_id: EpochId, node_id: NodeId },
/// Returns historical performance for particular node
#[cfg_attr(feature = "schema", returns(NodePerformancePagedResponse))]
NodePerformancePaged {
node_id: NodeId,
start_after: Option<EpochId>,
limit: Option<u32>,
},
/// Returns all submitted measurements for the particular node
#[cfg_attr(feature = "schema", returns(NodeMeasurementsResponse))]
NodeMeasurements { epoch_id: EpochId, node_id: NodeId },
/// Returns (paged) measurements for particular epoch
#[cfg_attr(feature = "schema", returns(EpochMeasurementsPagedResponse))]
EpochMeasurementsPaged {
epoch_id: EpochId,
start_after: Option<NodeId>,
limit: Option<u32>,
},
/// Returns (paged) performance for particular epoch
#[cfg_attr(feature = "schema", returns(EpochPerformancePagedResponse))]
EpochPerformancePaged {
epoch_id: EpochId,
start_after: Option<NodeId>,
limit: Option<u32>,
},
/// Returns full (paged) historical performance of the whole network
#[cfg_attr(feature = "schema", returns(FullHistoricalPerformancePagedResponse))]
FullHistoricalPerformancePaged {
start_after: Option<(EpochId, NodeId)>,
limit: Option<u32>,
},
/// Returns information about particular network monitor
#[cfg_attr(feature = "schema", returns(NetworkMonitorResponse))]
NetworkMonitor { address: String },
/// Returns information about all network monitors
#[cfg_attr(feature = "schema", returns(NetworkMonitorsPagedResponse))]
NetworkMonitorsPaged {
start_after: Option<String>,
limit: Option<u32>,
},
/// Returns information about all retired network monitors
#[cfg_attr(feature = "schema", returns(RetiredNetworkMonitorsPagedResponse))]
RetiredNetworkMonitorsPaged {
start_after: Option<String>,
limit: Option<u32>,
},
/// Returns information regarding the latest submitted performance data
#[cfg_attr(feature = "schema", returns(LastSubmission))]
LastSubmittedMeasurement {},
}
#[cw_serde]
pub struct MigrateMsg {
//
}
@@ -0,0 +1,258 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use cosmwasm_schema::cw_serde;
use cosmwasm_std::{Addr, Env, Timestamp};
use nym_contracts_common::Percent;
pub type EpochId = u32;
pub type NodeId = u32;
#[cw_serde]
pub struct LastSubmission {
pub block_height: u64,
pub block_time: Timestamp,
// not as relevant, but might as well store it
pub data: Option<LastSubmittedData>,
}
#[cw_serde]
pub struct LastSubmittedData {
pub sender: Addr,
pub epoch_id: EpochId,
pub data: NodePerformance,
}
#[cw_serde]
pub struct NetworkMonitorDetails {
pub address: Addr,
pub authorised_by: Addr,
pub authorised_at_height: u64,
}
impl NetworkMonitorDetails {
pub fn retire(self, env: &Env, sender: &Addr) -> RetiredNetworkMonitor {
RetiredNetworkMonitor {
details: self,
retired_by: sender.clone(),
retired_at_height: env.block.height,
}
}
}
#[cw_serde]
pub struct RetiredNetworkMonitor {
pub details: NetworkMonitorDetails,
pub retired_by: Addr,
pub retired_at_height: u64,
}
#[cw_serde]
#[derive(Copy)]
pub struct NodePerformance {
#[serde(rename = "n")]
pub node_id: NodeId,
// note: value is rounded to 2 decimal places.
#[serde(rename = "p")]
pub performance: Percent,
}
#[cw_serde]
pub struct NetworkMonitorSubmissionMetadata {
pub last_submitted_epoch_id: EpochId,
pub last_submitted_node_id: NodeId,
}
// the internal values are always sorted
#[cw_serde]
pub struct NodeResults(Vec<Percent>);
impl NodeResults {
pub fn new(initial: Percent) -> NodeResults {
NodeResults(vec![initial.round_to_two_decimal_places()])
}
// ASSUMPTION: number of NM will be relatively small, so loading the whole vector of values
// to insert new one and resave is cheap
pub fn insert_new(&mut self, result: Percent) {
let result = result.round_to_two_decimal_places();
let pos = self.0.binary_search(&result).unwrap_or_else(|e| e);
self.0.insert(pos, result);
}
// SAFETY: there are no codepaths that allow constructing empty struct
pub fn median(&self) -> Percent {
let len = self.0.len();
if len % 2 == 1 {
// odd number of elements: return the middle one
self.0[len / 2]
} else {
// even number: average the two middle elements
let mid1 = self.0[len / 2 - 1];
let mid2 = self.0[len / 2];
mid1.average(&mid2).round_to_two_decimal_places()
}
}
pub fn inner(&self) -> &[Percent] {
&self.0
}
}
#[cw_serde]
pub struct NodePerformanceResponse {
pub performance: Option<Percent>,
}
#[cw_serde]
pub struct NodeMeasurementsResponse {
pub measurements: Option<NodeResults>,
}
#[cw_serde]
#[derive(Copy)]
pub struct EpochNodePerformance {
pub epoch: EpochId,
pub performance: Option<Percent>,
}
#[cw_serde]
pub struct NodePerformancePagedResponse {
pub node_id: NodeId,
pub performance: Vec<EpochNodePerformance>,
pub start_next_after: Option<EpochId>,
}
#[cw_serde]
pub struct EpochPerformancePagedResponse {
pub epoch_id: EpochId,
pub performance: Vec<NodePerformance>,
pub start_next_after: Option<NodeId>,
}
#[cw_serde]
pub struct NodeMeasurement {
pub node_id: NodeId,
pub measurements: NodeResults,
}
#[cw_serde]
pub struct EpochMeasurementsPagedResponse {
pub epoch_id: EpochId,
pub measurements: Vec<NodeMeasurement>,
pub start_next_after: Option<NodeId>,
}
#[cw_serde]
#[derive(Copy)]
pub struct HistoricalPerformance {
pub epoch_id: EpochId,
pub node_id: NodeId,
pub performance: Percent,
}
#[cw_serde]
pub struct FullHistoricalPerformancePagedResponse {
pub performance: Vec<HistoricalPerformance>,
pub start_next_after: Option<(EpochId, NodeId)>,
}
#[cw_serde]
pub struct NetworkMonitorInformation {
pub details: NetworkMonitorDetails,
pub current_submission_metadata: NetworkMonitorSubmissionMetadata,
}
#[cw_serde]
pub struct NetworkMonitorResponse {
pub info: Option<NetworkMonitorInformation>,
}
#[cw_serde]
pub struct NetworkMonitorsPagedResponse {
pub info: Vec<NetworkMonitorInformation>,
pub start_next_after: Option<String>,
}
#[cw_serde]
pub struct RetiredNetworkMonitorsPagedResponse {
pub info: Vec<RetiredNetworkMonitor>,
pub start_next_after: Option<String>,
}
#[cw_serde]
pub struct RemoveEpochMeasurementsResponse {
pub additional_entries_to_remove_remaining: bool,
}
#[cw_serde]
#[derive(Default)]
pub struct BatchSubmissionResult {
pub accepted_scores: u64,
pub non_existent_nodes: Vec<NodeId>,
}
#[cfg(test)]
mod tests {
use super::*;
fn p(raw: impl AsRef<str>) -> Percent {
raw.as_ref().parse().unwrap()
}
fn ps(raw: &[&str]) -> Vec<Percent> {
raw.iter().map(p).collect()
}
#[test]
fn node_results_insertion() {
let initial = NodeResults::new(p("0.5"));
let mut smaller = initial.clone();
let mut greater = initial.clone();
smaller.insert_new(p("0.4"));
greater.insert_new(p("0.6"));
assert_eq!(smaller.0, ps(&["0.4", "0.5"]));
assert_eq!(greater.0, ps(&["0.5", "0.6"]));
let mut another = NodeResults(ps(&["0.1", "0.4", "0.5", "0.6", "0.6", "1.0"]));
another.insert_new(p("0.6"));
another.insert_new(p("0.2"));
another.insert_new(p("0.7"));
another.insert_new(p("0.3"));
another.insert_new(p("0.3"));
another.insert_new(p("0.55"));
assert_eq!(
another.0,
ps(&[
"0.1", "0.2", "0.3", "0.3", "0.4", "0.5", "0.55", "0.6", "0.6", "0.6", "0.7", "1.0"
])
);
}
#[test]
fn node_results_median() {
let results = NodeResults(ps(&["0.1"]));
assert_eq!(results.median(), p("0.1"));
let results = NodeResults(ps(&["0.1", "0.2"]));
assert_eq!(results.median(), p("0.15"));
let results = NodeResults(ps(&["0.1", "0.2", "0.3"]));
assert_eq!(results.median(), p("0.2"));
let results = NodeResults(ps(&["0.1", "0.2", "0.3", "0.4"]));
assert_eq!(results.median(), p("0.25"));
let results = NodeResults(ps(&["0.1", "0.2", "0.3", "0.4", "0.5"]));
assert_eq!(results.median(), p("0.3"));
let results = NodeResults(ps(&["0", "0", "1", "1", "1", "1", "1"]));
assert_eq!(results.median(), p("1"));
}
}
+9 -2
View File
@@ -20,6 +20,8 @@ nym-credentials = { path = "../credentials" }
nym-compact-ecash = { path = "../nym_offline_compact_ecash" }
nym-ecash-time = { path = "../ecash-time" }
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.sqlx-pool-guard]
path = "../../sqlx-pool-guard"
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.sqlx]
workspace = true
@@ -31,8 +33,13 @@ features = ["rt-multi-thread", "net", "signal", "fs"]
[build-dependencies]
sqlx = { workspace = true, features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate"] }
sqlx = { workspace = true, features = [
"runtime-tokio-rustls",
"sqlite",
"macros",
"migrate",
] }
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
[features]
persistent-storage = ["bincode", "serde"]
persistent-storage = ["bincode", "serde"]
@@ -0,0 +1,123 @@
/*
* Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
* SPDX-License-Identifier: Apache-2.0
*/
-- 1. add temporary `epoch_id` column
ALTER TABLE pending_issuance
ADD COLUMN epoch_id INTEGER;
-- 2. populate the value
UPDATE pending_issuance
SET epoch_id = (SELECT epoch_id
FROM expiration_date_signatures
WHERE expiration_date_signatures.expiration_date = pending_issuance.expiration_date);
-- 3. create new expiration_date_signatures table (with changed constraints)
CREATE TABLE expiration_date_signatures_new
(
expiration_date DATE NOT NULL,
epoch_id INTEGER NOT NULL,
serialization_revision INTEGER NOT NULL,
-- combined signatures for all tuples issued for given day
serialised_signatures BLOB NOT NULL,
PRIMARY KEY (epoch_id, expiration_date)
);
-- 4. migrate the data
INSERT INTO expiration_date_signatures_new (expiration_date, epoch_id, serialization_revision, serialised_signatures)
SELECT expiration_date, epoch_id, serialization_revision, serialised_signatures
FROM expiration_date_signatures;
-- 5. drop and recreate the table references (due to new FK)
-- 5.1.
-- (data for ticketbooks that have an associated deposit, but failed to get issued)
CREATE TABLE pending_issuance_new
(
deposit_id INTEGER NOT NULL PRIMARY KEY,
-- introduce a way for us to introduce breaking changes in serialization of data
serialization_revision INTEGER NOT NULL,
pending_ticketbook_data BLOB NOT NULL UNIQUE,
-- for each ticketbook we MUST have corresponding expiration date signatures
expiration_date DATE NOT NULL,
epoch_id INTEGER NOT NULL,
-- for each ticketbook we MUST have corresponding expiration date signatures
FOREIGN KEY (epoch_id, expiration_date) REFERENCES expiration_date_signatures_new (epoch_id, expiration_date)
);
INSERT INTO pending_issuance_new (deposit_id, serialization_revision, pending_ticketbook_data, expiration_date,
epoch_id)
SELECT deposit_id, serialization_revision, pending_ticketbook_data, expiration_date, epoch_id
FROM pending_issuance;
-- 5.2.
CREATE TABLE ecash_ticketbook_new
(
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
-- introduce a way for us to introduce breaking changes in serialization of data
serialization_revision INTEGER NOT NULL,
-- the type of the associated ticketbook
ticketbook_type TEXT NOT NULL,
-- the actual crypto data of the ticketbook (wallet, keys, etc.)
ticketbook_data BLOB NOT NULL UNIQUE,
-- for each ticketbook we MUST have corresponding expiration date signatures
expiration_date DATE NOT NULL,
-- for each ticketbook we MUST have corresponding coin index signatures
epoch_id INTEGER NOT NULL,
-- the initial number of tickets the wallet has been created for
total_tickets INTEGER NOT NULL,
-- how many tickets have been used so far (the `l` value of the wallet)
used_tickets INTEGER NOT NULL,
-- FOREIGN KEYS:
-- for each ticketbook we MUST have corresponding coin index signatures
FOREIGN KEY (epoch_id) REFERENCES coin_indices_signatures (epoch_id),
-- for each ticketbook we MUST have corresponding expiration date signatures
FOREIGN KEY (expiration_date, epoch_id) REFERENCES expiration_date_signatures_new (expiration_date, epoch_id)
);
INSERT INTO ecash_ticketbook_new (id, serialization_revision, ticketbook_type, ticketbook_data, expiration_date,
epoch_id, total_tickets, used_tickets)
SELECT id,
serialization_revision,
ticketbook_type,
ticketbook_data,
expiration_date,
epoch_id,
total_tickets,
used_tickets
FROM ecash_ticketbook;
-- 6. finally swap out the old tables
-- drop old tables
DROP TABLE pending_issuance;
DROP TABLE ecash_ticketbook;
DROP TABLE expiration_date_signatures;
-- rename new tables
ALTER TABLE pending_issuance_new
RENAME TO pending_issuance;
ALTER TABLE ecash_ticketbook_new
RENAME TO ecash_ticketbook;
ALTER TABLE expiration_date_signatures_new
RENAME TO expiration_date_signatures;
@@ -28,7 +28,7 @@ struct EcashCredentialManagerInner {
pending: HashMap<i64, RetrievedPendingTicketbook>,
master_vk: HashMap<u64, VerificationKeyAuth>,
coin_indices_sigs: HashMap<u64, Vec<AnnotatedCoinIndexSignature>>,
expiration_date_sigs: HashMap<Date, Vec<AnnotatedExpirationDateSignature>>,
expiration_date_sigs: HashMap<(u64, Date), Vec<AnnotatedExpirationDateSignature>>,
_next_id: i64,
}
@@ -242,10 +242,14 @@ impl MemoryEcachTicketbookManager {
pub(crate) async fn get_expiration_date_signatures(
&self,
expiration_date: Date,
epoch_id: u64,
) -> Option<Vec<AnnotatedExpirationDateSignature>> {
let guard = self.inner.read().await;
guard.expiration_date_sigs.get(&expiration_date).cloned()
guard
.expiration_date_sigs
.get(&(epoch_id, expiration_date))
.cloned()
}
pub(crate) async fn insert_expiration_date_signatures(
@@ -254,8 +258,9 @@ impl MemoryEcachTicketbookManager {
) {
let mut guard = self.inner.write().await;
guard
.expiration_date_sigs
.insert(sigs.expiration_date, sigs.signatures.clone());
guard.expiration_date_sigs.insert(
(sigs.epoch_id, sigs.expiration_date),
sigs.signatures.clone(),
);
}
}
@@ -7,10 +7,11 @@ use crate::models::{
};
use nym_ecash_time::Date;
use sqlx::{Executor, Sqlite, Transaction};
use sqlx_pool_guard::SqlitePoolGuard;
#[derive(Clone)]
pub struct SqliteEcashTicketbookManager {
connection_pool: sqlx::SqlitePool,
connection_pool: SqlitePoolGuard,
}
impl SqliteEcashTicketbookManager {
@@ -19,7 +20,7 @@ impl SqliteEcashTicketbookManager {
/// # Arguments
///
/// * `connection_pool`: database connection pool to use.
pub fn new(connection_pool: sqlx::SqlitePool) -> Self {
pub fn new(connection_pool: SqlitePoolGuard) -> Self {
SqliteEcashTicketbookManager { connection_pool }
}
@@ -33,12 +34,12 @@ impl SqliteEcashTicketbookManager {
"DELETE FROM ecash_ticketbook WHERE expiration_date <= ?",
deadline
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
pub(crate) async fn begin_storage_tx(&self) -> Result<Transaction<Sqlite>, sqlx::Error> {
pub(crate) async fn begin_storage_tx(&self) -> Result<Transaction<'_, Sqlite>, sqlx::Error> {
self.connection_pool.begin().await
}
@@ -60,7 +61,7 @@ impl SqliteEcashTicketbookManager {
data,
expiration_date,
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
@@ -90,7 +91,7 @@ impl SqliteEcashTicketbookManager {
epoch_id,
total_tickets,
used_tickets,
).execute(&self.connection_pool).await?;
).execute(&*self.connection_pool).await?;
Ok(())
}
@@ -105,7 +106,7 @@ impl SqliteEcashTicketbookManager {
"#,
)
.bind(data)
.fetch_optional(&self.connection_pool)
.fetch_optional(&*self.connection_pool)
.await?
.is_some();
@@ -121,7 +122,7 @@ impl SqliteEcashTicketbookManager {
FROM ecash_ticketbook
"#,
)
.fetch_all(&self.connection_pool)
.fetch_all(&*self.connection_pool)
.await
}
@@ -143,7 +144,7 @@ impl SqliteEcashTicketbookManager {
ticketbook_id,
expected_current_total_spent
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?
.rows_affected();
Ok(affected > 0)
@@ -153,7 +154,7 @@ impl SqliteEcashTicketbookManager {
&self,
) -> Result<Vec<StoredPendingTicketbook>, sqlx::Error> {
sqlx::query_as("SELECT * FROM pending_issuance")
.fetch_all(&self.connection_pool)
.fetch_all(&*self.connection_pool)
.await
}
@@ -165,7 +166,7 @@ impl SqliteEcashTicketbookManager {
"DELETE FROM pending_issuance WHERE deposit_id = ?",
pending_id
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
@@ -182,7 +183,7 @@ impl SqliteEcashTicketbookManager {
"#,
epoch_id
)
.fetch_optional(&self.connection_pool)
.fetch_optional(&*self.connection_pool)
.await
}
@@ -208,7 +209,7 @@ impl SqliteEcashTicketbookManager {
serialisation_revision,
epoch_id
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
@@ -225,7 +226,7 @@ impl SqliteEcashTicketbookManager {
"#,
epoch_id
)
.fetch_optional(&self.connection_pool)
.fetch_optional(&*self.connection_pool)
.await
}
@@ -251,7 +252,7 @@ impl SqliteEcashTicketbookManager {
serialisation_revision,
epoch_id,
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
@@ -259,17 +260,19 @@ impl SqliteEcashTicketbookManager {
pub(crate) async fn get_expiration_date_signatures(
&self,
expiration_date: Date,
epoch_id: i64,
) -> Result<Option<RawExpirationDateSignatures>, sqlx::Error> {
sqlx::query_as!(
RawExpirationDateSignatures,
r#"
SELECT epoch_id as "epoch_id: u32", serialised_signatures, serialization_revision as "serialization_revision: u8"
SELECT serialised_signatures, serialization_revision as "serialization_revision: u8"
FROM expiration_date_signatures
WHERE expiration_date = ?
WHERE expiration_date = ? AND epoch_id = ?
"#,
expiration_date
expiration_date,
epoch_id
)
.fetch_optional(&self.connection_pool)
.fetch_optional(&*self.connection_pool)
.await
}
@@ -298,7 +301,7 @@ impl SqliteEcashTicketbookManager {
serialisation_revision,
expiration_date
)
.execute(&self.connection_pool)
.execute(&*self.connection_pool)
.await?;
Ok(())
}
@@ -166,10 +166,11 @@ impl Storage for EphemeralStorage {
async fn get_expiration_date_signatures(
&self,
expiration_date: Date,
epoch_id: u64,
) -> Result<Option<Vec<AnnotatedExpirationDateSignature>>, Self::StorageError> {
Ok(self
.storage_manager
.get_expiration_date_signatures(expiration_date)
.get_expiration_date_signatures(expiration_date, epoch_id)
.await)
}

Some files were not shown because too many files have changed in this diff Show More