Compare commits

...

98 Commits

Author SHA1 Message Date
Tommy Verrall 55a0f80d73 Feat: implement supply chain attack mitigation
- Add yarn resolutions for vulnerable packages (chalk, strip-ansi, color-convert, etc.)
- Add .npmrc and .nvmrc security configurations
2025-09-10 18:51:38 +02:00
Jędrzej Stuczyński d3cdaf373b Feature/credential proxy crate (#6018)
* moved storage and deposits buffer to the common lib

* move more of the state into the shared lib

* extracted the rest of the features into the shared lib

* fixed test imports

* clippy
2025-09-10 09:28:38 +01:00
Jędrzej Stuczyński 7c5f10a219 refresh mixnet contract on epoch progression (#6023) 2025-09-09 09:59:54 +01:00
Simon Wicky f90fc4f2f0 Moving clients crate from vpn-client repo to here (#6015)
* moving crates as is

* changes due to crate moving

* cargo fmt
2025-09-08 10:50:18 +02:00
Jędrzej Stuczyński e95aca715c feat: use ShutdownToken (CancellationToken inside) for nym-api (#5997)
* make nym-api use ShutdownToken instead of TaskClient

* ignore public-api tests if env is not set

* removed default features to avoid pulling in openssl
2025-09-08 09:45:28 +01:00
Bogdan-Ștefan Neacşu 8e7d1d510d Use default value for the ports until api is deployed (#6007) 2025-09-04 15:55:56 +03:00
import this 4062734a31 [DOCs/operators]: NIP-2 tokenomics update & fix csv2md bug (#6008) 2025-09-04 11:29:44 +02:00
import this ccd8ff26a3 Feature: Delegation program stake checker and adjuster (#5980)
* initialise stake adjustment program

* add readme file with a simple guide

* syntax

* syntax

* FINISHED: faster and returning more data

* change dwl link to develop branch
2025-09-03 16:06:06 +00:00
import this 43d043a9cd Feature: Nym node autorun CLI (#5916)
* initial commit - add prereqs install script

* add env vars prompt

* automate latest binary url env var

* add install node script

* add modes to nym-node install script

* start main cli framework

* adding branch var for easier deployment and testing

* add systemd config

* add proxy and wss setup script

* add landing page stub and fix nginx script

* add nginx setup

* fix typo

* add checks for existing dir and wg prompt

* add nginx commands

* add service file check

* add service file check

* convention alignment

* add checks to nginx setup

* cleanup old code

* add bonding prompt and nym node run fns

* fix syntax

* fix syntax

* fix syntax

* fix syntax

* fix syntax

* fix syntax

* fix syntax

* fix syntax

* add service script to init

* fix syntax

* fix syntax

* add chmod

* fix script logic

* syntax fix

* syntax fix

* silent mode trial

* fix evn prompt script

* make scripts interactive

* indent fix

* correct node-install script

* initial mixnode setup working - gws need more love

* fix bonding function

* syntax fix

* improve run noide as service script

* improve service script

* improve run service fn

* fix logic

* beautify

* beautify

* create run node as service script

* syntax fix

* attempt to resolve memory running out issue

* attempt to resolve memory running out issue

* attempt to resolve memory running out issue

* attempt to resolve memory running out issue

* attempt to resolve memory running out issue

* attempt to resolve memory running out issue

* attempt to resolve memory running out issue

* attempt to resolve memory running out issue

* setting wireguard

* solved memory issues

* rename landing page template

* modify wireguard enabled fn

* layout change

* syntax fix

* modify node setup script

* sync up envs

* return missing function

* fix urls

* fix network manager script execution

* fix wss and nginx

* fix layout

* tweak WG contion

* syntax fix

* add init placeholder

* syntax fix

* redefine wireguard check logic

* check if node exists

* add argparse and dev option

* styling

* add panic

* add error message

* improve logic

* improve logic

* add arg

* add dev arg for all levels

* add confirmation loop

* styling

* fix bonding question

* syntax edit

* syntax edit

* syntax edit

* refactor for already bonded nodes

* add default branch on top and define metavar

* fix node install script

* clean and prepare for review

* indentation fix

* fix nginx setup

* fix nginx setup

* style cleanup

* fix try error logic

* tune --dev option to run before command correctly

* fix y/n convention across the modules

* add explorer URL to the message

* minor layout fixes
2025-09-02 20:34:24 +00:00
Drazen Urch 3d6cf730c2 NS-API: Cast to BIGINT to make i64 work (#6003) 2025-09-02 18:35:25 +01:00
Jędrzej Stuczyński c0f8d98b63 bugfix: return from MixTrafficController if client request channel has closed (#6002) 2025-09-02 10:23:25 +01:00
Jędrzej Stuczyński 91995da4f1 chore: use updated version of simulate endpoint (#5988) 2025-09-02 10:12:52 +01:00
Jędrzej Stuczyński 01fa1df66c feat: shared library for attempting to retrieve update mode attestation (#5954)
* feat: shared library for attempting to retrieve update mode attestation

* clippy

* add nym- prefix to the crate name

* use pure-rust impl for jwt-simple
2025-09-02 09:28:32 +01:00
Jędrzej Stuczyński baddaaac22 feat: nym signers monitor (#5933)
* initialise nym-signers-monitor

* creating nyxd client

* performing checks

* sending notifications on failure

* rate limitting on notifications + clippy
2025-09-02 09:27:09 +01:00
elsirion 2c4b5f168b fix: use WASM compatible time API in client (#5948) 2025-09-02 09:26:06 +01:00
Bogdan-Ștefan Neacşu a557ac22c7 Revert "Create an axum_test client for more integrated unit testing (#5956)" (#5999)
This reverts commit efd61eb47c.
2025-09-01 15:37:10 +03:00
Jędrzej Stuczyński 55ef89178b chore: upgraded syn to 2.0 and removed nym-execute (#5998) 2025-09-01 12:59:13 +01:00
Jędrzej Stuczyński d97be2d8ef bugfix: Recipient deserialisation for deserialisers missing bytes specialisation (#5991)
* bugfix: Recipient deserialisation for deserialisers missing bytes specialisation

for example toml or json will just default to visit_seq ignoring bytes related optimisations

* clippy
2025-09-01 11:30:35 +01:00
Bogdan-Ștefan Neacşu efd61eb47c Create an axum_test client for more integrated unit testing (#5956) 2025-09-01 13:27:06 +03:00
benedetta davico 4a01973b31 Merge pull request #5981 from nymtech/benny/ns-api-ci-fix
Fix the ns api ci workflow
2025-09-01 11:02:21 +02:00
Mark Sinclair 9ad9c3b8e7 Bug fix: NS API monikers (#5990)
* node-status-api: fix missing monikers because of deserialisation issues from unstructured data

* node-status-api: bump version after bug fix monikers

---------

Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
2025-09-01 09:48:37 +01:00
dependabot[bot] 6706500132 build(deps): bump pbkdf2 from 3.1.2 to 3.1.3 (#5869)
Bumps [pbkdf2](https://github.com/crypto-browserify/pbkdf2) from 3.1.2 to 3.1.3.
- [Changelog](https://github.com/browserify/pbkdf2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crypto-browserify/pbkdf2/compare/v3.1.2...v3.1.3)

---
updated-dependencies:
- dependency-name: pbkdf2
  dependency-version: 3.1.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 15:52:10 +01:00
dependabot[bot] 33fe059c28 Bump console from 0.15.11 to 0.16.0 (#5931)
Bumps [console](https://github.com/console-rs/console) from 0.15.11 to 0.16.0.
- [Release notes](https://github.com/console-rs/console/releases)
- [Changelog](https://github.com/console-rs/console/blob/main/CHANGELOG.md)
- [Commits](https://github.com/console-rs/console/compare/0.15.11...0.16.0)

---
updated-dependencies:
- dependency-name: console
  dependency-version: 0.16.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 14:24:03 +01:00
dependabot[bot] d6ed2b770b Bump indicatif from 0.17.11 to 0.18.0 (#5924)
Bumps [indicatif](https://github.com/console-rs/indicatif) from 0.17.11 to 0.18.0.
- [Release notes](https://github.com/console-rs/indicatif/releases)
- [Commits](https://github.com/console-rs/indicatif/compare/0.17.11...0.18.0)

---
updated-dependencies:
- dependency-name: indicatif
  dependency-version: 0.18.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 14:23:39 +01:00
dependabot[bot] 7c18a3dced Bump mock_instant from 0.5.3 to 0.6.0 (#5930)
Bumps [mock_instant](https://github.com/museun/mock_instant) from 0.5.3 to 0.6.0.
- [Commits](https://github.com/museun/mock_instant/commits/v0.6.0)

---
updated-dependencies:
- dependency-name: mock_instant
  dependency-version: 0.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 14:20:22 +01:00
dependabot[bot] 09475ab4e0 build(deps): bump mikefarah/yq from 4.45.4 to 4.47.1 (#5911)
Bumps [mikefarah/yq](https://github.com/mikefarah/yq) from 4.45.4 to 4.47.1.
- [Release notes](https://github.com/mikefarah/yq/releases)
- [Changelog](https://github.com/mikefarah/yq/blob/master/release_notes.txt)
- [Commits](https://github.com/mikefarah/yq/compare/v4.45.4...v4.47.1)

---
updated-dependencies:
- dependency-name: mikefarah/yq
  dependency-version: 4.47.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 13:36:39 +01:00
dependabot[bot] b7606cd2ef Bump defguard_wireguard_rs from v0.4.7 to v0.7.5 (#5928)
Bumps [defguard_wireguard_rs](https://github.com/DefGuard/wireguard-rs) from v0.4.7 to v0.7.5.
- [Release notes](https://github.com/DefGuard/wireguard-rs/releases)
- [Commits](https://github.com/DefGuard/wireguard-rs/compare/ef1cf3714629bf5016fb38cbb7320451dc69fb09...d090d2249e5bb3d4154f07de098387e2ab69bfdc)

---
updated-dependencies:
- dependency-name: defguard_wireguard_rs
  dependency-version: d090d2249e5bb3d4154f07de098387e2ab69bfdc
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 13:35:42 +01:00
dependabot[bot] 006a57312d Bump tokio from 1.46.1 to 1.47.1 (#5929)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.46.1 to 1.47.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.46.1...tokio-1.47.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.47.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 13:35:18 +01:00
dependabot[bot] 9b5aded8a5 build(deps): bump actions/download-artifact from 4 to 5 (#5939)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 5.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 13:33:31 +01:00
dependabot[bot] f4a69636fe build(deps): bump actions/first-interaction from 1 to 3 (#5950)
Bumps [actions/first-interaction](https://github.com/actions/first-interaction) from 1 to 3.
- [Release notes](https://github.com/actions/first-interaction/releases)
- [Commits](https://github.com/actions/first-interaction/compare/v1...v3)

---
updated-dependencies:
- dependency-name: actions/first-interaction
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 13:31:29 +01:00
dependabot[bot] 0463d88646 Bump slab from 0.4.10 to 0.4.11 (#5952)
Bumps [slab](https://github.com/tokio-rs/slab) from 0.4.10 to 0.4.11.
- [Release notes](https://github.com/tokio-rs/slab/releases)
- [Changelog](https://github.com/tokio-rs/slab/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/slab/compare/v0.4.10...v0.4.11)

---
updated-dependencies:
- dependency-name: slab
  dependency-version: 0.4.11
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 13:31:02 +01:00
dependabot[bot] 534bf5d824 build(deps): bump actions/setup-java from 4 to 5 (#5975)
Bumps [actions/setup-java](https://github.com/actions/setup-java) from 4 to 5.
- [Release notes](https://github.com/actions/setup-java/releases)
- [Commits](https://github.com/actions/setup-java/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-java
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 13:30:00 +01:00
dependabot[bot] 34684b14db Bump sha.js from 2.4.11 to 2.4.12 (#5983)
Bumps [sha.js](https://github.com/crypto-browserify/sha.js) from 2.4.11 to 2.4.12.
- [Changelog](https://github.com/browserify/sha.js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crypto-browserify/sha.js/compare/v2.4.11...v2.4.12)

---
updated-dependencies:
- dependency-name: sha.js
  dependency-version: 2.4.12
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-29 13:29:18 +01:00
Jędrzej Stuczyński b2266d04ef chore: internal hidden command to force advance nyx epoch (#5964) 2025-08-29 11:41:22 +01:00
Jędrzej Stuczyński 911b365609 chore: purge temp databases on build (#5984)
* purge any temp databases on build

* updated min rust version

* fixed clippy::manual_abs_diff' in verloc due to updated msrv

* wasm
2025-08-29 11:41:08 +01:00
Jędrzej Stuczyński e9acc014ed feat: credential proxy deposit pool (#5945)
* chore: rename VpnApiError to CredentialProxyError

* reorganise deposit flow

* updated sql tables et al.

* insert information about deposit usage failure

* remove old deposit maker

* nym credential proxy to monitor quorum state to stop issuance if it'd fail

* clippy

* target lock new modules

* windows clippy

* renamed migration file due to rebasing
2025-08-29 09:39:57 +01:00
Jędrzej Stuczyński 0f66e5a154 bugfix: make sure tables are removed in correct order to not trigger FK constraint issue (#5987) 2025-08-29 09:03:22 +01:00
Bogdan-Ștefan Neacşu f8337d9b38 Wireguard metadata client library (#5943) 2025-08-28 15:43:46 +03:00
Bogdan-Ștefan Neacşu 4fb252c44b Wireguard private metadata (#5915) 2025-08-28 15:14:52 +03:00
Jędrzej Stuczyński 17708cdf92 bugfix: manually calculate per node work on rewarded set changes (#5972) 2025-08-27 12:33:24 +01:00
Andrej Mihajlov a9c56ef9ac Merge pull request #5976 from nymtech/am/update-sysinfo
Update sysinfo to the latest
2025-08-27 10:58:06 +02:00
Jędrzej Stuczyński 724420f97c chore: move authenticator into gateway crate (#5982)
* removed unused bits of authenticator config

* moved authenticator into gateway

* cleaned up imports

* clippy
2025-08-27 09:05:02 +01:00
benedettadavico 66d0296f47 update dockerfile to pg 2025-08-26 16:02:11 +02:00
benedettadavico 03bbbf44e9 ns api ci fix 2025-08-26 16:02:11 +02:00
dynco-nym 0a48fa6172 Remove freshness check on testrun submit (#5977)
* Remove freshness check on testrun submit
- freshness is enforced by a background task
  that marks testruns as stale after a
  configured amount of time

* Move code around

* Add humantime

* Update launch script

* Fix typo

* Adjust agent run script

* Configure user agent

* Bump version
2025-08-26 12:26:13 +02:00
Andrej Mihajlov 5c8749a2e1 Update sysinfo to the latest
Shakes out windows@0.57 from the tree
2025-08-23 09:29:47 +02:00
p17o 18d9d807f2 Added VPS provider hostraha.com (#5959) 2025-08-22 12:31:31 +00:00
import this 3a7393d316 [DOCs/operators]: Roll back from v2025.15-gruyere to v2025.14-feta (#5973)
* release notes and version bump up

* update stats

* roll version back and comment new notes
2025-08-22 11:23:16 +00:00
import this 6ce5f707c6 [DOCs/operators]: Release notes for v2025.15 gruyere (#5969)
* release notes and version bump up

* update stats
2025-08-21 11:49:52 +00:00
benedetta davico 766a1d4497 Merge pull request #5965 from nymtech/benny/fix-ns-agent-ci
fixing the ci for ns agent
2025-08-21 12:22:05 +02:00
benedetta davico 35c83f0a31 Merge pull request #5967 from nymtech/release/2025.15-gruyere
merge gruyere to develop
2025-08-21 12:20:43 +02:00
benedetta davico 01dd4a7972 Merge pull request #5958 from nymtech/bugfix/linux-build-ci
bugfix: fix ci-build for linux (and use updated runner)
2025-08-21 10:32:51 +02:00
Jędrzej Stuczyński c2e335557e Feature/testing utils (#5963)
* helper wrapper for stream-sink channel

* similar helper for async read/write

* example tests and clippy
2025-08-20 16:17:09 +01:00
benedettadavico 40e1cbc7a9 update changelog 2025-08-20 12:34:04 +02:00
Jędrzej Stuczyński c133e0e88b chore: updated refs to cheddar rev of nym repo (#5955)
* chore: updated refs to cheddar rev of nym repo

* update statistics-api version
2025-08-19 09:28:42 +01:00
Andrej Mihajlov 5b716633de Merge pull request #5960 from nymtech/am/update-strum
Migrate strum to 0.27.2
2025-08-18 18:28:22 +02:00
Andrej Mihajlov 834538300d Migrate to strum 0.27.2 2025-08-18 18:02:41 +02:00
Jędrzej Stuczyński bd0d70f7cd bugfix: fix ci-build for linux (and use updated runner) 2025-08-15 09:37:41 +01:00
Jędrzej Stuczyński 979485c582 http api client adjustment (#5953)
* missing feature lock for attempting to clone client

* added helper macro to generate user agent without additional imports
2025-08-13 12:52:16 +01:00
Bogdan-Ștefan Neacşu d95f66bd90 Move credential verifier in peer controller (#5938)
* Move credential verifier in peer controller

* Send back errors of peer controller
2025-08-13 13:09:44 +03:00
Jędrzej Stuczyński 906dfb2fb0 change PK/FK on expiration date signatures tables (#5934)
* update nym-credential-proxy

* update credential-storage

* update nym-api

* clippy
2025-08-12 09:03:53 +01:00
Jędrzej Stuczyński 7daa726626 feat: introduce additional checks when attempting to send to bounded channels (#5941)
* feat: introduce additional checks when attempting to send to bounded channels

or to a fallible gateway

* return error rather than panic when merging socket during shutdown
2025-08-11 09:15:12 +01:00
Jędrzej Stuczyński 067f492ad6 chore: fix rust 1.89 clippy issues (#5944) 2025-08-08 13:03:05 +01:00
Jędrzej Stuczyński ed73ec9ce6 chore: remove unused import (#5942) 2025-08-07 09:56:08 +01:00
import this 61606630bd [DOCs/operators]: Release notes v2025.14-feta (#5935)
* update release

* fix typos
2025-08-06 08:24:38 +00:00
benedettadavico 2d3deeb424 bump versions 2025-08-06 09:56:08 +02:00
benedetta davico 3827dc357d Merge pull request #5936 from nymtech/release/2025.14-feta
Merge release/2025.14-feta to develop
2025-08-06 09:54:29 +02:00
benedetta davico a70e9e23d3 Merge branch 'develop' into release/2025.14-feta 2025-08-06 09:48:22 +02:00
Jędrzej Stuczyński dc59149a5d squashed feature/ecash-liveness-check (#5890) (#5890)
delay to gruyere

chore: delay to Feta

added threshold information to the response

nym api test clippy

bugfixes and endpoint improvements

expose results on api endpoints

wip: making nym api monitor network signers

added fallback legacy queries to get basic support idea

refactored the code to expose bool-only methods for status

ecash-signer-check lib for obtaining basic ecash signer information
2025-08-05 12:28:42 +01:00
benedetta davico e418c7587a Merge pull request #5914 from nymtech/feature/nym-node-gw-reset
nym-node debug command to reset providers db
2025-08-05 12:05:44 +02:00
import this 33339c085d [DOCs/operators]: Update ISP list (#5918)
* update ISP list

* remove typo
2025-07-31 13:47:27 +00:00
Sachin Kamath 863f329106 docs: update validator instructions and waitlist callout (#5922) 2025-07-30 15:03:39 +00:00
import this 314a37cabe WG exit policy scripts update (#5921)
* add NIP-3 ports to WG manager script

* add monero ports to local testing script

* console output snippet update
2025-07-30 09:43:39 +00:00
Jack Wampler 917f391948 Make DNS Resolver fallback optional (#5920)
default to no dns system fallback, but keep support
2025-07-29 11:00:24 -06:00
Jędrzej Stuczyński 0b4deda621 nym-node debug command to reset providers db 2025-07-25 13:33:12 +01:00
benedetta davico d01867ca8d save version 2025-07-25 11:27:42 +02:00
benedetta davico 502c63b291 Fix broken CI 2025-07-25 11:19:27 +02:00
Jędrzej Stuczyński a4e674c98b basic zulip client for sending messages (#5913) 2025-07-24 16:22:35 +01:00
Jędrzej Stuczyński 7f97f13799 chore: nym node tokio console (#5909)
* conditionally enable console-subscriber within nym-node

* Update ci-build-upload-binaries.yml

* Update ci-build-upload-binaries.yml

add features console

* updated feature name

* fixed filtering on tracing layers

* add track_caller when spawning futures for better tokio-console support

* allow [client] tasks to specify their names when used within tokio console

* clippy

* pre-emptively fix wasm clippy

---------

Co-authored-by: Tommy Verrall <60836166+tommyv1987@users.noreply.github.com>
2025-07-24 11:00:58 +01:00
Bogdan-Ștefan Neacşu b975d08342 Remove old free credential handle (#5864)
* Set cached storage counters to 0 (#5812)

* Set cached storage counters to 0

* u64 to i64 log possible error

* Check addition too

Debug commit

Remove more data from wg storage peer

Put actual ticket type in storage

Simplify add peer

Finish rebase

Pass defguard Peer

Cache less data for consumption

GatewayStorage traits

Wg API trait

Mock test structures

Unit test for peer controller

EcashManager trait

Init test of Authenticator

Remove peer test

* Fix windows different API

* Use make_bincode_serializer like in other places

* Add log_slow_statements to gateway storage

* Use correct LevelFilter

* Fix clippy

* More win fix

* Win clippy

* Use two error variants more

* Use only one Arc<RwLock<T>> instead of many more

* Remove commented test

* Specific trait import
2025-07-23 17:07:12 +03:00
Jędrzej Stuczyński 8e44f9f07f chore: allow compatibility with 'CDLA-Permissive-2.0' (#5910) 2025-07-23 14:48:40 +01:00
benedettadavico 85604e8305 bump versions 2025-07-23 10:18:45 +02:00
benedetta davico 8461d085a5 Merge pull request #5906 from nymtech/release/2025.13-emmental
merge release/2025.13-emmental to develop
2025-07-22 16:23:29 +02:00
Drazen Urch af9f6e5ca0 Allow PG database backend (#5880)
* feat(db): add SQL query wrapper for PostgreSQL placeholder conversion

- Created query_wrapper module with functions to automatically convert
  SQLite ? placeholders to PostgreSQL $1, $2, ... format
- Updated build.rs to handle mutually exclusive feature flags
- Modified one query in mixnodes.rs as proof of concept
- Added type conversions for PostgreSQL compatibility (u32->i64, u16->i32)

This is a checkpoint commit before converting all queries to use the wrapper.

* feat(nym-node-status-api): add PostgreSQL database support via feature flags

Implement dual database support for SQLite and PostgreSQL through Cargo feature flags.
The implementation uses a query wrapper that automatically converts SQLite-style ?
placeholders to PostgreSQL-style $1, $2, ... placeholders at runtime.

Key changes:
- Add query wrapper functions that handle placeholder conversion
- Convert all sqlx::query\! macros to use wrapper functions
- Handle type conversions between databases (i64 vs i32)
- Add feature-gated implementations for database-specific SQL syntax
- Update Makefile with clippy targets for both database features
- Document database support in README

* feat(nym-node-status-agent): add multi-API support with random selection

Agents can now connect to multiple APIs and randomly select one for each testrun:
- Accept multiple --server arguments in format "address:port:auth_key"
- Randomly shuffle server list before attempting connections
- Try each server until a testrun is obtained
- Submit results back only to the API that provided the testrun
- Continue to next server if one is down or has no testruns available

* feat(nym-node-status): implement primary/secondary server architecture

- Agent now requests testruns only from primary server (first in list)
- Results are submitted to all configured servers in parallel
- Secondary servers accept external testruns via new v2 endpoint
- Added auto-creation of gateway and testrun records on secondary servers
- New database queries: get_or_create_gateway, insert_external_testrun
- Client library enhanced with submit_results_with_context method

* Bump Node status API version

* Fix build workdir

* Bump to 3.1.4

* Fix types and queries

* 3.1.6

* Fix gateway perf, bump 3.1.7

* NodeId -> i32, 3.1.8

* Bump agent version

* i64 -> i32

* Use image yq

* Migration and more types

* Update remaining JSONB columns

* Simplify server config

* Update build path

* Change delimiter

* bump agent

* Split up pg and sqlite builds

* More typing fixes, build-and-push script

* Fix Dockerfile-pg

* Bump node-status-api

* TYping

* Agent build script

* More logging around testruns

* Fail loudly on read errors

* Cleanup

* Debug get gateways query

* Fix get_gateways query

* Use pg cert, 3.1.16

* Submit regular results to primary server

* Bump freshenss cutoff

* Update Cargo.lock

* fix: resolve rebase conflicts and compilation errors

After rebasing onto develop, fixed several issues:
- Fixed borrowed data escapes error by using sqlx::query directly in transaction functions
- Removed unused imports and cleaned up code
- Maintained database-specific implementations for transaction functions

* fmt

* Make PG default to make lives easier

* Performance improvements for Explorer v2

* Fix sqlite build

* Fix PG migration

* Tests round 1

* DB tests

* More tests

* And some more tests

* And some more, more tests

* cargo fmt

* Fix some failing lints

* Fix lioness version problems

* Clippy in tests

---------

Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
2025-07-22 15:25:43 +02:00
import this a9ae2017f5 [DOCs/operators]: Release notes/v2025.13 emmental & NIP-3 announcement (#5908)
* initialise PR, add dev notes and bump node version

* add operators tool and update api stats
2025-07-22 12:10:43 +00:00
Bogdan-Ștefan Neacşu 09ebe7f9e9 Support mnemonic in the NS agent (#5883)
Co-authored-by: benedettadavico <benedetta.davico@gmail.com>
2025-07-22 14:21:12 +03:00
Andrej Mihajlov b72915c224 Merge pull request #5905 from nymtech/am/sqlx-guard-obtain-db-path-from-pool
sqlx-pool-guard: obtain filename from connect options
2025-07-22 11:57:55 +02:00
Andrej Mihajlov add3e864e3 sqlx-pool-guard: obtain filename from connect options 2025-07-22 11:09:39 +02:00
benedettadavico 578c9b0567 update changelog 2025-07-22 11:09:35 +02:00
Andrej Mihajlov 8f6f696f36 Merge pull request #5896 from nymtech/am/handle-table-allocate-more-memory 2025-07-22 11:09:11 +02:00
Jędrzej Stuczyński e9165763b6 Feature/dkg snapshot epoch (#5900)
* define storage item for holding historical DKG state

* make all epoch storage operations go through proxy functions

* make each saving action also apply to the historical item

* removed usage of update_epoch function

* test correct save heights

* exposed query for the epoch state at specified height

* regenerated contract schema

* restored default cw-plus behaviour as in hindsight it makes more sense
2025-07-21 17:32:57 +01:00
mfahampshire 6c1149708b GW Probe docs: Go dep. + new required mnemonic (#5897)
* add note on go dep

* updated -h and useage doc
2025-07-18 12:36:30 +00:00
Mark Sinclair aaf6931d78 nym-node-status-ui placeholder (#5902)
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
2025-07-17 20:04:45 +01:00
Jędrzej Stuczyński 97804f2fe5 Feature/dkg epoch dealers query (#5899)
* feat: add GetEpochDealers and GetEpochDealersAddresses queries to the DKG contract

* extended DkgQueryClient with new queries

* updated contract schema

* unit tests
2025-07-17 12:26:01 +01:00
Jędrzej Stuczyński 802d9b69ca fix: don't allow mixnode running in exit mode (#5898)
* fix: don't allow mixnode running in exit mode

* fixed error message
2025-07-17 10:57:16 +01:00
Andrej Mihajlov 7313857bc8 Allocate more memory to account for a drift in handle table size in between calls 2025-07-16 13:29:45 +02:00
benedettadavico 779174ada5 update wallet changelog 2025-07-15 14:57:49 +02:00
benedettadavico 329ad83fc0 bump versions 2025-07-15 10:04:51 +02:00
644 changed files with 39938 additions and 16889 deletions
@@ -38,15 +38,14 @@ jobs:
rm -rf ci-builds || true
mkdir -p $OUTPUT_DIR
echo $OUTPUT_DIR
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install libudev-dev
- name: Sets env vars for tokio if set in manual dispatch inputs
run: |
echo 'RUSTFLAGS="--cfg tokio_unstable"' >> $GITHUB_ENV
if: github.event_name == 'workflow_dispatch' && inputs.add_tokio_unstable == true
run: |
echo "RUSTFLAGS=--cfg tokio_unstable" >> $GITHUB_ENV
echo "CARGO_FEATURES=--features tokio-console" >> $GITHUB_ENV
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
@@ -103,7 +102,6 @@ jobs:
if [ ${{ github.event_name == 'workflow_dispatch' && inputs.enable_deb == true }} = true ]; then
cp target/debian/*.deb $OUTPUT_DIR
fi
- name: Deploy branch to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
+7 -7
View File
@@ -38,7 +38,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ arc-ubuntu-22.04, custom-windows-11, custom-macos-15 ]
os: [ arc-linux-latest, custom-windows-11, custom-macos-15 ]
runs-on: ${{ matrix.os }}
env:
CARGO_TERM_COLOR: always
@@ -46,9 +46,9 @@ jobs:
RUSTUP_PERMIT_COPY_RENAME: 1
steps:
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get -y install libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools protobuf-compiler
run: sudo apt-get update && sudo apt-get -y install libwebkit2gtk-4.0-dev build-essential curl wget libssl-dev libgtk-3-dev libudev-dev squashfs-tools protobuf-compiler cmake
continue-on-error: true
if: contains(matrix.os, 'ubuntu')
if: contains(matrix.os, 'linux')
- name: Check out repository code
uses: actions/checkout@v4
@@ -63,7 +63,7 @@ jobs:
# To avoid running out of disk space, skip generating debug symbols
- name: Set debug to false (unix)
if: contains(matrix.os, 'ubuntu') || contains(matrix.os, 'mac')
if: contains(matrix.os, 'linux') || contains(matrix.os, 'mac')
run: |
sed -i.bak 's/\[profile.dev\]/\[profile.dev\]\ndebug = false/' Cargo.toml
git diff
@@ -93,14 +93,14 @@ jobs:
command: build
- name: Build all examples
if: contains(matrix.os, 'ubuntu')
if: contains(matrix.os, 'linux')
uses: actions-rs/cargo@v1
with:
command: build
args: --workspace --examples
- name: Run all tests
if: contains(matrix.os, 'ubuntu')
if: contains(matrix.os, 'linux')
uses: actions-rs/cargo@v1
env:
NYM_API: https://sandbox-nym-api1.nymtech.net/api
@@ -109,7 +109,7 @@ jobs:
args: --workspace
- name: Run expensive tests
if: (github.ref == 'refs/heads/develop' || github.event.pull_request.base.ref == 'develop' || github.event.pull_request.base.ref == 'master') && contains(matrix.os, 'ubuntu')
if: (github.ref == 'refs/heads/develop' || github.event.pull_request.base.ref == 'develop' || github.event.pull_request.base.ref == 'master') && contains(matrix.os, 'linux')
uses: actions-rs/cargo@v1
with:
command: test
@@ -16,7 +16,7 @@ jobs:
uses: actions/checkout@v4
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
@@ -16,7 +16,7 @@ jobs:
uses: actions/checkout@v4
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
+1 -1
View File
@@ -6,7 +6,7 @@ jobs:
greeting:
runs-on: ubuntu-latest
steps:
- uses: actions/first-interaction@v1
- uses: actions/first-interaction@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
issue-message: 'Thank you for raising this issue'
+1 -1
View File
@@ -31,7 +31,7 @@ jobs:
- name: Check out repository code
uses: actions/checkout@v4
- name: Download report from previous job
uses: actions/download-artifact@v4
uses: actions/download-artifact@v5
with:
name: report
path: .github/workflows/support-files/notifications
@@ -25,7 +25,7 @@ jobs:
uses: actions/checkout@v4
- name: Install Java
uses: actions/setup-java@v4
uses: actions/setup-java@v5
with:
distribution: "temurin"
java-version: "17"
@@ -91,7 +91,7 @@ jobs:
- name: Checkout
uses: actions/checkout@v4
- name: Download binary artifact
uses: actions/download-artifact@v4
uses: actions/download-artifact@v5
with:
name: nyms5-apk-arch64
path: apk
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-credential-proxy/Cargo.toml
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-network-monitor/Cargo.toml
@@ -38,10 +38,10 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
run: |
VERSION=$(yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml)
echo "result=$VERSION" >> $GITHUB_OUTPUT
- name: cleanup-gateway-probe-ref
id: cleanup_gateway_probe_ref
@@ -53,13 +53,16 @@ jobs:
- name: Set GIT_TAG variable
run: echo "GIT_TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}" >> $GITHUB_ENV
- name: Set RELEASE_TAG variable
- name: Initialize RELEASE_TAG
run: echo "RELEASE_TAG=" >> $GITHUB_ENV
- name: Set RELEASE_TAG for release
if: github.event.inputs.release_image == 'true'
run: echo "RELEASE_TAG=golden-" >> $GITHUB_ENV
- name: Set IMAGE_NAME_AND_TAGS variable
run: echo "IMAGE_NAME_AND_TAGS=${{ env.CONTAINER_NAME }}:${{ env.RELEASE_TAG }}${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}" >> $GITHUB_ENV
- name: New env vars
run: echo "RELEASE_TAG='$RELEASE_TAG' GIT_TAG='$GIT_TAG' IMAGE_NAME_AND_TAGS='$IMAGE_NAME_AND_TAGS'"
+10 -7
View File
@@ -32,21 +32,24 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
run: |
VERSION=$(yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml)
echo "result=$VERSION" >> $GITHUB_OUTPUT
- name: Set GIT_TAG variable
run: echo "GIT_TAG=${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}" >> $GITHUB_ENV
- name: Set RELEASE_TAG variable
- name: Initialise RELEASE_TAG
run: echo "RELEASE_TAG=" >> $GITHUB_ENV
- name: Set RELEASE_TAG for release
if: github.event.inputs.release_image == 'true'
run: echo "RELEASE_TAG=golden-" >> $GITHUB_ENV
- name: Set IMAGE_NAME_AND_TAGS variable
run: echo "IMAGE_NAME_AND_TAGS=${{ env.CONTAINER_NAME }}:${{ env.RELEASE_TAG }}${{ steps.get_version.outputs.result }}" >> $GITHUB_ENV
- name: New env vars
run: echo "RELEASE_TAG='$RELEASE_TAG' GIT_TAG='$GIT_TAG' IMAGE_NAME_AND_TAGS='$IMAGE_NAME_AND_TAGS'"
@@ -66,6 +69,6 @@ jobs:
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.IMAGE_NAME_AND_TAGS }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile-pg . -t harbor.nymte.ch/nym/${{ env.IMAGE_NAME_AND_TAGS }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-api/Cargo.toml
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
@@ -26,7 +26,7 @@ jobs:
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.45.4
uses: mikefarah/yq@v4.47.1
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
+58
View File
@@ -0,0 +1,58 @@
# Security and sensitive files
.env*
*.key
*.pem
*.p12
*.pfx
secrets/
private/
config/secrets/
# Development files
node_modules/
.npm/
.npmrc
.nvmrc
*.log
*.tmp
.DS_Store
Thumbs.db
# Build artifacts
dist/
build/
target/
*.tgz
*.tar.gz
# IDE files
.vscode/
.idea/
*.swp
*.swo
*~
# Test files
test/
tests/
__tests__/
*.test.js
*.test.ts
*.spec.js
*.spec.ts
# Documentation
docs/
*.md
!README.md
# CI/CD files
.github/
.gitlab-ci.yml
.travis.yml
.circleci/
azure-pipelines.yml
# Scripts
scripts/
!scripts/security-check.sh
+21
View File
@@ -0,0 +1,21 @@
audit-level=moderate
fund=false
update-notifier=false
ignore-scripts=false
strict-ssl=true
registry=https://registry.npmjs.org/
audit=true
package-lock=true
package-lock-only=false
save-exact=false
# use npm ci for production builds (faster and more secure)
# this will be enforced in CI/CD scripts
# prevent installation of optional dependencies that might contain vulnerabilities
optional=false
audit=true
update-notifier=false
save-exact=false
+1
View File
@@ -0,0 +1 @@
20.18.0
+84
View File
@@ -4,6 +4,90 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
## [Unreleased]
## [2025.15-gruyere] (2025-08-20)
- Migrate strum to 0.27.2 ([#5960])
- WG exit policy scripts update ([#5921])
- Make DNS Resolver fallback optional ([#5920])
- nym-node debug command to reset providers db ([#5914])
- basic zulip client for sending messages ([#5913])
- chore: allow compatibility with 'CDLA-Permissive-2.0' ([#5910])
- feat: ecash liveness check ([#5890])
- Remove old free credential handle ([#5864])
[#5960]: https://github.com/nymtech/nym/pull/5960
[#5921]: https://github.com/nymtech/nym/pull/5921
[#5920]: https://github.com/nymtech/nym/pull/5920
[#5914]: https://github.com/nymtech/nym/pull/5914
[#5913]: https://github.com/nymtech/nym/pull/5913
[#5910]: https://github.com/nymtech/nym/pull/5910
[#5890]: https://github.com/nymtech/nym/pull/5890
[#5864]: https://github.com/nymtech/nym/pull/5864
## [2025.14-feta] (2025-08-05)
- chore: nym node tokio console ([#5909])
- Feature/dkg snapshot epoch ([#5900])
- Feature/dkg epoch dealers query ([#5899])
- sqlx-pool-guard: allocate more memory on windows ([#5896])
- Support mnemonic in the NS agent ([#5883])
- Allow PG database backend ([#5880])
[#5909]: https://github.com/nymtech/nym/pull/5909
[#5900]: https://github.com/nymtech/nym/pull/5900
[#5899]: https://github.com/nymtech/nym/pull/5899
[#5896]: https://github.com/nymtech/nym/pull/5896
[#5883]: https://github.com/nymtech/nym/pull/5883
[#5880]: https://github.com/nymtech/nym/pull/5880
## [2025.13-emmental] (2025-07-22)
- fix: don't allow mixnode running in exit mode ([#5898])
- fix contract build process in Makefile ([#5892])
- bugfix: ignore 'Send' responses when claiming bandwidth ([#5884])
- Update push-node-status-agent.yaml ([#5882])
- listen for shutdown signals during nym-node startup ([#5879])
- feat: forbid running mixnode + entry on the same node ([#5878])
- chore: 1.88 clippy ([#5877])
- Batch SQL writes for packet stats ([#5874])
- fix the broken link ([#5873])
- Set busy_timeout in sqlx ([#5872])
- feat: basic performance contract integration [within Nym API] ([#5871])
- scraper bugfix: ignore precommits from missing validators ([#5867])
- Return true remaining ([#5866])
- Make Mix hops optional for Mixnet Client SURBs ([#5861])
- Check gateway supported versions ([#5860])
- Add build info endpoints ([#5857])
- Clear out screaming logs ([#5856])
- fix removal of qa env ([#5855])
- Use display when printing paths ([#5853])
- feat: initial performance contract ([#5833])
- Security patches for the `dkg` crate ([#5828])
- HTTP Discovery objects & network defaults ([#5814])
[#5898]: https://github.com/nymtech/nym/pull/5898
[#5892]: https://github.com/nymtech/nym/pull/5892
[#5884]: https://github.com/nymtech/nym/pull/5884
[#5882]: https://github.com/nymtech/nym/pull/5882
[#5879]: https://github.com/nymtech/nym/pull/5879
[#5878]: https://github.com/nymtech/nym/pull/5878
[#5877]: https://github.com/nymtech/nym/pull/5877
[#5874]: https://github.com/nymtech/nym/pull/5874
[#5873]: https://github.com/nymtech/nym/pull/5873
[#5872]: https://github.com/nymtech/nym/pull/5872
[#5871]: https://github.com/nymtech/nym/pull/5871
[#5867]: https://github.com/nymtech/nym/pull/5867
[#5866]: https://github.com/nymtech/nym/pull/5866
[#5861]: https://github.com/nymtech/nym/pull/5861
[#5860]: https://github.com/nymtech/nym/pull/5860
[#5857]: https://github.com/nymtech/nym/pull/5857
[#5856]: https://github.com/nymtech/nym/pull/5856
[#5855]: https://github.com/nymtech/nym/pull/5855
[#5853]: https://github.com/nymtech/nym/pull/5853
[#5833]: https://github.com/nymtech/nym/pull/5833
[#5828]: https://github.com/nymtech/nym/pull/5828
[#5814]: https://github.com/nymtech/nym/pull/5814
## [2025.12-dolcelatte] (2025-07-07)
- bugfix: key-rotation + reply SURBs ([#5876])
+686
View File
@@ -0,0 +1,686 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Nym is a privacy platform that uses mixnet technology to protect against metadata surveillance. The platform consists of several key components:
- Mixnet nodes (mixnodes) for packet mixing
- Gateways (entry/exit points for the network)
- Clients for interacting with the network
- Network monitoring tools
- Validators for network consensus
- Various service providers and integrations
## Build Commands
### Rust Components
```bash
# Default build (debug)
cargo build
# Release build
cargo build --release
# Build a specific package
cargo build -p <package-name>
# Build main components
make build
# Build release versions of main binaries and contracts
make build-release
# Build specific binaries
make build-nym-cli
cargo build -p nym-node --release
cargo build -p nym-api --release
```
### Testing
```bash
# Run clippy, unit tests, and formatting
make test
# Run all tests including slow tests
make test-all
# Run clippy on all workspaces
make clippy
# Run unit tests for a specific package
cargo test -p <package-name>
# Run only expensive/ignored tests
cargo test --workspace -- --ignored
# Run API tests
dotenv -f envs/sandbox.env -- cargo test --test public-api-tests
# Run tests with specific log level
RUST_LOG=debug cargo test -p <package-name>
# Run specific test scripts
./nym-node/tests/test_apis.sh
./scripts/wireguard-exit-policy/exit-policy-tests.sh
```
### Linting and Formatting
```bash
# Run rustfmt on all code
make fmt
# Check formatting without modifying
cargo fmt --all -- --check
# Run clippy with all targets
cargo clippy --workspace --all-targets -- -D warnings
# TypeScript linting
yarn lint
yarn lint:fix
yarn types:lint:fix
# Check dependencies for security/licensing issues
cargo deny check
```
### WASM Components
```bash
# Build all WASM components
make sdk-wasm-build
# Build TypeScript SDK
yarn build:sdk
npx lerna run --scope @nymproject/sdk build --stream
# Build and test WASM components
make sdk-wasm
# Build specific WASM packages
cd wasm/client && make
cd wasm/mix-fetch && make
cd wasm/node-tester && make
```
### Contract Development
```bash
# Build all contracts
make contracts
# Build contracts in release mode
make build-release-contracts
# Generate contract schemas
make contract-schema
# Run wasm-opt on contracts
make wasm-opt-contracts
# Check contracts with cosmwasm-check
make cosmwasm-check-contracts
```
### Running Components
```bash
# Run nym-node as a mixnode
cargo run -p nym-node -- run --mode mixnode
# Run nym-node as a gateway
cargo run -p nym-node -- run --mode gateway
# Run the network monitor
cargo run -p nym-network-monitor
# Run the API server
cargo run -p nym-api
# Run with specific environment
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Start a local network
./scripts/localnet_start.sh
```
## Architecture
The Nym platform consists of various components organized as a monorepo:
1. **Core Mixnet Infrastructure**:
- `nym-node`: Core binary supporting mixnode and gateway modes
- `common/nymsphinx`: Implementation of the Sphinx packet format
- `common/topology`: Network topology management
- `common/types`: Shared data types across components
2. **Network Monitoring**:
- `nym-network-monitor`: Monitors the network's reliability and performance
- `nym-api`: API server for network stats and monitoring data
- Metrics tracking for nodes, routes, and overall network health
3. **Client Implementations**:
- `clients/native`: Native Rust client implementation
- `clients/socks5`: SOCKS5 proxy client for standard applications
- `wasm`: WebAssembly client implementations (for browsers)
- `nym-connect`: Desktop and mobile clients
4. **Blockchain & Smart Contracts**:
- `common/cosmwasm-smart-contracts`: Smart contract implementations
- `contracts`: CosmWasm contracts for the Nym network
- `common/ledger`: Blockchain integration
5. **Utilities & Tools**:
- `tools`: Various CLI tools and utilities
- `sdk`: SDKs for different languages and platforms
- `documentation`: Documentation generation and management
## Packet System
Nym uses a modified Sphinx packet format for its mixnet:
1. **Message Chunking**:
- Messages are divided into "sets" and "fragments"
- Each fragment fits in a single Sphinx packet
- The `common/nymsphinx/chunking` module handles message fragmentation
2. **Routing**:
- Packets traverse through 3 layers of mixnodes
- Routing information is encrypted in layers (onion routing)
- The final gateway receives and processes the messages
3. **Monitoring**:
- Monitoring system tracks packet delivery through the network
- Routes are analyzed for reliability statistics
- Node performance metrics are collected
## Network Protocol
Nym implements the Loopix mixnet design with several key privacy features:
1. **Continuous-time Mixing**:
- Each mixnode delays messages independently with an exponential distribution
- This creates random reordering of packets, destroying timing correlations
- Offers better anonymity properties than batch mixing approaches
2. **Cover Traffic**:
- Clients and nodes generate dummy "loop" packets that circulate through the network
- These packets are indistinguishable from real traffic
- Creates a baseline level of traffic that hides actual communication patterns
- Provides unobservability (hiding when and how much real traffic is being sent)
3. **Stratified Network Architecture**:
- Traffic flows through Entry Gateway → 3 Mixnode Layers → Exit Gateway
- Path selection is independent per-message (unlike Tor)
- Each node connects only to adjacent layers
4. **Anonymous Replies**:
- Single-Use Reply Blocks (SURBs) allow receiving messages without revealing identity
- Enables bidirectional communication while maintaining privacy
## Network Monitoring Architecture
The network monitoring system is a core component that measures mixnet reliability:
1. The `nym-network-monitor` sends test packets through the network
2. These packets follow predefined routes through multiple mixnodes
3. Metrics are collected about:
- Successful and failed packet deliveries
- Node reliability (percentage of successful packet handling)
- Route reliability (which specific route combinations work best)
4. Results are stored in the database and used by `nym-api` to:
- Present node performance statistics
- Determine network rewards
- Provide route selection guidance to clients
In the current branch, metrics collection is being enhanced with a fanout approach to submit to multiple API endpoints.
## Development Environment
### Required Dependencies
- Rust toolchain (stable, 1.80+)
- Node.js (v20+) and yarn for TypeScript components
- SQLite for local database development
- PostgreSQL for API database (optional, for full API functionality)
- CosmWasm tools for contract development
- For building contracts: `wasm-opt` tool from `binaryen`
- Python 3.8+ for some scripts
- Docker (optional, for containerized development)
- protoc (Protocol Buffers compiler) for some components
### Environment Configurations
The `envs/` directory contains pre-configured environments:
#### Available Environments
- **`local.env`**: Local development environment
- Points to local services (localhost)
- Uses test mnemonics and keys
- Ideal for testing without external dependencies
- **`sandbox.env`**: Sandbox test network
- Public test network with real nodes
- Test tokens available from faucet
- Contract addresses for sandbox deployment
- API: https://sandbox-nym-api1.nymtech.net
- **`mainnet.env`**: Production mainnet
- Real network with real tokens
- Production contract addresses
- API: https://validator.nymtech.net
- Use with caution!
- **`canary.env`**: Canary deployment
- Pre-release testing environment
- Tests new features before mainnet
- **`mainnet-local-api.env`**: Hybrid environment
- Uses mainnet contracts but local API
- Useful for API development against mainnet data
#### Key Environment Variables
```bash
# Network configuration
NETWORK_NAME=sandbox # Network identifier
BECH32_PREFIX=n # Address prefix (n for sandbox, n for mainnet)
NYM_API=https://sandbox-nym-api1.nymtech.net/api
NYXD=https://rpc.sandbox.nymtech.net
NYM_API_NETWORK=sandbox
# Contract addresses (network-specific)
MIXNET_CONTRACT_ADDRESS=n1xr3rq8yvd7qplsw5yx90ftsr2zdhg4e9z60h5duusgxpv72hud3sjkxkav
VESTING_CONTRACT_ADDRESS=n1unyuj8qnmygvzuex3dwmg9yzt9alhvyeat0uu0jedg2wj33efl5qackslz
# ... other contract addresses
# Mnemonic for testing (NEVER use in production)
MNEMONIC="clutch captain shoe salt awake harvest setup primary inmate ugly among become"
# API Keys and tokens
IPINFO_API_TOKEN=your_token_here
AUTHENTICATOR_PASSWORD=password_here
# Logging
RUST_LOG=info # Options: error, warn, info, debug, trace
RUST_BACKTRACE=1 # Enable backtraces
# Database
DATABASE_URL=postgresql://user:pass@localhost/nym_api
```
#### Using Environment Files
```bash
# Load environment and run command
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Export to shell
source envs/sandbox.env
# Use with make targets
dotenv -f envs/sandbox.env -- make run-api-tests
```
## Initial Setup
### First Time Setup
1. **Install Prerequisites**
```bash
# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install Node.js and yarn
# Via nvm (recommended):
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
nvm install 20
npm install -g yarn
# Install build tools
# Ubuntu/Debian:
sudo apt-get install build-essential pkg-config libssl-dev protobuf-compiler libpq-dev
# macOS:
brew install protobuf postgresql
# Install wasm-opt for contract builds
npm install -g wasm-opt
# Add wasm target for Rust
rustup target add wasm32-unknown-unknown
```
2. **Clone and Setup Repository**
```bash
git clone https://github.com/nymtech/nym.git
cd nym/nym
# Install JavaScript dependencies
yarn install
# Build the project
make build
```
3. **Database Setup (Optional, for API development)**
```bash
# Install PostgreSQL
# Create database
createdb nym_api
# Run migrations (from nym-api directory)
cd nym-api
sqlx migrate run
```
### Quick Start
```bash
# Run a mixnode locally
dotenv -f envs/sandbox.env -- cargo run -p nym-node -- run --mode mixnode --id my-mixnode
# Run a gateway locally
dotenv -f envs/sandbox.env -- cargo run -p nym-node -- run --mode gateway --id my-gateway
# Run the API server
dotenv -f envs/sandbox.env -- cargo run -p nym-api
# Run a client
cargo run -p nym-client -- init --id my-client
cargo run -p nym-client -- run --id my-client
```
## CI/CD Pipeline
The project uses GitHub Actions for CI/CD with several key workflows:
1. **Build and Test**:
- `ci-build.yml`: Main build workflow for Rust components
- Tests are run on multiple platforms (Linux, Windows, macOS)
- Includes formatting check (rustfmt) and linting (clippy)
2. **Release Process**:
- Binary artifacts are published on release tags
- Multiple platform builds are created
3. **Documentation**:
- Documentation is automatically built and deployed
## Database Structure
The system uses SQLite databases with tables like:
- `mixnode_status`: Status information about mixnodes
- `gateway_status`: Status information about gateways
- `routes`: Route performance information (success/failure of specific paths)
- `monitor_run`: Information about monitoring test runs
## Development Workflows
### Running a Node
To run the mixnode or gateway:
```bash
# Run nym-node as a mixnode with specified identity
cargo run -p nym-node -- run --mode mixnode --id my-mixnode
# Run nym-node as a gateway
cargo run -p nym-node -- run --mode gateway --id my-gateway
```
### Configuration
Nodes can be configured with files in various locations:
- Command-line arguments
- Environment variables
- `.env` files specified with `--config-env-file`
### Monitoring
To monitor the health of your node:
- View logs for real-time information
- Use the node's HTTP API for status information
- Check the explorer for public node statistics
## Common Libraries
- `common/types`: Shared data types across all components
- `common/crypto`: Cryptographic primitives and wrappers
- `common/client-core`: Core client functionality
- `common/gateway-client`: Client-gateway communication
- `common/task`: Task management and concurrency utilities
- `common/nymsphinx`: Sphinx packet implementation for mixnet
- `common/topology`: Network topology management
- `common/credentials`: Credential system for privacy-preserving authentication
- `common/bandwidth-controller`: Bandwidth management and accounting
## Code Conventions
- Error handling: Use anyhow/thiserror for structured error handling
- Logging: Use the tracing framework for logging and diagnostics
- State management: Generally use Tokio/futures for async code
- Configuration: Use the config crate and env vars with defaults
- Database: Use sqlx for type-safe database queries
- Follow clippy recommendations and rustfmt formatting
- Use semantic commit messages: feat, fix, docs, refactor, test, chore
## When Making Changes
- Run `make test` before submitting PRs
- Follow Rust naming conventions
- Use `clippy` to check for common issues
- Update SQLx query caches when modifying DB queries: `cargo sqlx prepare`
- Consider backward compatibility for protocol changes
- Use lefthook pre-commit hooks for TypeScript formatting
- Run `cargo deny check` to verify dependency compliance
- Test against both sandbox and local environments when possible
- Update relevant documentation and CHANGELOG.md
## Development Tools
### Useful Cargo Commands
```bash
# Check for outdated dependencies
cargo outdated
# Analyze binary size
cargo bloat --release -p nym-node
# Generate dependency graph
cargo tree -p nym-api
# Run with instrumentation
cargo run --features profiling -p nym-node
# Check for security advisories
cargo audit
```
### Database Tools
```bash
# SQLx CLI for migrations
cargo install sqlx-cli
# Create new migration
cd nym-api && sqlx migrate add <migration_name>
# Prepare query metadata for offline compilation
cargo sqlx prepare --workspace
# View database schema
./nym-api/enter_db.sh
```
### Development Scripts
- `scripts/build_topology.py`: Generate network topology files
- `scripts/node_api_check.py`: Verify node API endpoints
- `scripts/network_tunnel_manager.sh`: Manage network tunnels
- `scripts/localnet_start.sh`: Start a local test network
- Various deployment scripts in `deployment/` for different environments
## Debugging
- Enable more verbose logging with the RUST_LOG environment variable:
```
RUST_LOG=debug,nym_node=trace cargo run -p nym-node -- run --mode mixnode
```
- Use the HTTP API endpoints for status information
- Check monitoring data in the database for network performance metrics
- For complex issues, use tracing tools to follow packet flow
- Enable backtraces: `RUST_BACKTRACE=full`
- For WASM debugging: Use browser developer tools with source maps
## Deployment and Advanced Configurations
### Deployment Structure
The `deployment/` directory contains Ansible playbooks and configurations for various deployment scenarios:
- **`aws/`**: AWS-specific deployment configurations
- **`mixnode/`**: Mixnode deployment playbooks
- **`gateway/`**: Gateway deployment playbooks
- **`validator/`**: Validator node deployment
- **`sandbox-v2/`**: Complete sandbox environment setup
- **`big-dipper-2/`**: Block explorer deployment
### Sandbox V2 Deployment
The sandbox-v2 deployment (`deployment/sandbox-v2/`) provides a complete test environment:
```bash
# Key playbooks:
- deploy.yaml # Main deployment orchestrator
- deploy-mixnodes.yaml # Deploy mixnodes
- deploy-gateways.yaml # Deploy gateways
- deploy-validators.yaml # Deploy validator nodes
- deploy-nym-api.yaml # Deploy API services
```
### Custom Environment Setup
To create a custom environment:
1. Copy an existing env file: `cp envs/sandbox.env envs/custom.env`
2. Modify the network endpoints and contract addresses
3. Update the `NETWORK_NAME` to your identifier
4. Set appropriate mnemonics and keys (use fresh ones for production!)
### Contract Addresses
Contract addresses are network-specific and defined in environment files:
- Mixnet contract: Manages mixnode/gateway registry
- Vesting contract: Handles token vesting schedules
- Coconut contracts: Privacy-preserving credentials
- Name service: Human-readable address mapping
- Ecash contract: Electronic cash functionality
### Local Network Setup
For a completely local network:
```bash
# Start local chain
./scripts/localnet_start.sh
# Deploy contracts
cd contracts
make deploy-local
# Start nodes with local config
dotenv -f envs/local.env -- cargo run -p nym-node -- run --mode mixnode
```
## Common Issues and Troubleshooting
### Database Issues
- When modifying database queries, you must update SQLx query caches:
```bash
cargo sqlx prepare
```
- If you see SQLx errors about missing query files, this is likely the cause
- For "database is locked" errors with SQLite, ensure only one process accesses the DB
- For PostgreSQL connection issues, verify DATABASE_URL and that the server is running
### API Connection Issues
- Check the environment variables pointing to the APIs (NYM_API, NYXD)
- Verify network connectivity and API health endpoints
- For authentication issues, check node keys and credentials
- Common endpoints to verify:
- API health: `$NYM_API/health`
- Chain status: `$NYXD/status`
- Contract info: `$NYXD/cosmwasm/wasm/v1/contract/$CONTRACT_ADDRESS`
### Build Problems
- Clean dependencies with `cargo clean` for a fresh build
- Check for compatible Rust version (1.80+ recommended)
- For smart contract builds, ensure wasm-opt is installed: `npm install -g wasm-opt`
- For cross-compilation issues, check target-specific dependencies
- WASM build issues: Ensure wasm32-unknown-unknown target is installed:
```bash
rustup target add wasm32-unknown-unknown
```
- For "cannot find -lpq" errors, install PostgreSQL development files:
```bash
# Ubuntu/Debian
sudo apt-get install libpq-dev
# macOS
brew install postgresql
```
### Environment Issues
- Contract address mismatches: Ensure you're using the correct environment file
- "Account sequence mismatch": The account nonce is out of sync, wait and retry
- Token decimal issues: Sandbox uses different decimal places than mainnet
- API version mismatches: Ensure your local API version matches the network
- "Insufficient funds": Get test tokens from faucet (sandbox) or check balance
- Gateway/mixnode bonding issues: Verify minimum stake requirements
## Working with Routes and Monitoring
1. Route monitoring metrics are stored in a `routes` table with:
- Layer node IDs (layer1, layer2, layer3, gw)
- Success flag (boolean)
- Timestamp
2. To analyze routes:
- Check `NetworkAccount` and `AccountingRoute` in `nym-network-monitor/src/accounting.rs`
- View monitoring logic in `common/nymsphinx/chunking/monitoring.rs`
- Observe how routes are submitted to the database in the `submit_accounting_routes_to_db` function
## Performance Optimization
### Profiling and Benchmarking
```bash
# Run benchmarks
cargo bench -p nym-node
# Profile with perf (Linux)
cargo build --release --features profiling
perf record --call-graph=dwarf ./target/release/nym-node run --mode mixnode
perf report
# Generate flamegraph
cargo install flamegraph
cargo flamegraph --bin nym-node -- run --mode mixnode
```
### Common Performance Considerations
- Use bounded channels for backpressure
- Batch database operations where possible
- Monitor memory usage with `RUST_LOG=nym_node::metrics=debug`
- Use connection pooling for database connections
- Consider using `jemalloc` for better memory allocation performance
Generated
+1934 -2393
View File
File diff suppressed because it is too large Load Diff
+31 -16
View File
@@ -39,9 +39,11 @@ members = [
"common/cosmwasm-smart-contracts/ecash-contract",
"common/cosmwasm-smart-contracts/group-contract",
"common/cosmwasm-smart-contracts/mixnet-contract",
"common/cosmwasm-smart-contracts/multisig-contract", "common/cosmwasm-smart-contracts/nym-performance-contract",
"common/cosmwasm-smart-contracts/multisig-contract",
"common/cosmwasm-smart-contracts/nym-performance-contract",
"common/cosmwasm-smart-contracts/nym-pool-contract",
"common/cosmwasm-smart-contracts/vesting-contract",
"common/credential-proxy",
"common/credential-storage",
"common/credential-utils",
"common/credential-verification",
@@ -49,8 +51,9 @@ members = [
"common/credentials-interface",
"common/crypto",
"common/dkg",
"common/ecash-signer-check",
"common/ecash-signer-check-types",
"common/ecash-time",
"common/execute",
"common/exit-policy",
"common/gateway-requests",
"common/gateway-stats-storage",
@@ -89,25 +92,32 @@ members = [
"common/socks5/requests",
"common/statistics",
"common/store-cipher",
"common/task",
"common/task", "common/test-utils",
"common/ticketbooks-merkle",
"common/topology",
"common/tun",
"common/types",
"common/types", "common/upgrade-mode-check",
"common/verloc",
"common/wasm/client-core",
"common/wasm/storage",
"common/wasm/utils",
"common/wireguard",
"common/wireguard-private-metadata/client",
"common/wireguard-private-metadata/server",
"common/wireguard-private-metadata/shared",
"common/wireguard-private-metadata/tests",
"common/wireguard-types",
"common/zulip-client",
"documentation/autodoc",
"gateway",
"nym-api",
"nym-api/nym-api-requests",
"nym-authenticator-client",
"nym-browser-extension/storage",
"nym-credential-proxy/nym-credential-proxy",
"nym-credential-proxy/nym-credential-proxy-requests",
"nym-credential-proxy/vpn-api-lib-wasm",
"nym-ip-packet-client",
"nym-network-monitor",
"nym-node",
"nym-node-status-api/nym-node-status-agent",
@@ -115,15 +125,15 @@ members = [
"nym-node-status-api/nym-node-status-client",
"nym-node/nym-node-metrics",
"nym-node/nym-node-requests",
"nym-outfox",
"nym-outfox", "nym-signers-monitor",
"nym-statistics-api",
"nym-validator-rewarder",
"nym-wg-gateway-client",
"nyx-chain-watcher",
"sdk/ffi/cpp",
"sdk/ffi/go",
"sdk/ffi/shared",
"sdk/rust/nym-sdk",
"service-providers/authenticator",
"service-providers/common",
"service-providers/ip-packet-router",
"service-providers/network-requester",
@@ -161,7 +171,6 @@ default-members = [
"nym-statistics-api",
"nym-validator-rewarder",
"nyx-chain-watcher",
"service-providers/authenticator",
"service-providers/ip-packet-router",
"service-providers/network-requester",
"tools/nymvisor",
@@ -176,7 +185,7 @@ homepage = "https://nymtech.net"
documentation = "https://nymtech.net"
edition = "2021"
license = "Apache-2.0"
rust-version = "1.80"
rust-version = "1.81"
readme = "README.md"
[workspace.dependencies]
@@ -217,8 +226,8 @@ clap_complete = "4.5"
clap_complete_fig = "4.5"
colored = "2.2"
comfy-table = "7.1.4"
console = "0.15.11"
console-subscriber = "0.1.1"
console = "0.16.0"
console-subscriber = "0.4.1"
console_error_panic_hook = "0.1"
const-str = "0.5.6"
const_format = "0.2.34"
@@ -234,6 +243,7 @@ digest = "0.10.7"
dirs = "5.0"
doc-comment = "0.3"
dotenvy = "0.15.6"
dyn-clone = "1.0.19"
ecdsa = "0.16"
ed25519-dalek = "2.1"
encoding_rs = "0.8.35"
@@ -263,11 +273,12 @@ humantime = "2.2.0"
humantime-serde = "1.1.1"
hyper = "1.6.0"
hyper-util = "0.1"
indicatif = "0.17.11"
indicatif = "0.18.0"
inquire = "0.6.2"
ip_network = "0.4.1"
ipnetwork = "0.20"
itertools = "0.14.0"
jwt-simple = { version = "0.12.12", default-features = false, features = ["pure-rust"] }
k256 = "0.13"
lazy_static = "1.5.0"
ledger-transport = "0.10.0"
@@ -316,17 +327,18 @@ si-scale = "0.2.3"
snow = "0.9.6"
sphinx-packet = "=0.6.0"
sqlx = "0.8.6"
strum = "0.26"
strum_macros = "0.26"
strum = "0.27.2"
strum_macros = "0.27.2"
subtle-encoding = "0.5"
syn = "1"
sysinfo = "0.33.0"
syn = "2"
sysinfo = "0.37.0"
tap = "1.0.1"
tar = "0.4.44"
test-with = { version = "0.15.4", default-features = false }
tempfile = "3.20"
thiserror = "2.0"
time = "0.3.41"
tokio = "1.45"
tokio = "1.47"
tokio-postgres = "0.7"
tokio-stream = "0.1.17"
tokio-test = "0.4.4"
@@ -434,6 +446,9 @@ opt-level = 'z'
# lto = true
opt-level = 'z'
[workspace.lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tokio_unstable)'] }
[workspace.lints.clippy]
unwrap_used = "deny"
expect_used = "deny"
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-client"
version = "1.1.58"
version = "1.1.61"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
description = "Implementation of the Nym Client"
edition = "2021"
+1 -1
View File
@@ -111,7 +111,7 @@ impl SocketClient {
let dkg_query_client = if self.config.base.client.disabled_credentials_mode {
None
} else {
Some(default_query_dkg_client_from_config(&self.config.base))
Some(default_query_dkg_client_from_config(&self.config.base)?)
};
let storage = self.initialise_storage().await?;
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-socks5-client"
version = "1.1.58"
version = "1.1.61"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
description = "A SOCKS5 localhost proxy that converts incoming messages to Sphinx and sends them to a Nym address"
edition = "2021"
@@ -28,8 +28,6 @@ pub type HmacSha256 = Hmac<Sha256>;
pub type Nonce = u64;
pub type Taken = Option<SystemTime>;
pub const BANDWIDTH_CAP_PER_DAY: u64 = 250 * 1024 * 1024 * 1024; // 250 GB
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct IpPair {
pub ipv4: Ipv4Addr,
+2 -2
View File
@@ -207,7 +207,7 @@ where
<St as Storage>::StorageError: Send + Sync + 'static,
{
if let Some(stored) = storage
.get_expiration_date_signatures(expiration_date)
.get_expiration_date_signatures(expiration_date, epoch_id)
.await
.map_err(BandwidthControllerError::credential_storage_error)?
{
@@ -220,7 +220,7 @@ where
ecash_apis,
|api| async move {
api.api_client
.global_expiration_date_signatures(Some(expiration_date))
.global_expiration_date_signatures(Some(expiration_date), Some(epoch_id))
.await
},
format!("aggregated coin index signatures for date {expiration_date}"),
+4
View File
@@ -13,6 +13,7 @@ async-trait = { workspace = true }
base64 = { workspace = true }
bs58 = { workspace = true }
clap = { workspace = true, optional = true }
cfg-if = { workspace = true }
comfy-table = { workspace = true, optional = true }
futures = { workspace = true }
humantime = { workspace = true }
@@ -123,3 +124,6 @@ fs-surb-storage = ["nym-client-core-surb-storage/fs-surb-storage"]
fs-gateways-storage = ["nym-client-core-gateways-storage/fs-gateways-storage"]
wasm = ["nym-gateway-client/wasm"]
metrics-server = []
[lints]
workspace = true
@@ -3,6 +3,7 @@ name = "nym-client-core-gateways-storage"
version = "0.1.0"
edition = "2021"
license.workspace = true
rust-version.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -26,6 +27,7 @@ features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate", "time"]
optional = true
[build-dependencies]
anyhow = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
sqlx = { workspace = true, features = [
"runtime-tokio-rustls",
+13 -4
View File
@@ -2,23 +2,30 @@
// SPDX-License-Identifier: Apache-2.0
#[tokio::main]
async fn main() {
async fn main() -> anyhow::Result<()> {
#[cfg(feature = "fs-gateways-storage")]
{
use anyhow::Context;
use sqlx::{Connection, SqliteConnection};
use std::env;
let out_dir = env::var("OUT_DIR").unwrap();
let out_dir = env::var("OUT_DIR")?;
let database_path = format!("{out_dir}/gateways-storage-example.sqlite");
// remove the db file if it already existed from previous build
// in case it was from a different branch
if std::fs::exists(&database_path)? {
std::fs::remove_file(&database_path)?;
}
let mut conn = SqliteConnection::connect(&format!("sqlite://{database_path}?mode=rwc"))
.await
.expect("Failed to create SQLx database connection");
.context("Failed to create SQLx database connection")?;
sqlx::migrate!("./fs_gateways_migrations")
.run(&mut conn)
.await
.expect("Failed to perform SQLx migrations");
.context("Failed to perform SQLx migrations")?;
#[cfg(target_family = "unix")]
println!("cargo:rustc-env=DATABASE_URL=sqlite://{}", &database_path);
@@ -28,4 +35,6 @@ async fn main() {
// not a valid windows path... but hey, it works...
println!("cargo:rustc-env=DATABASE_URL=sqlite:///{}", &database_path);
}
Ok(())
}
@@ -58,6 +58,7 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.signatures_path.unwrap())?
}
};
@@ -64,6 +64,7 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.credential_path.unwrap())?
}
};
@@ -58,6 +58,7 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.signatures_path.unwrap())?
}
};
@@ -58,6 +58,7 @@ where
Some(data) => data,
None => {
// SAFETY: one of those arguments must have been set
#[allow(clippy::unwrap_used)]
fs::read(common_args.key_path.unwrap())?
}
};
@@ -135,9 +135,11 @@ pub enum ClientInputStatus {
}
impl ClientInputStatus {
#[allow(clippy::panic)]
pub fn register_producer(&mut self) -> ClientInput {
match std::mem::replace(self, ClientInputStatus::Connected) {
ClientInputStatus::AwaitingProducer { client_input } => client_input,
// critical failure implying misuse of software
ClientInputStatus::Connected => panic!("producer was already registered before"),
}
}
@@ -149,9 +151,11 @@ pub enum ClientOutputStatus {
}
impl ClientOutputStatus {
#[allow(clippy::panic)]
pub fn register_consumer(&mut self) -> ClientOutput {
match std::mem::replace(self, ClientOutputStatus::Connected) {
ClientOutputStatus::AwaitingConsumer { client_output } => client_output,
// critical failure implying misuse of software
ClientOutputStatus::Connected => panic!("consumer was already registered before"),
}
}
@@ -707,11 +711,14 @@ where
})?;
let store_clone = mem_store.clone();
spawn_future(async move {
persistent_storage
.flush_on_shutdown(store_clone, shutdown)
.await
});
spawn_future!(
async move {
persistent_storage
.flush_on_shutdown(store_clone, shutdown)
.await
},
"PersistentReplyStorage::flush_on_shutdown"
);
Ok(mem_store)
}
@@ -732,7 +739,7 @@ where
let mut rng = OsRng;
let keys = if let Some(derivation_material) = derivation_material {
ClientKeys::from_master_key(&mut rng, &derivation_material)
.map_err(|_| ClientCoreError::HkdfDerivationError {})?
.map_err(|_| ClientCoreError::HkdfDerivationError)?
} else {
ClientKeys::generate_new(&mut rng)
};
@@ -114,41 +114,32 @@ pub async fn setup_fs_gateways_storage<P: AsRef<Path>>(
})
}
pub fn create_bandwidth_controller<St: CredentialStorage>(
config: &Config,
storage: St,
) -> BandwidthController<QueryHttpRpcNyxdClient, St> {
let nyxd_url = config
.get_validator_endpoints()
.pop()
.expect("No nyxd validator endpoint provided");
create_bandwidth_controller_with_urls(nyxd_url, storage)
}
pub fn create_bandwidth_controller_with_urls<St: CredentialStorage>(
nyxd_url: Url,
storage: St,
) -> BandwidthController<QueryHttpRpcNyxdClient, St> {
let client = default_query_dkg_client(nyxd_url);
) -> Result<BandwidthController<QueryHttpRpcNyxdClient, St>, ClientCoreError> {
let client = default_query_dkg_client(nyxd_url)?;
BandwidthController::new(storage, client)
Ok(BandwidthController::new(storage, client))
}
pub fn default_query_dkg_client_from_config(config: &Config) -> QueryHttpRpcNyxdClient {
pub fn default_query_dkg_client_from_config(
config: &Config,
) -> Result<QueryHttpRpcNyxdClient, ClientCoreError> {
let nyxd_url = config
.get_validator_endpoints()
.pop()
.expect("No nyxd validator endpoint provided");
.ok_or(ClientCoreError::RpcClientMissingUrl)?;
default_query_dkg_client(nyxd_url)
}
pub fn default_query_dkg_client(nyxd_url: Url) -> QueryHttpRpcNyxdClient {
pub fn default_query_dkg_client(nyxd_url: Url) -> Result<QueryHttpRpcNyxdClient, ClientCoreError> {
let details = nym_network_defaults::NymNetworkDetails::new_from_env();
let client_config = nyxd::Config::try_from_nym_network_details(&details)
.expect("failed to construct validator client config");
.map_err(|source| ClientCoreError::InvalidNetworkDetails { source })?;
// overwrite env configuration with config URLs
QueryHttpRpcNyxdClient::connect(client_config, nyxd_url.as_str())
.expect("Could not construct query client")
.map_err(|source| ClientCoreError::RpcClientCreationFailure { source })
}
@@ -235,6 +235,7 @@ impl LoopCoverTrafficStream<OsRng> {
tokio::task::yield_now().await;
}
#[allow(clippy::panic)]
pub fn start(mut self) {
if self.cover_traffic.disable_loop_cover_traffic_stream {
// we should have never got here in the first place - the task should have never been created to begin with
@@ -251,27 +252,30 @@ impl LoopCoverTrafficStream<OsRng> {
let mut shutdown = self.task_client.fork("select");
spawn_future(async move {
debug!("Started LoopCoverTrafficStream with graceful shutdown support");
spawn_future!(
async move {
debug!("Started LoopCoverTrafficStream with graceful shutdown support");
while !shutdown.is_shutdown() {
tokio::select! {
biased;
_ = shutdown.recv() => {
tracing::trace!("LoopCoverTrafficStream: Received shutdown");
}
next = self.next() => {
if next.is_some() {
self.on_new_message().await;
} else {
tracing::trace!("LoopCoverTrafficStream: Stopping since channel closed");
break;
while !shutdown.is_shutdown() {
tokio::select! {
biased;
_ = shutdown.recv() => {
tracing::trace!("LoopCoverTrafficStream: Received shutdown");
}
next = self.next() => {
if next.is_some() {
self.on_new_message().await;
} else {
tracing::trace!("LoopCoverTrafficStream: Stopping since channel closed");
break;
}
}
}
}
}
shutdown.recv_timeout().await;
tracing::debug!("LoopCoverTrafficStream: Exiting");
})
shutdown.recv_timeout().await;
tracing::debug!("LoopCoverTrafficStream: Exiting");
},
"LoopCoverTrafficStream"
)
}
}
@@ -96,72 +96,94 @@ impl MixTrafficController {
mut mix_packets: Vec<MixPacket>,
) -> Result<(), ErasedGatewayError> {
debug_assert!(!mix_packets.is_empty());
let result = if mix_packets.len() == 1 {
let send_future = if mix_packets.len() == 1 {
// SAFETY: we just checked we have one packet
#[allow(clippy::unwrap_used)]
let mix_packet = mix_packets.pop().unwrap();
self.gateway_transceiver.send_mix_packet(mix_packet).await
self.gateway_transceiver.send_mix_packet(mix_packet)
} else {
self.gateway_transceiver
.batch_send_mix_packets(mix_packets)
.await
self.gateway_transceiver.batch_send_mix_packets(mix_packets)
};
if result.is_err() {
self.consecutive_gateway_failure_count += 1;
} else {
trace!("We *might* have managed to forward sphinx packet(s) to the gateway!");
self.consecutive_gateway_failure_count = 0;
}
tokio::select! {
biased;
_ = self.task_client.recv() => {
trace!("received shutdown while handling messages");
Ok(())
}
result = send_future => {
if result.is_err() {
self.consecutive_gateway_failure_count += 1;
} else {
trace!("We *might* have managed to forward sphinx packet(s) to the gateway!");
self.consecutive_gateway_failure_count = 0;
}
result
result
}
}
}
async fn on_client_request(&mut self, client_request: ClientRequest) {
tokio::select! {
biased;
_ = self.task_client.recv() => {
trace!("received shutdown while handling client request");
}
result = self.gateway_transceiver.send_client_request(client_request) => {
if let Err(err) = result {
error!("Failed to send client request: {err}")
}
}
}
}
pub fn start(mut self) {
spawn_future(async move {
debug!("Started MixTrafficController with graceful shutdown support");
while !self.task_client.is_shutdown() {
tokio::select! {
mix_packets = self.mix_rx.recv() => match mix_packets {
Some(mix_packets) => {
if let Err(err) = self.on_messages(mix_packets).await {
error!("Failed to send sphinx packet(s) to the gateway: {err}");
if self.consecutive_gateway_failure_count == MAX_FAILURE_COUNT {
// Disconnect from the gateway. If we should try to re-connect
// is handled at a higher layer.
error!("Failed to send sphinx packet to the gateway {MAX_FAILURE_COUNT} times in a row - assuming the gateway is dead");
// Do we need to handle the embedded mixnet client case
// separately?
self.task_client.send_we_stopped(Box::new(ClientCoreError::GatewayFailedToForwardMessages));
break;
}
}
},
None => {
tracing::trace!("MixTrafficController: Stopping since channel closed");
spawn_future!(
async move {
debug!("Started MixTrafficController with graceful shutdown support");
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = self.task_client.recv() => {
tracing::trace!("MixTrafficController: Received shutdown");
break;
}
},
client_request = self.client_rx.recv() => match client_request {
Some(client_request) => {
match self.gateway_transceiver.send_client_request(client_request).await {
Ok(_) => (),
Err(e) => error!("Failed to send client request: {e}"),
};
mix_packets = self.mix_rx.recv() => match mix_packets {
Some(mix_packets) => {
if let Err(err) = self.on_messages(mix_packets).await {
error!("Failed to send sphinx packet(s) to the gateway: {err}");
if self.consecutive_gateway_failure_count == MAX_FAILURE_COUNT {
// Disconnect from the gateway. If we should try to re-connect
// is handled at a higher layer.
error!("Failed to send sphinx packet to the gateway {MAX_FAILURE_COUNT} times in a row - assuming the gateway is dead");
// Do we need to handle the embedded mixnet client case
// separately?
self.task_client.send_we_stopped(Box::new(ClientCoreError::GatewayFailedToForwardMessages));
break;
}
}
},
None => {
tracing::trace!("MixTrafficController: Stopping since channel closed");
break;
}
},
client_request = self.client_rx.recv() => match client_request {
Some(client_request) => {
self.on_client_request(client_request).await;
},
None => {
tracing::trace!("MixTrafficController, client request channel closed");
break
}
},
None => {
tracing::trace!("MixTrafficController, client request channel closed");
}
},
_ = self.task_client.recv() => {
tracing::trace!("MixTrafficController: Received shutdown");
break;
}
}
}
self.task_client.recv_timeout().await;
tracing::debug!("MixTrafficController: Exiting");
});
self.task_client.recv_timeout().await;
tracing::debug!("MixTrafficController: Exiting");
},
"MixTrafficController"
);
}
}
@@ -269,6 +269,8 @@ pub struct MockGateway {
}
impl Default for MockGateway {
// test code
#[allow(clippy::unwrap_used)]
fn default() -> Self {
MockGateway {
dummy_identity: "3ebjp1Fb9hdcS1AR6AZihgeJiMHkB5jjJUsvqNnfQwU7"
@@ -194,10 +194,11 @@ impl ActionController {
trace!("{frag_id} is updating its delay");
// TODO: is it possible to solve this without either locking or temporarily removing the value?
if let Some((pending_ack_data, queue_key)) = self.pending_acks_data.remove(&frag_id) {
// this Action is triggered by `RetransmissionRequestListener` (for 'normal' packets)
// SAFETY: this Action is triggered by `RetransmissionRequestListener` (for 'normal' packets)
// or `ReplyController` (for 'reply' packets) which held the other potential
// reference to this Arc. HOWEVER, before the Action was pushed onto the queue, the reference
// was dropped hence this unwrap is safe.
#[allow(clippy::unwrap_used)]
let mut inner_data = Arc::try_unwrap(pending_ack_data).unwrap();
inner_data.update_retransmitted(delay);
@@ -209,6 +210,7 @@ impl ActionController {
}
// note: when the entry expires it's automatically removed from pending_acks_timers
#[allow(clippy::panic)]
fn handle_expired_ack_timer(&mut self, expired_ack: Expired<FragmentIdentifier>) {
let frag_id = expired_ack.into_inner();
@@ -120,6 +120,7 @@ where
}
}
#[allow(clippy::panic)]
async fn on_input_message(&mut self, msg: InputMessage) {
match msg {
InputMessage::Regular {
@@ -213,7 +214,9 @@ where
self.handle_premade_packets(msgs, lane).await
}
// MessageWrappers can't be nested
InputMessage::MessageWrapper { .. } => unimplemented!(),
InputMessage::MessageWrapper { .. } => {
panic!("attempted to use nested MessageWrapper")
}
},
};
}
@@ -223,6 +226,11 @@ where
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = self.task_client.recv() => {
tracing::trace!("InputMessageListener: Received shutdown");
break;
}
input_msg = self.input_receiver.recv() => match input_msg {
Some(input_msg) => {
self.on_input_message(input_msg).await;
@@ -232,9 +240,7 @@ where
break;
}
},
_ = self.task_client.recv() => {
tracing::trace!("InputMessageListener: Received shutdown");
}
}
}
self.task_client.recv_timeout().await;
@@ -298,29 +298,44 @@ where
let mut sent_notification_listener = self.sent_notification_listener;
let mut action_controller = self.action_controller;
spawn_future(async move {
acknowledgement_listener.run().await;
debug!("The acknowledgement listener has finished execution!");
});
spawn_future!(
async move {
acknowledgement_listener.run().await;
debug!("The acknowledgement listener has finished execution!");
},
"AcknowledgementController::AcknowledgementListener"
);
spawn_future(async move {
input_message_listener.run().await;
debug!("The input listener has finished execution!");
});
spawn_future!(
async move {
input_message_listener.run().await;
debug!("The input listener has finished execution!");
},
"AcknowledgementController::InputMessageListener"
);
spawn_future(async move {
retransmission_request_listener.run(packet_type).await;
debug!("The retransmission request listener has finished execution!");
});
spawn_future!(
async move {
retransmission_request_listener.run(packet_type).await;
debug!("The retransmission request listener has finished execution!");
},
"AcknowledgementController::RetransmissionRequestListener"
);
spawn_future(async move {
sent_notification_listener.run().await;
debug!("The sent notification listener has finished execution!");
});
spawn_future!(
async move {
sent_notification_listener.run().await;
debug!("The sent notification listener has finished execution!");
},
"AcknowledgementController::SentNotificationListener"
);
spawn_future(async move {
action_controller.run().await;
debug!("The controller has finished execution!");
});
spawn_future!(
async move {
action_controller.run().await;
debug!("The controller has finished execution!");
},
"AcknowledgementController::ActionController"
);
}
}
@@ -179,6 +179,11 @@ where
while !self.task_client.is_shutdown() {
tokio::select! {
biased;
_ = self.task_client.recv() => {
tracing::trace!("RetransmissionRequestListener: Received shutdown");
break;
}
timed_out_ack = self.request_receiver.next() => match timed_out_ack {
Some(timed_out_ack) => self.on_retransmission_request(timed_out_ack, packet_type).await,
None => {
@@ -186,9 +191,7 @@ where
break;
}
},
_ = self.task_client.recv() => {
tracing::trace!("RetransmissionRequestListener: Received shutdown");
}
}
}
self.task_client.recv_timeout().await;
@@ -35,6 +35,9 @@ pub enum PreparationError {
#[error(transparent)]
NymTopologyError(#[from] NymTopologyError),
#[error("message wasn't split into any fragments!")]
EmptyFragments,
#[error("message too long for a single SURB, splitting into {fragments} fragments.")]
MessageTooLongForSingleSurb { fragments: usize },
@@ -320,6 +323,16 @@ where
});
}
if fragment.is_empty() {
error!("CRITICAL FAILURE: our split message didn't result in any sendable fragments");
return Err(SurbWrappedPreparationError {
source: PreparationError::EmptyFragments,
returned_surbs: Some(vec![reply_surb]),
});
}
// SAFETY: we just checked we have one fragment
#[allow(clippy::unwrap_used)]
let chunk = fragment.pop().unwrap();
let chunk_clone = chunk.clone();
let prepared_fragment = self
@@ -535,6 +548,7 @@ where
pending_acks.push(pending_ack);
}
drop(topology_permit);
self.insert_pending_acks(pending_acks);
self.forward_messages(real_messages, lane).await;
@@ -657,6 +671,7 @@ where
.zip(reply_surbs.into_iter())
.map(|(fragment, reply_surb)| {
// unwrap here is fine as we know we have a valid topology
#[allow(clippy::unwrap_used)]
self.message_preparer
.prepare_reply_chunk_for_sending(
fragment,
@@ -716,17 +731,21 @@ where
// tells real message sender (with the poisson timer) to send this to the mix network
pub(crate) async fn forward_messages(
&self,
&mut self,
messages: Vec<RealMessage>,
transmission_lane: TransmissionLane,
) {
if let Err(err) = self
.real_message_sender
.send((messages, transmission_lane))
.await
{
if !self.task_client.is_shutdown_poll() {
error!("Failed to forward messages to the real message sender: {err}");
tokio::select! {
biased;
_ = self.task_client.recv() => {
trace!("received shutdown while attempting to forward mixnet messages");
}
sending_res = self.real_message_sender.send((messages, transmission_lane)) => {
if sending_res.is_err() {
error!(
"failed to forward mixnet messages due to closed channel (outside of shutdown!)"
);
}
}
}
}
@@ -224,14 +224,20 @@ impl RealMessagesController<OsRng> {
let ack_control = self.ack_control;
let mut reply_control = self.reply_control;
spawn_future(async move {
out_queue_control.run().await;
debug!("The out queue controller has finished execution!");
});
spawn_future(async move {
reply_control.run().await;
debug!("The reply controller has finished execution!");
});
spawn_future!(
async move {
out_queue_control.run().await;
debug!("The out queue controller has finished execution!");
},
"RealMessagesController::OutQueueControl)"
);
spawn_future!(
async move {
reply_control.run().await;
debug!("The reply controller has finished execution!");
},
"RealMessagesController::ReplyController"
);
ack_control.start(packet_type);
}
@@ -249,6 +249,8 @@ where
}
};
// SAFETY: our topology must be valid at this point
#[allow(clippy::expect_used)]
(
generate_loop_cover_packet(
&mut self.rng,
@@ -278,17 +280,33 @@ where
}
};
if let Err(err) = self.mix_tx.send(vec![next_message]).await {
if !self.task_client.is_shutdown_poll() {
tracing::error!("Failed to send: {err}");
let sending_res = tokio::select! {
biased;
_ = self.task_client.recv() => {
trace!("received shutdown signal while attempting to send mix message");
return
}
sending_res = self.mix_tx.send(vec![next_message]) => {
sending_res
}
};
match sending_res {
Err(_) => {
if !self.task_client.is_shutdown_poll() {
tracing::error!(
"failed to send mixnet packet due to closed channel (outside of shutdown!)"
);
}
}
Ok(_) => {
let event = if fragment_id.is_some() {
PacketStatisticsEvent::RealPacketSent(packet_size)
} else {
PacketStatisticsEvent::CoverPacketSent(packet_size)
};
self.stats_tx.report(event.into());
}
} else {
let event = if fragment_id.is_some() {
PacketStatisticsEvent::RealPacketSent(packet_size)
} else {
PacketStatisticsEvent::CoverPacketSent(packet_size)
};
self.stats_tx.report(event.into());
}
// notify ack controller about sending our message only after we actually managed to push it
@@ -439,6 +457,8 @@ where
tracing::trace!("handling real_messages: size: {}", real_messages.len());
self.transmission_buffer.store(&conn_id, real_messages);
// SAFETY: we just stored the message
#[allow(clippy::expect_used)]
let real_next = self.pop_next_message().expect("Just stored one");
Poll::Ready(Some(StreamMessage::Real(Box::new(real_next))))
@@ -487,6 +507,8 @@ where
// First store what we got for the given connection id
self.transmission_buffer.store(&conn_id, real_messages);
// SAFETY: we just stored the message
#[allow(clippy::expect_used)]
let real_next = self.pop_next_message().expect("we just added one");
Poll::Ready(Some(StreamMessage::Real(Box::new(real_next))))
@@ -1,6 +1,7 @@
// Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::helpers::get_time_now;
use crate::client::replies::{
reply_controller::ReplyControllerSender, reply_storage::SentReplyKeys,
};
@@ -22,7 +23,7 @@ use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, C
use nym_task::TaskClient;
use std::collections::HashSet;
use std::sync::Arc;
use std::time::{Duration, Instant};
use std::time::Duration;
use tracing::*;
// The interval at which we check for stale buffers
@@ -54,7 +55,7 @@ struct ReceivedMessagesBufferInner<R: MessageReceiver> {
stats_tx: ClientStatsSender,
// Periodically check for stale buffers to clean up
last_stale_check: Instant,
last_stale_check: crate::client::helpers::Instant,
}
impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
@@ -154,7 +155,7 @@ impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
}
fn cleanup_stale_buffers(&mut self) {
let now = Instant::now();
let now = get_time_now();
if now - self.last_stale_check > STALE_BUFFER_CHECK_INTERVAL {
self.last_stale_check = now;
self.message_receiver
@@ -190,7 +191,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
message_sender: None,
recently_reconstructed: HashSet::new(),
stats_tx,
last_stale_check: Instant::now(),
last_stale_check: get_time_now(),
})),
reply_key_storage,
reply_controller_sender,
@@ -198,6 +199,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
}
}
#[allow(clippy::panic)]
async fn disconnect_sender(&mut self) {
let mut guard = self.inner.lock().await;
if guard.message_sender.is_none() {
@@ -208,6 +210,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
guard.message_sender = None;
}
#[allow(clippy::panic)]
async fn connect_sender(&mut self, sender: ReconstructedMessagesSender) {
let mut guard = self.inner.lock().await;
if guard.message_sender.is_some() {
@@ -599,14 +602,20 @@ impl<R: MessageReceiver + Clone + Send + 'static> ReceivedMessagesBufferControll
let mut fragmented_message_receiver = self.fragmented_message_receiver;
let mut request_receiver = self.request_receiver;
spawn_future(async move {
match fragmented_message_receiver.run().await {
Ok(_) => {}
Err(e) => error!("{e}"),
}
});
spawn_future(async move {
request_receiver.run().await;
});
spawn_future!(
async move {
match fragmented_message_receiver.run().await {
Ok(_) => {}
Err(e) => error!("{e}"),
}
},
"ReceivedMessagesBufferController::FragmentedMessageReceiver"
);
spawn_future!(
async move {
request_receiver.run().await;
},
"ReceivedMessagesBufferController::RequestReceiver"
);
}
}
@@ -155,8 +155,9 @@ where
data: Vec<Arc<PendingAcknowledgement>>,
) {
trace!("re-inserting pending retransmissions for {recipient}");
// the underlying entry MUST exist as we've just got data from there
// SAFETY: the underlying entry MUST exist as we've just got data from there
// and we hold a mut reference
#[allow(clippy::expect_used)]
let map_entry = &mut self
.surb_senders
.get_mut(recipient)
@@ -429,6 +430,7 @@ where
.pop_at_most_n_next_messages_at_random(amount)
}
#[allow(clippy::panic)]
async fn try_clear_pending_queue(&mut self, target: AnonymousSenderTag) {
trace!("trying to clear pending queue");
let available_surbs = self.surbs_storage.available_surbs(&target);
@@ -165,9 +165,12 @@ impl StatisticsControl {
}
pub(crate) fn start(mut self) {
spawn_future(async move {
self.run().await;
})
spawn_future!(
async move {
self.run().await;
},
"StatisticsControl"
)
}
pub(crate) fn create_and_start(
@@ -126,7 +126,7 @@ impl TopologyAccessor {
.map(|p| p.topology.clone())
}
pub async fn current_route_provider(&self) -> Option<RwLockReadGuard<NymRouteProvider>> {
pub async fn current_route_provider(&self) -> Option<RwLockReadGuard<'_, NymRouteProvider>> {
let provider = self.inner.topology.read().await;
if provider.topology.is_empty() {
None
@@ -145,36 +145,39 @@ impl TopologyRefresher {
}
pub fn start(mut self) {
spawn_future(async move {
debug!("Started TopologyRefresher with graceful shutdown support");
spawn_future!(
async move {
debug!("Started TopologyRefresher with graceful shutdown support");
#[cfg(not(target_arch = "wasm32"))]
let mut interval = tokio_stream::wrappers::IntervalStream::new(tokio::time::interval(
self.refresh_rate,
));
#[cfg(not(target_arch = "wasm32"))]
let mut interval = tokio_stream::wrappers::IntervalStream::new(
tokio::time::interval(self.refresh_rate),
);
#[cfg(target_arch = "wasm32")]
let mut interval =
gloo_timers::future::IntervalStream::new(self.refresh_rate.as_millis() as u32);
#[cfg(target_arch = "wasm32")]
let mut interval =
gloo_timers::future::IntervalStream::new(self.refresh_rate.as_millis() as u32);
// We already have an initial topology, so no need to refresh it immediately.
// My understanding is that js setInterval does not fire immediately, so it's not
// needed there.
#[cfg(not(target_arch = "wasm32"))]
interval.next().await;
// We already have an initial topology, so no need to refresh it immediately.
// My understanding is that js setInterval does not fire immediately, so it's not
// needed there.
#[cfg(not(target_arch = "wasm32"))]
interval.next().await;
while !self.task_client.is_shutdown() {
tokio::select! {
_ = interval.next() => {
self.try_refresh().await;
},
_ = self.task_client.recv() => {
tracing::trace!("TopologyRefresher: Received shutdown");
},
while !self.task_client.is_shutdown() {
tokio::select! {
_ = interval.next() => {
self.try_refresh().await;
},
_ = self.task_client.recv() => {
tracing::trace!("TopologyRefresher: Received shutdown");
},
}
}
}
self.task_client.recv_timeout().await;
tracing::debug!("TopologyRefresher: Exiting");
})
self.task_client.recv_timeout().await;
tracing::debug!("TopologyRefresher: Exiting");
},
"TopologyRefresher"
)
}
}
+15 -1
View File
@@ -7,7 +7,9 @@ use nym_gateway_client::error::GatewayClientError;
use nym_topology::node::RoutingNodeError;
use nym_topology::{NodeId, NymTopologyError};
use nym_validator_client::nym_api::error::NymAPIError;
use nym_validator_client::nyxd::error::NyxdError;
use nym_validator_client::ValidatorClientError;
use rand::distributions::WeightedError;
use std::error::Error;
use std::path::PathBuf;
@@ -230,7 +232,19 @@ pub enum ClientCoreError {
UnexpectedKeyUpgrade { gateway_id: String },
#[error("failed to derive keys from master key")]
HkdfDerivationError {},
HkdfDerivationError,
#[error("missing url for constructing RPC client")]
RpcClientMissingUrl,
#[error("provided nym network details were malformed: {source}")]
InvalidNetworkDetails { source: NyxdError },
#[error("failed to construct RPC client: {source}")]
RpcClientCreationFailure { source: NyxdError },
#[error("failed to select valid gateway due to incomputable latency")]
GatewaySelectionFailure { source: WeightedError },
}
impl From<tungstenite::Error> for ClientCoreError {
+2 -2
View File
@@ -148,7 +148,7 @@ async fn connect(endpoint: &str) -> Result<WsConn, ClientCoreError> {
JSWebsocket::new(endpoint).map_err(|_| ClientCoreError::GatewayJsConnectionFailure)
}
async fn measure_latency<G>(gateway: &G) -> Result<GatewayWithLatency<G>, ClientCoreError>
async fn measure_latency<G>(gateway: &G) -> Result<GatewayWithLatency<'_, G>, ClientCoreError>
where
G: ConnectableGateway,
{
@@ -245,7 +245,7 @@ pub async fn choose_gateway_by_latency<R: Rng, G: ConnectableGateway + Clone>(
let gateways_with_latency = gateways_with_latency.lock().await;
let chosen = gateways_with_latency
.choose_weighted(rng, |item| 1. / item.latency.as_secs_f32())
.expect("invalid selection weight!");
.map_err(|source| ClientCoreError::GatewaySelectionFailure { source })?;
info!(
"chose gateway {} with average latency of {:?}",
+38 -2
View File
@@ -18,18 +18,54 @@ pub use nym_topology::{
};
#[cfg(target_arch = "wasm32")]
pub(crate) fn spawn_future<F>(future: F)
pub fn spawn_future<F>(future: F)
where
F: Future<Output = ()> + 'static,
{
wasm_bindgen_futures::spawn_local(future);
}
// TODO: expose similar API to the rest of the codebase,
// perhaps with some simple trait for a task to define its name
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn spawn_future<F>(future: F)
#[track_caller]
pub fn spawn_future<F>(future: F)
where
F: Future + Send + 'static,
F::Output: Send + 'static,
{
tokio::spawn(future);
}
#[cfg(not(target_arch = "wasm32"))]
#[track_caller]
pub fn spawn_named_future<F>(future: F, name: &str)
where
F: Future + Send + 'static,
F::Output: Send + 'static,
{
cfg_if::cfg_if! {if #[cfg(tokio_unstable)] {
#[allow(clippy::expect_used)]
tokio::task::Builder::new().name(name).spawn(future).expect("failed to spawn future");
} else {
let _ = name;
tracing::debug!(r#"the underlying binary hasn't been built with `RUSTFLAGS="--cfg tokio_unstable"` - the future naming won't do anything"#);
spawn_future(future);
}}
}
#[macro_export]
macro_rules! spawn_future {
($future:expr) => {{
$crate::spawn_future($future)
}};
($future:expr, $name:expr) => {{
cfg_if::cfg_if! {if #[cfg(not(target_arch = "wasm32"))] {
$crate::spawn_named_future($future, $name)
} else {
let _ = $name;
$crate::spawn_future($future)
}}
}};
}
@@ -30,6 +30,7 @@ optional = true
path = "../../../sqlx-pool-guard"
[build-dependencies]
anyhow = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
sqlx = { workspace = true, features = [
"runtime-tokio-rustls",
+7 -4
View File
@@ -2,23 +2,24 @@
// SPDX-License-Identifier: Apache-2.0
#[tokio::main]
async fn main() {
async fn main() -> anyhow::Result<()> {
#[cfg(feature = "fs-surb-storage")]
{
use anyhow::Context;
use sqlx::{Connection, SqliteConnection};
use std::env;
let out_dir = env::var("OUT_DIR").unwrap();
let out_dir = env::var("OUT_DIR")?;
let database_path = format!("{out_dir}/fs-surbs-example.sqlite");
let mut conn = SqliteConnection::connect(&format!("sqlite://{database_path}?mode=rwc"))
.await
.expect("Failed to create SQLx database connection");
.context("Failed to create SQLx database connection")?;
sqlx::migrate!("./fs_surbs_migrations")
.run(&mut conn)
.await
.expect("Failed to perform SQLx migrations");
.context("Failed to perform SQLx migrations")?;
#[cfg(target_family = "unix")]
println!("cargo:rustc-env=DATABASE_URL=sqlite://{}", &database_path);
@@ -28,4 +29,6 @@ async fn main() {
// not a valid windows path... but hey, it works...
println!("cargo:rustc-env=DATABASE_URL=sqlite:///{}", &database_path);
}
Ok(())
}
@@ -37,7 +37,7 @@ impl StorageManager {
.journal_mode(sqlx::sqlite::SqliteJournalMode::Wal)
.synchronous(SqliteSynchronous::Normal)
.auto_vacuum(SqliteAutoVacuum::Incremental)
.filename(&database_path)
.filename(database_path)
.create_if_missing(fresh)
.disable_statement_logging();
@@ -49,8 +49,7 @@ impl StorageManager {
}
};
let connection_pool =
SqlitePoolGuard::new(database_path.as_ref().to_path_buf(), connection_pool);
let connection_pool = SqlitePoolGuard::new(connection_pool);
if let Err(err) = sqlx::migrate!("./fs_surbs_migrations")
.run(&*connection_pool)
@@ -201,7 +201,7 @@ impl<C, St> GatewayClient<C, St> {
#[cfg(not(target_arch = "wasm32"))]
pub async fn establish_connection(&mut self) -> Result<(), GatewayClientError> {
debug!(
"Attemting to establish connection to gateway at: {}",
"Attempting to establish connection to gateway at: {}",
self.gateway_address
);
let (ws_stream, _) = connect_async(
@@ -337,7 +337,7 @@ impl PartiallyDelegatedHandle {
// check if the split stream didn't error out
let receive_res = stream_receiver
.try_recv()
.expect("stream sender was somehow dropped without sending anything!");
.map_err(|_| GatewayClientError::ConnectionAbruptlyClosed)?;
if let Some(res) = receive_res {
let _res = res?;
@@ -719,10 +719,11 @@ impl NymApiClient {
pub async fn partial_expiration_date_signatures(
&self,
expiration_date: Option<Date>,
epoch_id: Option<EpochId>,
) -> Result<PartialExpirationDateSignatureResponse, ValidatorClientError> {
Ok(self
.nym_api
.partial_expiration_date_signatures(expiration_date)
.partial_expiration_date_signatures(expiration_date, epoch_id)
.await?)
}
@@ -739,10 +740,11 @@ impl NymApiClient {
pub async fn global_expiration_date_signatures(
&self,
expiration_date: Option<Date>,
epoch_id: Option<EpochId>,
) -> Result<AggregatedExpirationDateSignatureResponse, ValidatorClientError> {
Ok(self
.nym_api
.global_expiration_date_signatures(expiration_date)
.global_expiration_date_signatures(expiration_date, epoch_id)
.await?)
}
@@ -6,16 +6,18 @@ use crate::nym_api::routes::{ecash, CORE_STATUS_COUNT, SINCE_ARG};
use async_trait::async_trait;
use nym_api_requests::ecash::models::{
AggregatedCoinIndicesSignatureResponse, AggregatedExpirationDateSignatureResponse,
BatchRedeemTicketsBody, EcashBatchTicketRedemptionResponse, EcashTicketVerificationResponse,
IssuedTicketbooksChallengeCommitmentRequest, IssuedTicketbooksChallengeCommitmentResponse,
IssuedTicketbooksDataRequest, IssuedTicketbooksDataResponse, IssuedTicketbooksForCountResponse,
IssuedTicketbooksForResponse, VerifyEcashTicketBody,
BatchRedeemTicketsBody, EcashBatchTicketRedemptionResponse, EcashSignerStatusResponse,
EcashTicketVerificationResponse, IssuedTicketbooksChallengeCommitmentRequest,
IssuedTicketbooksChallengeCommitmentResponse, IssuedTicketbooksDataRequest,
IssuedTicketbooksDataResponse, IssuedTicketbooksForCountResponse, IssuedTicketbooksForResponse,
VerifyEcashTicketBody,
};
use nym_api_requests::ecash::VerificationKeyResponse;
use nym_api_requests::models::{
AnnotationResponse, ApiHealthResponse, BinaryBuildInformationOwned, ChainStatusResponse,
KeyRotationInfoResponse, LegacyDescribedMixNode, NodePerformanceResponse, NodeRefreshBody,
NymNodeDescription, PerformanceHistoryResponse, RewardedSetResponse,
AnnotationResponse, ApiHealthResponse, BinaryBuildInformationOwned, ChainBlocksStatusResponse,
ChainStatusResponse, KeyRotationInfoResponse, LegacyDescribedMixNode, NodePerformanceResponse,
NodeRefreshBody, NymNodeDescription, PerformanceHistoryResponse, RewardedSetResponse,
SignerInformationResponse,
};
use nym_api_requests::nym_nodes::{
NodesByAddressesRequestBody, NodesByAddressesResponse, PaginatedCachedNodesResponseV1,
@@ -1101,8 +1103,9 @@ pub trait NymApiClientExt: ApiClient {
async fn partial_expiration_date_signatures(
&self,
expiration_date: Option<Date>,
epoch_id: Option<EpochId>,
) -> Result<PartialExpirationDateSignatureResponse, NymAPIError> {
let params = match expiration_date {
let mut params = match expiration_date {
None => Vec::new(),
Some(exp) => vec![(
ecash::EXPIRATION_DATE_PARAM,
@@ -1110,6 +1113,10 @@ pub trait NymApiClientExt: ApiClient {
)],
};
if let Some(epoch_id) = epoch_id {
params.push((ecash::EPOCH_ID_PARAM, epoch_id.to_string()));
}
self.get_json(
&[
routes::V1_API_VERSION,
@@ -1146,8 +1153,9 @@ pub trait NymApiClientExt: ApiClient {
async fn global_expiration_date_signatures(
&self,
expiration_date: Option<Date>,
epoch_id: Option<EpochId>,
) -> Result<AggregatedExpirationDateSignatureResponse, NymAPIError> {
let params = match expiration_date {
let mut params = match expiration_date {
None => Vec::new(),
Some(exp) => vec![(
ecash::EXPIRATION_DATE_PARAM,
@@ -1155,6 +1163,10 @@ pub trait NymApiClientExt: ApiClient {
)],
};
if let Some(epoch_id) = epoch_id {
params.push((ecash::EPOCH_ID_PARAM, epoch_id.to_string()));
}
self.get_json(
&[
routes::V1_API_VERSION,
@@ -1331,6 +1343,22 @@ pub trait NymApiClientExt: ApiClient {
.await
}
async fn get_chain_blocks_status(&self) -> Result<ChainBlocksStatusResponse, NymAPIError> {
self.get_json("/v1/network/chain-blocks-status", NO_PARAMS)
.await
}
#[instrument(level = "debug", skip(self))]
async fn get_signer_status(&self) -> Result<EcashSignerStatusResponse, NymAPIError> {
self.get_json("/v1/ecash/signer-status", NO_PARAMS).await
}
#[instrument(level = "debug", skip(self))]
async fn get_signer_information(&self) -> Result<SignerInformationResponse, NymAPIError> {
self.get_json("/v1/api-status/signer-information", NO_PARAMS)
.await
}
#[instrument(level = "debug", skip(self))]
async fn get_key_rotation_info(&self) -> Result<KeyRotationInfoResponse, NymAPIError> {
self.get_json(
@@ -8,11 +8,11 @@ use crate::nyxd::CosmWasmClient;
use async_trait::async_trait;
use cosmrs::AccountId;
use cosmwasm_std::Addr;
use nym_coconut_dkg_common::dealer::RegisteredDealerDetails;
use nym_coconut_dkg_common::types::{ChunkIndex, NodeIndex, StateAdvanceResponse};
use serde::Deserialize;
use tracing::trace;
use nym_coconut_dkg_common::dealer::RegisteredDealerDetails;
pub use nym_coconut_dkg_common::{
dealer::{DealerDetailsResponse, PagedDealerIndexResponse, PagedDealerResponse},
dealing::{
@@ -21,7 +21,9 @@ pub use nym_coconut_dkg_common::{
},
msg::QueryMsg as DkgQueryMsg,
types::{DealerDetails, DealingIndex, Epoch, EpochId, EpochState, State},
verification_key::{ContractVKShare, PagedVKSharesResponse, VkShareResponse},
verification_key::{
ContractVKShare, PagedVKSharesResponse, VerificationKeyShare, VkShareResponse,
},
};
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
@@ -41,6 +43,11 @@ pub trait DkgQueryClient {
self.query_dkg_contract(request).await
}
async fn get_epoch_at_height(&self, height: u64) -> Result<Option<Epoch>, NyxdError> {
let request = DkgQueryMsg::GetEpochStateAtHeight { height };
self.query_dkg_contract(request).await
}
async fn can_advance_state(&self) -> Result<StateAdvanceResponse, NyxdError> {
let request = DkgQueryMsg::CanAdvanceState {};
self.query_dkg_contract(request).await
@@ -87,6 +94,34 @@ pub trait DkgQueryClient {
self.query_dkg_contract(request).await
}
async fn get_epoch_dealers_paged(
&self,
epoch_id: EpochId,
start_after: Option<String>,
limit: Option<u32>,
) -> Result<PagedDealerResponse, NyxdError> {
let request = DkgQueryMsg::GetEpochDealers {
epoch_id,
start_after,
limit,
};
self.query_dkg_contract(request).await
}
async fn get_epoch_dealers_addresses_paged(
&self,
epoch_id: EpochId,
start_after: Option<String>,
limit: Option<u32>,
) -> Result<PagedDealerResponse, NyxdError> {
let request = DkgQueryMsg::GetEpochDealersAddresses {
epoch_id,
start_after,
limit,
};
self.query_dkg_contract(request).await
}
async fn get_dealer_indices_paged(
&self,
start_after: Option<String>,
@@ -208,6 +243,20 @@ pub trait PagedDkgQueryClient: DkgQueryClient {
collect_paged!(self, get_current_dealers_paged, dealers)
}
async fn get_all_epoch_dealers(
&self,
epoch_id: EpochId,
) -> Result<Vec<DealerDetails>, NyxdError> {
collect_paged!(self, get_epoch_dealers_paged, dealers, epoch_id)
}
async fn get_all_epoch_dealers_addresses(
&self,
epoch_id: EpochId,
) -> Result<Vec<DealerDetails>, NyxdError> {
collect_paged!(self, get_epoch_dealers_addresses_paged, dealers, epoch_id)
}
async fn get_all_dealer_indices(&self) -> Result<Vec<(Addr, NodeIndex)>, NyxdError> {
collect_paged!(self, get_dealer_indices_paged, indices)
}
@@ -257,6 +306,9 @@ mod tests {
match msg {
DkgQueryMsg::GetState {} => client.get_state().ignore(),
DkgQueryMsg::GetCurrentEpochState {} => client.get_current_epoch().ignore(),
DkgQueryMsg::GetEpochStateAtHeight { height } => {
client.get_epoch_at_height(height).ignore()
}
DkgQueryMsg::CanAdvanceState {} => client.can_advance_state().ignore(),
DkgQueryMsg::GetCurrentEpochThreshold {} => {
client.get_current_epoch_threshold().ignore()
@@ -276,6 +328,20 @@ mod tests {
DkgQueryMsg::GetCurrentDealers { limit, start_after } => client
.get_current_dealers_paged(start_after, limit)
.ignore(),
QueryMsg::GetEpochDealers {
epoch_id,
limit,
start_after,
} => client
.get_epoch_dealers_paged(epoch_id, start_after, limit)
.ignore(),
QueryMsg::GetEpochDealersAddresses {
epoch_id,
limit,
start_after,
} => client
.get_epoch_dealers_addresses_paged(epoch_id, start_after, limit)
.ignore(),
DkgQueryMsg::GetDealerIndices { limit, start_after } => {
client.get_dealer_indices_paged(start_after, limit).ignore()
}
@@ -28,7 +28,7 @@ use cosmrs::proto::cosmwasm::wasm::v1::{
QueryRawContractStateResponse, QuerySmartContractStateRequest, QuerySmartContractStateResponse,
};
use cosmrs::tendermint::{block, chain, Hash};
use cosmrs::{AccountId, Coin as CosmosCoin, Tx};
use cosmrs::{AccountId, Coin as CosmosCoin};
use prost::Message;
use serde::{Deserialize, Serialize};
@@ -556,23 +556,12 @@ pub trait CosmWasmClient: TendermintRpcClient {
Ok(serde_json::from_slice(&res.data)?)
}
// deprecation warning is due to the fact the protobuf files built were based on cosmos-sdk 0.44,
// where they prefer using tx_bytes directly. However, in 0.42, which we are using at the time
// of writing this, the option does not work
// TODO: we should really stop using the `tx` argument here and use `tx_bytes` exlusively,
// however, at the time of writing this update, while our QA and mainnet networks do support it,
// sandbox is still running old version of wasmd that lacks support for `tx_bytes`
#[allow(deprecated)]
async fn query_simulate(
&self,
tx: Option<Tx>,
tx_bytes: Vec<u8>,
) -> Result<SimulateResponse, NyxdError> {
async fn query_simulate(&self, tx_bytes: Vec<u8>) -> Result<SimulateResponse, NyxdError> {
let path = Some("/cosmos.tx.v1beta1.Service/Simulate".to_owned());
let req = SimulateRequest {
tx: tx.map(Into::into),
tx_bytes,
..Default::default()
};
let res = self
@@ -81,17 +81,14 @@ where
auth_info: single_unspecified_signer_auth(public_key, sequence_response.sequence),
signatures: vec![Vec::new()],
};
self.query_simulate(Some(partial_tx), Vec::new()).await
// for completion sake, once we're able to transition into using `tx_bytes`,
// we might want to use something like this instead:
// let tx_raw: tx::Raw = cosmrs::proto::cosmos::tx::v1beta1::TxRaw {
// body_bytes: partial_tx.body.into_bytes().unwrap(),
// auth_info_bytes: partial_tx.auth_info.into_bytes().unwrap(),
// signatures: partial_tx.signatures,
// }
// .into();
// self.query_simulate(None, tx_raw.to_bytes().unwrap()).await
let tx_raw: tx::Raw = cosmrs::proto::cosmos::tx::v1beta1::TxRaw {
body_bytes: partial_tx.body.into_bytes()?,
auth_info_bytes: partial_tx.auth_info.into_bytes()?,
signatures: partial_tx.signatures,
}
.into();
self.query_simulate(tx_raw.to_bytes()?).await
}
async fn upload(
@@ -139,12 +139,22 @@ impl NyxdClient<HttpClient> {
})
}
pub fn connect_with_network_details<U>(
endpoint: U,
network_details: NymNetworkDetails,
) -> Result<QueryHttpRpcNyxdClient, NyxdError>
where
U: TryInto<HttpClientUrl, Error = TendermintRpcError>,
{
let config = Config::try_from_nym_network_details(&network_details)?;
Self::connect(config, endpoint)
}
pub fn connect_to_default_env<U>(endpoint: U) -> Result<QueryHttpRpcNyxdClient, NyxdError>
where
U: TryInto<HttpClientUrl, Error = TendermintRpcError>,
{
let config = Config::try_from_nym_network_details(&NymNetworkDetails::new_from_env())?;
Self::connect(config, endpoint)
Self::connect_with_network_details(endpoint, NymNetworkDetails::new_from_env())
}
}
+1 -1
View File
@@ -86,7 +86,7 @@ pub async fn execute(args: Args) -> anyhow::Result<()> {
anyhow!("ticketbook got incorrectly imported - the master verification key is missing")
})?;
let expiration_signatures = persistent_storage
.get_expiration_date_signatures(expiration_date)
.get_expiration_date_signatures(expiration_date, epoch_id)
.await?
.ok_or_else(|| {
anyhow!(
@@ -120,7 +120,7 @@ async fn issue_to_file(args: Args, client: SigningClient) -> anyhow::Result<()>
if args.include_expiration_date_signatures {
let signatures = credentials_store
.get_expiration_date_signatures(expiration_date)
.get_expiration_date_signatures(expiration_date, epoch_id)
.await?
.ok_or(anyhow!("missing expiration date signatures!"))?;
+3
View File
@@ -4,6 +4,7 @@
use clap::{Args, Subcommand};
pub mod ecash;
pub mod nyx;
#[derive(Debug, Args)]
#[clap(args_conflicts_with_subcommands = true, subcommand_required = true)]
@@ -16,4 +17,6 @@ pub struct Internal {
pub enum InternalCommands {
/// Ecash related internal commands
Ecash(ecash::InternalEcash),
Nyx(nyx::InternalNyx),
}
@@ -0,0 +1,116 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::context::SigningClient;
use anyhow::bail;
use clap::Parser;
use nym_mixnet_contract_common::nym_node::Role;
use nym_mixnet_contract_common::reward_params::NodeRewardingParameters;
use nym_mixnet_contract_common::{
EpochRewardedSet, EpochState, NodeId, RewardingParams, RoleAssignment,
};
use nym_validator_client::nyxd::contract_traits::mixnet_query_client::MixnetQueryClientExt;
use nym_validator_client::nyxd::contract_traits::{MixnetQueryClient, MixnetSigningClient};
use rand::prelude::*;
use rand::thread_rng;
#[derive(Debug, Parser)]
pub struct Args {}
fn choose_new_nodes(
params: &RewardingParams,
rewarded_set: &EpochRewardedSet,
role: Role,
) -> Vec<NodeId> {
let mut rng = thread_rng();
match role {
Role::EntryGateway => rewarded_set
.assignment
.entry_gateways
.choose_multiple(&mut rng, params.rewarded_set.entry_gateways as usize)
.copied()
.collect(),
Role::Layer1 => rewarded_set
.assignment
.layer1
.choose_multiple(&mut rng, params.rewarded_set.mixnodes as usize / 3)
.copied()
.collect(),
Role::Layer2 => rewarded_set
.assignment
.layer2
.choose_multiple(&mut rng, params.rewarded_set.mixnodes as usize / 3)
.copied()
.collect(),
Role::Layer3 => rewarded_set
.assignment
.layer3
.choose_multiple(&mut rng, params.rewarded_set.mixnodes as usize / 3)
.copied()
.collect(),
Role::ExitGateway => rewarded_set
.assignment
.exit_gateways
.choose_multiple(&mut rng, params.rewarded_set.exit_gateways as usize)
.copied()
.collect(),
Role::Standby => rewarded_set
.assignment
.standby
.choose_multiple(&mut rng, params.rewarded_set.standby as usize)
.copied()
.collect(),
}
}
pub async fn force_advance_epoch(_: Args, client: SigningClient) -> anyhow::Result<()> {
let current_epoch = client.get_current_interval_details().await?;
let epoch_status = client.get_current_epoch_status().await?;
if epoch_status.being_advanced_by.as_str() != client.address().to_string() {
bail!(
"this client is not authorised to perform any epoch operations. we need {}",
client.address()
)
}
let rewarding_params = client.get_rewarding_parameters().await?;
let current_rewarded_set = client.get_rewarded_set().await?;
if !current_epoch.is_current_epoch_over {
println!("the current epoch is not over yet - there's nothing to do")
}
// is this most efficient? no. but it's simple
loop {
let epoch_status = client.get_current_epoch_status().await?;
match epoch_status.state {
EpochState::InProgress => break,
EpochState::Rewarding { final_node_id, .. } => {
println!("rewarding {final_node_id} with big fat 0...");
client
.reward_node(
final_node_id,
NodeRewardingParameters::new(Default::default(), Default::default()),
None,
)
.await?;
}
EpochState::ReconcilingEvents => {
println!("trying to reconcile events...");
client.reconcile_epoch_events(None, None).await?;
}
EpochState::RoleAssignment { next } => {
let nodes = choose_new_nodes(&rewarding_params, &current_rewarded_set, next);
println!("assigning {nodes:?} as {next}");
client
.assign_roles(RoleAssignment { role: next, nodes }, None)
.await?;
}
}
}
Ok(())
}
+19
View File
@@ -0,0 +1,19 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use clap::{Args, Subcommand};
pub mod force_advance_epoch;
#[derive(Debug, Args)]
#[clap(args_conflicts_with_subcommands = true, subcommand_required = true)]
pub struct InternalNyx {
#[clap(subcommand)]
pub command: InternalNyxCommands,
}
#[derive(Debug, Subcommand)]
pub enum InternalNyxCommands {
/// Attempt to force advance the current epoch
ForceAdvanceEpoch(force_advance_epoch::Args),
}
@@ -241,7 +241,7 @@ pub async fn delegate_to_multiple_mixnodes(args: Args, client: SigningClient) {
let node_id = row.node_id.clone().parse::<u32>().unwrap();
let coins: Vec<Coin> = vec![];
undelegation_msgs.push((ExecuteMsg::Undelegate { node_id }, coins));
undelegation_table.add_row(&[row.node_id.clone()]);
undelegation_table.add_row(std::slice::from_ref(&row.node_id));
if row.amount.amount > 0 {
delegation_msgs
@@ -55,6 +55,14 @@ impl DealerDetailsResponse {
}
}
#[cw_serde]
pub struct PagedDealerAddressesResponse {
pub dealers: Vec<Addr>,
/// Field indicating paging information for the following queries if the caller wishes to get further entries.
pub start_next_after: Option<Addr>,
}
#[cw_serde]
pub struct PagedDealerResponse {
pub dealers: Vec<DealerDetails>,
@@ -12,8 +12,8 @@ use cosmwasm_schema::cw_serde;
#[cfg(feature = "schema")]
use crate::{
dealer::{
DealerDetailsResponse, PagedDealerIndexResponse, PagedDealerResponse,
RegisteredDealerDetails,
DealerDetailsResponse, PagedDealerAddressesResponse, PagedDealerIndexResponse,
PagedDealerResponse, RegisteredDealerDetails,
},
dealing::{
DealerDealingsStatusResponse, DealingChunkResponse, DealingChunkStatusResponse,
@@ -84,6 +84,9 @@ pub enum QueryMsg {
#[cfg_attr(feature = "schema", returns(Epoch))]
GetCurrentEpochState {},
#[cfg_attr(feature = "schema", returns(Option<Epoch>))]
GetEpochStateAtHeight { height: u64 },
#[cfg_attr(feature = "schema", returns(u64))]
GetCurrentEpochThreshold {},
@@ -102,6 +105,20 @@ pub enum QueryMsg {
#[cfg_attr(feature = "schema", returns(DealerDetailsResponse))]
GetDealerDetails { dealer_address: String },
#[cfg_attr(feature = "schema", returns(PagedDealerAddressesResponse))]
GetEpochDealersAddresses {
epoch_id: EpochId,
limit: Option<u32>,
start_after: Option<String>,
},
#[cfg_attr(feature = "schema", returns(PagedDealerResponse))]
GetEpochDealers {
epoch_id: EpochId,
limit: Option<u32>,
start_after: Option<String>,
},
#[cfg_attr(feature = "schema", returns(PagedDealerResponse))]
GetCurrentDealers {
limit: Option<u32>,
@@ -188,7 +188,7 @@ impl<C> ContractTesterBuilder<C> {
*self.app.api()
}
pub fn querier(&self) -> QuerierWrapper {
pub fn querier(&self) -> QuerierWrapper<'_> {
self.app.wrap()
}
}
@@ -57,7 +57,7 @@ pub trait NodeBond {
fn is_unbonding(&self) -> bool;
fn identity(&self) -> IdentityKeyRef;
fn identity(&self) -> IdentityKeyRef<'_>;
fn original_pledge(&self) -> &Coin;
@@ -125,7 +125,7 @@ impl NodeBond for MixNodeBond {
self.is_unbonding
}
fn identity(&self) -> IdentityKeyRef {
fn identity(&self) -> IdentityKeyRef<'_> {
self.identity()
}
@@ -178,7 +178,7 @@ impl NodeBond for NymNodeBond {
self.is_unbonding
}
fn identity(&self) -> IdentityKeyRef {
fn identity(&self) -> IdentityKeyRef<'_> {
self.identity()
}
@@ -58,7 +58,7 @@ impl<'a> PrimaryKey<'a> for Role {
type Suffix = <u8 as PrimaryKey<'a>>::Suffix;
type SuperSuffix = <u8 as PrimaryKey<'a>>::SuperSuffix;
fn key(&self) -> Vec<Key> {
fn key(&self) -> Vec<Key<'_>> {
// I'm not sure why it wasn't possible to delegate the call to
// `(*self as u8).key()` directly...
// I guess because of the `Key::Ref(&'a [u8])` variant?
@@ -86,6 +86,25 @@ impl IntervalRewardParams {
pub fn to_inline_json(&self) -> String {
to_json_string(self).unwrap_or_else(|_| "serialisation failure".into())
}
pub fn active_node_work(&self, standby_node_work: Decimal) -> WorkFactor {
self.active_set_work_factor * standby_node_work
}
pub fn standby_node_work(
&self,
rewarded_set_size: Decimal,
standby_set_size: Decimal,
) -> WorkFactor {
let f = self.active_set_work_factor;
let k = rewarded_set_size;
let one = Decimal::one();
// nodes in reserve
let k_r = standby_set_size;
one / (f * k - (f - one) * k_r)
}
}
/// Parameters used for reward calculation.
@@ -109,18 +128,15 @@ pub struct RewardingParams {
impl RewardingParams {
pub fn active_node_work(&self) -> WorkFactor {
self.interval.active_set_work_factor * self.standby_node_work()
let standby_work = self.standby_node_work();
self.interval.active_node_work(standby_work)
}
pub fn standby_node_work(&self) -> WorkFactor {
let f = self.interval.active_set_work_factor;
let k = self.dec_rewarded_set_size();
let one = Decimal::one();
// nodes in reserve
let k_r = self.dec_standby_set_size();
one / (f * k - (f - one) * k_r)
let rewarded_set_size = self.dec_rewarded_set_size();
let standby_set_size = self.dec_standby_set_size();
self.interval
.standby_node_work(rewarded_set_size, standby_set_size)
}
pub fn rewarded_set_size(&self) -> u32 {
@@ -3,6 +3,7 @@
use crate::config_score::{ConfigScoreParams, OutdatedVersionWeights, VersionScoreFormulaParams};
use crate::nym_node::Role;
use crate::reward_params::RewardedSetParams;
use crate::EpochId;
use contracts_common::Percent;
use cosmwasm_schema::cw_serde;
@@ -85,7 +86,11 @@ impl RewardedSet {
}
pub fn rewarded_set_size(&self) -> usize {
self.active_set_size() + self.standby.len()
self.active_set_size() + self.standby_set_size()
}
pub fn standby_set_size(&self) -> usize {
self.standby.len()
}
pub fn get_role(&self, node_id: NodeId) -> Option<Role> {
@@ -110,6 +115,13 @@ impl RewardedSet {
}
None
}
pub fn matches_parameters(&self, params: RewardedSetParams) -> bool {
self.entry_gateways.len() <= params.entry_gateways as usize
&& self.exit_gateways.len() <= params.exit_gateways as usize
&& self.layer1.len() + self.layer2.len() + self.layer3.len() <= params.mixnodes as usize
&& self.standby.len() <= params.standby as usize
}
}
#[cw_serde]
+55
View File
@@ -0,0 +1,55 @@
[package]
name = "nym-credential-proxy-lib"
version = "0.1.0"
authors.workspace = true
repository.workspace = true
homepage.workspace = true
documentation.workspace = true
edition.workspace = true
license.workspace = true
rust-version.workspace = true
readme.workspace = true
[dependencies]
anyhow = { workspace = true }
axum = { workspace = true }
bip39 = { workspace = true, features = ["zeroize"] }
bs58 = { workspace = true }
futures = { workspace = true }
humantime = { workspace = true }
rand = { workspace = true }
reqwest = { workspace = true, features = ["rustls-tls"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
strum = { workspace = true, features = ["derive"] }
strum_macros = { workspace = true }
sqlx = { workspace = true, features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate", "time"] }
time = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["sync"] }
tokio-util = { workspace = true, features = ["rt"] }
tracing = { workspace = true }
uuid = { workspace = true, features = ["serde"] }
url = { workspace = true }
zeroize = { workspace = true }
nym-credentials = { path = "../credentials" }
nym-crypto = { path = "../crypto", features = ["asymmetric", "rand", "serde"] }
nym-credentials-interface = { path = "../credentials-interface" }
nym-credential-proxy-requests = { path = "../../nym-credential-proxy/nym-credential-proxy-requests" }
nym-ecash-signer-check = { path = "../ecash-signer-check" }
nym-ecash-contract-common = { path = "../cosmwasm-smart-contracts/ecash-contract" }
nym-compact-ecash = { path = "../nym_offline_compact_ecash" }
nym-validator-client = { path = "../client-libs/validator-client" }
nym-network-defaults = { path = "../network-defaults" }
[dev-dependencies]
tempfile = { workspace = true }
[build-dependencies]
anyhow = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
sqlx = { workspace = true, features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate"] }
[lints]
workspace = true
@@ -1,22 +1,31 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use anyhow::Context;
#[tokio::main]
async fn main() {
async fn main() -> anyhow::Result<()> {
use sqlx::{Connection, SqliteConnection};
use std::env;
let out_dir = env::var("OUT_DIR").unwrap();
let out_dir = env::var("OUT_DIR")?;
let database_path = format!("{out_dir}/nym-credential-proxy-example.sqlite");
// remove the db file if it already existed from previous build
// in case it was from a different branch
if std::fs::exists(&database_path)? {
std::fs::remove_file(&database_path)?;
}
let mut conn = SqliteConnection::connect(&format!("sqlite://{database_path}?mode=rwc"))
.await
.expect("Failed to create SQLx database connection");
.context("Failed to create SQLx database connection")?;
sqlx::migrate!("./migrations")
.run(&mut conn)
.await
.expect("Failed to perform SQLx migrations");
.context("Failed to perform SQLx migrations")?;
println!("cargo:rustc-env=DATABASE_URL=sqlite://{}", &database_path);
Ok(())
}
@@ -0,0 +1,18 @@
/*
* Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
* SPDX-License-Identifier: GPL-3.0-only
*/
DROP TABLE global_expiration_date_signatures;
CREATE TABLE global_expiration_date_signatures
(
expiration_date DATE NOT NULL,
epoch_id INTEGER NOT NULL,
serialization_revision INTEGER NOT NULL,
-- combined signatures for all tuples issued for given day
serialised_signatures BLOB NOT NULL,
PRIMARY KEY (epoch_id, expiration_date)
)
@@ -0,0 +1,81 @@
/*
* Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
* SPDX-License-Identifier: GPL-3.0-only
*/
CREATE TABLE ecash_deposit
(
-- id assigned [by the contract] to the deposit
deposit_id INTEGER PRIMARY KEY NOT NULL,
-- associated tx hash
deposit_tx_hash TEXT NOT NULL,
-- indication of when the deposit request has been created
-- (so that based on block timestamp we could potentially determine latency)
requested_on TIMESTAMP WITHOUT TIME ZONE NOT NULL,
-- the amount put in the deposit (informative, as we expect this to change in the future)
deposit_amount TEXT NOT NULL,
-- the private key generated for the purposes of the deposit (the public component has been put in the transaction)
ed25519_deposit_private_key BLOB NOT NULL
);
INSERT INTO ecash_deposit(deposit_id, deposit_tx_hash, requested_on, deposit_amount, ed25519_deposit_private_key)
SELECT deposit_id, deposit_tx_hash, requested_on, deposit_amount, ed25519_deposit_private_key
FROM ticketbook_deposit;
CREATE TABLE ecash_deposit_usage
(
deposit_id INTEGER PRIMARY KEY REFERENCES ecash_deposit (deposit_id),
ticketbooks_requested_on TIMESTAMP WITHOUT TIME ZONE NOT NULL,
client_pubkey BLOB NOT NULL,
request_uuid TEXT UNIQUE NOT NULL,
-- this has to be improved later on to resume issuance or something, but for now it's fine
ticketbook_request_error TEXT
);
INSERT INTO ecash_deposit_usage(deposit_id, ticketbooks_requested_on, client_pubkey, request_uuid)
SELECT deposit_id, 0, client_pubkey, request_uuid
FROM ticketbook_deposit;
CREATE TABLE partial_blinded_wallet_new
(
corresponding_deposit INTEGER NOT NULL REFERENCES ecash_deposit_usage (deposit_id),
epoch_id INTEGER NOT NULL,
expiration_date DATE NOT NULL,
node_id INTEGER NOT NULL,
created TIMESTAMP WITHOUT TIME ZONE NOT NULL,
blinded_signature BLOB NOT NULL
);
CREATE TABLE partial_blinded_wallet_failure_new
(
corresponding_deposit INTEGER NOT NULL REFERENCES ecash_deposit_usage (deposit_id),
epoch_id INTEGER NOT NULL,
expiration_date DATE NOT NULL,
node_id INTEGER NOT NULL,
created TIMESTAMP WITHOUT TIME ZONE NOT NULL,
failure_message TEXT NOT NULL
);
INSERT INTO partial_blinded_wallet_new
SELECT *
FROM partial_blinded_wallet;
INSERT INTO partial_blinded_wallet_failure_new
SELECT *
FROM partial_blinded_wallet_failure;
DROP TABLE partial_blinded_wallet;
DROP TABLE partial_blinded_wallet_failure;
DROP TABLE ticketbook_deposit;
ALTER TABLE partial_blinded_wallet_new
RENAME TO partial_blinded_wallet;
ALTER TABLE partial_blinded_wallet_failure_new
RENAME TO partial_blinded_wallet_failure;
@@ -0,0 +1,101 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use crate::error::CredentialProxyError;
use crate::storage::models::StorableEcashDeposit;
use nym_compact_ecash::WithdrawalRequest;
use nym_credentials::IssuanceTicketBook;
use nym_crypto::asymmetric::ed25519;
use nym_validator_client::nyxd::{Coin, Hash};
use time::OffsetDateTime;
use zeroize::Zeroizing;
pub struct BufferedDeposit {
pub deposit_id: u32,
// note: this type implements `ZeroizeOnDrop`
pub ed25519_private_key: ed25519::PrivateKey,
}
impl TryFrom<StorableEcashDeposit> for BufferedDeposit {
type Error = CredentialProxyError;
fn try_from(deposit: StorableEcashDeposit) -> Result<Self, Self::Error> {
let ed25519_private_key = ed25519::PrivateKey::from_bytes(
deposit.ed25519_deposit_private_key.as_ref(),
)
.map_err(|err| CredentialProxyError::DatabaseInconsistency {
reason: format!("one of the stored deposit ed25519 private keys is malformed: {err}"),
})?;
Ok(BufferedDeposit {
deposit_id: deposit.deposit_id,
ed25519_private_key,
})
}
}
impl BufferedDeposit {
pub fn new(deposit_id: u32, ed25519_private_key: ed25519::PrivateKey) -> Self {
BufferedDeposit {
deposit_id,
ed25519_private_key,
}
}
pub fn sign_ticketbook_plaintext(
&self,
withdrawal_request: &WithdrawalRequest,
) -> ed25519::Signature {
let plaintext = IssuanceTicketBook::request_plaintext(withdrawal_request, self.deposit_id);
self.ed25519_private_key.sign(plaintext)
}
}
pub struct PerformedDeposits {
pub deposits_data: Vec<BufferedDeposit>,
// shared by all performed deposits as they were included in the same tx
pub tx_hash: Hash,
pub requested_on: OffsetDateTime,
pub deposit_amount: Coin,
}
impl PerformedDeposits {
pub(crate) fn to_storable(&self) -> Vec<StorableEcashDeposit> {
self.deposits_data
.iter()
.map(|d| StorableEcashDeposit {
deposit_id: d.deposit_id,
deposit_tx_hash: self.tx_hash.to_string(),
requested_on: self.requested_on,
deposit_amount: self.deposit_amount.to_string(),
ed25519_deposit_private_key: Zeroizing::new(d.ed25519_private_key.to_bytes()),
})
.collect()
}
}
pub(super) fn request_sizes(total: usize, max_request_size: usize) -> impl Iterator<Item = usize> {
(0..total)
.step_by(max_request_size)
.map(move |start| std::cmp::min(max_request_size, total - start))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn request_sizes_test() {
assert_eq!(
request_sizes(100, 32).collect::<Vec<_>>(),
vec![32, 32, 32, 4]
);
assert_eq!(request_sizes(10, 32).collect::<Vec<_>>(), vec![10]);
assert_eq!(request_sizes(32, 32).collect::<Vec<_>>(), vec![32]);
assert_eq!(request_sizes(33, 32).collect::<Vec<_>>(), vec![32, 1]);
assert_eq!(request_sizes(1, 32).collect::<Vec<_>>(), vec![1]);
}
}
@@ -0,0 +1,308 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use crate::deposits_buffer::helpers::request_sizes;
use crate::deposits_buffer::refill_task::RefillTask;
use crate::error::CredentialProxyError;
use crate::shared_state::nyxd_client::ChainClient;
use crate::shared_state::required_deposit_cache::RequiredDepositCache;
use crate::storage::CredentialProxyStorage;
use nym_compact_ecash::PublicKeyUser;
use nym_crypto::asymmetric::ed25519;
use nym_ecash_contract_common::deposit::DepositId;
use nym_validator_client::nyxd::cosmwasm_client::ContractResponseData;
use nym_validator_client::nyxd::Coin;
use rand::rngs::OsRng;
use std::sync::Arc;
use std::time::Duration;
use time::OffsetDateTime;
use tokio::sync::Mutex as AsyncMutex;
use tokio_util::sync::CancellationToken;
use tracing::{debug, error, info, instrument, warn};
use uuid::Uuid;
pub use helpers::{BufferedDeposit, PerformedDeposits};
pub(crate) mod helpers;
mod refill_task;
// TODO: I guess make it configurable
const DEPOSITS_THRESHOLD_P: f32 = 0.1;
struct DepositsBufferInner {
client: ChainClient,
required_deposit_cache: RequiredDepositCache,
storage: CredentialProxyStorage,
target_amount: usize,
max_concurrent_deposits: usize,
unused_deposits: AsyncMutex<Vec<BufferedDeposit>>,
deposits_refill_task: RefillTask,
short_sha: &'static str,
cancellation_token: CancellationToken,
}
#[derive(Clone)]
pub struct DepositsBuffer {
inner: Arc<DepositsBufferInner>,
}
impl DepositsBuffer {
pub async fn new(
storage: CredentialProxyStorage,
client: ChainClient,
required_deposit_cache: RequiredDepositCache,
short_sha: &'static str,
target_amount: usize,
max_concurrent_deposits: usize,
cancellation_token: CancellationToken,
) -> Result<Self, CredentialProxyError> {
let unused_deposits = storage.load_unused_deposits().await?;
info!("managed to load {} deposits", unused_deposits.len());
Ok(DepositsBuffer {
inner: Arc::new(DepositsBufferInner {
client,
required_deposit_cache,
storage,
target_amount,
max_concurrent_deposits,
unused_deposits: AsyncMutex::new(unused_deposits),
deposits_refill_task: RefillTask::default(),
short_sha,
cancellation_token,
}),
})
}
async fn deposit_amount(&self) -> Result<Coin, CredentialProxyError> {
self.inner
.required_deposit_cache
.get_or_update(&self.inner.client)
.await
}
#[instrument(skip(self), err(Display))]
async fn make_deposits_request(
&self,
amount: usize,
) -> Result<PerformedDeposits, CredentialProxyError> {
let requested_on = OffsetDateTime::now_utc();
let chain_write_permit = self.inner.client.start_chain_tx().await;
let mut rng = OsRng;
let deposit_amount = self.deposit_amount().await?;
let keys = (0..amount)
.map(|_| ed25519::PrivateKey::new(&mut rng))
.collect::<Vec<_>>();
info!("starting {amount} deposits");
let mut contents = Vec::new();
for key in &keys {
let public_key: ed25519::PublicKey = key.into();
contents.push((public_key.to_base58_string(), deposit_amount.clone()));
}
let execute_res = chain_write_permit
.make_deposits(self.inner.short_sha, contents)
.await?;
let tx_hash = execute_res.transaction_hash;
info!("{amount} deposits made in transaction: {tx_hash}");
let contract_data = match execute_res.to_contract_data() {
Ok(contract_data) => contract_data,
Err(err) => {
// that one is tricky. deposits technically got made, but we somehow failed to parse response,
// in this case terminate the proxy with 0 exit code so it wouldn't get automatically restarted
// because it requires some serious MANUAL intervention
error!("CRITICAL FAILURE: failed to parse out deposit information from the contract transaction. either the chain got upgraded and the schema changed or the ecash contract got changed! terminating the process. it has to be inspected manually. error was: {err}");
self.inner.cancellation_token.cancel();
return Err(CredentialProxyError::DepositFailure);
}
};
if contract_data.len() != amount {
// another critical failure, that one should be quite impossible and thus has to be manually inspected
error!("CRITICAL FAILURE: failed to parse out all deposit information from the contract transaction. got {} responses while we sent {amount} deposits! either the chain got upgraded and the schema changed or the ecash contract got changed! terminating the process. it has to be inspected manually", contract_data.len());
self.inner.cancellation_token.cancel();
return Err(CredentialProxyError::DepositFailure);
}
let mut deposits_data = Vec::new();
for (key, response) in keys.into_iter().zip(contract_data) {
let response_index = response.message_index;
let deposit_id = match response.parse_singleton_u32_contract_data() {
Ok(deposit_id) => deposit_id,
Err(err) => {
// another impossibility
error!("CRITICAL FAILURE: failed to parse out deposit id out of the response at index {response_index}: {err}. either the chain got upgraded and the schema changed or the ecash contract got changed! terminating the process. it has to be inspected manually");
self.inner.cancellation_token.cancel();
return Err(CredentialProxyError::DepositFailure);
}
};
deposits_data.push(BufferedDeposit::new(deposit_id, key));
}
Ok(PerformedDeposits {
deposits_data,
tx_hash,
requested_on,
deposit_amount,
})
}
async fn insert_new_deposits(
&self,
mut deposits: PerformedDeposits,
) -> Result<(), CredentialProxyError> {
// 1. insert into the db
self.inner.storage.insert_new_deposits(&deposits).await?;
// 2. update the buffer
self.inner
.unused_deposits
.lock()
.await
.append(&mut deposits.deposits_data);
Ok(())
}
/// Start refilling our deposit buffer.
/// It chunks the amount required based on the configured maximum request size
/// and updates global state after each successful transaction.
async fn refill_deposits(&self) -> Result<(), CredentialProxyError> {
let available = self.inner.unused_deposits.lock().await.len();
let target = self.deposits_upper_threshold();
let to_request = target - available;
for request_chunk in request_sizes(to_request, self.inner.max_concurrent_deposits) {
// note: we check for cancellation between individual requests
// as opposed to wrapping that in tokio::select! so that we would never abandon chain operations
// as we wouldn't want to lose funds
if self.inner.cancellation_token.is_cancelled() {
info!("received cancellation during deposits refilling");
return Ok(());
}
// make sure to insert deposits into db/vec as we get them so on initial run,
// we'd start trickling down data as soon as possible
let deposits = self.make_deposits_request(request_chunk).await?;
self.insert_new_deposits(deposits).await?;
}
Ok(())
}
// if we're here, we know we're below the threshold
fn maybe_refill_deposits(&self) {
if let Some(mut guard) = self.inner.deposits_refill_task.try_get_new_task_guard() {
let this = self.clone();
*guard = Some(tokio::spawn(async move { this.refill_deposits().await }));
}
}
fn deposits_lower_threshold(&self) -> usize {
self.inner.target_amount - (self.inner.target_amount as f32 * DEPOSITS_THRESHOLD_P) as usize
}
fn deposits_upper_threshold(&self) -> usize {
self.inner.target_amount + (self.inner.target_amount as f32 * DEPOSITS_THRESHOLD_P) as usize
}
async fn mark_deposit_as_used(
&self,
deposit_id: DepositId,
requested_on: OffsetDateTime,
client_pubkey: PublicKeyUser,
request_uuid: Uuid,
) -> Result<(), CredentialProxyError> {
self.inner
.storage
.insert_deposit_usage(deposit_id, requested_on, client_pubkey, request_uuid)
.await
}
async fn wait_for_deposit(
&self,
request_uuid: Uuid,
requested_on: OffsetDateTime,
client_pubkey: PublicKeyUser,
) -> Result<BufferedDeposit, CredentialProxyError> {
loop {
tokio::time::sleep(Duration::from_millis(500)).await;
if let Some(buffered_deposit) = self.inner.unused_deposits.lock().await.pop() {
// if the db call fails, we technically don't lose the deposit (we'll 'recover' it on restart)
self.mark_deposit_as_used(
buffered_deposit.deposit_id,
requested_on,
client_pubkey,
request_uuid,
)
.await?;
return Ok(buffered_deposit);
} else {
// make sure there's always a task working in the background in case deposits get used up too quickly
self.maybe_refill_deposits()
}
}
}
pub async fn get_valid_deposit(
&self,
request_uuid: Uuid,
requested_on: OffsetDateTime,
client_pubkey: PublicKeyUser,
) -> Result<BufferedDeposit, CredentialProxyError> {
let mut deposits_guard = self.inner.unused_deposits.lock().await;
let deposits_available = deposits_guard.len();
debug!("we have {deposits_available} unused deposits available");
let maybe_deposit = deposits_guard.pop();
drop(deposits_guard);
if deposits_available < self.deposits_lower_threshold() {
// if we're below threshold, start refill task
self.maybe_refill_deposits()
}
match maybe_deposit {
None => {
warn!("we currently don't have any usable deposits! are we using them up faster than we request them?");
// we have to wait until refill task has completed (either initiated by this or another fn call)
self.wait_for_deposit(request_uuid, requested_on, client_pubkey)
.await
}
Some(buffered_deposit) => {
self.mark_deposit_as_used(
buffered_deposit.deposit_id,
requested_on,
client_pubkey,
request_uuid,
)
.await?;
Ok(buffered_deposit)
}
}
}
pub async fn wait_for_shutdown(&self) {
let task_handle = self.inner.deposits_refill_task.take_task_join_handle();
if let Some(task_handle) = task_handle {
if !task_handle.is_finished() {
info!("the deposit refill task is currently in progress - waiting for the current transaction to finish before concluding shutdown");
let _ = task_handle.await;
}
}
}
}
impl DepositsBufferInner {
//
}
@@ -0,0 +1,56 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use crate::error::CredentialProxyError;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Mutex as StdMutex, MutexGuard};
use tokio::task::JoinHandle;
use tracing::{debug, error};
pub(super) type RefillTaskResult = Result<(), CredentialProxyError>;
#[derive(Default)]
pub(super) struct RefillTask {
// note that we can only have a single transaction in progress (or it'd mess up with our sequence numbers)
// if we find that we're using up deposits more quickly than we're refilling them,
// we'll have to increase the number of deposits per transaction
join_handle: StdMutex<Option<JoinHandle<RefillTaskResult>>>,
in_progress: AtomicBool,
}
impl RefillTask {
/// Attempt to set the `in_progress` value to `true` if it's not already `true`.
/// Returns boolean indicating whether it was successful
fn try_set_in_progress(&self) -> bool {
self.in_progress
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
.is_ok()
}
pub(super) fn try_get_new_task_guard(
&self,
) -> Option<MutexGuard<'_, Option<JoinHandle<RefillTaskResult>>>> {
// sanity check for concurrent request
if !self.try_set_in_progress() {
debug!("another task has already started deposit refill request");
return None;
}
#[allow(clippy::expect_used)]
let guard = self.join_handle.lock().expect("mutex got poisoned");
if let Some(existing_handle) = guard.as_ref() {
if !existing_handle.is_finished() {
error!("CRITICAL BUG: there was already a deposit refill task spawned that hasn't yet finished")
}
}
Some(guard)
}
pub(super) fn take_task_join_handle(&self) -> Option<JoinHandle<RefillTaskResult>> {
#[allow(clippy::expect_used)]
self.join_handle.lock().expect("mutex got poisoned").take()
}
}
@@ -1,6 +1,7 @@
// Copyright 2024 Nym Technologies SA <contact@nymtech.net>
// Copyright 2025 Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use nym_ecash_signer_check::SignerCheckError;
use nym_validator_client::coconut::EcashApiError;
use nym_validator_client::nym_api::EpochId;
use nym_validator_client::nyxd::error::NyxdError;
@@ -10,7 +11,7 @@ use thiserror::Error;
use time::OffsetDateTime;
#[derive(Debug, Error)]
pub enum VpnApiError {
pub enum CredentialProxyError {
#[error("encountered an internal io error: {source}")]
IoError {
#[from]
@@ -118,11 +119,44 @@ pub enum VpnApiError {
#[error("failed to create deposit")]
DepositFailure,
#[error("can't obtain sufficient number of credential shares due to unavailable quorum")]
UnavailableSigningQuorum,
#[error("failed to perform quorum check: {source}")]
QuorumCheckFailure {
#[from]
source: SignerCheckError,
},
#[error(
"this operation couldn't be completed as the program is in the process of shutting down"
)]
ShutdownInProgress,
#[error("failed to obtain wallet shares with id {id}: {message}")]
ShareByIdLoadError { message: String, id: i64 },
#[error("failed to obtain wallet shares with device_id {device_id} and credential_id: {credential_id}: {message}")]
ShareByDeviceLoadError {
message: String,
device_id: String,
credential_id: String,
},
#[error("could not find shares with id {id}")]
SharesByIdNotFound { id: i64 },
#[error("could not find shares with device_id {device_id} and credential_id: {credential_id}")]
SharesByDeviceNotFound {
device_id: String,
credential_id: String,
},
}
impl VpnApiError {
pub fn database_inconsistency<S: Into<String>>(reason: S) -> VpnApiError {
VpnApiError::DatabaseInconsistency {
impl CredentialProxyError {
pub fn database_inconsistency<S: Into<String>>(reason: S) -> CredentialProxyError {
CredentialProxyError::DatabaseInconsistency {
reason: reason.into(),
}
}
+67
View File
@@ -0,0 +1,67 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use rand::rngs::OsRng;
use rand::RngCore;
use time::OffsetDateTime;
use tracing::{debug, info, warn};
use uuid::Uuid;
pub fn random_uuid() -> Uuid {
let mut bytes = [0u8; 16];
let mut rng = OsRng;
rng.fill_bytes(&mut bytes);
Uuid::from_bytes(bytes)
}
pub struct LockTimer {
created: OffsetDateTime,
message: String,
}
impl LockTimer {
pub fn new<S: Into<String>>(message: S) -> Self {
LockTimer {
message: message.into(),
..Default::default()
}
}
}
impl Drop for LockTimer {
fn drop(&mut self) {
let time_taken = OffsetDateTime::now_utc() - self.created;
let time_taken_formatted = humantime::format_duration(time_taken.unsigned_abs());
if time_taken > time::Duration::SECOND * 10 {
warn!(time_taken = %time_taken_formatted, "{}", self.message)
} else if time_taken > time::Duration::SECOND * 5 {
info!(time_taken = %time_taken_formatted, "{}", self.message)
} else {
debug!(time_taken = %time_taken_formatted, "{}", self.message)
};
}
}
impl Default for LockTimer {
fn default() -> Self {
LockTimer {
created: OffsetDateTime::now_utc(),
message: "released the lock".to_string(),
}
}
}
// #[allow(clippy::panic)]
// fn build_sha_short() -> &'static str {
// let bin_info = bin_info!();
// if bin_info.commit_sha.len() < 7 {
// panic!("unavailable build commit sha")
// }
//
// if bin_info.commit_sha == "VERGEN_IDEMPOTENT_OUTPUT" {
// error!("the binary hasn't been built correctly. it doesn't have a commit sha information");
// return "unknown";
// }
//
// &bin_info.commit_sha[..7]
// }
@@ -1,11 +1,12 @@
// Copyright 2024 Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::error::VpnApiError;
use crate::error::CredentialProxyError;
use axum::http::StatusCode;
use axum::response::{IntoResponse, Response};
use axum::Json;
use nym_credential_proxy_requests::api::v1::ErrorResponse;
use tracing::warn;
use uuid::Uuid;
#[derive(Debug, Clone)]
@@ -35,7 +36,11 @@ impl RequestError {
}
}
pub fn new_server_error(err: VpnApiError, uuid: Uuid) -> Self {
pub fn new_plain_error(err: CredentialProxyError) -> Self {
Self::from_err(err, StatusCode::INTERNAL_SERVER_ERROR)
}
pub fn new_server_error(err: CredentialProxyError, uuid: Uuid) -> Self {
RequestError::new_with_uuid(err.to_string(), uuid, StatusCode::INTERNAL_SERVER_ERROR)
}
@@ -59,3 +64,12 @@ impl IntoResponse for RequestError {
(self.status, Json(self.inner)).into_response()
}
}
pub fn db_failure<T>(err: CredentialProxyError, uuid: Uuid) -> Result<T, RequestError> {
warn!("db failure: {err}");
Err(RequestError::new_with_uuid(
format!("oh no, something went wrong {err}"),
uuid,
StatusCode::INTERNAL_SERVER_ERROR,
))
}
+13
View File
@@ -0,0 +1,13 @@
// Copyright 2025 Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
pub mod deposits_buffer;
pub mod error;
pub mod helpers;
pub mod http_helpers;
pub mod nym_api_helpers;
pub mod quorum_checker;
pub mod shared_state;
pub mod storage;
pub mod ticketbook_manager;
pub mod webhook;
@@ -4,7 +4,7 @@
// TODO: this was just copied from nym-api;
// it should have been therefore extracted to a common crate instead and imported as dependency
use crate::error::VpnApiError;
use crate::error::CredentialProxyError;
use futures::{stream, StreamExt};
use nym_credentials::ecash::utils::{cred_exp_date, ecash_today, EcashTime};
use nym_validator_client::nym_api::EpochId;
@@ -19,9 +19,9 @@ use time::{Date, OffsetDateTime};
use tokio::sync::{Mutex, RwLock, RwLockReadGuard};
use tracing::warn;
pub(crate) struct CachedEpoch {
pub struct CachedEpoch {
valid_until: OffsetDateTime,
pub(crate) current_epoch: Epoch,
pub current_epoch: Epoch,
}
impl Default for CachedEpoch {
@@ -34,11 +34,11 @@ impl Default for CachedEpoch {
}
impl CachedEpoch {
pub(crate) fn is_valid(&self) -> bool {
pub fn is_valid(&self) -> bool {
self.valid_until > OffsetDateTime::now_utc()
}
pub(crate) fn update(&mut self, epoch: Epoch) {
pub fn update(&mut self, epoch: Epoch) {
let now = OffsetDateTime::now_utc();
let validity_duration = if let Some(epoch_finish) = epoch.deadline {
@@ -58,13 +58,13 @@ impl CachedEpoch {
}
// a map of items that never change for given key
pub(crate) struct CachedImmutableItems<K, V> {
pub struct CachedImmutableItems<K, V> {
// I wonder if there's a more efficient structure with OnceLock or OnceCell or something
inner: RwLock<HashMap<K, V>>,
}
// an item that stays constant throughout given epoch
pub(crate) type CachedImmutableEpochItem<T> = CachedImmutableItems<EpochId, T>;
pub type CachedImmutableEpochItem<T> = CachedImmutableItems<EpochId, T>;
impl<K, V> Default for CachedImmutableItems<K, V> {
fn default() -> Self {
@@ -86,7 +86,7 @@ impl<K, V> CachedImmutableItems<K, V>
where
K: Eq + Hash,
{
pub(crate) async fn get_or_init<F, U, E>(&self, key: K, f: F) -> Result<RwLockReadGuard<V>, E>
pub async fn get_or_init<F, U, E>(&self, key: K, f: F) -> Result<RwLockReadGuard<'_, V>, E>
where
F: FnOnce() -> U,
U: Future<Output = Result<V, E>>,
@@ -125,29 +125,29 @@ where
}
}
pub(crate) fn ensure_sane_expiration_date(expiration_date: Date) -> Result<(), VpnApiError> {
pub fn ensure_sane_expiration_date(expiration_date: Date) -> Result<(), CredentialProxyError> {
let today = ecash_today();
if expiration_date < today.date() {
// what's the point of signatures with expiration in the past?
return Err(VpnApiError::ExpirationDateTooEarly);
return Err(CredentialProxyError::ExpirationDateTooEarly);
}
if expiration_date > cred_exp_date().ecash_date() {
return Err(VpnApiError::ExpirationDateTooLate);
return Err(CredentialProxyError::ExpirationDateTooLate);
}
Ok(())
}
pub(crate) async fn query_all_threshold_apis<F, T, U>(
pub async fn query_all_threshold_apis<F, T, U>(
all_apis: Vec<EcashApiClient>,
threshold: u64,
f: F,
) -> Result<Vec<T>, VpnApiError>
) -> Result<Vec<T>, CredentialProxyError>
where
F: Fn(EcashApiClient) -> U,
U: Future<Output = Result<T, VpnApiError>>,
U: Future<Output = Result<T, CredentialProxyError>>,
{
let shares = Mutex::new(Vec::with_capacity(all_apis.len()));
@@ -168,7 +168,7 @@ where
let shares = shares.into_inner();
if shares.len() < threshold as usize {
return Err(VpnApiError::InsufficientNumberOfSigners {
return Err(CredentialProxyError::InsufficientNumberOfSigners {
threshold,
available: shares.len(),
});
@@ -0,0 +1,102 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use crate::error::CredentialProxyError;
use crate::shared_state::nyxd_client::ChainClient;
use nym_ecash_signer_check::{check_known_dealers, dkg_details_with_client};
use std::ops::Deref;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::Duration;
use tokio_util::sync::CancellationToken;
use tracing::{error, info, warn};
#[derive(Clone)]
pub struct QuorumState {
available: Arc<AtomicBool>,
}
impl QuorumState {
pub fn available(&self) -> bool {
self.available.load(Ordering::Acquire)
}
}
pub struct QuorumStateChecker {
client: ChainClient,
cancellation_token: CancellationToken,
check_interval: Duration,
quorum_state: QuorumState,
}
impl QuorumStateChecker {
pub async fn new(
client: ChainClient,
check_interval: Duration,
cancellation_token: CancellationToken,
) -> Result<Self, CredentialProxyError> {
let this = QuorumStateChecker {
client,
cancellation_token,
check_interval,
quorum_state: QuorumState {
available: Arc::new(Default::default()),
},
};
// first check MUST succeed, otherwise we shouldn't start
let quorum_available = this.check_quorum_state().await?;
this.quorum_state
.available
.store(quorum_available, Ordering::Relaxed);
Ok(this)
}
pub fn quorum_state_ref(&self) -> QuorumState {
self.quorum_state.clone()
}
async fn check_quorum_state(&self) -> Result<bool, CredentialProxyError> {
let client_guard = self.client.query_chain().await;
// split the operation as we only need to hold the reference to chain client for the first part
// and the second half doesn't rely on it (and takes way longer)
let dkg_details = dkg_details_with_client(client_guard.deref()).await?;
drop(client_guard);
let res = check_known_dealers(dkg_details).await?;
let Some(signing_threshold) = res.threshold else {
warn!("signing threshold is currently unavailable and we have not yet implemented credential issuance during DKG transition");
return Ok(false);
};
let mut working_issuer = 0;
for result in res.results {
if result.chain_available() && result.signing_available() {
working_issuer += 1;
}
}
Ok((working_issuer as u64) >= signing_threshold)
}
pub async fn run_forever(self) {
info!("starting quorum state checker");
loop {
tokio::select! {
biased;
_ = self.cancellation_token.cancelled() => {
break
}
_ = tokio::time::sleep(self.check_interval) => {
match self.check_quorum_state().await {
Ok(available) => self.quorum_state.available.store(available, Ordering::SeqCst),
Err(err) => error!("failed to check current quorum state: {err}"),
}
}
}
}
}
}
@@ -0,0 +1,49 @@
// Copyright 2025 Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use crate::nym_api_helpers::{CachedEpoch, CachedImmutableEpochItem, CachedImmutableItems};
use crate::quorum_checker::QuorumState;
use crate::shared_state::required_deposit_cache::RequiredDepositCache;
use nym_compact_ecash::VerificationKeyAuth;
use nym_credentials::{AggregatedCoinIndicesSignatures, AggregatedExpirationDateSignatures};
use nym_validator_client::nym_api::EpochId;
use nym_validator_client::EcashApiClient;
use time::Date;
use tokio::sync::RwLock;
pub struct EcashState {
pub required_deposit_cache: RequiredDepositCache,
pub quorum_state: QuorumState,
pub cached_epoch: RwLock<CachedEpoch>,
pub master_verification_key: CachedImmutableEpochItem<VerificationKeyAuth>,
pub threshold_values: CachedImmutableEpochItem<u64>,
pub epoch_clients: CachedImmutableEpochItem<Vec<EcashApiClient>>,
pub coin_index_signatures: CachedImmutableEpochItem<AggregatedCoinIndicesSignatures>,
pub expiration_date_signatures:
CachedImmutableItems<(EpochId, Date), AggregatedExpirationDateSignatures>,
}
impl EcashState {
pub fn new(
required_deposit_cache: RequiredDepositCache,
quorum_state: QuorumState,
) -> EcashState {
EcashState {
required_deposit_cache,
quorum_state,
cached_epoch: Default::default(),
master_verification_key: Default::default(),
threshold_values: Default::default(),
epoch_clients: Default::default(),
coin_index_signatures: Default::default(),
expiration_date_signatures: Default::default(),
}
}
}
@@ -0,0 +1,495 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use crate::deposits_buffer::{BufferedDeposit, DepositsBuffer};
use crate::error::CredentialProxyError;
use crate::nym_api_helpers::{ensure_sane_expiration_date, query_all_threshold_apis};
use crate::shared_state::ecash_state::EcashState;
use crate::shared_state::nyxd_client::ChainClient;
use crate::storage::CredentialProxyStorage;
use nym_compact_ecash::scheme::coin_indices_signatures::{
aggregate_annotated_indices_signatures, CoinIndexSignatureShare,
};
use nym_compact_ecash::scheme::expiration_date_signatures::{
aggregate_annotated_expiration_signatures, ExpirationDateSignatureShare,
};
use nym_compact_ecash::{Base58, PublicKeyUser, VerificationKeyAuth};
use nym_credential_proxy_requests::api::v1::ticketbook::models::{
AggregatedCoinIndicesSignaturesResponse, AggregatedExpirationDateSignaturesResponse,
GlobalDataParams, MasterVerificationKeyResponse,
};
use nym_credentials::ecash::utils::EcashTime;
use nym_credentials::{
AggregatedCoinIndicesSignatures, AggregatedExpirationDateSignatures, EpochVerificationKey,
};
use nym_ecash_contract_common::deposit::DepositId;
use nym_validator_client::coconut::EcashApiError;
use nym_validator_client::nym_api::EpochId;
use nym_validator_client::nyxd::contract_traits::dkg_query_client::Epoch;
use nym_validator_client::nyxd::contract_traits::{DkgQueryClient, PagedDkgQueryClient};
use nym_validator_client::nyxd::Coin;
use nym_validator_client::{DirectSigningHttpRpcNyxdClient, EcashApiClient};
use std::sync::Arc;
use std::time::Duration;
use time::{Date, OffsetDateTime};
use tokio::sync::RwLockReadGuard;
use tokio::time::Instant;
use tracing::{debug, error, info, warn};
use uuid::Uuid;
pub mod ecash_state;
pub mod nyxd_client;
pub mod required_deposit_cache;
#[derive(Clone)]
pub struct CredentialProxyState {
inner: Arc<CredentialProxyStateInner>,
}
impl CredentialProxyState {
pub fn new(
storage: CredentialProxyStorage,
client: ChainClient,
deposits_buffer: DepositsBuffer,
ecash_state: EcashState,
) -> Self {
CredentialProxyState {
inner: Arc::new(CredentialProxyStateInner {
storage,
client,
deposits_buffer,
ecash_state,
}),
}
}
pub fn storage(&self) -> &CredentialProxyStorage {
&self.inner.storage
}
pub async fn deposit_amount(&self) -> Result<Coin, CredentialProxyError> {
self.ecash_state()
.required_deposit_cache
.get_or_update(self.client())
.await
}
pub fn client(&self) -> &ChainClient {
&self.inner.client
}
pub fn deposits_buffer(&self) -> &DepositsBuffer {
&self.inner.deposits_buffer
}
pub fn ecash_state(&self) -> &EcashState {
&self.inner.ecash_state
}
pub(crate) async fn query_chain(&self) -> RwLockReadGuard<'_, DirectSigningHttpRpcNyxdClient> {
self.inner.client.query_chain().await
}
pub async fn ensure_credentials_issuable(&self) -> Result<(), CredentialProxyError> {
let epoch = self.current_epoch().await?;
if epoch.state.is_final() {
Ok(())
} else if let Some(final_timestamp) = epoch.final_timestamp_secs() {
// SAFETY: the timestamp values in our DKG contract should be valid timestamps,
// otherwise it means the chain is seriously misbehaving
#[allow(clippy::unwrap_used)]
let finish_dt = OffsetDateTime::from_unix_timestamp(final_timestamp as i64).unwrap();
Err(CredentialProxyError::CredentialsNotYetIssuable {
availability: finish_dt,
})
} else if epoch.state.is_waiting_initialisation() {
Err(CredentialProxyError::UninitialisedDkg)
} else {
Err(CredentialProxyError::UnknownEcashFailure)
}
}
pub async fn get_deposit(
&self,
request_uuid: Uuid,
requested_on: OffsetDateTime,
client_pubkey: PublicKeyUser,
) -> Result<BufferedDeposit, CredentialProxyError> {
let start = Instant::now();
let deposit = self
.deposits_buffer()
.get_valid_deposit(request_uuid, requested_on, client_pubkey)
.await;
let time_taken = start.elapsed();
let formatted = humantime::format_duration(time_taken);
if time_taken > Duration::from_secs(10) {
warn!("attempting to get buffered deposit took {formatted}. perhaps the buffer is too small or the process/chain is overloaded?")
} else {
debug!("attempting to get buffered deposit took {formatted}")
};
deposit
}
pub async fn insert_deposit_usage_error(&self, deposit_id: DepositId, error: String) {
if let Err(err) = self
.storage()
.insert_deposit_usage_error(deposit_id, error)
.await
{
error!("failed to insert information about deposit (id: {deposit_id}) usage failure: {err}")
}
}
pub async fn current_epoch_id(&self) -> Result<EpochId, CredentialProxyError> {
let read_guard = self.inner.ecash_state.cached_epoch.read().await;
if read_guard.is_valid() {
return Ok(read_guard.current_epoch.epoch_id);
}
// update cache
drop(read_guard);
let mut write_guard = self.inner.ecash_state.cached_epoch.write().await;
let epoch = self.query_chain().await.get_current_epoch().await?;
write_guard.update(epoch);
Ok(epoch.epoch_id)
}
pub async fn current_epoch(&self) -> Result<Epoch, CredentialProxyError> {
let read_guard = self.ecash_state().cached_epoch.read().await;
if read_guard.is_valid() {
return Ok(read_guard.current_epoch);
}
// update cache
drop(read_guard);
let mut write_guard = self.ecash_state().cached_epoch.write().await;
let epoch = self.query_chain().await.get_current_epoch().await?;
write_guard.update(epoch);
Ok(epoch)
}
pub async fn global_data(
&self,
global_data: GlobalDataParams,
epoch_id: EpochId,
expiration_date: Date,
) -> Result<
(
Option<MasterVerificationKeyResponse>,
Option<AggregatedExpirationDateSignaturesResponse>,
Option<AggregatedCoinIndicesSignaturesResponse>,
),
CredentialProxyError,
> {
let master_verification_key = if global_data.include_master_verification_key {
debug!("including master verification key in the response");
Some(
self.master_verification_key(Some(epoch_id))
.await
.map(|key| MasterVerificationKeyResponse {
epoch_id,
bs58_encoded_key: key.to_bs58(),
})
.inspect_err(|err| warn!("request failure: {err}"))?,
)
} else {
None
};
let aggregated_expiration_date_signatures =
if global_data.include_expiration_date_signatures {
debug!("including expiration date signatures in the response");
Some(
self.master_expiration_date_signatures(epoch_id, expiration_date)
.await
.map(|signatures| AggregatedExpirationDateSignaturesResponse {
signatures: signatures.clone(),
})
.inspect_err(|err| warn!("request failure: {err}"))?,
)
} else {
None
};
let aggregated_coin_index_signatures = if global_data.include_coin_index_signatures {
debug!("including coin index signatures in the response");
Some(
self.master_coin_index_signatures(Some(epoch_id))
.await
.map(|signatures| AggregatedCoinIndicesSignaturesResponse {
signatures: signatures.clone(),
})
.inspect_err(|err| warn!("request failure: {err}"))?,
)
} else {
None
};
Ok((
master_verification_key,
aggregated_expiration_date_signatures,
aggregated_coin_index_signatures,
))
}
pub async fn master_verification_key(
&self,
epoch_id: Option<EpochId>,
) -> Result<RwLockReadGuard<'_, VerificationKeyAuth>, CredentialProxyError> {
let epoch_id = match epoch_id {
Some(id) => id,
None => self.current_epoch_id().await?,
};
self.inner
.ecash_state
.master_verification_key
.get_or_init(epoch_id, || async {
// 1. check the storage
if let Some(stored) = self
.inner
.storage
.get_master_verification_key(epoch_id)
.await?
{
return Ok(stored.key);
}
info!("attempting to establish master verification key for epoch {epoch_id}...");
// 2. perform actual aggregation
let all_apis = self.ecash_clients(epoch_id).await?;
let threshold = self.ecash_threshold(epoch_id).await?;
if all_apis.len() < threshold as usize {
return Err(CredentialProxyError::InsufficientNumberOfSigners {
threshold,
available: all_apis.len(),
});
}
let master_key = nym_credentials::aggregate_verification_keys(&all_apis)?;
let epoch = EpochVerificationKey {
epoch_id,
key: master_key,
};
// 3. save the key in the storage for when we reboot
self.inner
.storage
.insert_master_verification_key(&epoch)
.await?;
Ok(epoch.key)
})
.await
}
pub async fn master_coin_index_signatures(
&self,
epoch_id: Option<EpochId>,
) -> Result<RwLockReadGuard<'_, AggregatedCoinIndicesSignatures>, CredentialProxyError> {
let epoch_id = match epoch_id {
Some(id) => id,
None => self.current_epoch_id().await?,
};
self.inner
.ecash_state
.coin_index_signatures
.get_or_init(epoch_id, || async {
// 1. check the storage
if let Some(master_sigs) = self
.inner
.storage
.get_master_coin_index_signatures(epoch_id)
.await?
{
return Ok(master_sigs);
}
info!(
"attempting to establish master coin index signatures for epoch {epoch_id}..."
);
// 2. go around APIs and attempt to aggregate the data
let master_vk = self.master_verification_key(Some(epoch_id)).await?;
let all_apis = self.ecash_clients(epoch_id).await?;
let threshold = self.ecash_threshold(epoch_id).await?;
let get_partial_signatures = |api: EcashApiClient| async {
// move the api into the closure
let api = api;
let node_index = api.node_id;
let partial_vk = api.verification_key;
let partial = api
.api_client
.partial_coin_indices_signatures(Some(epoch_id))
.await?
.signatures;
Ok(CoinIndexSignatureShare {
index: node_index,
key: partial_vk,
signatures: partial,
})
};
let shares =
query_all_threshold_apis(all_apis.clone(), threshold, get_partial_signatures)
.await?;
let aggregated = aggregate_annotated_indices_signatures(
nym_credentials_interface::ecash_parameters(),
&master_vk,
&shares,
)?;
let sigs = AggregatedCoinIndicesSignatures {
epoch_id,
signatures: aggregated,
};
// 3. save the signatures in the storage for when we reboot
self.inner
.storage
.insert_master_coin_index_signatures(&sigs)
.await?;
Ok(sigs)
})
.await
}
pub async fn master_expiration_date_signatures(
&self,
epoch_id: EpochId,
expiration_date: Date,
) -> Result<RwLockReadGuard<'_, AggregatedExpirationDateSignatures>, CredentialProxyError> {
self.inner.ecash_state
.expiration_date_signatures
.get_or_init((epoch_id, expiration_date), || async {
// 1. sanity check to see if the expiration_date is not nonsense
ensure_sane_expiration_date(expiration_date)?;
// 2. check the storage
if let Some(master_sigs) = self
.storage()
.get_master_expiration_date_signatures(expiration_date, epoch_id)
.await?
{
return Ok(master_sigs);
}
info!(
"attempting to establish master expiration date signatures for {expiration_date} and epoch {epoch_id}..."
);
// 3. go around APIs and attempt to aggregate the data
let epoch_id = self.current_epoch_id().await?;
let master_vk = self.master_verification_key(Some(epoch_id)).await?;
let all_apis = self.ecash_clients(epoch_id).await?;
let threshold = self.ecash_threshold(epoch_id).await?;
let get_partial_signatures = |api: EcashApiClient| async {
// move the api into the closure
let api = api;
let node_index = api.node_id;
let partial_vk = api.verification_key;
let partial = api
.api_client
.partial_expiration_date_signatures(Some(expiration_date), Some(epoch_id))
.await?
.signatures;
Ok(ExpirationDateSignatureShare {
index: node_index,
key: partial_vk,
signatures: partial,
})
};
let shares =
query_all_threshold_apis(all_apis.clone(), threshold, get_partial_signatures)
.await?;
let aggregated = aggregate_annotated_expiration_signatures(
&master_vk,
expiration_date.ecash_unix_timestamp(),
&shares,
)?;
let sigs = AggregatedExpirationDateSignatures {
epoch_id,
expiration_date,
signatures: aggregated,
};
// 4. save the signatures in the storage for when we reboot
self.inner.storage
.insert_master_expiration_date_signatures(&sigs)
.await?;
Ok(sigs)
})
.await
}
pub async fn ecash_clients(
&self,
epoch_id: EpochId,
) -> Result<RwLockReadGuard<'_, Vec<EcashApiClient>>, CredentialProxyError> {
self.inner
.ecash_state
.epoch_clients
.get_or_init(epoch_id, || async {
Ok(self
.query_chain()
.await
.get_all_verification_key_shares(epoch_id)
.await?
.into_iter()
.map(TryInto::try_into)
.collect::<anyhow::Result<Vec<_>, EcashApiError>>()?)
})
.await
}
pub async fn ecash_threshold(&self, epoch_id: EpochId) -> Result<u64, CredentialProxyError> {
self.inner
.ecash_state
.threshold_values
.get_or_init(epoch_id, || async {
if let Some(threshold) = self
.query_chain()
.await
.get_epoch_threshold(epoch_id)
.await?
{
Ok(threshold)
} else {
Err(CredentialProxyError::UnavailableThreshold { epoch_id })
}
})
.await
.map(|t| *t)
}
}
struct CredentialProxyStateInner {
storage: CredentialProxyStorage,
client: ChainClient,
deposits_buffer: DepositsBuffer,
ecash_state: EcashState,
}
@@ -0,0 +1,126 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::error::CredentialProxyError;
use crate::helpers::LockTimer;
use nym_ecash_contract_common::msg::ExecuteMsg;
use nym_validator_client::nyxd::contract_traits::NymContractsProvider;
use nym_validator_client::nyxd::cosmwasm_client::types::ExecuteResult;
use nym_validator_client::nyxd::{Coin, CosmWasmClient, NyxdClient};
use nym_validator_client::{nyxd, DirectSigningHttpRpcNyxdClient};
use std::ops::Deref;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::{RwLock, RwLockReadGuard, RwLockWriteGuard};
use tracing::{instrument, warn};
#[derive(Clone)]
pub struct ChainClient(Arc<RwLock<DirectSigningHttpRpcNyxdClient>>);
impl ChainClient {
pub fn new(mnemonic: bip39::Mnemonic) -> Result<Self, CredentialProxyError> {
let network_details = nym_network_defaults::NymNetworkDetails::new_from_env();
let client_config = nyxd::Config::try_from_nym_network_details(&network_details)?;
let nyxd_url = network_details
.endpoints
.first()
.ok_or_else(|| CredentialProxyError::NoNyxEndpointsAvailable)?
.nyxd_url
.as_str();
let client = NyxdClient::connect_with_mnemonic(client_config, nyxd_url, mnemonic)?;
if client.ecash_contract_address().is_none() {
return Err(CredentialProxyError::UnavailableEcashContract);
}
if client.dkg_contract_address().is_none() {
return Err(CredentialProxyError::UnavailableDKGContract);
}
Ok(ChainClient(Arc::new(RwLock::new(client))))
}
pub async fn query_chain(&self) -> ChainReadPermit<'_> {
let _acquire_timer = LockTimer::new("acquire chain query permit");
self.0.read().await
}
pub async fn start_chain_tx(&self) -> ChainWritePermit<'_> {
let _acquire_timer = LockTimer::new("acquire exclusive chain write permit");
ChainWritePermit {
lock_timer: LockTimer::new("exclusive chain access permit"),
inner: self.0.write().await,
}
}
}
pub type ChainReadPermit<'a> = RwLockReadGuard<'a, DirectSigningHttpRpcNyxdClient>;
// explicitly wrap the WriteGuard for extra information regarding time taken
pub struct ChainWritePermit<'a> {
// it's not really dead, we only care about it being dropped
#[allow(dead_code)]
lock_timer: LockTimer,
inner: RwLockWriteGuard<'a, DirectSigningHttpRpcNyxdClient>,
}
impl ChainWritePermit<'_> {
#[instrument(skip(self, short_sha, info), err(Display))]
pub async fn make_deposits(
self,
short_sha: &'static str,
info: Vec<(String, Coin)>,
) -> Result<ExecuteResult, CredentialProxyError> {
let address = self.inner.address();
let starting_sequence = self.inner.get_sequence(&address).await?.sequence;
let deposits = info.len();
let ecash_contract = self
.inner
.ecash_contract_address()
.ok_or(CredentialProxyError::UnavailableEcashContract)?;
let deposit_messages = info
.into_iter()
.map(|(identity_key, amount)| {
(
ExecuteMsg::DepositTicketBookFunds { identity_key },
vec![amount],
)
})
.collect::<Vec<_>>();
let res = self
.inner
.execute_multiple(
ecash_contract,
deposit_messages,
None,
format!("cp-{short_sha}: performing {deposits} deposits"),
)
.await?;
loop {
let updated_sequence = self.inner.get_sequence(&address).await?.sequence;
if updated_sequence > starting_sequence {
break;
}
warn!("wrong sequence number... waiting before releasing chain lock");
tokio::time::sleep(Duration::from_millis(50)).await;
}
Ok(res)
}
}
impl Deref for ChainWritePermit<'_> {
type Target = DirectSigningHttpRpcNyxdClient;
fn deref(&self) -> &Self::Target {
self.inner.deref()
}
}
@@ -0,0 +1,71 @@
// Copyright 2025 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use crate::error::CredentialProxyError;
use crate::shared_state::nyxd_client::ChainClient;
use nym_validator_client::nyxd::contract_traits::EcashQueryClient;
use nym_validator_client::nyxd::Coin;
use std::sync::Arc;
use time::OffsetDateTime;
use tokio::sync::RwLock;
pub struct CachedDeposit {
valid_until: OffsetDateTime,
required_amount: Coin,
}
impl CachedDeposit {
const MAX_VALIDITY: time::Duration = time::Duration::MINUTE;
fn is_valid(&self) -> bool {
self.valid_until > OffsetDateTime::now_utc()
}
fn update(&mut self, required_amount: Coin) {
self.valid_until = OffsetDateTime::now_utc() + Self::MAX_VALIDITY;
self.required_amount = required_amount;
}
}
impl Default for CachedDeposit {
fn default() -> Self {
CachedDeposit {
valid_until: OffsetDateTime::UNIX_EPOCH,
required_amount: Coin {
amount: u128::MAX,
denom: "unym".to_string(),
},
}
}
}
#[derive(Clone, Default)]
pub struct RequiredDepositCache {
inner: Arc<RwLock<CachedDeposit>>,
}
impl RequiredDepositCache {
pub async fn get_or_update(
&self,
chain_client: &ChainClient,
) -> Result<Coin, CredentialProxyError> {
let read_guard = self.inner.read().await;
if read_guard.is_valid() {
return Ok(read_guard.required_amount.clone());
}
// update cache
drop(read_guard);
let mut write_guard = self.inner.write().await;
let deposit_amount = chain_client
.query_chain()
.await
.get_required_deposit_amount()
.await?;
let nym_coin: Coin = deposit_amount.into();
write_guard.update(nym_coin.clone());
Ok(nym_coin)
}
}

Some files were not shown because too many files have changed in this diff Show More