Compare commits

..

78 Commits

Author SHA1 Message Date
Bogdan-Ștefan Neacşu 3aa2d859a2 Gateway ipv6 separation of peers 2024-11-13 16:57:23 +00:00
Jon Häggblad 556ea76cf8 Allow Unicode-3.0 license in cargo-deny (#5123) 2024-11-13 09:57:45 +01:00
Simon Wicky b424c6a8ff [Product Data] Add stats reporting configuration in client config (#5115)
* add stats_reporting_config in config and env var

* fix serializazion issue

* remove duplicate stats reporting config

* cargo toml cleanup

* more cleanup

* draft of wasm sdk for stats reporting

* fix wasm sdk?

* again

* make stats sending possible from outside the sdk

* make sure stats_id from client and gateway reported ared different
2024-11-13 08:38:35 +01:00
Tommy Verrall 100eea8f64 Merge pull request #5119 from nymtech/feat/add-gh-workflow-validator-rewarder
feat: add GH workflow for nym-validator-rewarder
2024-11-12 15:57:28 +00:00
Tommy Verrall e52d977d46 Merge pull request #5118 from nymtech/feat/add-dockerfile-add-env-vars
feat: add Dockerfile and add env vars for clap arguments
2024-11-12 15:57:07 +00:00
Fran Arbanas 30133a06ec feat: add GH workflow for nym-validator-rewarder 2024-11-12 12:09:05 +01:00
Fran Arbanas 261caae7f6 feat: add Dockerfile and add env vars for clap arguments 2024-11-12 11:59:09 +01:00
Bogdan-Ștefan Neacşu 84fff02e12 Correct IPv6 address generation (#5113)
* Correct IPv6 addres generation

* Fix clippy
2024-11-11 13:57:44 +02:00
Tommy Verrall f005693643 Merge pull request #4789 from nymtech/dependabot/npm_and_yarn/nym-wallet/webdriver/micromatch-4.0.8
build(deps): bump micromatch from 4.0.4 to 4.0.8 in /nym-wallet/webdriver
2024-11-08 17:24:23 +00:00
Tommy Verrall 5a0b20683e Merge pull request #4768 from nymtech/dependabot/npm_and_yarn/testnet-faucet/elliptic-6.5.7
Bump elliptic from 6.5.4 to 6.5.7 in /testnet-faucet
2024-11-08 17:23:42 +00:00
Tommy Verrall 69c36e8cb0 Merge pull request #4790 from nymtech/dependabot/npm_and_yarn/nym-api/tests/axios-1.7.5
build(deps): bump axios from 1.6.0 to 1.7.5 in /nym-api/tests
2024-11-08 17:23:00 +00:00
Simon Wicky b09ac57597 [Product Data] Client-side stats collection (#5107)
* draft of client data collection

* refactor gateway stats collection to fit client stats collection in same common crate

* moved client stats event and reporter to common crate

* basic os reporting

* add stats reporting address in sdk

* integrate stats scaffolding changes

* remove tokio spawn to potentially accomodate wasm32

* fmt

* fix typo

* add client_stats_id

* unify stats reporting

* avoid shutdown handle drop

* add client_type to stats reporting

* better way to build statsReportingconfig

* disarm shutdown on sink

* remove sink reporter and env dev-dependency

* cherrypick from jon/send-packet-stats

* uncoditionally start controller + licensing

* improve ClientStatsReport serialization

* better time handling

* reintroduce proper local reporting

* Let task wait for shutdown when exiting

* Log tweak

---------

Co-authored-by: jmwample <jmwample@users.noreply.github.com>
Co-authored-by: Jon Häggblad <jon.haggblad@gmail.com>
2024-11-08 14:00:29 +01:00
Tommy Verrall d8322d696b Merge pull request #5111 from nymtech/feature/granular_node_log
Add granular log on nym-node
2024-11-08 12:15:06 +00:00
Bogdan-Ștefan Neacşu 430255fea5 Add granular log on nym-node 2024-11-08 13:39:34 +02:00
Jon Häggblad 85a122f3eb Send mixnet packet stats using task client (#5109) 2024-11-08 10:49:36 +01:00
Jędrzej Stuczyński 3147d6aef7 chore: ecash contract migration to remove unused 'redemption_gateway_share' (#5104) 2024-11-08 09:34:04 +00:00
Bogdan-Ștefan Neacşu 74db9ab779 Expose time range (#5108) 2024-11-08 11:10:22 +02:00
Jon Häggblad 0bb287af89 Merge pull request #4919 from nymtech/jon/create-task-event-trait
Create TaskStatusEvent trait instead of piggybacking on Error
2024-11-08 10:09:51 +01:00
Jon Häggblad f4a2cec5aa Remove outdated comment 2024-11-08 09:07:00 +01:00
Jon Häggblad 7f434b2b26 Creat event mod 2024-11-07 23:54:38 +01:00
Jon Häggblad 3722c6c47d Remove requiring error trait 2024-11-07 23:54:38 +01:00
Jon Häggblad 903a60e7c1 Rename trait 2024-11-07 23:54:38 +01:00
Jon Häggblad 2d34a5ec3d Create trait for status events 2024-11-07 23:54:38 +01:00
mx 0bdf750be9 Max/fix links (#5106)
* fix twitter footer link

* fix old link of mdbook variables
2024-11-07 11:11:34 +00:00
Bogdan-Ștefan Neacşu 44ae29b06d IPv6 support for wireguard (#5059)
* Add ipv6 in configs

* Make v4 latest

* Fix linux

* IPv6 prefix in config

* Fix template of private ip

* Fix clippy

* Fix v6 cidr

* Move from 2001:db8::/32 to fc00::/7 addresses

* Fix version number on conversion
2024-11-07 12:31:01 +02:00
Dinko Zdravac bfd7240dcd Sync code with .env in build.rs (#4876)
* Sync code with .env in build.rs

* PR feedback
2024-11-06 22:28:04 +01:00
Fran Arbanas 9c680fd7b4 feat: add functionality to specify a git_ref for gateway probe when b… (#5094)
* feat: add functionality to specify a git_ref for gateway probe when building node status agent

* remove mac specifics

* fix: remove unused part of code, add gateway probe git ref to tag name

* fix: add cleaning gateway probe git ref

* fix: incorrect bash

* fix: incorrect output name

* workflow fix
2024-11-06 13:11:42 +01:00
mx c7d025baba try fix scan redirect failures (#5100)
* try fix scan redirect failures

* yet more redirects
2024-11-06 09:53:49 +00:00
Simon Wicky ec7482e417 publishing list of actvie client hashes (#5084) 2024-11-05 09:28:17 +01:00
Dinko Zdravac 307d326f82 Merge pull request #5085 from nymtech/develop-release-merge
Merge release into develop
2024-11-05 08:58:57 +01:00
mx 2e746e9890 [DOCs]: Fix all redirects (#5086)
* add new redirect

* add new redirect fix

* add new redirect fix

* try remove prepend from redirects

* more tweaks config redirects

* new tweaks

* move root redirect to bottom

* tweak

* tweak

* test the difference in config

* correct all redirects based on testing - ready to review

* adding one more redirect

---------

Co-authored-by: import this <97586125+serinko@users.noreply.github.com>
2024-11-04 18:36:10 +00:00
dynco-nym e840c1fe93 Merge branch 'release/2024.13-magura' into develop-release-merge 2024-11-04 16:50:44 +01:00
mx d6599b2933 [DOCs]: New docs framework (#5017)
* started todo list for rework

* startd long todo list

* startd long todo list

* remove ts docs from ts sdk dir

* started new docs draft

* rearranged code example dir structure

* modified code component filepaths

* first pass rust sdk

* small shift typescript org

* updated todo list

* consolidating images folders in one place

* first pass @ operator docs

* gen updates

* sdk in its own dir

* first pass developers structure

* first pass network structure

* structure

* add licensing

* moved old docs -> old_docs dir for clarity when devving

* moving around new docs - think this is the final dir structure

* updated todo list

* new autodoc version (#4781)

* Update rework_todo.md

* quick first sketch of landing page

* rework of structure of developers

* added arch and concepts stubs

* add new bits to todo list

* new list

* tweak to overview

* mixnet node overview

* tweak overview

* first pass new arch

* first pass concepts

* first pass traffic

* more network docs

* moved some chain files to the dev portal stubs

* removed old reference to archive

* note to client

* concepts 2nd pass

* crypto first proper pass, sphinx

* stub for not p2p

* structure change

* traffic 2nd pass

* misc

* hid root index

* overhaul arch

* overhaul arch

* add links + tweaks

* update todo list

* updating nyx section

* added zknym docs

* added zknym docs

* note on where to find deployed info

* smart contracts done

* started moving integrations docs over from ts sdk

* pass @ integration page

* todo for the tldr overview

* added ffi stub files

* updated todo list

* move sdks to developers

* initial pass at new clients overview for developers

* rework intro

* add echo serv to tools

* sidebar autocollapse

* integration overview work + tools

* concepts overview for devporta

* stub

* more for networking pages

* added to concepts in dev portal

* updated arch

* crypto overview page

* typo fix

* add credential stub

* first pass concepts done

* start reorg of rust sdk docs

* reorg + added FFI table

* added no scroll to inline code

* finished ffi overview page

* first pass @ rest of rust sdk doc

* first pass ffi

* tweaks

* added testnet example + note to custom topology example overview

* stripped unnecessary stuff from TS

* tweaks to ffi

* updated faq

* first pass tcpproxy

* commit before moving image dir

* moved images/ to correct place

* started on client redo

* chain first pass

* moved cli wallet out of tools

* first pass new ws client

* new chain info, left todo links in

* links

* more links

* chain registry

* added echo server to tools

* rust sdk links

* ts sdk links

* final linkchecks

* redo acks diagram as mermaid

* add mermaid flow diagram

* added links for codecs + full flow diagram

* removed todo

* remove forced dark mode

* diagram + concepts overview

* small correction re tcpproxy ffi

* remove diagram title

* new sock5 diagram, minor client docs tweaks

* diagrams

* change order in list

* added note for standalone: can be accessed via sdk

* tweaks

* replaced old diagram with mermaid

* fixed link

* hardcoded import version for the moment

* update deps

* remove test component

* recreated tools dir

* remove tools dir moved to wrong palce

* prebuild and predev script for autodoc commands

* make script own command instead of prebuild

* made code blocks sh

* updated autogenerated docs

* temp

* auto commit generated command files

* add link to autodoc generated files

* updated autodoc for committing changing else exit

* auto commit generated command files

* updated readme

* make subcommand headers smaller

* removed mdbook related scripts

* update readme

* update readme

* removed backups of root meta.json

* cherry pick yana commits + some extra config in theme

* update readme

* update theme: width of page and padding

* some more themeing

* changed erroneous note

* docs redirects first pass

* tweaking

* new pages + rest of redirects for old docs/

* brought in archive + done rewrites for devportal

* cherry pick yana landingpage

* tweaked landing page component

* changed theme of mermaid diagram to match everything else

* updated todo list

* [DOCs]: Operators rework to next.js (#4930)

* initialise operators guides v2

* new introduction page

* add variables csv and page

* add baseurl to allow short path

* add sandbox page

* added building from source page

* add binary pages

* add preliminary steps

* clean preliminary steps dir

* syntax edit

* syntax edit

* add configuration page

* create new proxy configuration page

* create new proxy configuration page

* create bonding.mdx page

* correct images path

* syntax edit

* add new validator setup page

* add api setup page

* add nyx configuration page

* add nym node and maintenance pages

* finish maintenance and add nymvisor conf page

* add manual upgrade page

* add nymvisor upgrade page

* add performance testing page and dir

* add node api check page

* add explore nym scripts page

* add testing pages

* fix menu issue by moving snippets to coomponents

* add all troubleshooting pages

* add general faq page

* add nym node faq page

* add nyx faq page

* revamp legal forum to community counsel and add all pages

* rewire relative paths to new structure

* simplify setup and remove lock file

* syntax fix

* rm package.json

* re add package.json, rm package-lock.json

* removed old books from commit

* address review comments

---------

Co-authored-by: mfahampshire <maxhampshire@pm.me>
Co-authored-by: mx <33262279+mfahampshire@users.noreply.github.com>

* tweak client links

* also moved matrix images to correct place

* Max/fix links new docs framework (#4989)

* tweak client links
* standardise images in public/
* old images move to public/archive

* rename overview to more descriptive

* links (#4990)

* links
* removed todos
* updated todo list

* minor themeing

* operator redirects

* pick yana's edits: remove specified callout theming

* added todo comments for old ts sdk redirects

* [new/docs/operators]: Create archive section - PR ready to merge (#5004)

* [new-docs/operators] : Fix callout syntax (#5006)

* fix callout syntax from color to type

* correct callout from danger to warning

* update footer

* updated footer

* finalised rewrites

* tweaks to clients and reintroduced old examples page

* update todo

* Max/individual command autodocs (#5015)


* auto commit generated command files

* added to autodoc.sh: build all binaries before running

* autodoc move individual command outputs to components

* Max/individual command autodocs (#5018)


* updated autodoc script

* updated autodoc script for fix + reintroduced gitignore file for generated markdown

* auto commit generated command files

* auto commit generated command files

* added command-outputs to autodoc script

* fix merge conflicts

* repush components

* remove old docs dirs

* auto commit generated command files

* auto commit generated command files

* updated messages paradigm with the standalone proxies

* [NEW-DOCs/operators]: Command output, accordion, api scraping & all final tasks (#5026)

* add custom scripts, create prebuild to import data to pages

* update after latest prebuild

* auto commit generated command files

* add accordion component

* add changbelog page

* add node_api_check outputs

* finish all command outputs

* more accordions beautifications

* finish accordion

* PR ready to go

* address review comments

---------

Co-authored-by: mfahampshire <maxhampshire@pm.me>

* Adjust padding

* Fix responsive design

* cherry pick yana landingpage flex update

* reremove old docs

* added dependencies to readme

* pushing build attempt changes

* fix merge errors, path errors, dump uselss dinosaurs - BUILT THE F*N DOCS w success

* moved prebuild to its own script

* generate timenow

* auto commit generated command files

* remove comment

* auto commit generated command files

* auto commit generated command files

* auto commit generated command files

* build from new configs

* add mdx type as explicit dep

* remove rc from version in package

* change predev script

* update readme with scripts

* update general info

* add license

* auto commit generated command files

* add updated components

* removed old examples page for the moment

* remove old list will reintroduce hidden behind gitignore for future

* reintroduce todo list behind gitignore

* added standalone tcpproxy binary info

* nothing change for redeploy test

* make build standalone

* updated readme

* working on new cd

* remove export

* updated ci/cd for docs

* added ci script for dist

* hide text on laptop wide screen

* add pnpm to ci/cd

* add pnpm version to ci/cd

* add default dir to ci/cd

* change path to script

* update projct name ci

* lint ci branch ignore

* add basePath to next.config.js

* update doc rewrites

* revert basePath addition

* update basePath

* add mobile styles

* fix responsive style

* remove old ts sdk docs workflow

* temp remove autodoc from workspace

* update sidebar for clarity: crypto = cryptography

* ignore documentation in pr-validation workflow

---------

Co-authored-by: Yana <yanok87@users.noreply.github.com>
Co-authored-by: import this <97586125+serinko@users.noreply.github.com>
Co-authored-by: fmtabbara <fmtabbara@hotmail.co.uk>
2024-11-04 11:42:56 +00:00
Bogdan-Ștefan Neacşu 5cefa7fdd4 Don't increase bandwidth again (#5081) 2024-11-04 13:15:27 +02:00
Fran Arbanas 5e0417ebe7 feat: add nym node GH workflow (#5080) 2024-11-04 10:41:40 +01:00
Fran Arbanas de4239a5dd fix: update dockerfile env vars description (#5079) 2024-11-04 10:35:49 +01:00
Mark Sinclair 80b590d50d bug-fix: nym-credential-proxy webhook request is the correct shape and added reporting errors via webhook (#5077)
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
2024-11-01 21:48:04 +01:00
Bogdan-Ștefan Neacşu f9b363648f Fix expiration date as today + 7 days (#5076) 2024-11-01 16:01:24 +02:00
Bogdan-Ștefan Neacşu b73561f1c9 Fix gateway decreasing bandwidth (#5075)
* Update storage peers after periodic check

* Reset storage bytes on restart

* Fix clippy
2024-11-01 15:40:22 +02:00
Dinko Zdravac 09b68a8204 Cherry pick NS API from develop (#5074)
* Revert "NS API with directory v2 (#5068)"

This reverts commit cf4fe5f875.

* Merge pull request #5050 from nymtech/dz-node-status-api

Node Status API

* Ns agent workflow (#5055)

* feat: add dockerfile

* add github workflow for node status agent

---------

Co-authored-by: Fran Arbanas <arbanasfran@gmail.com>

* NS API with directory v2 (#5058)

* Use unstable explorer client

* Clean up stale testruns & logging
- log gw identity key
- better agent testrun logging
- log responses
- change response code for agents

* Better logging on agent

* Testrun stores gw identity key instead of gw pk

* Agent 0.1.3

* Agent 0.1.4

* Sqlx offline query data + clippy

* Compatible with directory v2

* Point to internal deps + rebase + v0.1.5

* self described field not null

* Fix build.rs typo

* Fix clippy

---------

Co-authored-by: Fran Arbanas <arbanasfran@gmail.com>
2024-11-01 01:24:41 +01:00
Fouad 0374626960 Allow custom http port to be reset (#5073)
* allow custom port to be reset in wallet
2024-10-31 16:53:55 +00:00
Dinko Zdravac cf4fe5f875 NS API with directory v2 (#5068)
* Use unstable explorer client

* Clean up stale testruns & logging
- log gw identity key
- better agent testrun logging
- log responses
- change response code for agents

* Better logging on agent

* Testrun stores gw identity key instead of gw pk

* Agent 0.1.3

* Agent 0.1.4

* Sqlx offline query data + clippy

* Compatible with directory v2

* Point to internal deps + rebase + v0.1.5

* self described field not null

* Fix build.rs typo
2024-10-31 13:52:20 +01:00
Jędrzej Stuczyński 9f8bf2d080 bugfix: wallet backend fixes (#5070)
* fixed simulation arguments

* make sure 'try_convert_pubkey_to_node_id' checks for native nymnodes
2024-10-31 12:23:20 +00:00
Jędrzej Stuczyński b9d1fc40e7 deprecated old nym-api client methods and replaced them when possible (#5069) 2024-10-31 12:08:58 +00:00
Dinko Zdravac c740f84336 NS API with directory v2 (#5058)
* Use unstable explorer client

* Clean up stale testruns & logging
- log gw identity key
- better agent testrun logging
- log responses
- change response code for agents

* Better logging on agent

* Testrun stores gw identity key instead of gw pk

* Agent 0.1.3

* Agent 0.1.4

* Sqlx offline query data + clippy

* Compatible with directory v2

* Point to internal deps + rebase + v0.1.5

* self described field not null

* Fix build.rs typo
2024-10-31 04:32:41 +01:00
Jędrzej Stuczyński 16de47ba57 Merge pull request #5063 from nymtech/merge2/release/2024.13-magura
Merge2/release/2024.13 magura
2024-10-30 14:30:11 +00:00
Jędrzej Stuczyński 54a823311b Merge branch 'release/2024.13-magura' into develop 2024-10-30 14:16:07 +00:00
dependabot[bot] 2ca7c7a252 build(deps): bump lazy_static from 1.4.0 to 1.5.0 (#4913) 2024-10-30 07:07:39 +01:00
dependabot[bot] e680e8dc49 build(deps): bump once_cell from 1.19.0 to 1.20.2 (#4952)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.19.0 to 1.20.2.
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.19.0...v1.20.2)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-29 23:27:10 +01:00
Jon Häggblad 242bc93807 Merge pull request #5027 from nymtech/jon/integrate-credential-proxy-into-workspace
Integrate nym-credential-proxy into workspace
2024-10-29 20:47:07 +01:00
dynco-nym 94c6cdc7b2 Type coercion into time::Date 2024-10-29 17:46:35 +01:00
Jon Häggblad fce322c789 Remove unused workflow 2024-10-29 17:46:35 +01:00
Jon Häggblad ac5baab693 Add to default workspace 2024-10-29 17:46:35 +01:00
Jon Häggblad 23da0f4d8e Workspace updates 2024-10-29 17:46:35 +01:00
Jon Häggblad 25e3b4cd83 Delete old Cargo files 2024-10-29 17:46:35 +01:00
Jon Häggblad 8e4d72a565 Update for rebase 2024-10-29 17:46:34 +01:00
Jon Häggblad ad84a6d85d Add nym-vpn-api crates to main workspace 2024-10-29 17:45:56 +01:00
Jędrzej Stuczyński 34c5f23684 Merge pull request #5061 from nymtech/merge1/release/2024.13-magura
checkpoint merge release/2024.13-magura into develop
2024-10-29 16:17:16 +00:00
Jędrzej Stuczyński 000f2f1c29 Merge branch 'release/2024.13-magura' into develop 2024-10-29 15:31:51 +00:00
Dinko Zdravac b747308f74 Add subcommand to image (#5056) 2024-10-29 10:52:33 +01:00
Dinko Zdravac afdd721cc3 Ns agent workflow (#5055)
* feat: add dockerfile

* add github workflow for node status agent

---------

Co-authored-by: Fran Arbanas <arbanasfran@gmail.com>
2024-10-29 10:39:58 +01:00
Dinko Zdravac 9f5c4c5968 Merge pull request #5050 from nymtech/dz-node-status-api
Node Status API
2024-10-29 00:43:33 +01:00
dynco-nym 9583a5c6c8 Fix build script 2024-10-29 00:24:18 +01:00
Tommy Verrall da60fc0ade Merge pull request #5052 from nymtech/feat/add-node-status-agent-workflow
feat: add simple node-status-agent
2024-10-28 19:30:52 +00:00
Fran Arbanas 96b54c455e feat: add simple node-status-agent 2024-10-28 19:16:46 +01:00
Dinko Zdravac cc983963d4 Fully functional network scores (#5048)
* Compile & copy wg probe

* Node status agent WIP

* Enable debug logging

* Agent submits results
- add clap to agent
- agent runs network probe
- /submit endpoint on NS API

* Build clients with timeouts

* Update logging and dev scripts

* Replace /blaclisted endpoint

* Testruns fully functional
- task that queues testruns periodically
- testruns read/write in DB

* Probe scores fully working
- testruns are assigned on API
- submit updates testruns correctly on NS API side
- agent registers with API
- agent submits results correctly

* Clippy fixes

* PR feedback

* Clippy again

* PR feedback

* Run clippy earlier in CI

* Make refresh delay configurable in server & agent
2024-10-28 17:31:43 +01:00
Fran Arbanas 40d9321aec Node status API dockerfile and env vars (#4986)
* feat: add dockerfile and env variables

* Added workflow for pushing node status api on harbor

* Misc changes to pathing and using yq instead of jq

* fix: change the way we read env vars for nyxd, nym api and explorer

* fix: docker build workflow

* Remove config in favor of clap args

* Added naming and tags

* change from value to result

---------

Co-authored-by: Lawrence Stalder <lawrence@nymtech.net>
Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
2024-10-28 17:07:38 +01:00
dynco-nym e5a29cc76e Work with directory pre-v2.1
Rebase + point to earlier network client code

Adjust to new Nym API types

Refer to earlier client code

Revert "Rebase + point to earlier network client code"

This reverts commit dd75e7dc0695c25b0883e2f5dd15b7d70165e9e8.

Point to earlier commit
2024-10-28 17:04:22 +01:00
Dinko Zdravac 56c55f6b95 Working HTTP server (#4941)
* Server file structure

* Create HTTP server
- graceful shutdown
- routes
- logging, CORS

* gateways WIP

* gateways API + swagger docs complete

* Mixnodes API + swagger docs complete

* Services API + swagger docs complete

* Commit summary insert

* Make troubleshooting DB easier

* Summary API + swagger docs

* Client log changes

* QOL improvements

- remove implicit panics via `as`
- safer DTO conversions
- add logging
- new config
2024-10-28 16:59:12 +01:00
Dinko Zdravac 2f051fd943 Node Status API background task (#4854)
* Setup new package

* Setup DB

* Fetch & store mixnodes/GWs
- refactor db package structure
- finally solve DATABASE_URL: absolute path works best

* Additional query functionality
- missing only daily summary, which requires type refactoring

* Replace type alias tuples with structs

* Insert summary

* Add github job to build package

* Build script for sqlx

* Remove data dir
- useless now that sqlx DB sits in OUT_DIR

* PR feedback
2024-10-28 16:54:26 +01:00
Bogdan-Ștefan Neacşu c03cf86000 Authenticator CLI client mode (#5044) 2024-10-28 16:42:05 +02:00
Simon Wicky ab11508235 [Product Data] Introduce data persistence on gateways (#5022)
* add stats storage to gateways

* config fix

* add stats storage model and logic

* adapt stats collection to new storage

* stats cleanup on start

* change to linux only code

* tweaks

* modified stats cleanup + change session started

* change wrong table name

* store crashed session as 0 duration

* adapt for sqlx 0.7

* remove unused dependencies

* revert changes from gateway config, as it is broken anyway

* copyright and misc stuff

---------

Co-authored-by: Simon Wicky <simon@linode2-2.net>
2024-10-28 09:25:37 +01:00
Fran Arbanas e65bfaeb31 Fix/nym data observatory dockerfile (#5021)
* fix: added needed env vars to dockerfile, updated db env for a bit

* feat: add github workflow for pushing data observatory

* feat: split the postgresql connection string into multiple variables

* fix docker compose

* fix workflow

* fix: short in clap
2024-10-24 18:10:34 +02:00
Jędrzej Stuczyński 5a6982fd10 Merge pull request #5011 from nymtech/dependabot/cargo/patch-updates-9a83837eff
Bump the patch-updates group across 1 directory with 10 updates
2024-10-23 10:51:55 +01:00
dependabot[bot] 7abe1f505c Bump the patch-updates group across 1 directory with 10 updates
Bumps the patch-updates group with 9 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [anyhow](https://github.com/dtolnay/anyhow) | `1.0.89` | `1.0.90` |
| [clap](https://github.com/clap-rs/clap) | `4.5.18` | `4.5.20` |
| [clap_complete](https://github.com/clap-rs/clap) | `4.5.29` | `4.5.33` |
| [pin-project](https://github.com/taiki-e/pin-project) | `1.1.5` | `1.1.6` |
| [serde](https://github.com/serde-rs/serde) | `1.0.210` | `1.0.211` |
| [serde_json](https://github.com/serde-rs/json) | `1.0.128` | `1.0.132` |
| [wasm-bindgen](https://github.com/rustwasm/wasm-bindgen) | `0.2.93` | `0.2.95` |
| [wasm-bindgen-futures](https://github.com/rustwasm/wasm-bindgen) | `0.4.43` | `0.4.45` |
| [web-sys](https://github.com/rustwasm/wasm-bindgen) | `0.3.70` | `0.3.72` |



Updates `anyhow` from 1.0.89 to 1.0.90
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.89...1.0.90)

Updates `clap` from 4.5.18 to 4.5.20
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.18...clap_complete-v4.5.20)

Updates `clap_complete` from 4.5.29 to 4.5.33
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.29...clap_complete-v4.5.33)

Updates `pin-project` from 1.1.5 to 1.1.6
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.1.5...v1.1.6)

Updates `serde` from 1.0.210 to 1.0.211
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.210...v1.0.211)

Updates `serde_derive` from 1.0.210 to 1.0.211
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.210...v1.0.211)

Updates `serde_json` from 1.0.128 to 1.0.132
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/1.0.128...1.0.132)

Updates `wasm-bindgen` from 0.2.93 to 0.2.95
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/compare/0.2.93...0.2.95)

Updates `wasm-bindgen-futures` from 0.4.43 to 0.4.45
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/commits)

Updates `web-sys` from 0.3.70 to 0.3.72
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/commits)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: clap_complete
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: pin-project
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: wasm-bindgen
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: wasm-bindgen-futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: web-sys
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-22 14:56:39 +00:00
dependabot[bot] 1f748ecbe8 build(deps): bump axios from 1.6.0 to 1.7.5 in /nym-api/tests
Bumps [axios](https://github.com/axios/axios) from 1.6.0 to 1.7.5.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.6.0...v1.7.5)

---
updated-dependencies:
- dependency-name: axios
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-26 12:27:07 +00:00
dependabot[bot] 2dbfdf377a build(deps): bump micromatch in /nym-wallet/webdriver
Bumps [micromatch](https://github.com/micromatch/micromatch) from 4.0.4 to 4.0.8.
- [Release notes](https://github.com/micromatch/micromatch/releases)
- [Changelog](https://github.com/micromatch/micromatch/blob/4.0.8/CHANGELOG.md)
- [Commits](https://github.com/micromatch/micromatch/compare/4.0.4...4.0.8)

---
updated-dependencies:
- dependency-name: micromatch
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-26 12:26:35 +00:00
dependabot[bot] 45f9ffa3a3 Bump elliptic from 6.5.4 to 6.5.7 in /testnet-faucet
Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.4 to 6.5.7.
- [Commits](https://github.com/indutny/elliptic/compare/v6.5.4...v6.5.7)

---
updated-dependencies:
- dependency-name: elliptic
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-20 12:53:09 +00:00
946 changed files with 49577 additions and 16187 deletions
+15 -28
View File
@@ -6,20 +6,27 @@ on:
jobs:
build:
runs-on: arc-ubuntu-20.04
defaults:
run:
working-directory: documentation/docs
steps:
- uses: actions/checkout@v4
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get install -y build-essential curl wget libssl-dev libudev-dev squashfs-tools protobuf-compiler git python3 && sudo apt-get update --fix-missing
- name: Install pip3
run: sudo apt install -y python3-pip
run: sudo apt install -y python3-pip
- name: Install Python3 modules
run: sudo pip3 install pandas tabulate
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
- name: Setup pnpm
uses: pnpm/action-setup@v4.0.0
with:
version: 9
- uses: actions/setup-node@v4
with:
node-version: 18
node-version: 20
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
@@ -29,33 +36,13 @@ jobs:
with:
command: build
args: --workspace --release
- name: Install mdbook and plugins
run: cd documentation && ./install_mdbook_deps.sh
- name: Remove existing Nym config directory (`~/.nym/`)
run: cd documentation && ./remove_existing_config.sh
continue-on-error: false
# This is the original flow
# - name: Build all projects in documentation/ & move to ~/dist/docs/
# run: cd documentation && ./build_all_to_dist.sh
# This is a workaround replacement which builds on the last working commit b332a6b55668f60988e36961f3f62a794ba82ddb and then on current branch
- name: Save current branch to ~/current_branch
run: git rev-parse --abbrev-ref HEAD > ~/current_branch
- name: Git pull, reset & switch to b332a6b55668f60988e36961f3f62a794ba82ddb
run: git pull && git reset --hard && git checkout b332a6b55668f60988e36961f3f62a794ba82ddb
- name: Build all projects in documentation/ & move to ~/dist/docs/ from b332a6b55668f60988e36961f3f62a794ba82ddb
run: cd documentation && ./build_all_to_dist.sh
- name: Switch to current branch
run: git checkout $echo "$(cat ~/current_branch)"
- name: Build all projects in documentation/ & move to ~/dist/docs/ on current branch
run: cd documentation && ./build_all_to_dist.sh && rm ~/current_branch
# End of replacemet
- name: Post process
run: cd documentation && ./post_process.sh
continue-on-error: false
- name: Install project dependencies
run: pnpm i
- name: Build project
run: pnpm run build
- name: Move files to /dist/
run: ../scripts/move-to-dist.sh
- name: Create Vercel project file
uses: mobiledevops/secret-to-file-action@v1
+6 -6
View File
@@ -57,6 +57,12 @@ jobs:
command: fmt
args: --all -- --check
- name: Clippy
uses: actions-rs/cargo@v1
with:
command: clippy
args: --workspace --all-targets -- -D warnings
- name: Build all binaries
uses: actions-rs/cargo@v1
with:
@@ -82,9 +88,3 @@ jobs:
with:
command: test
args: --workspace -- --ignored
- name: Clippy
uses: actions-rs/cargo@v1
with:
command: clippy
args: --workspace --all-targets -- -D warnings
+19 -29
View File
@@ -3,28 +3,35 @@ name: ci-docs
on:
workflow_dispatch:
push:
branches-ignore: master
branches-ignore: [master]
paths:
- 'documentation/docs/**'
- '.github/workflows/ci-docs.yml'
- "documentation/docs/**"
- ".github/workflows/ci-docs.yml"
jobs:
build:
runs-on: arc-ubuntu-20.04
defaults:
run:
working-directory: documentation/docs
steps:
- uses: actions/checkout@v4
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get install -y build-essential curl wget libssl-dev libudev-dev squashfs-tools protobuf-compiler git python3 && sudo apt-get update --fix-missing
- name: Install pip3
run: sudo apt install -y python3-pip
run: sudo apt install -y python3-pip
- name: Install Python3 modules
run: sudo pip3 install pandas tabulate
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
- name: Setup pnpm
uses: pnpm/action-setup@v4.0.0
with:
version: 9
- uses: actions/setup-node@v4
with:
node-version: 18
node-version: 20
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
@@ -34,30 +41,13 @@ jobs:
with:
command: build
args: --workspace --release
- name: Install mdbook and plugins
run: cd documentation && ./install_mdbook_deps.sh
- name: Remove existing Nym config directory (`~/.nym/`)
run: cd documentation && ./remove_existing_config.sh
continue-on-error: false
# This is the original flow
# - name: Build all projects in documentation/ & move to ~/dist/docs/
# run: cd documentation && ./build_all_to_dist.sh
# This is a workaround replacement which builds on the last working commit b332a6b55668f60988e36961f3f62a794ba82ddb and then on current branch
- name: Save current branch to ~/current_branch
run: git rev-parse --abbrev-ref HEAD > ~/current_branch
- name: Git pull, reset & switch to b332a6b55668f60988e36961f3f62a794ba82ddb
run: git pull && git reset --hard && git checkout b332a6b55668f60988e36961f3f62a794ba82ddb
- name: Build all projects in documentation/ & move to ~/dist/docs/ from b332a6b55668f60988e36961f3f62a794ba82ddb
run: cd documentation && ./build_all_to_dist.sh
- name: Switch to current branch
run: git checkout $echo "$(cat ~/current_branch)"
- name: Build all projects in documentation/ & move to ~/dist/docs/ on current branch
run: cd documentation && ./build_all_to_dist.sh && rm ~/current_branch
# End of replacemet
- name: Install project dependencies
run: pnpm i
- name: Build project
run: pnpm run build
- name: Move files to /dist/
run: ../scripts/move-to-dist.sh
- name: Deploy branch to CI www
continue-on-error: true
@@ -68,5 +58,5 @@ jobs:
SOURCE: "dist/docs/"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/docs-${{ env.GITHUB_REF_SLUG }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/docs-nextra-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/node_modules/"
@@ -1,45 +0,0 @@
name: ci-nym-credential-proxy
on:
pull_request:
paths:
- 'common/**'
- 'nym-credential-proxy/**'
- '.github/workspace/ci-nym-credential-proxy.yml'
workflow_dispatch:
jobs:
build:
runs-on: arc-ubuntu-22.04
env:
CARGO_TERM_COLOR: always
MANIFEST_PATH: "--manifest-path nym-credential-proxy/Cargo.toml"
steps:
- name: Check out repository code
uses: actions/checkout@v4
- name: Install rust toolchain
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
components: rustfmt, clippy
- name: Check formatting
uses: actions-rs/cargo@v1
with:
command: fmt
args: ${{ env.MANIFEST_PATH }} --all -- --check
- name: Build
uses: actions-rs/cargo@v1
with:
command: build
args: ${{ env.MANIFEST_PATH }} --workspace --all-targets
- name: Clippy
uses: actions-rs/cargo@v1
with:
command: clippy
args: ${{ env.MANIFEST_PATH }} --workspace --all-targets -- -D warnings
@@ -1,79 +0,0 @@
name: ci-sdk-docs-typescript
on:
pull_request:
paths:
- "sdk/typescript/**"
- "wasm/**"
- '.github/workflows/ci-sdk-docs-typescript.yml'
jobs:
build:
runs-on: custom-linux
steps:
- uses: actions/checkout@v4
- name: Install rsync
run: sudo apt-get install rsync
continue-on-error: true
- uses: rlespinasse/github-slug-action@v3.x
- uses: actions/setup-node@v4
with:
node-version: 18.17
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: Setup yarn
run: npm install -g yarn
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.20'
- name: Install wasm-pack
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
- name: Install wasm-opt
uses: ./.github/actions/install-wasm-opt
with:
version: '116'
- name: Build branch WASM packages
run: make sdk-wasm-build
- name: Install
run: yarn
- name: Build
run: yarn docs:prod:build
- name: Deploy branch to CI www (docs)
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-rltgoDzvO --delete"
SOURCE: "dist/ts/"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/sdk-ts-docs-${{ env.GITHUB_REF_SLUG }}
EXCLUDE: "/dist/, /node_modules/"
- name: Matrix - Node Install
run: npm install
working-directory: .github/workflows/support-files
- name: Matrix - Send Notification
env:
NYM_NOTIFICATION_KIND: ts-packages
NYM_PROJECT_NAME: "sdk-ts-docs"
NYM_CI_WWW_BASE: "${{ secrets.NYM_CI_WWW_BASE }}/docs/sdk/typescript"
NYM_CI_WWW_LOCATION: "sdk-ts-docs-${{ env.GITHUB_REF_SLUG }}"
GIT_COMMIT_MESSAGE: "${{ github.event.head_commit.message }}"
GIT_BRANCH: "${GITHUB_REF##*/}"
IS_SUCCESS: "${{ job.status == 'success' }}"
MATRIX_SERVER: "${{ secrets.MATRIX_SERVER }}"
MATRIX_ROOM: "${{ secrets.MATRIX_ROOM }}"
MATRIX_USER_ID: "${{ secrets.MATRIX_USER_ID }}"
MATRIX_TOKEN: "${{ secrets.MATRIX_TOKEN }}"
MATRIX_DEVICE_ID: "${{ secrets.MATRIX_DEVICE_ID }}"
uses: docker://keybaseio/client:stable-node
with:
args: .github/workflows/support-files/notifications/entry_point.sh
+3 -1
View File
@@ -4,7 +4,9 @@ on:
pull_request:
branches:
- develop
- 'release/**'
- "release/**"
paths-ignore:
- "documentation/**"
types:
- labeled
- unlabeled
@@ -0,0 +1,55 @@
name: Build and upload Data observatory container to harbor.nymte.ch
on:
workflow_dispatch:
env:
WORKING_DIRECTORY: "nym-data-observatory"
CONTAINER_NAME: "data-observatory"
jobs:
build-container:
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: harbor.nymte.ch
username: ${{ secrets.HARBOR_ROBOT_USERNAME }}
password: ${{ secrets.HARBOR_ROBOT_SECRET }}
- name: Checkout repo
uses: actions/checkout@v4
- name: Configure git identity
run: |
git config --global user.email "lawrence@nymtech.net"
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.44.3
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Check if tag exists
run: |
if git rev-parse ${{ steps.get_version.outputs.value }} >/dev/null 2>&1; then
echo "Tag ${{ steps.get_version.outputs.value }} already exists"
fi
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
fi
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
@@ -0,0 +1,61 @@
name: Build and upload Node Status agent container to harbor.nymte.ch
on:
workflow_dispatch:
inputs:
gateway_probe_git_ref:
type: string
description: Which gateway probe git ref to build the image with
env:
WORKING_DIRECTORY: "nym-node-status-agent"
CONTAINER_NAME: "node-status-agent"
jobs:
build-container:
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: harbor.nymte.ch
username: ${{ secrets.HARBOR_ROBOT_USERNAME }}
password: ${{ secrets.HARBOR_ROBOT_SECRET }}
- name: Checkout repo
uses: actions/checkout@v4
- name: Configure git identity
run: |
git config --global user.email "lawrence@nymtech.net"
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.44.3
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: cleanup-gateway-probe-ref
id: cleanup_gateway_probe_ref
run: |
GATEWAY_PROBE_GIT_REF=${{ github.event.inputs.gateway_probe_git_ref }}
GIT_REF_SLUG="${GATEWAY_PROBE_GIT_REF//\//-}"
echo "git_ref=${GIT_REF_SLUG}" >> $GITHUB_OUTPUT
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
fi
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }} -m "Version ${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
- name: BuildAndPushImageOnHarbor
run: |
docker build --build-arg GIT_REF=${{ github.event.inputs.gateway_probe_git_ref }} -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }}-${{ steps.cleanup_gateway_probe_ref.outputs.git_ref }}
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+49 -5
View File
@@ -1,11 +1,55 @@
name: Build and upload Node Status API container to harbor.nymte.ch
on:
workflow_dispatch:
env:
WORKING_DIRECTORY: "nym-node-status-api"
CONTAINER_NAME: "node-status-api"
jobs:
my-job:
runs-on: arc-ubuntu-22.04
build-container:
runs-on: arc-ubuntu-22.04-dind
steps:
- name: my-step
run: echo "Hello World!"
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: harbor.nymte.ch
username: ${{ secrets.HARBOR_ROBOT_USERNAME }}
password: ${{ secrets.HARBOR_ROBOT_SECRET }}
- name: Checkout repo
uses: actions/checkout@v4
- name: Configure git identity
run: |
git config --global user.email "lawrence@nymtech.net"
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.44.3
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Check if tag exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
echo "Tag ${{ steps.get_version.outputs.result }} already exists"
fi
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
fi
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
+55
View File
@@ -0,0 +1,55 @@
name: Build and upload nym node container to harbor.nymte.ch
on:
workflow_dispatch:
env:
WORKING_DIRECTORY: "nym-node"
CONTAINER_NAME: "nym-node"
jobs:
build-container:
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: harbor.nymte.ch
username: ${{ secrets.HARBOR_ROBOT_USERNAME }}
password: ${{ secrets.HARBOR_ROBOT_SECRET }}
- name: Checkout repo
uses: actions/checkout@v4
- name: Configure git identity
run: |
git config --global user.email "lawrence@nymtech.net"
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.44.3
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/Cargo.toml
- name: Check if tag exists
run: |
if git rev-parse ${{ steps.get_version.outputs.value }} >/dev/null 2>&1; then
echo "Tag ${{ steps.get_version.outputs.value }} already exists"
fi
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
fi
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
@@ -0,0 +1,49 @@
name: Build and upload Validator Rewarder container to harbor.nymte.ch
on:
workflow_dispatch:
env:
WORKING_DIRECTORY: "nym-validator-rewarder"
CONTAINER_NAME: "validator-rewarder"
jobs:
build-container:
runs-on: arc-ubuntu-22.04-dind
steps:
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: harbor.nymte.ch
username: ${{ secrets.HARBOR_ROBOT_USERNAME }}
password: ${{ secrets.HARBOR_ROBOT_SECRET }}
- name: Checkout repo
uses: actions/checkout@v4
- name: Configure git identity
run: |
git config --global user.email "lawrence@nymtech.net"
git config --global user.name "Lawrence Stalder"
- name: Get version from cargo.toml
uses: mikefarah/yq@v4.44.3
id: get_version
with:
cmd: yq -oy '.package.version' ${{ env.WORKING_DIRECTORY }}/nym-credential-proxy/Cargo.toml
- name: Remove existing tag if exists
run: |
if git rev-parse ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} >/dev/null 2>&1; then
git push --delete origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
git tag -d ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
fi
- name: Create tag
run: |
git tag -a ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }} -m "Version ${{ steps.get_version.outputs.result }}"
git push origin ${{ env.WORKING_DIRECTORY }}-${{ steps.get_version.outputs.result }}
- name: BuildAndPushImageOnHarbor
run: |
docker build -f ${{ env.WORKING_DIRECTORY }}/Dockerfile . -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:${{ steps.get_version.outputs.result }} -t harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }}:latest
docker push harbor.nymte.ch/nym/${{ env.CONTAINER_NAME }} --all-tags
Generated
+485 -134
View File
File diff suppressed because it is too large Load Diff
+43 -29
View File
@@ -19,33 +19,33 @@ members = [
"clients/native",
"clients/native/websocket-requests",
"clients/socks5",
"common/authenticator-requests",
"common/async-file-watcher",
"common/authenticator-requests",
"common/bandwidth-controller",
"common/bin-common",
"common/client-core",
"common/client-core/config-types",
"common/client-core/surb-storage",
"common/client-core/gateways-storage",
"common/client-core/surb-storage",
"common/client-libs/gateway-client",
"common/client-libs/mixnet-client",
"common/client-libs/validator-client",
"common/commands",
"common/config",
"common/cosmwasm-smart-contracts/coconut-bandwidth-contract",
"common/cosmwasm-smart-contracts/ecash-contract",
"common/cosmwasm-smart-contracts/coconut-dkg",
"common/cosmwasm-smart-contracts/contracts-common",
"common/cosmwasm-smart-contracts/ecash-contract",
"common/cosmwasm-smart-contracts/group-contract",
"common/cosmwasm-smart-contracts/mixnet-contract",
"common/cosmwasm-smart-contracts/multisig-contract",
"common/cosmwasm-smart-contracts/vesting-contract",
"common/country-group",
"common/credential-storage",
"common/credentials",
"common/credential-utils",
"common/credentials-interface",
"common/credential-verification",
"common/credentials",
"common/credentials-interface",
"common/crypto",
"common/dkg",
"common/ecash-double-spending",
@@ -54,19 +54,21 @@ members = [
"common/exit-policy",
"common/gateway-requests",
"common/gateway-storage",
"common/gateway-stats-storage",
"common/http-api-client",
"common/http-api-common",
"common/inclusion-probability",
"common/ip-packet-requests",
"common/ledger",
"common/mixnode-common",
"common/models",
"common/network-defaults",
"common/node-tester-utils",
"common/nonexhaustive-delayqueue",
"common/nymcoconut",
"common/nym_offline_compact_ecash",
"common/nym-id",
"common/nym-metrics",
"common/nym_offline_compact_ecash",
"common/nymcoconut",
"common/nymsphinx",
"common/nymsphinx/acknowledgements",
"common/nymsphinx/addressing",
@@ -96,29 +98,35 @@ members = [
"common/wasm/utils",
"common/wireguard",
"common/wireguard-types",
# "documentation/autodoc",
"explorer-api",
"explorer-api/explorer-api-requests",
"explorer-api/explorer-client",
"gateway",
"integrations/bity",
"mixnode",
"sdk/ffi/cpp",
"sdk/ffi/go",
"sdk/ffi/shared",
"sdk/lib/socks5-listener",
"sdk/rust/nym-sdk",
"sdk/ffi/shared",
"sdk/ffi/go",
"sdk/ffi/cpp",
"service-providers/authenticator",
"service-providers/common",
"service-providers/ip-packet-router",
"service-providers/network-requester",
"nym-network-monitor",
"nym-api",
"nym-browser-extension/storage",
"nym-api/nym-api-requests",
"nym-browser-extension/storage",
"nym-credential-proxy/nym-credential-proxy",
"nym-credential-proxy/nym-credential-proxy-requests",
"nym-credential-proxy/vpn-api-lib-wasm",
"nym-data-observatory",
"nym-network-monitor",
"nym-node",
"nym-node/nym-node-http-api",
"nym-node/nym-node-requests",
"nym-node-status-api",
"nym-node-status-agent",
"nym-outfox",
"nym-validator-rewarder",
"tools/echo-server",
@@ -136,23 +144,27 @@ members = [
"wasm/mix-fetch",
"wasm/node-tester",
"wasm/zknym-lib",
"tools/internal/testnet-manager",
"tools/internal/testnet-manager/dkg-bypass-contract",
"tools/echo-server",
"tools/internal/contract-state-importer/importer-cli",
"tools/internal/contract-state-importer/importer-contract",
"tools/internal/testnet-manager",
"tools/internal/testnet-manager/dkg-bypass-contract",
]
default-members = [
"clients/native",
"clients/socks5",
"common/models",
"explorer-api",
"gateway",
"mixnode",
"nym-api",
"nym-credential-proxy/nym-credential-proxy",
"nym-data-observatory",
"nym-node",
"nym-node-status-api",
"nym-validator-rewarder",
"nym-node-status-api",
"service-providers/authenticator",
"service-providers/ip-packet-router",
"service-providers/network-requester",
@@ -183,7 +195,7 @@ aes = "0.8.1"
aes-gcm = "0.10.1"
aes-gcm-siv = "0.11.1"
aead = "0.5.2"
anyhow = "1.0.89"
anyhow = "1.0.90"
argon2 = "0.5.0"
async-trait = "0.1.83"
axum-client-ip = "0.6.1"
@@ -192,11 +204,8 @@ axum-extra = "0.9.4"
base64 = "0.22.1"
bincode = "1.3.3"
bip39 = { version = "2.0.0", features = ["zeroize"] }
# can we unify those?
bit-vec = "0.7.0"
bit-vec = "0.7.0" # can we unify those?
bitvec = "1.0.0"
blake3 = "1.5.4"
bloomfilter = "1.0.14"
bs58 = "0.5.1"
@@ -209,7 +218,7 @@ chacha20 = "0.9.0"
chacha20poly1305 = "0.10.1"
chrono = "0.4.31"
cipher = "0.4.3"
clap = "4.5.18"
clap = "4.5.20"
clap_complete = "4.5"
clap_complete_fig = "4.5"
colored = "2.0"
@@ -234,10 +243,12 @@ dotenvy = "0.15.6"
ecdsa = "0.16"
ed25519-dalek = "2.1"
etherparse = "0.13.0"
envy = "0.4"
eyre = "0.6.9"
fastrand = "2.1.1"
flate2 = "1.0.34"
futures = "0.3.28"
futures-util = "0.3"
generic-array = "0.14.7"
getrandom = "0.2.10"
getset = "0.1.3"
@@ -261,22 +272,23 @@ ipnetwork = "0.20"
isocountry = "0.3.2"
itertools = "0.13.0"
k256 = "0.13"
lazy_static = "1.4.0"
lazy_static = "1.5.0"
ledger-transport = "0.10.0"
ledger-transport-hid = "0.10.0"
log = "0.4"
maxminddb = "0.23.0"
mime = "0.3.17"
moka = { version = "0.12", features = ["future"] }
nix = "0.27.1"
notify = "5.1.0"
okapi = "0.7.0"
once_cell = "1.7.2"
once_cell = "1.20.2"
opentelemetry = "0.19.0"
opentelemetry-jaeger = "0.18.0"
parking_lot = "0.12.3"
pem = "0.8"
petgraph = "0.6.5"
pin-project = "1.0"
pin-project = "1.1"
pin-project-lite = "0.2.14"
pretty_env_logger = "0.4.0"
publicsuffix = "2.2.3"
@@ -296,10 +308,11 @@ rocket_okapi = "0.8.0"
safer-ffi = "0.1.13"
schemars = "0.8.21"
semver = "1.0.23"
serde = "1.0.210"
serde = "1.0.211"
serde_bytes = "0.11.15"
serde_derive = "1.0"
serde_json = "1.0.128"
serde_json = "1.0.132"
serde_json_path = "0.6.7"
serde_repr = "0.1"
serde_with = "3.9.0"
serde_yaml = "0.9.25"
@@ -308,6 +321,7 @@ si-scale = "0.2.3"
sphinx-packet = "0.1.1"
sqlx = "0.7.4"
strum = "0.26"
strum_macros = "0.26"
subtle-encoding = "0.5"
syn = "1"
sysinfo = "0.30.13"
@@ -329,6 +343,7 @@ tracing = "0.1.37"
tracing-opentelemetry = "0.19.0"
tracing-subscriber = "0.3.16"
tracing-tree = "0.2.2"
tracing-log = "0.2"
ts-rs = "10.0.0"
tungstenite = { version = "0.20.1", default-features = false }
url = "2.5"
@@ -390,11 +405,10 @@ indexed_db_futures = { git = "https://github.com/TiemenSch/rust-indexed-db", bra
js-sys = "0.3.70"
serde-wasm-bindgen = "0.6.5"
tsify = "0.4.5"
wasm-bindgen = "0.2.93"
wasm-bindgen-futures = "0.4.43"
wasm-bindgen = "0.2.95"
wasm-bindgen-futures = "0.4.45"
wasmtimer = "0.2.0"
web-sys = "0.3.70"
web-sys = "0.3.72"
# Profile settings for individual crates
@@ -102,5 +102,10 @@ average_ack_delay = '{{ debug.acknowledgements.average_ack_delay }}'
[debug.cover_traffic]
loop_cover_traffic_average_delay = '{{ debug.cover_traffic.loop_cover_traffic_average_delay }}'
[debug.stats_reporting]
enabled = {{ debug.stats_reporting.enabled }}
provider_address = '{{ debug.stats_reporting.provider_address }}'
reporting_interval = '{{ debug.stats_reporting.reporting_interval }}'
"#;
+1
View File
@@ -81,6 +81,7 @@ impl From<Init> for OverrideConfig {
nyxd_urls: init_config.common_args.nyxd_urls,
enabled_credentials_mode: init_config.common_args.enabled_credentials_mode,
stats_reporting_address: init_config.common_args.stats_reporting_address,
}
}
}
+7
View File
@@ -13,6 +13,7 @@ use clap::{Parser, Subcommand};
use log::{error, info};
use nym_bin_common::bin_info;
use nym_bin_common::completions::{fig_generate, ArgShell};
use nym_client::client::Recipient;
use nym_client_core::cli_helpers::CliClient;
use nym_client_core::client::base_client::storage::migration_helpers::v1_1_33;
use nym_config::OptionalSet;
@@ -104,6 +105,7 @@ pub(crate) struct OverrideConfig {
no_cover: bool,
nyxd_urls: Option<Vec<url::Url>>,
enabled_credentials_mode: Option<bool>,
stats_reporting_address: Option<Recipient>,
}
pub(crate) async fn execute(args: Cli) -> Result<(), Box<dyn Error + Send + Sync>> {
@@ -149,6 +151,11 @@ pub(crate) fn override_config(config: Config, args: OverrideConfig) -> Config {
BaseClientConfig::with_disabled_credentials,
args.enabled_credentials_mode.map(|b| !b),
)
.with_optional_env_ext(
BaseClientConfig::with_enabled_stats_reporting_address,
args.stats_reporting_address,
nym_network_defaults::var_names::CLIENT_STATS_COLLECTION_PROVIDER,
)
}
async fn try_upgrade_v1_1_13_config(id: &str) -> Result<bool, ClientError> {
+1
View File
@@ -43,6 +43,7 @@ impl From<Run> for OverrideConfig {
no_cover: run_config.common_args.no_cover,
nyxd_urls: run_config.common_args.nyxd_urls,
enabled_credentials_mode: run_config.common_args.enabled_credentials_mode,
stats_reporting_address: run_config.common_args.stats_reporting_address,
}
}
}
+1
View File
@@ -92,6 +92,7 @@ impl From<Init> for OverrideConfig {
nyxd_urls: init_config.common_args.nyxd_urls,
enabled_credentials_mode: init_config.common_args.enabled_credentials_mode,
outfox: false,
stats_reporting_address: init_config.common_args.stats_reporting_address,
}
}
}
+7
View File
@@ -19,6 +19,7 @@ use nym_client_core::client::base_client::storage::migration_helpers::v1_1_33;
use nym_client_core::client::topology_control::geo_aware_provider::CountryGroup;
use nym_client_core::config::{GroupBy, TopologyStructure};
use nym_config::OptionalSet;
use nym_sphinx::addressing::Recipient;
use nym_sphinx::params::{PacketSize, PacketType};
use std::error::Error;
use std::net::IpAddr;
@@ -111,6 +112,7 @@ pub(crate) struct OverrideConfig {
nyxd_urls: Option<Vec<url::Url>>,
enabled_credentials_mode: Option<bool>,
outfox: bool,
stats_reporting_address: Option<Recipient>,
}
pub(crate) async fn execute(args: Cli) -> Result<(), Box<dyn Error + Send + Sync>> {
@@ -196,6 +198,11 @@ pub(crate) fn override_config(config: Config, args: OverrideConfig) -> Config {
BaseClientConfig::with_disabled_credentials,
args.enabled_credentials_mode.map(|b| !b),
)
.with_optional_base_env(
BaseClientConfig::with_enabled_stats_reporting_address,
args.stats_reporting_address,
nym_network_defaults::var_names::CLIENT_STATS_COLLECTION_PROVIDER,
)
}
async fn try_upgrade_v1_1_13_config(id: &str) -> Result<bool, Socks5ClientError> {
+1
View File
@@ -70,6 +70,7 @@ impl From<Run> for OverrideConfig {
nyxd_urls: run_config.common_args.nyxd_urls,
enabled_credentials_mode: run_config.common_args.enabled_credentials_mode,
outfox: run_config.outfox,
stats_reporting_address: run_config.common_args.stats_reporting_address,
}
}
}
+5
View File
@@ -108,4 +108,9 @@ average_ack_delay = '{{ core.debug.acknowledgements.average_ack_delay }}'
[core.debug.cover_traffic]
loop_cover_traffic_average_delay = '{{ core.debug.cover_traffic.loop_cover_traffic_average_delay }}'
[core.debug.stats_reporting]
enabled = {{ core.debug.stats_reporting.enabled }}
provider_address = '{{ core.debug.stats_reporting.provider_address }}'
reporting_interval = '{{ core.debug.stats_reporting.reporting_interval }}'
"#;
+1
View File
@@ -17,6 +17,7 @@ thiserror = { workspace = true }
nym-credentials-interface = { path = "../credentials-interface" }
nym-crypto = { path = "../crypto", features = ["asymmetric"] }
nym-network-defaults = { path = "../network-defaults" }
nym-service-provider-requests-common = { path = "../service-provider-requests-common" }
nym-sphinx = { path = "../nymsphinx" }
nym-wireguard-types = { path = "../wireguard-types" }
+3 -2
View File
@@ -4,13 +4,14 @@
pub mod v1;
pub mod v2;
pub mod v3;
pub mod v4;
mod error;
pub use error::Error;
pub use v3 as latest;
pub use v4 as latest;
pub const CURRENT_VERSION: u8 = 3;
pub const CURRENT_VERSION: u8 = 4;
fn make_bincode_serializer() -> impl bincode::Options {
use bincode::Options;
@@ -9,7 +9,7 @@ impl From<v2::request::AuthenticatorRequest> for v3::request::AuthenticatorReque
fn from(authenticator_request: v2::request::AuthenticatorRequest) -> Self {
Self {
protocol: Protocol {
version: 2,
version: 3,
service_provider_type: ServiceProviderType::Authenticator,
},
data: authenticator_request.data.into(),
@@ -0,0 +1,200 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use nym_service_provider_requests_common::{Protocol, ServiceProviderType};
use crate::{v3, v4};
impl From<v3::request::AuthenticatorRequest> for v4::request::AuthenticatorRequest {
fn from(authenticator_request: v3::request::AuthenticatorRequest) -> Self {
Self {
protocol: Protocol {
version: 4,
service_provider_type: ServiceProviderType::Authenticator,
},
data: authenticator_request.data.into(),
reply_to: authenticator_request.reply_to,
request_id: authenticator_request.request_id,
}
}
}
impl From<v3::request::AuthenticatorRequestData> for v4::request::AuthenticatorRequestData {
fn from(authenticator_request_data: v3::request::AuthenticatorRequestData) -> Self {
match authenticator_request_data {
v3::request::AuthenticatorRequestData::Initial(init_msg) => {
v4::request::AuthenticatorRequestData::Initial(init_msg.into())
}
v3::request::AuthenticatorRequestData::Final(gw_client) => {
v4::request::AuthenticatorRequestData::Final(gw_client.into())
}
v3::request::AuthenticatorRequestData::QueryBandwidth(pub_key) => {
v4::request::AuthenticatorRequestData::QueryBandwidth(pub_key)
}
v3::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message) => {
v4::request::AuthenticatorRequestData::TopUpBandwidth(top_up_message.into())
}
}
}
}
impl From<v3::registration::InitMessage> for v4::registration::InitMessage {
fn from(init_msg: v3::registration::InitMessage) -> Self {
Self {
pub_key: init_msg.pub_key,
}
}
}
impl From<Box<v3::registration::FinalMessage>> for Box<v4::registration::FinalMessage> {
fn from(gw_client: Box<v3::registration::FinalMessage>) -> Self {
Box::new(v4::registration::FinalMessage {
gateway_client: gw_client.gateway_client.into(),
credential: gw_client.credential,
})
}
}
impl From<Box<v3::topup::TopUpMessage>> for Box<v4::topup::TopUpMessage> {
fn from(top_up_message: Box<v3::topup::TopUpMessage>) -> Self {
Box::new(v4::topup::TopUpMessage {
pub_key: top_up_message.pub_key,
credential: top_up_message.credential,
})
}
}
impl From<v3::registration::GatewayClient> for v4::registration::GatewayClient {
fn from(gw_client: v3::registration::GatewayClient) -> Self {
Self {
pub_key: gw_client.pub_key,
private_ips: gw_client.private_ip.into(),
mac: gw_client.mac.into(),
}
}
}
impl From<v4::registration::GatewayClient> for v3::registration::GatewayClient {
fn from(gw_client: v4::registration::GatewayClient) -> Self {
Self {
pub_key: gw_client.pub_key,
private_ip: gw_client.private_ips.ipv4.into(),
mac: gw_client.mac.into(),
}
}
}
impl From<v3::registration::ClientMac> for v4::registration::ClientMac {
fn from(mac: v3::registration::ClientMac) -> Self {
Self::new(mac.to_vec())
}
}
impl From<v4::registration::ClientMac> for v3::registration::ClientMac {
fn from(mac: v4::registration::ClientMac) -> Self {
Self::new(mac.to_vec())
}
}
impl TryFrom<v4::response::AuthenticatorResponse> for v3::response::AuthenticatorResponse {
type Error = crate::Error;
fn try_from(
authenticator_response: v4::response::AuthenticatorResponse,
) -> Result<Self, Self::Error> {
Ok(Self {
data: authenticator_response.data.try_into()?,
reply_to: authenticator_response.reply_to,
protocol: authenticator_response.protocol,
})
}
}
impl TryFrom<v4::response::AuthenticatorResponseData> for v3::response::AuthenticatorResponseData {
type Error = crate::Error;
fn try_from(
authenticator_response_data: v4::response::AuthenticatorResponseData,
) -> Result<Self, Self::Error> {
match authenticator_response_data {
v4::response::AuthenticatorResponseData::PendingRegistration(
pending_registration_response,
) => Ok(
v3::response::AuthenticatorResponseData::PendingRegistration(
pending_registration_response.into(),
),
),
v4::response::AuthenticatorResponseData::Registered(registered_response) => Ok(
v3::response::AuthenticatorResponseData::Registered(registered_response.into()),
),
v4::response::AuthenticatorResponseData::RemainingBandwidth(
remaining_bandwidth_response,
) => Ok(v3::response::AuthenticatorResponseData::RemainingBandwidth(
remaining_bandwidth_response.into(),
)),
v4::response::AuthenticatorResponseData::TopUpBandwidth(_) => {
Err(Self::Error::Conversion(
"a v3 request couldn't produce a v4 only type of response".to_string(),
))
}
}
}
}
impl From<v4::response::PendingRegistrationResponse> for v3::response::PendingRegistrationResponse {
fn from(value: v4::response::PendingRegistrationResponse) -> Self {
Self {
request_id: value.request_id,
reply_to: value.reply_to,
reply: value.reply.into(),
}
}
}
impl From<v4::response::RegisteredResponse> for v3::response::RegisteredResponse {
fn from(value: v4::response::RegisteredResponse) -> Self {
Self {
request_id: value.request_id,
reply_to: value.reply_to,
reply: value.reply.into(),
}
}
}
impl From<v4::response::RemainingBandwidthResponse> for v3::response::RemainingBandwidthResponse {
fn from(value: v4::response::RemainingBandwidthResponse) -> Self {
Self {
request_id: value.request_id,
reply_to: value.reply_to,
reply: value.reply.map(Into::into),
}
}
}
impl From<v4::registration::RegistrationData> for v3::registration::RegistrationData {
fn from(value: v4::registration::RegistrationData) -> Self {
Self {
nonce: value.nonce,
gateway_data: value.gateway_data.into(),
wg_port: value.wg_port,
}
}
}
impl From<v4::registration::RegistredData> for v3::registration::RegistredData {
fn from(value: v4::registration::RegistredData) -> Self {
Self {
pub_key: value.pub_key,
private_ip: value.private_ips.ipv4.into(),
wg_port: value.wg_port,
}
}
}
impl From<v4::registration::RemainingBandwidthData> for v3::registration::RemainingBandwidthData {
fn from(value: v4::registration::RemainingBandwidthData) -> Self {
Self {
available_bandwidth: value.available_bandwidth,
}
}
}
@@ -0,0 +1,10 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
pub mod conversion;
pub mod registration;
pub mod request;
pub mod response;
pub mod topup;
pub const VERSION: u8 = 4;
@@ -0,0 +1,281 @@
// -2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::error::Error;
use base64::{engine::general_purpose, Engine};
use nym_credentials_interface::CredentialSpendingData;
use nym_network_defaults::constants::{WG_TUN_DEVICE_IP_ADDRESS_V4, WG_TUN_DEVICE_IP_ADDRESS_V6};
use nym_wireguard_types::PeerPublicKey;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
use std::time::SystemTime;
use std::{fmt, ops::Deref, str::FromStr};
#[cfg(feature = "verify")]
use hmac::{Hmac, Mac};
#[cfg(feature = "verify")]
use nym_crypto::asymmetric::encryption::PrivateKey;
#[cfg(feature = "verify")]
use sha2::Sha256;
pub type PendingRegistrations = HashMap<PeerPublicKey, RegistrationData>;
pub type PrivateIPs = HashMap<IpPair, Taken>;
#[cfg(feature = "verify")]
pub type HmacSha256 = Hmac<Sha256>;
pub type Nonce = u64;
pub type Taken = Option<SystemTime>;
pub const BANDWIDTH_CAP_PER_DAY: u64 = 1024 * 1024 * 1024; // 1 GB
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct IpPair {
pub ipv4: Ipv4Addr,
pub ipv6: Ipv6Addr,
}
impl IpPair {
pub fn new(ipv4: Ipv4Addr, ipv6: Ipv6Addr) -> Self {
IpPair { ipv4, ipv6 }
}
}
impl fmt::Display for IpPair {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "({}, {})", self.ipv4, self.ipv6)
}
}
impl From<IpAddr> for IpPair {
fn from(value: IpAddr) -> Self {
let (before_last_byte, last_byte) = match value {
std::net::IpAddr::V4(ipv4_addr) => (ipv4_addr.octets()[2], ipv4_addr.octets()[3]),
std::net::IpAddr::V6(ipv6_addr) => (ipv6_addr.octets()[14], ipv6_addr.octets()[15]),
};
let last_bytes = (before_last_byte as u16) << 8 | last_byte as u16;
let ipv4 = Ipv4Addr::new(
WG_TUN_DEVICE_IP_ADDRESS_V4.octets()[0],
WG_TUN_DEVICE_IP_ADDRESS_V4.octets()[1],
before_last_byte,
last_byte,
);
let ipv6 = Ipv6Addr::new(
WG_TUN_DEVICE_IP_ADDRESS_V6.segments()[0],
WG_TUN_DEVICE_IP_ADDRESS_V6.segments()[1],
WG_TUN_DEVICE_IP_ADDRESS_V6.segments()[2],
WG_TUN_DEVICE_IP_ADDRESS_V6.segments()[3],
WG_TUN_DEVICE_IP_ADDRESS_V6.segments()[4],
WG_TUN_DEVICE_IP_ADDRESS_V6.segments()[5],
WG_TUN_DEVICE_IP_ADDRESS_V6.segments()[6],
last_bytes,
);
IpPair::new(ipv4, ipv6)
}
}
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct InitMessage {
/// Base64 encoded x25519 public key
pub pub_key: PeerPublicKey,
}
impl InitMessage {
pub fn new(pub_key: PeerPublicKey) -> Self {
InitMessage { pub_key }
}
}
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct FinalMessage {
/// Gateway client data
pub gateway_client: GatewayClient,
/// Ecash credential
pub credential: Option<CredentialSpendingData>,
}
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct RegistrationData {
pub nonce: u64,
pub gateway_data: GatewayClient,
pub wg_port: u16,
}
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct RegistredData {
pub pub_key: PeerPublicKey,
pub private_ips: IpPair,
pub wg_port: u16,
}
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct RemainingBandwidthData {
pub available_bandwidth: i64,
}
/// Client that wants to register sends its PublicKey bytes mac digest encrypted with a DH shared secret.
/// Gateway/Nym node can then verify pub_key payload using the same process
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct GatewayClient {
/// Base64 encoded x25519 public key
pub pub_key: PeerPublicKey,
/// Assigned private IPs (v4 and v6)
pub private_ips: IpPair,
/// Sha256 hmac on the data (alongside the prior nonce)
pub mac: ClientMac,
}
impl GatewayClient {
#[cfg(feature = "verify")]
pub fn new(
local_secret: &PrivateKey,
remote_public: x25519_dalek::PublicKey,
private_ips: IpPair,
nonce: u64,
) -> Self {
// convert from 1.0 x25519-dalek private key into 2.0 x25519-dalek
#[allow(clippy::expect_used)]
let static_secret = x25519_dalek::StaticSecret::from(local_secret.to_bytes());
let local_public: x25519_dalek::PublicKey = (&static_secret).into();
let dh = static_secret.diffie_hellman(&remote_public);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
let mut mac = HmacSha256::new_from_slice(dh.as_bytes())
.expect("x25519 shared secret is always 32 bytes long");
mac.update(local_public.as_bytes());
mac.update(private_ips.to_string().as_bytes());
mac.update(&nonce.to_le_bytes());
GatewayClient {
pub_key: PeerPublicKey::new(local_public),
private_ips,
mac: ClientMac(mac.finalize().into_bytes().to_vec()),
}
}
// Reusable secret should be gateways Wireguard PK
// Client should perform this step when generating its payload, using its own WG PK
#[cfg(feature = "verify")]
pub fn verify(&self, gateway_key: &PrivateKey, nonce: u64) -> Result<(), Error> {
// convert from 1.0 x25519-dalek private key into 2.0 x25519-dalek
#[allow(clippy::expect_used)]
let static_secret = x25519_dalek::StaticSecret::from(gateway_key.to_bytes());
let dh = static_secret.diffie_hellman(&self.pub_key);
// TODO: change that to use our nym_crypto::hmac module instead
#[allow(clippy::expect_used)]
let mut mac = HmacSha256::new_from_slice(dh.as_bytes())
.expect("x25519 shared secret is always 32 bytes long");
mac.update(self.pub_key.as_bytes());
mac.update(self.private_ips.to_string().as_bytes());
mac.update(&nonce.to_le_bytes());
mac.verify_slice(&self.mac)
.map_err(|source| Error::FailedClientMacVerification {
client: self.pub_key.to_string(),
source,
})
}
pub fn pub_key(&self) -> PeerPublicKey {
self.pub_key
}
}
// TODO: change the inner type into generic array of size HmacSha256::OutputSize
// TODO2: rely on our internal crypto/hmac
#[derive(Debug, Clone)]
pub struct ClientMac(Vec<u8>);
impl fmt::Display for ClientMac {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", general_purpose::STANDARD.encode(&self.0))
}
}
impl ClientMac {
#[allow(dead_code)]
pub fn new(mac: Vec<u8>) -> Self {
ClientMac(mac)
}
}
impl Deref for ClientMac {
type Target = Vec<u8>;
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl FromStr for ClientMac {
type Err = Error;
fn from_str(s: &str) -> Result<Self, Self::Err> {
let mac_bytes: Vec<u8> =
general_purpose::STANDARD
.decode(s)
.map_err(|source| Error::MalformedClientMac {
mac: s.to_string(),
source,
})?;
Ok(ClientMac(mac_bytes))
}
}
impl Serialize for ClientMac {
fn serialize<S: serde::Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
let encoded_key = general_purpose::STANDARD.encode(self.0.clone());
serializer.serialize_str(&encoded_key)
}
}
impl<'de> Deserialize<'de> for ClientMac {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
let encoded_key = String::deserialize(deserializer)?;
ClientMac::from_str(&encoded_key).map_err(serde::de::Error::custom)
}
}
#[cfg(test)]
mod tests {
use super::*;
use nym_crypto::asymmetric::encryption;
#[test]
fn create_ip_pair() {
let ipv4: IpAddr = Ipv4Addr::from_str("10.1.10.50").unwrap().into();
let ipv6: IpAddr = Ipv6Addr::from_str("fc01::0a32").unwrap().into();
assert_eq!(IpPair::from(ipv4), IpPair::from(ipv6));
}
#[test]
#[cfg(feature = "verify")]
fn client_request_roundtrip() {
let mut rng = rand::thread_rng();
let gateway_key_pair = encryption::KeyPair::new(&mut rng);
let client_key_pair = encryption::KeyPair::new(&mut rng);
let nonce = 1234567890;
let client = GatewayClient::new(
client_key_pair.private_key(),
x25519_dalek::PublicKey::from(gateway_key_pair.public_key().to_bytes()),
IpPair::new("10.0.0.42".parse().unwrap(), "fc00::42".parse().unwrap()),
nonce,
);
assert!(client.verify(gateway_key_pair.private_key(), nonce).is_ok())
}
}
@@ -0,0 +1,136 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use super::{
registration::{FinalMessage, InitMessage},
topup::TopUpMessage,
};
use nym_service_provider_requests_common::{Protocol, ServiceProviderType};
use nym_sphinx::addressing::Recipient;
use nym_wireguard_types::PeerPublicKey;
use serde::{Deserialize, Serialize};
use crate::make_bincode_serializer;
use super::VERSION;
fn generate_random() -> u64 {
use rand::RngCore;
let mut rng = rand::rngs::OsRng;
rng.next_u64()
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct AuthenticatorRequest {
pub protocol: Protocol,
pub data: AuthenticatorRequestData,
pub reply_to: Recipient,
pub request_id: u64,
}
impl AuthenticatorRequest {
pub fn from_reconstructed_message(
message: &nym_sphinx::receiver::ReconstructedMessage,
) -> Result<Self, bincode::Error> {
use bincode::Options;
make_bincode_serializer().deserialize(&message.message)
}
pub fn new_initial_request(init_message: InitMessage, reply_to: Recipient) -> (Self, u64) {
let request_id = generate_random();
(
Self {
protocol: Protocol {
service_provider_type: ServiceProviderType::Authenticator,
version: VERSION,
},
data: AuthenticatorRequestData::Initial(init_message),
reply_to,
request_id,
},
request_id,
)
}
pub fn new_final_request(final_message: FinalMessage, reply_to: Recipient) -> (Self, u64) {
let request_id = generate_random();
(
Self {
protocol: Protocol {
service_provider_type: ServiceProviderType::Authenticator,
version: VERSION,
},
data: AuthenticatorRequestData::Final(Box::new(final_message)),
reply_to,
request_id,
},
request_id,
)
}
pub fn new_query_request(peer_public_key: PeerPublicKey, reply_to: Recipient) -> (Self, u64) {
let request_id = generate_random();
(
Self {
protocol: Protocol {
service_provider_type: ServiceProviderType::Authenticator,
version: VERSION,
},
data: AuthenticatorRequestData::QueryBandwidth(peer_public_key),
reply_to,
request_id,
},
request_id,
)
}
pub fn new_topup_request(top_up_message: TopUpMessage, reply_to: Recipient) -> (Self, u64) {
let request_id = generate_random();
(
Self {
protocol: Protocol {
service_provider_type: ServiceProviderType::Authenticator,
version: VERSION,
},
data: AuthenticatorRequestData::TopUpBandwidth(Box::new(top_up_message)),
reply_to,
request_id,
},
request_id,
)
}
pub fn to_bytes(&self) -> Result<Vec<u8>, bincode::Error> {
use bincode::Options;
make_bincode_serializer().serialize(self)
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub enum AuthenticatorRequestData {
Initial(InitMessage),
Final(Box<FinalMessage>),
QueryBandwidth(PeerPublicKey),
TopUpBandwidth(Box<TopUpMessage>),
}
#[cfg(test)]
mod tests {
use super::*;
use std::str::FromStr;
#[test]
fn check_first_bytes_protocol() {
let version = 4;
let data = AuthenticatorRequest {
protocol: Protocol { version, service_provider_type: ServiceProviderType::Authenticator },
data: AuthenticatorRequestData::Initial(InitMessage::new(
PeerPublicKey::from_str("yvNUDpT5l7W/xDhiu6HkqTHDQwbs/B3J5UrLmORl1EQ=").unwrap(),
)),
reply_to: Recipient::try_from_base58_string("D1rrpsysCGCYXy9saP8y3kmNpGtJZUXN9SvFoUcqAsM9.9Ssso1ea5NfkbMASdiseDSjTN1fSWda5SgEVjdSN4CvV@GJqd3ZxpXWSNxTfx7B1pPtswpetH4LnJdFeLeuY5KUuN").unwrap(),
request_id: 1,
};
let bytes = *data.to_bytes().unwrap().first_chunk::<2>().unwrap();
assert_eq!(bytes, [version, ServiceProviderType::Authenticator as u8]);
}
}
@@ -0,0 +1,157 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use super::registration::{RegistrationData, RegistredData, RemainingBandwidthData};
use nym_service_provider_requests_common::{Protocol, ServiceProviderType};
use nym_sphinx::addressing::Recipient;
use serde::{Deserialize, Serialize};
use crate::make_bincode_serializer;
use super::VERSION;
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct AuthenticatorResponse {
pub protocol: Protocol,
pub data: AuthenticatorResponseData,
pub reply_to: Recipient,
}
impl AuthenticatorResponse {
pub fn new_pending_registration_success(
registration_data: RegistrationData,
request_id: u64,
reply_to: Recipient,
) -> Self {
Self {
protocol: Protocol {
service_provider_type: ServiceProviderType::Authenticator,
version: VERSION,
},
data: AuthenticatorResponseData::PendingRegistration(PendingRegistrationResponse {
reply: registration_data,
reply_to,
request_id,
}),
reply_to,
}
}
pub fn new_registered(
registred_data: RegistredData,
reply_to: Recipient,
request_id: u64,
) -> Self {
Self {
protocol: Protocol {
service_provider_type: ServiceProviderType::Authenticator,
version: VERSION,
},
data: AuthenticatorResponseData::Registered(RegisteredResponse {
reply: registred_data,
reply_to,
request_id,
}),
reply_to,
}
}
pub fn new_remaining_bandwidth(
remaining_bandwidth_data: Option<RemainingBandwidthData>,
reply_to: Recipient,
request_id: u64,
) -> Self {
Self {
protocol: Protocol {
service_provider_type: ServiceProviderType::Authenticator,
version: VERSION,
},
data: AuthenticatorResponseData::RemainingBandwidth(RemainingBandwidthResponse {
reply: remaining_bandwidth_data,
reply_to,
request_id,
}),
reply_to,
}
}
pub fn new_topup_bandwidth(
remaining_bandwidth_data: RemainingBandwidthData,
reply_to: Recipient,
request_id: u64,
) -> Self {
Self {
protocol: Protocol {
service_provider_type: ServiceProviderType::Authenticator,
version: VERSION,
},
data: AuthenticatorResponseData::TopUpBandwidth(TopUpBandwidthResponse {
reply: remaining_bandwidth_data,
reply_to,
request_id,
}),
reply_to,
}
}
pub fn recipient(&self) -> Recipient {
self.reply_to
}
pub fn to_bytes(&self) -> Result<Vec<u8>, bincode::Error> {
use bincode::Options;
make_bincode_serializer().serialize(self)
}
pub fn from_reconstructed_message(
message: &nym_sphinx::receiver::ReconstructedMessage,
) -> Result<Self, bincode::Error> {
use bincode::Options;
make_bincode_serializer().deserialize(&message.message)
}
pub fn id(&self) -> Option<u64> {
match &self.data {
AuthenticatorResponseData::PendingRegistration(response) => Some(response.request_id),
AuthenticatorResponseData::Registered(response) => Some(response.request_id),
AuthenticatorResponseData::RemainingBandwidth(response) => Some(response.request_id),
AuthenticatorResponseData::TopUpBandwidth(response) => Some(response.request_id),
}
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub enum AuthenticatorResponseData {
PendingRegistration(PendingRegistrationResponse),
Registered(RegisteredResponse),
RemainingBandwidth(RemainingBandwidthResponse),
TopUpBandwidth(TopUpBandwidthResponse),
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct PendingRegistrationResponse {
pub request_id: u64,
pub reply_to: Recipient,
pub reply: RegistrationData,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct RegisteredResponse {
pub request_id: u64,
pub reply_to: Recipient,
pub reply: RegistredData,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct RemainingBandwidthResponse {
pub request_id: u64,
pub reply_to: Recipient,
pub reply: Option<RemainingBandwidthData>,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct TopUpBandwidthResponse {
pub request_id: u64,
pub reply_to: Recipient,
pub reply: RemainingBandwidthData,
}
@@ -0,0 +1,15 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use nym_credentials_interface::CredentialSpendingData;
use nym_wireguard_types::PeerPublicKey;
use serde::{Deserialize, Serialize};
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct TopUpMessage {
/// Base64 encoded x25519 public key
pub pub_key: PeerPublicKey,
/// Ecash credential
pub credential: CredentialSpendingData,
}
+4 -3
View File
@@ -14,14 +14,15 @@ thiserror = { workspace = true }
url = { workspace = true }
zeroize = { workspace = true }
nym-ecash-time = { path = "../ecash-time" }
nym-credential-storage = { path = "../credential-storage" }
nym-credentials = { path = "../credentials" }
nym-credentials-interface = { path = "../credentials-interface" }
nym-crypto = { path = "../crypto", features = ["rand", "asymmetric", "stream_cipher", "aes", "hashing"] }
nym-network-defaults = { path = "../network-defaults" }
nym-validator-client = { path = "../client-libs/validator-client", default-features = false }
nym-ecash-contract-common = { path = "../cosmwasm-smart-contracts/ecash-contract" }
nym-ecash-time = { path = "../ecash-time" }
nym-network-defaults = { path = "../network-defaults" }
nym-task = { path = "../task" }
nym-validator-client = { path = "../client-libs/validator-client", default-features = false }
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.nym-validator-client]
path = "../client-libs/validator-client"
+18 -6
View File
@@ -1,13 +1,25 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
// See other comments for other TaskStatus message enumds about abusing the Error trait when we
// should have a new trait for TaskStatus messages
#[derive(Debug, thiserror::Error)]
#[derive(Debug)]
pub enum BandwidthStatusMessage {
#[error("remaining bandwidth: {0}")]
RemainingBandwidth(i64),
#[error("no bandwidth left")]
NoBandwidth,
}
impl std::fmt::Display for BandwidthStatusMessage {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
BandwidthStatusMessage::RemainingBandwidth(b) => {
write!(f, "remaining bandwidth: {}", b)
}
BandwidthStatusMessage::NoBandwidth => write!(f, "no bandwidth left"),
}
}
}
impl nym_task::TaskStatusEvent for BandwidthStatusMessage {
fn as_any(&self) -> &dyn std::any::Any {
self
}
}
+1
View File
@@ -45,3 +45,4 @@ tracing = [
"opentelemetry",
]
clap = [ "dep:clap", "dep:clap_complete", "dep:clap_complete_fig" ]
models = []
+1
View File
@@ -43,6 +43,7 @@ nym-gateway-requests = { path = "../gateway-requests" }
nym-metrics = { path = "../nym-metrics" }
nym-nonexhaustive-delayqueue = { path = "../nonexhaustive-delayqueue" }
nym-sphinx = { path = "../nymsphinx" }
nym-statistics-common = { path = "../statistics" }
nym-pemstore = { path = "../pemstore" }
nym-topology = { path = "../topology", features = ["serializable"] }
nym-validator-client = { path = "../client-libs/validator-client", default-features = false }
+2 -1
View File
@@ -9,6 +9,7 @@ license.workspace = true
[dependencies]
humantime-serde = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_with = { workspace = true, features = ["macros"] }
thiserror.workspace = true
url = { workspace = true, features = ["serde"] }
@@ -23,4 +24,4 @@ nym-sphinx-addressing = { path = "../../nymsphinx/addressing" }
[features]
disk-persistence = ["nym-pemstore"]
disk-persistence = ["nym-pemstore"]
@@ -5,6 +5,7 @@ use nym_config::defaults::NymNetworkDetails;
use nym_sphinx_addressing::Recipient;
use nym_sphinx_params::{PacketSize, PacketType};
use serde::{Deserialize, Serialize};
use serde_with::{serde_as, DisplayFromStr};
use std::time::Duration;
use url::Url;
@@ -61,6 +62,11 @@ const DEFAULT_MAXIMUM_REPLY_SURB_AGE: Duration = Duration::from_secs(12 * 60 * 6
// 24 hours
const DEFAULT_MAXIMUM_REPLY_KEY_AGE: Duration = Duration::from_secs(24 * 60 * 60);
// stats reporting related
/// Time interval between reporting statistics to the given provider if it exist
const STATS_REPORT_INTERVAL_SECS: Duration = Duration::from_secs(300);
use crate::error::InvalidTrafficModeFailure;
pub use nym_country_group::CountryGroup;
@@ -133,6 +139,12 @@ impl Config {
self
}
pub fn with_enabled_stats_reporting_address(mut self, address: Recipient) -> Self {
self.debug.stats_reporting.provider_address = Some(address);
self.debug.stats_reporting.enabled = true; //since we are overriding the address, we assume the reporting should be enabled
self
}
// TODO: this should be refactored properly
// as of 12.09.23 the below is true (not sure how this comment will rot in the future)
// medium_toggle:
@@ -631,6 +643,32 @@ impl Default for ReplySurbs {
}
}
#[serde_as]
#[derive(Debug, Clone, Copy, Deserialize, PartialEq, Serialize)]
#[serde(default, deny_unknown_fields)]
pub struct StatsReporting {
/// Is stats reporting enabled
pub enabled: bool,
/// Address of the stats collector. If this is none, no reporting will happen, regardless of `enabled`
#[serde_as(as = "Option<DisplayFromStr>")]
pub provider_address: Option<Recipient>,
/// With what frequence will statistics be sent
#[serde(with = "humantime_serde")]
pub reporting_interval: Duration,
}
impl Default for StatsReporting {
fn default() -> Self {
StatsReporting {
enabled: true,
provider_address: None,
reporting_interval: STATS_REPORT_INTERVAL_SECS,
}
}
}
#[derive(Debug, Clone, Copy, Deserialize, PartialEq, Serialize)]
#[serde(default, deny_unknown_fields)]
pub struct DebugConfig {
@@ -651,6 +689,9 @@ pub struct DebugConfig {
/// Defines all configuration options related to reply SURBs.
pub reply_surbs: ReplySurbs,
/// Defines all configuration options related to stats reporting.
pub stats_reporting: StatsReporting,
}
impl DebugConfig {
@@ -672,6 +713,7 @@ impl Default for DebugConfig {
acknowledgements: Default::default(),
topology: Default::default(),
reply_surbs: Default::default(),
stats_reporting: Default::default(),
}
}
}
@@ -181,6 +181,7 @@ impl From<ConfigV5> for Config {
maximum_reply_key_age: value.debug.reply_surbs.maximum_reply_key_age,
surb_mix_hops: value.debug.reply_surbs.surb_mix_hops,
},
stats_reporting: Default::default(),
},
}
}
@@ -15,6 +15,7 @@ use crate::{
use log::info;
use nym_client_core_gateways_storage::GatewayDetails;
use nym_crypto::asymmetric::identity;
use nym_sphinx::addressing::Recipient;
use nym_topology::NymTopology;
use nym_validator_client::UserAgent;
use rand::rngs::OsRng;
@@ -88,6 +89,10 @@ pub struct CommonClientInitArgs {
/// Disable loop cover traffic and the Poisson rate limiter (for debugging only)
#[cfg_attr(feature = "cli", clap(long, hide = true))]
pub no_cover: bool,
/// Sets the address to report statistics
#[cfg_attr(feature = "cli", clap(long, hide = true))]
pub stats_reporting_address: Option<Recipient>,
}
pub struct InitResultsWithConfig<T> {
@@ -2,6 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use nym_crypto::asymmetric::identity;
use nym_sphinx::addressing::Recipient;
use std::path::PathBuf;
#[cfg_attr(feature = "cli", derive(clap::Args))]
@@ -56,4 +57,8 @@ pub struct CommonClientRunArgs {
// has defined the conflict on that field itself
#[cfg_attr(feature = "cli", clap(long, hide = true))]
pub no_cover: bool,
/// Sets the address to report statistics
#[cfg_attr(feature = "cli", clap(long, hide = true))]
pub stats_reporting_address: Option<Recipient>,
}
@@ -1,8 +1,8 @@
// Copyright 2022-2023 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use super::packet_statistics_control::PacketStatisticsReporter;
use super::received_buffer::ReceivedBufferMessage;
use super::statistics_control::StatisticsControl;
use super::topology_control::geo_aware_provider::GeoAwareTopologyProvider;
use crate::client::base_client::storage::helpers::store_client_keys;
use crate::client::base_client::storage::MixnetClientStorage;
@@ -12,7 +12,6 @@ use crate::client::key_manager::persistence::KeyStore;
use crate::client::key_manager::ClientKeys;
use crate::client::mix_traffic::transceiver::{GatewayReceiver, GatewayTransceiver, RemoteGateway};
use crate::client::mix_traffic::{BatchMixMessageSender, MixTrafficController};
use crate::client::packet_statistics_control::PacketStatisticsControl;
use crate::client::real_messages_control;
use crate::client::real_messages_control::RealMessagesController;
use crate::client::received_buffer::{
@@ -49,16 +48,19 @@ use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::addressing::nodes::NodeIdentity;
use nym_sphinx::params::PacketType;
use nym_sphinx::receiver::{ReconstructedMessage, SphinxMessageReceiver};
use nym_statistics_common::clients::ClientStatsSender;
use nym_task::connections::{ConnectionCommandReceiver, ConnectionCommandSender, LaneQueueLengths};
use nym_task::{TaskClient, TaskHandle};
use nym_topology::provider_trait::TopologyProvider;
use nym_topology::HardcodedTopologyProvider;
use nym_validator_client::{nyxd::contract_traits::DkgQueryClient, UserAgent};
use rand::rngs::OsRng;
use sha2::Digest;
use std::fmt::Debug;
use std::os::raw::c_int as RawFd;
use std::path::Path;
use std::sync::Arc;
use tokio::sync::mpsc::Sender;
use url::Url;
#[cfg(all(
@@ -273,7 +275,7 @@ where
self_address: Recipient,
topology_accessor: TopologyAccessor,
mix_tx: BatchMixMessageSender,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
shutdown: TaskClient,
) {
info!("Starting loop cover traffic stream...");
@@ -306,7 +308,7 @@ where
client_connection_rx: ConnectionCommandReceiver,
shutdown: TaskClient,
packet_type: PacketType,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
) {
info!("Starting real traffic stream...");
@@ -335,7 +337,7 @@ where
reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender,
shutdown: TaskClient,
packet_statistics_control: PacketStatisticsReporter,
metrics_reporter: ClientStatsSender,
) {
info!("Starting received messages buffer controller...");
let controller: ReceivedMessagesBufferController<SphinxMessageReceiver> =
@@ -345,7 +347,7 @@ where
mixnet_receiver,
reply_key_storage,
reply_controller_sender,
packet_statistics_control,
metrics_reporter,
);
controller.start_with_shutdown(shutdown)
}
@@ -586,11 +588,23 @@ where
Ok(())
}
fn start_packet_statistics_control(shutdown: TaskClient) -> PacketStatisticsReporter {
info!("Starting packet statistics control...");
let (packet_statistics_control, packet_stats_reporter) = PacketStatisticsControl::new();
packet_statistics_control.start_with_shutdown(shutdown);
packet_stats_reporter
fn start_statistics_control(
config: &Config,
user_agent: Option<UserAgent>,
client_stats_id: String,
input_sender: Sender<InputMessage>,
shutdown: TaskClient,
) -> ClientStatsSender {
info!("Starting statistics control...");
StatisticsControl::create_and_start_with_shutdown(
config.debug.stats_reporting,
user_agent
.map(|u| u.application)
.unwrap_or("unknown".to_string()),
client_stats_id,
input_sender.clone(),
shutdown.with_suffix("controller"),
)
}
fn start_mix_traffic_controller(
@@ -720,6 +734,19 @@ where
self.user_agent.clone(),
);
//make sure we don't accidentally get the same id as gateways are reporting
let client_stats_id = format!(
"stats_id_{:x}",
sha2::Sha256::digest(self_address.identity().to_bytes())
);
let stats_reporter = Self::start_statistics_control(
self.config,
self.user_agent.clone(),
client_stats_id,
input_sender.clone(),
shutdown.fork("statistics_control"),
);
// needs to be started as the first thing to block if required waiting for the gateway
Self::start_topology_refresher(
topology_provider,
@@ -731,9 +758,6 @@ where
)
.await?;
let packet_stats_reporter =
Self::start_packet_statistics_control(shutdown.fork("packet_statistics_control"));
let gateway_packet_router = PacketRouter::new(
ack_sender,
mixnet_messages_sender,
@@ -765,7 +789,7 @@ where
reply_storage.key_storage(),
reply_controller_sender.clone(),
shutdown.fork("received_messages_buffer"),
packet_stats_reporter.clone(),
stats_reporter.clone(),
);
// The message_sender is the transmitter for any component generating sphinx packets
@@ -804,7 +828,7 @@ where
client_connection_rx,
shutdown.fork("real_traffic_controller"),
self.config.debug.traffic.packet_type,
packet_stats_reporter.clone(),
stats_reporter.clone(),
);
if !self
@@ -819,7 +843,7 @@ where
self_address,
shared_topology_accessor.clone(),
message_sender,
packet_stats_reporter,
stats_reporter.clone(),
shutdown.fork("cover_traffic_stream"),
);
}
@@ -847,6 +871,7 @@ where
topology_accessor: shared_topology_accessor,
gateway_connection: GatewayConnection { gateway_ws_fd },
},
stats_reporter,
task_handle: shutdown,
})
}
@@ -858,6 +883,7 @@ pub struct BaseClient {
pub client_input: ClientInputStatus,
pub client_output: ClientOutputStatus,
pub client_state: ClientState,
pub stats_reporter: ClientStatsSender,
pub task_handle: TaskHandle,
}
@@ -2,7 +2,6 @@
// SPDX-License-Identifier: Apache-2.0
use crate::client::mix_traffic::BatchMixMessageSender;
use crate::client::packet_statistics_control::{PacketStatisticsEvent, PacketStatisticsReporter};
use crate::client::topology_control::TopologyAccessor;
use crate::{config, spawn_future};
use futures::task::{Context, Poll};
@@ -13,6 +12,7 @@ use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::cover::generate_loop_cover_packet;
use nym_sphinx::params::{PacketSize, PacketType};
use nym_sphinx::utils::sample_poisson_duration;
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use rand::{rngs::OsRng, CryptoRng, Rng};
use std::pin::Pin;
use std::sync::Arc;
@@ -63,7 +63,7 @@ where
packet_type: PacketType,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
}
impl<R> Stream for LoopCoverTrafficStream<R>
@@ -109,7 +109,7 @@ impl LoopCoverTrafficStream<OsRng> {
topology_access: TopologyAccessor,
traffic_config: config::Traffic,
cover_config: config::CoverTraffic,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
) -> Self {
let rng = OsRng;
@@ -198,9 +198,9 @@ impl LoopCoverTrafficStream<OsRng> {
}
}
} else {
self.stats_tx.report(PacketStatisticsEvent::CoverPacketSent(
cover_traffic_packet_size.size(),
));
self.stats_tx.report(
PacketStatisticsEvent::CoverPacketSent(cover_traffic_packet_size.size()).into(),
);
}
// TODO: I'm not entirely sure whether this is really required, because I'm not 100%
+1 -1
View File
@@ -7,9 +7,9 @@ pub(crate) mod helpers;
pub mod inbound_messages;
pub mod key_manager;
pub mod mix_traffic;
pub(crate) mod packet_statistics_control;
pub mod real_messages_control;
pub mod received_buffer;
pub mod replies;
pub mod statistics_control;
pub mod topology_control;
pub(crate) mod transmission_buffer;
@@ -1,9 +1,9 @@
// Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::packet_statistics_control::{PacketStatisticsEvent, PacketStatisticsReporter};
use super::action_controller::{AckActionSender, Action};
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use futures::StreamExt;
use log::*;
use nym_gateway_client::AcknowledgementReceiver;
@@ -19,7 +19,7 @@ pub(super) struct AcknowledgementListener {
ack_key: Arc<AckKey>,
ack_receiver: AcknowledgementReceiver,
action_sender: AckActionSender,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
}
impl AcknowledgementListener {
@@ -27,7 +27,7 @@ impl AcknowledgementListener {
ack_key: Arc<AckKey>,
ack_receiver: AcknowledgementReceiver,
action_sender: AckActionSender,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
) -> Self {
AcknowledgementListener {
ack_key,
@@ -40,7 +40,7 @@ impl AcknowledgementListener {
async fn on_ack(&mut self, ack_content: Vec<u8>) {
trace!("Received an ack");
self.stats_tx
.report(PacketStatisticsEvent::AckReceived(ack_content.len()));
.report(PacketStatisticsEvent::AckReceived(ack_content.len()).into());
let frag_id = match recover_identifier(&self.ack_key, &ack_content)
.map(FragmentIdentifier::try_from_bytes)
@@ -57,13 +57,13 @@ impl AcknowledgementListener {
if frag_id == COVER_FRAG_ID {
trace!("Received an ack for a cover message - no need to do anything");
self.stats_tx
.report(PacketStatisticsEvent::CoverAckReceived(ack_content.len()));
.report(PacketStatisticsEvent::CoverAckReceived(ack_content.len()).into());
return;
}
trace!("Received {} from the mix network", frag_id);
self.stats_tx
.report(PacketStatisticsEvent::RealAckReceived(ack_content.len()));
.report(PacketStatisticsEvent::RealAckReceived(ack_content.len()).into());
self.action_sender
.unbounded_send(Action::new_remove(frag_id))
.unwrap();
@@ -8,7 +8,6 @@ use self::{
sent_notification_listener::SentNotificationListener,
};
use crate::client::inbound_messages::InputMessageReceiver;
use crate::client::packet_statistics_control::PacketStatisticsReporter;
use crate::client::real_messages_control::message_handler::MessageHandler;
use crate::client::replies::reply_controller::ReplyControllerSender;
use crate::spawn_future;
@@ -24,6 +23,7 @@ use nym_sphinx::{
chunking::fragment::{Fragment, FragmentIdentifier},
Delay as SphinxDelay,
};
use nym_statistics_common::clients::ClientStatsSender;
use rand::{CryptoRng, Rng};
use std::{
sync::{Arc, Weak},
@@ -209,7 +209,7 @@ where
connectors: AcknowledgementControllerConnectors,
message_handler: MessageHandler<R>,
reply_controller_sender: ReplyControllerSender,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
) -> Self {
let (retransmission_tx, retransmission_rx) = mpsc::unbounded();
@@ -35,7 +35,7 @@ use crate::client::replies::reply_controller;
use crate::config;
pub(crate) use acknowledgement_control::{AckActionSender, Action};
use super::packet_statistics_control::PacketStatisticsReporter;
use nym_statistics_common::clients::ClientStatsSender;
pub(crate) mod acknowledgement_control;
pub(crate) mod message_handler;
@@ -145,7 +145,7 @@ impl RealMessagesController<OsRng> {
reply_controller_receiver: ReplyControllerReceiver,
lane_queue_lengths: LaneQueueLengths,
client_connection_rx: ConnectionCommandReceiver,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
) -> Self {
let rng = OsRng;
@@ -3,7 +3,6 @@
use self::sending_delay_controller::SendingDelayController;
use crate::client::mix_traffic::BatchMixMessageSender;
use crate::client::packet_statistics_control::{PacketStatisticsEvent, PacketStatisticsReporter};
use crate::client::real_messages_control::acknowledgement_control::SentPacketNotificationSender;
use crate::client::topology_control::TopologyAccessor;
use crate::client::transmission_buffer::TransmissionBuffer;
@@ -19,6 +18,7 @@ use nym_sphinx::forwarding::packet::MixPacket;
use nym_sphinx::params::PacketSize;
use nym_sphinx::preparer::PreparedFragment;
use nym_sphinx::utils::sample_poisson_duration;
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use nym_task::connections::{
ConnectionCommand, ConnectionCommandReceiver, ConnectionId, LaneQueueLengths, TransmissionLane,
};
@@ -115,8 +115,8 @@ where
/// Report queue lengths so that upstream can backoff sending data, and keep connections open.
lane_queue_lengths: LaneQueueLengths,
/// Channel used for sending statistics events to `PacketStatisticsControl`.
stats_tx: PacketStatisticsReporter,
/// Channel used for sending metrics events (specifically `PacketStatistics` events) to the metrics tracker.
stats_tx: ClientStatsSender,
}
#[derive(Debug)]
@@ -175,7 +175,7 @@ where
topology_access: TopologyAccessor,
lane_queue_lengths: LaneQueueLengths,
client_connection_rx: ConnectionCommandReceiver,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
) -> Self {
OutQueueControl {
config,
@@ -277,7 +277,7 @@ where
} else {
PacketStatisticsEvent::CoverPacketSent(packet_size)
};
self.stats_tx.report(event);
self.stats_tx.report(event.into());
}
// notify ack controller about sending our message only after we actually managed to push it
@@ -373,13 +373,13 @@ where
TransmissionLane::Retransmission => Some(PacketStatisticsEvent::RetransmissionQueued),
};
if let Some(stat_event) = stat_event {
self.stats_tx.report(stat_event);
self.stats_tx.report(stat_event.into());
}
// To avoid comparing apples to oranges when presenting the fraction of packets that are
// retransmissions, we also need to keep track to the total number of real messages queued,
// even though we also track the actual number of messages sent later in the pipeline.
self.stats_tx
.report(PacketStatisticsEvent::RealPacketQueued);
.report(PacketStatisticsEvent::RealPacketQueued.into());
Some(real_next)
}
@@ -1,9 +1,8 @@
// Copyright 2021 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::client::{
packet_statistics_control::{PacketStatisticsEvent, PacketStatisticsReporter},
replies::{reply_controller::ReplyControllerSender, reply_storage::SentReplyKeys},
use crate::client::replies::{
reply_controller::ReplyControllerSender, reply_storage::SentReplyKeys,
};
use crate::spawn_future;
use futures::channel::mpsc;
@@ -20,6 +19,7 @@ use nym_sphinx::anonymous_replies::{encryption_key::EncryptionKeyDigest, SurbEnc
use nym_sphinx::message::{NymMessage, PlainMessage};
use nym_sphinx::params::ReplySurbKeyDigestAlgorithm;
use nym_sphinx::receiver::{MessageReceiver, MessageRecoveryError, ReconstructedMessage};
use nym_statistics_common::clients::{packet_statistics::PacketStatisticsEvent, ClientStatsSender};
use std::collections::HashSet;
use std::sync::Arc;
@@ -46,7 +46,7 @@ struct ReceivedMessagesBufferInner<R: MessageReceiver> {
// and every now and then remove ids older than X
recently_reconstructed: HashSet<i32>,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
}
impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
@@ -61,16 +61,12 @@ impl<R: MessageReceiver> ReceivedMessagesBufferInner<R> {
// received and sent packets due to the sphinx layers being removed by the exit gateway
// before it reaches the mixnet client.
self.stats_tx
.report(PacketStatisticsEvent::CoverPacketReceived(
fragment_data_size,
));
.report(PacketStatisticsEvent::CoverPacketReceived(fragment_data_size).into());
return None;
}
self.stats_tx
.report(PacketStatisticsEvent::RealPacketReceived(
fragment_data_size,
));
.report(PacketStatisticsEvent::RealPacketReceived(fragment_data_size).into());
let fragment = match self.message_receiver.recover_fragment(fragment_data) {
Err(err) => {
@@ -163,7 +159,7 @@ impl<R: MessageReceiver> ReceivedMessagesBuffer<R> {
local_encryption_keypair: Arc<encryption::KeyPair>,
reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender,
stats_tx: PacketStatisticsReporter,
stats_tx: ClientStatsSender,
) -> Self {
ReceivedMessagesBuffer {
inner: Arc::new(Mutex::new(ReceivedMessagesBufferInner {
@@ -504,13 +500,13 @@ impl<R: MessageReceiver + Clone + Send + 'static> ReceivedMessagesBufferControll
mixnet_packet_receiver: MixnetMessageReceiver,
reply_key_storage: SentReplyKeys,
reply_controller_sender: ReplyControllerSender,
packet_statistics_reporter: PacketStatisticsReporter,
metrics_reporter: ClientStatsSender,
) -> Self {
let received_buffer = ReceivedMessagesBuffer::new(
local_encryption_keypair,
reply_key_storage,
reply_controller_sender,
packet_statistics_reporter,
metrics_reporter,
);
ReceivedMessagesBufferController {
@@ -0,0 +1,151 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
//! # Statistics collection and reporting.
//!
//! Modular metrics collection and reporting system. submodules can be added to collect different types of metrics.
//! On creation the Statistics controller will start a task that will listen for incoming stats events and
//! multiplex them out to the appropriate metrics module based on type.
//!
//! Adding A new module you need to write a new module that implements the `StatsObj` trait and add it to
//! the `stats` hashmap in the `StatisticsControl` struct during it's initialization in the `new` function in
//! this file.
#![warn(clippy::expect_used)]
#![warn(clippy::unwrap_used)]
#![warn(clippy::todo)]
#![warn(clippy::dbg_macro)]
use std::time::Duration;
use nym_client_core_config_types::StatsReporting;
use nym_sphinx::addressing::Recipient;
use nym_statistics_common::clients::{
ClientStatsController, ClientStatsReceiver, ClientStatsSender,
};
use nym_task::connections::TransmissionLane;
use crate::{
client::inbound_messages::{InputMessage, InputMessageSender},
spawn_future,
};
/// Time interval between reporting statistics locally (logging/task_client)
const LOCAL_REPORT_INTERVAL: Duration = Duration::from_secs(2);
/// Interval for taking snapshots of the statistics
const SNAPSHOT_INTERVAL: Duration = Duration::from_millis(500);
/// Launches and manages metrics collection and reporting.
///
/// This is designed to be generic to allow for multiple types of metrics to be collected and
/// reported.
pub(crate) struct StatisticsControl {
/// Keep store the different types of metrics collectors
stats: ClientStatsController,
/// Incoming packet stats events from other tasks
stats_rx: ClientStatsReceiver,
/// Channel to send stats report through the mixnet
report_tx: InputMessageSender,
/// Config for stats reporting (enabled, address, interval)
reporting_config: StatsReporting,
}
impl StatisticsControl {
pub(crate) fn create(
reporting_config: StatsReporting,
client_type: String,
client_stats_id: String,
report_tx: InputMessageSender,
) -> (Self, ClientStatsSender) {
let (stats_tx, stats_rx) = tokio::sync::mpsc::unbounded_channel();
let stats = ClientStatsController::new(client_stats_id, client_type);
(
StatisticsControl {
stats,
stats_rx,
report_tx,
reporting_config,
},
ClientStatsSender::new(Some(stats_tx)),
)
}
async fn report_stats(&mut self, recipient: Recipient) {
let stats_report = self.stats.build_report();
let report_message = InputMessage::new_regular(
recipient,
stats_report.into(),
TransmissionLane::General,
None,
);
if let Err(err) = self.report_tx.send(report_message).await {
log::error!("Failed to report client stats: {:?}", err);
} else {
self.stats.reset();
}
}
async fn run_with_shutdown(&mut self, mut task_client: nym_task::TaskClient) {
log::debug!("Started StatisticsControl with graceful shutdown support");
let mut stats_report_interval =
tokio::time::interval(self.reporting_config.reporting_interval);
let mut local_report_interval = tokio::time::interval(LOCAL_REPORT_INTERVAL);
let mut snapshot_interval = tokio::time::interval(SNAPSHOT_INTERVAL);
loop {
tokio::select! {
stats_event = self.stats_rx.recv() => match stats_event {
Some(stats_event) => self.stats.handle_event(stats_event),
None => {
log::trace!("StatisticsControl: shutting down due to closed stats channel");
break;
}
},
_ = snapshot_interval.tick() => {
self.stats.snapshot();
}
_ = stats_report_interval.tick(), if self.reporting_config.enabled && self.reporting_config.provider_address.is_some() => {
// SAFTEY : this branch executes only if reporting is not none, so unwrapp is fine
#[allow(clippy::unwrap_used)]
self.report_stats(self.reporting_config.provider_address.unwrap()).await;
}
_ = local_report_interval.tick() => {
self.stats.local_report(&mut task_client);
}
_ = task_client.recv_with_delay() => {
log::trace!("StatisticsControl: Received shutdown");
break;
},
}
}
task_client.recv_timeout().await;
log::debug!("StatisticsControl: Exiting");
}
pub(crate) fn start_with_shutdown(mut self, task_client: nym_task::TaskClient) {
spawn_future(async move {
self.run_with_shutdown(task_client).await;
})
}
pub(crate) fn create_and_start_with_shutdown(
reporting_config: StatsReporting,
client_type: String,
client_stats_id: String,
report_tx: InputMessageSender,
task_client: nym_task::TaskClient,
) -> ClientStatsSender {
let (controller, sender) =
Self::create(reporting_config, client_type, client_stats_id, report_tx);
controller.start_with_shutdown(task_client);
sender
}
}
+22 -5
View File
@@ -212,12 +212,29 @@ pub enum ClientCoreError {
}
/// Set of messages that the client can send to listeners via the task manager
#[derive(thiserror::Error, Debug)]
#[derive(Debug)]
pub enum ClientCoreStatusMessage {
// NOTE: The nym-connect frontend listens for these strings, so don't change them until we have a more robust mechanism in place
#[error("The connected gateway is slow, or the connection to it is slow")]
GatewayIsSlow,
// NOTE: The nym-connect frontend listens for these strings, so don't change them until we have a more robust mechanism in place
#[error("The connected gateway is very slow, or the connection to it is very slow")]
GatewayIsVerySlow,
}
impl std::fmt::Display for ClientCoreStatusMessage {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
ClientCoreStatusMessage::GatewayIsSlow => write!(
f,
"The connected gateway is slow, or the connection to it is slow"
),
ClientCoreStatusMessage::GatewayIsVerySlow => write!(
f,
"The connected gateway is very slow, or the connection to it is very slow"
),
}
}
}
impl nym_task::TaskStatusEvent for ClientCoreStatusMessage {
fn as_any(&self) -> &dyn std::any::Any {
self
}
}
@@ -25,7 +25,7 @@ serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
nym-http-api-client = { path = "../../../common/http-api-client" }
thiserror = { workspace = true }
log = { workspace = true }
tracing = { workspace = true }
url = { workspace = true, features = ["serde"] }
tokio = { workspace = true, features = ["sync", "time"] }
time = { workspace = true, features = ["formatting"] }
@@ -18,7 +18,7 @@ use nym_api_requests::ecash::{
PartialExpirationDateSignatureResponse, VerificationKeyResponse,
};
use nym_api_requests::models::{
GatewayCoreStatusResponse, MixnodeCoreStatusResponse, MixnodeStatusResponse,
ApiHealthResponse, GatewayCoreStatusResponse, MixnodeCoreStatusResponse, MixnodeStatusResponse,
NymNodeDescription, RewardEstimationResponse, StakeSaturationResponse,
};
use nym_api_requests::models::{LegacyDescribedGateway, MixNodeBondAnnotated};
@@ -192,6 +192,8 @@ impl<C, S> Client<C, S> {
}
// validator-api wrappers
// we have to allow the use of deprecated method here as they're calling the deprecated trait methods
#[allow(deprecated)]
impl<C, S> Client<C, S> {
pub fn api_url(&self) -> &Url {
self.nym_api.current_url()
@@ -201,46 +203,54 @@ impl<C, S> Client<C, S> {
self.nym_api.change_base_url(new_endpoint)
}
#[deprecated]
pub async fn get_cached_mixnodes(&self) -> Result<Vec<MixNodeDetails>, ValidatorClientError> {
Ok(self.nym_api.get_mixnodes().await?)
}
#[deprecated]
pub async fn get_cached_mixnodes_detailed(
&self,
) -> Result<Vec<MixNodeBondAnnotated>, ValidatorClientError> {
Ok(self.nym_api.get_mixnodes_detailed().await?)
}
#[deprecated]
pub async fn get_cached_mixnodes_detailed_unfiltered(
&self,
) -> Result<Vec<MixNodeBondAnnotated>, ValidatorClientError> {
Ok(self.nym_api.get_mixnodes_detailed_unfiltered().await?)
}
#[deprecated]
pub async fn get_cached_rewarded_mixnodes(
&self,
) -> Result<Vec<MixNodeDetails>, ValidatorClientError> {
Ok(self.nym_api.get_rewarded_mixnodes().await?)
}
#[deprecated]
pub async fn get_cached_rewarded_mixnodes_detailed(
&self,
) -> Result<Vec<MixNodeBondAnnotated>, ValidatorClientError> {
Ok(self.nym_api.get_rewarded_mixnodes_detailed().await?)
}
#[deprecated]
pub async fn get_cached_active_mixnodes(
&self,
) -> Result<Vec<MixNodeDetails>, ValidatorClientError> {
Ok(self.nym_api.get_active_mixnodes().await?)
}
#[deprecated]
pub async fn get_cached_active_mixnodes_detailed(
&self,
) -> Result<Vec<MixNodeBondAnnotated>, ValidatorClientError> {
Ok(self.nym_api.get_active_mixnodes_detailed().await?)
}
#[deprecated]
pub async fn get_cached_gateways(&self) -> Result<Vec<GatewayBond>, ValidatorClientError> {
Ok(self.nym_api.get_gateways().await?)
}
@@ -304,6 +314,8 @@ pub struct NymApiClient {
// we could re-implement the communication with the REST API on port 1317
}
// we have to allow the use of deprecated method here as they're calling the deprecated trait methods
#[allow(deprecated)]
impl NymApiClient {
pub fn new(api_url: Url) -> Self {
let nym_api = nym_api::Client::new(api_url, None);
@@ -311,6 +323,13 @@ impl NymApiClient {
NymApiClient { nym_api }
}
#[cfg(not(target_arch = "wasm32"))]
pub fn new_with_timeout(api_url: Url, timeout: std::time::Duration) -> Self {
let nym_api = nym_api::Client::new(api_url, Some(timeout));
NymApiClient { nym_api }
}
pub fn new_with_user_agent(api_url: Url, user_agent: UserAgent) -> Self {
let nym_api = nym_api::Client::builder::<_, ValidatorClientError>(api_url)
.expect("invalid api url")
@@ -417,6 +436,38 @@ impl NymApiClient {
Ok(nodes)
}
/// retrieve basic information for nodes are capable of operating as a mixnode
/// this includes legacy mixnodes and nym-nodes
pub async fn get_all_basic_mixing_capable_nodes(
&self,
semver_compatibility: Option<String>,
) -> Result<Vec<SkimmedNode>, ValidatorClientError> {
// TODO: deal with paging in macro or some helper function or something, because it's the same pattern everywhere
let mut page = 0;
let mut nodes = Vec::new();
loop {
let mut res = self
.nym_api
.get_basic_mixing_capable_nodes(
semver_compatibility.clone(),
false,
Some(page),
None,
)
.await?;
nodes.append(&mut res.nodes.data);
if nodes.len() < res.nodes.pagination.total {
page += 1
} else {
break;
}
}
Ok(nodes)
}
/// retrieve basic information for all bonded nodes on the network
pub async fn get_all_basic_nodes(
&self,
@@ -443,26 +494,35 @@ impl NymApiClient {
Ok(nodes)
}
pub async fn health(&self) -> Result<ApiHealthResponse, ValidatorClientError> {
Ok(self.nym_api.health().await?)
}
#[deprecated]
pub async fn get_cached_active_mixnodes(
&self,
) -> Result<Vec<MixNodeDetails>, ValidatorClientError> {
Ok(self.nym_api.get_active_mixnodes().await?)
}
#[deprecated]
pub async fn get_cached_rewarded_mixnodes(
&self,
) -> Result<Vec<MixNodeDetails>, ValidatorClientError> {
Ok(self.nym_api.get_rewarded_mixnodes().await?)
}
#[deprecated]
pub async fn get_cached_mixnodes(&self) -> Result<Vec<MixNodeDetails>, ValidatorClientError> {
Ok(self.nym_api.get_mixnodes().await?)
}
#[deprecated]
pub async fn get_cached_gateways(&self) -> Result<Vec<GatewayBond>, ValidatorClientError> {
Ok(self.nym_api.get_gateways().await?)
}
#[deprecated]
pub async fn get_cached_described_gateways(
&self,
) -> Result<Vec<LegacyDescribedGateway>, ValidatorClientError> {
@@ -511,6 +571,7 @@ impl NymApiClient {
Ok(bonds)
}
#[deprecated]
pub async fn get_gateway_core_status_count(
&self,
identity: IdentityKeyRef<'_>,
@@ -522,6 +583,7 @@ impl NymApiClient {
.await?)
}
#[deprecated]
pub async fn get_mixnode_core_status_count(
&self,
mix_id: NodeId,
@@ -533,6 +595,7 @@ impl NymApiClient {
.await?)
}
#[deprecated]
pub async fn get_mixnode_status(
&self,
mix_id: NodeId,
@@ -540,6 +603,7 @@ impl NymApiClient {
Ok(self.nym_api.get_mixnode_status(mix_id).await?)
}
#[deprecated]
pub async fn get_mixnode_reward_estimation(
&self,
mix_id: NodeId,
@@ -547,6 +611,7 @@ impl NymApiClient {
Ok(self.nym_api.get_mixnode_reward_estimation(mix_id).await?)
}
#[deprecated]
pub async fn get_mixnode_stake_saturation(
&self,
mix_id: NodeId,
@@ -578,6 +643,7 @@ impl NymApiClient {
.await?)
}
#[deprecated]
pub async fn spent_credentials_filter(
&self,
) -> Result<SpentCredentialsResponse, ValidatorClientError> {
@@ -121,36 +121,36 @@ async fn test_nyxd_connection(
{
Ok(Err(NyxdError::TendermintErrorRpc(e))) => {
// If we get a tendermint-rpc error, we classify the node as not contactable
log::warn!("Checking: nyxd url: {url}: {}: {}", "failed".red(), e);
tracing::warn!("Checking: nyxd url: {url}: {}: {}", "failed".red(), e);
false
}
Ok(Err(NyxdError::AbciError { code, log, .. })) => {
// We accept the mixnet contract not found as ok from a connection standpoint. This happens
// for example on a pre-launch network.
log::debug!(
tracing::debug!(
"Checking: nyxd url: {url}: {}, but with abci error: {code}: {log}",
"success".green()
);
code == 18
}
Ok(Err(error @ NyxdError::NoContractAddressAvailable(_))) => {
log::warn!("Checking: nyxd url: {url}: {}: {error}", "failed".red());
tracing::warn!("Checking: nyxd url: {url}: {}: {error}", "failed".red());
false
}
Ok(Err(e)) => {
// For any other error, we're optimistic and just try anyway.
log::warn!(
tracing::warn!(
"Checking: nyxd_url: {url}: {}, but with error: {e}",
"success".green()
);
true
}
Ok(Ok(_)) => {
log::debug!("Checking: nyxd_url: {url}: {}", "success".green());
tracing::debug!("Checking: nyxd_url: {url}: {}", "success".green());
true
}
Err(e) => {
log::warn!("Checking: nyxd_url: {url}: {}: {e}", "failed".red());
tracing::warn!("Checking: nyxd_url: {url}: {}: {e}", "failed".red());
false
}
};
@@ -164,20 +164,20 @@ async fn test_nym_api_connection(
) -> ConnectionResult {
let result = match timeout(
Duration::from_secs(CONNECTION_TEST_TIMEOUT_SEC),
client.get_cached_mixnodes(),
client.health(),
)
.await
{
Ok(Ok(_)) => {
log::debug!("Checking: api_url: {url}: {}", "success".green());
tracing::debug!("Checking: api_url: {url}: {}", "success".green());
true
}
Ok(Err(e)) => {
log::debug!("Checking: api_url: {url}: {}: {e}", "failed".red());
tracing::debug!("Checking: api_url: {url}: {}: {e}", "failed".red());
false
}
Err(e) => {
log::debug!("Checking: api_url: {url}: {}: {e}", "failed".red());
tracing::debug!("Checking: api_url: {url}: {}: {e}", "failed".red());
false
}
};
@@ -11,7 +11,8 @@ use nym_api_requests::ecash::models::{
};
use nym_api_requests::ecash::VerificationKeyResponse;
use nym_api_requests::models::{
AnnotationResponse, LegacyDescribedMixNode, NodePerformanceResponse, NymNodeDescription,
AnnotationResponse, ApiHealthResponse, LegacyDescribedMixNode, NodePerformanceResponse,
NymNodeDescription,
};
use nym_api_requests::nym_nodes::PaginatedCachedNodesResponse;
use nym_api_requests::pagination::PaginatedResponse;
@@ -42,6 +43,7 @@ use nym_mixnet_contract_common::mixnode::MixNodeDetails;
use nym_mixnet_contract_common::{GatewayBond, IdentityKeyRef, NodeId, NymNodeDetails};
use time::format_description::BorrowedFormatItem;
use time::Date;
use tracing::instrument;
pub mod error;
pub mod routes;
@@ -53,11 +55,27 @@ pub fn rfc_3339_date() -> Vec<BorrowedFormatItem<'static>> {
#[cfg_attr(target_arch = "wasm32", async_trait(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
pub trait NymApiClientExt: ApiClient {
async fn health(&self) -> Result<ApiHealthResponse, NymAPIError> {
self.get_json(
&[
routes::API_VERSION,
routes::API_STATUS_ROUTES,
routes::HEALTH,
],
NO_PARAMS,
)
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnodes(&self) -> Result<Vec<MixNodeDetails>, NymAPIError> {
self.get_json(&[routes::API_VERSION, routes::MIXNODES], NO_PARAMS)
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnodes_detailed(&self) -> Result<Vec<MixNodeBondAnnotated>, NymAPIError> {
self.get_json(
&[
@@ -71,6 +89,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_gateways_detailed(&self) -> Result<Vec<GatewayBondAnnotated>, NymAPIError> {
self.get_json(
&[
@@ -84,6 +104,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnodes_detailed_unfiltered(
&self,
) -> Result<Vec<MixNodeBondAnnotated>, NymAPIError> {
@@ -99,11 +121,15 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_gateways(&self) -> Result<Vec<GatewayBond>, NymAPIError> {
self.get_json(&[routes::API_VERSION, routes::GATEWAYS], NO_PARAMS)
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_gateways_described(&self) -> Result<Vec<LegacyDescribedGateway>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::GATEWAYS, routes::DESCRIBED],
@@ -112,6 +138,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnodes_described(&self) -> Result<Vec<LegacyDescribedMixNode>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::MIXNODES, routes::DESCRIBED],
@@ -120,6 +148,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[tracing::instrument(level = "debug", skip_all)]
async fn get_nodes_described(
&self,
page: Option<u32>,
@@ -139,6 +168,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[tracing::instrument(level = "debug", skip_all)]
async fn get_nym_nodes(
&self,
page: Option<u32>,
@@ -158,6 +188,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[tracing::instrument(level = "debug", skip_all)]
async fn get_basic_mixnodes(
&self,
semver_compatibility: Option<String>,
@@ -181,6 +213,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_basic_gateways(
&self,
semver_compatibility: Option<String>,
@@ -206,6 +240,7 @@ pub trait NymApiClientExt: ApiClient {
/// retrieve basic information for nodes are capable of operating as an entry gateway
/// this includes legacy gateways and nym-nodes
#[instrument(level = "debug", skip(self))]
async fn get_basic_entry_assigned_nodes(
&self,
semver_compatibility: Option<String>,
@@ -247,6 +282,7 @@ pub trait NymApiClientExt: ApiClient {
/// retrieve basic information for nodes that got assigned 'mixing' node in this epoch
/// this includes legacy mixnodes and nym-nodes
#[instrument(level = "debug", skip(self))]
async fn get_basic_active_mixing_assigned_nodes(
&self,
semver_compatibility: Option<String>,
@@ -286,6 +322,49 @@ pub trait NymApiClientExt: ApiClient {
.await
}
/// retrieve basic information for nodes that got assigned 'mixing' node in this epoch
/// this includes legacy mixnodes and nym-nodes
#[instrument(level = "debug", skip(self))]
async fn get_basic_mixing_capable_nodes(
&self,
semver_compatibility: Option<String>,
no_legacy: bool,
page: Option<u32>,
per_page: Option<u32>,
) -> Result<PaginatedCachedNodesResponse<SkimmedNode>, NymAPIError> {
let mut params = Vec::new();
if let Some(arg) = &semver_compatibility {
params.push(("semver_compatibility", arg.clone()))
}
if no_legacy {
params.push(("no_legacy", "true".to_string()))
}
if let Some(page) = page {
params.push(("page", page.to_string()))
}
if let Some(per_page) = per_page {
params.push(("per_page", per_page.to_string()))
}
self.get_json(
&[
routes::API_VERSION,
"unstable",
"nym-nodes",
"skimmed",
"mixnodes",
"all",
],
&params,
)
.await
}
#[instrument(level = "debug", skip(self))]
async fn get_basic_nodes(
&self,
semver_compatibility: Option<String>,
@@ -318,6 +397,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_active_mixnodes(&self) -> Result<Vec<MixNodeDetails>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::MIXNODES, routes::ACTIVE],
@@ -326,6 +407,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_active_mixnodes_detailed(&self) -> Result<Vec<MixNodeBondAnnotated>, NymAPIError> {
self.get_json(
&[
@@ -340,6 +423,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_rewarded_mixnodes(&self) -> Result<Vec<MixNodeDetails>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::MIXNODES, routes::REWARDED],
@@ -348,6 +433,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnode_report(
&self,
mix_id: NodeId,
@@ -365,6 +452,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_gateway_report(
&self,
identity: IdentityKeyRef<'_>,
@@ -382,6 +471,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnode_history(
&self,
mix_id: NodeId,
@@ -399,6 +490,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_gateway_history(
&self,
identity: IdentityKeyRef<'_>,
@@ -416,6 +509,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_rewarded_mixnodes_detailed(
&self,
) -> Result<Vec<MixNodeBondAnnotated>, NymAPIError> {
@@ -432,6 +527,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_gateway_core_status_count(
&self,
identity: IdentityKeyRef<'_>,
@@ -463,6 +560,8 @@ pub trait NymApiClientExt: ApiClient {
}
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnode_core_status_count(
&self,
mix_id: NodeId,
@@ -495,6 +594,8 @@ pub trait NymApiClientExt: ApiClient {
}
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnode_status(
&self,
mix_id: NodeId,
@@ -512,6 +613,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnode_reward_estimation(
&self,
mix_id: NodeId,
@@ -529,6 +632,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn compute_mixnode_reward_estimation(
&self,
mix_id: NodeId,
@@ -548,6 +653,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnode_stake_saturation(
&self,
mix_id: NodeId,
@@ -565,6 +672,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnode_inclusion_probability(
&self,
mix_id: NodeId,
@@ -582,6 +691,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self))]
async fn get_current_node_performance(
&self,
node_id: NodeId,
@@ -598,6 +708,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
async fn get_mixnode_avg_uptime(&self, mix_id: NodeId) -> Result<UptimeResponse, NymAPIError> {
self.get_json(
&[
@@ -612,6 +723,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_mixnodes_blacklisted(&self) -> Result<Vec<NodeId>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::MIXNODES, routes::BLACKLISTED],
@@ -620,6 +733,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn get_gateways_blacklisted(&self) -> Result<Vec<IdentityKey>, NymAPIError> {
self.get_json(
&[routes::API_VERSION, routes::GATEWAYS, routes::BLACKLISTED],
@@ -628,6 +743,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self, request_body))]
async fn blind_sign(
&self,
request_body: &BlindSignRequestBody,
@@ -644,6 +760,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self, request_body))]
async fn verify_ecash_ticket(
&self,
request_body: &VerifyEcashTicketBody,
@@ -660,6 +777,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self, request_body))]
async fn batch_redeem_ecash_tickets(
&self,
request_body: &BatchRedeemTicketsBody,
@@ -676,6 +794,8 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[deprecated]
#[instrument(level = "debug", skip(self))]
async fn double_spending_filter_v1(&self) -> Result<SpentCredentialsResponse, NymAPIError> {
self.get_json(
&[
@@ -688,6 +808,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self))]
async fn partial_expiration_date_signatures(
&self,
expiration_date: Option<Date>,
@@ -711,6 +832,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self))]
async fn partial_coin_indices_signatures(
&self,
epoch_id: Option<EpochId>,
@@ -731,6 +853,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self))]
async fn global_expiration_date_signatures(
&self,
expiration_date: Option<Date>,
@@ -754,6 +877,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self))]
async fn global_coin_indices_signatures(
&self,
epoch_id: Option<EpochId>,
@@ -774,6 +898,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self))]
async fn master_verification_key(
&self,
epoch_id: Option<EpochId>,
@@ -793,6 +918,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self))]
async fn epoch_credentials(
&self,
dkg_epoch: EpochId,
@@ -809,6 +935,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self))]
async fn issued_credential(
&self,
credential_id: i64,
@@ -825,6 +952,7 @@ pub trait NymApiClientExt: ApiClient {
.await
}
#[instrument(level = "debug", skip(self))]
async fn issued_credentials(
&self,
credential_ids: Vec<i64>,
@@ -36,6 +36,8 @@ pub mod ecash {
}
pub const STATUS_ROUTES: &str = "status";
pub const API_STATUS_ROUTES: &str = "api-status";
pub const HEALTH: &str = "health";
pub const MIXNODE: &str = "mixnode";
pub const GATEWAY: &str = "gateway";
pub const NYM_NODES: &str = "nym-nodes";
@@ -8,9 +8,9 @@ use crate::nyxd::CosmWasmClient;
use async_trait::async_trait;
use cosmrs::AccountId;
use cosmwasm_std::Addr;
use log::trace;
use nym_coconut_dkg_common::types::{ChunkIndex, NodeIndex, StateAdvanceResponse};
use serde::Deserialize;
use tracing::trace;
use nym_coconut_dkg_common::dealer::RegisteredDealerDetails;
pub use nym_coconut_dkg_common::{
@@ -29,7 +29,6 @@ use cosmrs::proto::cosmwasm::wasm::v1::{
};
use cosmrs::tendermint::{block, chain, Hash};
use cosmrs::{AccountId, Coin as CosmosCoin, Tx};
use log::trace;
use prost::Message;
use serde::{Deserialize, Serialize};
@@ -68,7 +67,7 @@ pub trait CosmWasmClient: TendermintRpcClient {
Res: Message + Default,
{
if let Some(ref abci_path) = path {
trace!("performing query on abci path {abci_path}")
tracing::trace!("performing query on abci path {abci_path}")
}
let mut buf = Vec::with_capacity(req.encoded_len());
req.encode(&mut buf)?;
@@ -297,7 +296,7 @@ pub trait CosmWasmClient: TendermintRpcClient {
let start = Instant::now();
loop {
log::debug!(
tracing::debug!(
"Polling for result of including {} in a block...",
broadcasted.hash
);
@@ -522,7 +521,7 @@ pub trait CosmWasmClient: TendermintRpcClient {
.make_abci_query::<_, QuerySmartContractStateResponse>(path, req)
.await?;
trace!("raw query response: {}", String::from_utf8_lossy(&res.data));
tracing::trace!("raw query response: {}", String::from_utf8_lossy(&res.data));
Ok(serde_json::from_slice(&res.data)?)
}
@@ -25,12 +25,12 @@ use cosmrs::proto::cosmos::tx::signing::v1beta1::SignMode;
use cosmrs::staking::{MsgDelegate, MsgUndelegate};
use cosmrs::tx::{self, Msg};
use cosmrs::{cosmwasm, AccountId, Any, Tx};
use log::debug;
use serde::Serialize;
use sha2::Digest;
use sha2::Sha256;
use std::time::SystemTime;
use tendermint_rpc::endpoint::broadcast;
use tracing::debug;
fn empty_fee() -> tx::Fee {
tx::Fee {
@@ -7,9 +7,9 @@ use base64::Engine;
use cosmrs::abci::TxMsgData;
use cosmrs::cosmwasm::MsgExecuteContractResponse;
use cosmrs::proto::cosmos::base::query::v1beta1::{PageRequest, PageResponse};
use log::error;
use prost::bytes::Bytes;
use tendermint_rpc::endpoint::broadcast;
use tracing::error;
pub use cosmrs::abci::MsgResponse;
@@ -2,10 +2,9 @@
// SPDX-License-Identifier: Apache-2.0
use crate::context::QueryClientWithNyxd;
use crate::utils::{pretty_cosmwasm_coin, show_error};
use crate::utils::show_error;
use clap::Parser;
use comfy_table::Table;
use nym_validator_client::client::NymApiClientExt;
#[derive(Debug, Parser)]
pub struct Args {
@@ -15,12 +14,11 @@ pub struct Args {
}
pub async fn query(args: Args, client: &QueryClientWithNyxd) {
match client.nym_api.get_gateways().await {
match client.get_all_cached_described_nodes().await {
Ok(res) => match args.identity_key {
Some(identity_key) => {
let node = res.iter().find(|node| {
node.gateway
.identity_key
node.ed25519_identity_key()
.to_string()
.eq_ignore_ascii_case(&identity_key)
});
@@ -32,14 +30,16 @@ pub async fn query(args: Args, client: &QueryClientWithNyxd) {
None => {
let mut table = Table::new();
table.set_header(vec!["Identity Key", "Owner", "Host", "Bond", "Version"]);
for node in res {
table.set_header(vec!["Node Id", "Identity Key", "Version", "Is Legacy"]);
for node in res
.into_iter()
.filter(|node| node.description.declared_role.entry)
{
table.add_row(vec![
node.gateway.identity_key.to_string(),
node.owner.to_string(),
node.gateway.host.to_string(),
pretty_cosmwasm_coin(&node.pledge_amount),
node.gateway.version.clone(),
node.node_id.to_string(),
node.ed25519_identity_key().to_base58_string(),
node.description.build_information.build_version,
(!node.contract_node_type.is_nym_node()).to_string(),
]);
}
@@ -2,10 +2,9 @@
// SPDX-License-Identifier: Apache-2.0
use crate::context::QueryClientWithNyxd;
use crate::utils::{pretty_decimal_with_denom, show_error};
use crate::utils::show_error;
use clap::Parser;
use comfy_table::Table;
use nym_validator_client::client::NymApiClientExt;
#[derive(Debug, Parser)]
pub struct Args {
@@ -15,13 +14,11 @@ pub struct Args {
}
pub async fn query(args: Args, client: &QueryClientWithNyxd) {
match client.nym_api.get_mixnodes().await {
match client.get_all_cached_described_nodes().await {
Ok(res) => match args.identity_key {
Some(identity_key) => {
let node = res.iter().find(|node| {
node.bond_information
.mix_node
.identity_key
node.ed25519_identity_key()
.to_string()
.eq_ignore_ascii_case(&identity_key)
});
@@ -33,25 +30,16 @@ pub async fn query(args: Args, client: &QueryClientWithNyxd) {
None => {
let mut table = Table::new();
table.set_header(vec![
"Mix id",
"Identity Key",
"Owner",
"Host",
"Bond",
"Total Delegations",
"Version",
]);
for node in res {
let denom = &node.bond_information.original_pledge().denom;
table.set_header(vec!["Node Id", "Identity Key", "Version", "Is Legacy"]);
for node in res
.into_iter()
.filter(|node| node.description.declared_role.mixnode)
{
table.add_row(vec![
node.mix_id().to_string(),
node.bond_information.mix_node.identity_key.clone(),
node.bond_information.owner.clone().into_string(),
node.bond_information.mix_node.host.clone(),
pretty_decimal_with_denom(node.rewarding_details.operator, denom),
pretty_decimal_with_denom(node.rewarding_details.delegates, denom),
node.bond_information.mix_node.version,
node.node_id.to_string(),
node.ed25519_identity_key().to_base58_string(),
node.description.build_information.build_version,
(!node.contract_node_type.is_nym_node()).to_string(),
]);
}
@@ -21,7 +21,7 @@ pub mod error;
mod helpers;
mod state;
const TIME_RANGE_SEC: i64 = 30;
pub const TIME_RANGE_SEC: i64 = 30;
pub struct EcashManager<S> {
shared_state: SharedState<S>,
+2 -2
View File
@@ -6,7 +6,7 @@ use std::sync::Arc;
use time::{Date, OffsetDateTime};
use tracing::*;
use nym_credentials::ecash::utils::{ecash_today, EcashTime};
use nym_credentials::ecash::utils::{cred_exp_date, ecash_today, EcashTime};
use nym_credentials_interface::{Bandwidth, ClientTicket, TicketType};
use nym_gateway_requests::models::CredentialSpendingRequest;
use nym_gateway_storage::Storage;
@@ -131,7 +131,7 @@ impl<S: Storage + Clone + 'static> CredentialVerifier<S> {
let bandwidth = Bandwidth::ticket_amount(credential_type.into());
self.bandwidth_storage_manager
.increase_bandwidth(bandwidth, spend_date)
.increase_bandwidth(bandwidth, cred_exp_date())
.await?;
Ok(self
+34
View File
@@ -0,0 +1,34 @@
[package]
name = "nym-gateway-stats-storage"
version = "0.1.0"
authors.workspace = true
repository.workspace = true
homepage.workspace = true
documentation.workspace = true
edition.workspace = true
license.workspace = true
[dependencies]
sqlx = { workspace = true, features = [
"runtime-tokio-rustls",
"sqlite",
"macros",
"migrate",
"time",
] }
time = { workspace = true }
thiserror = { workspace = true }
tracing = { workspace = true }
nym-sphinx = { path = "../nymsphinx" }
nym-credentials-interface = { path = "../credentials-interface" }
[build-dependencies]
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
sqlx = { workspace = true, features = [
"runtime-tokio-rustls",
"sqlite",
"macros",
"migrate",
] }
+28
View File
@@ -0,0 +1,28 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use sqlx::{Connection, SqliteConnection};
use std::env;
#[tokio::main]
async fn main() {
let out_dir = env::var("OUT_DIR").unwrap();
let database_path = format!("{}/gateway-stats-example.sqlite", out_dir);
let mut conn = SqliteConnection::connect(&format!("sqlite://{}?mode=rwc", database_path))
.await
.expect("Failed to create SQLx database connection");
sqlx::migrate!("./migrations")
.run(&mut conn)
.await
.expect("Failed to perform SQLx migrations");
#[cfg(target_family = "unix")]
println!("cargo:rustc-env=DATABASE_URL=sqlite://{}", &database_path);
#[cfg(target_family = "windows")]
// for some strange reason we need to add a leading `/` to the windows path even though it's
// not a valid windows path... but hey, it works...
println!("cargo:rustc-env=DATABASE_URL=sqlite:///{}", &database_path);
}
@@ -0,0 +1,26 @@
/*
* Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
* SPDX-License-Identifier: GPL-3.0-only
*/
CREATE TABLE sessions_active
(
client_address TEXT NOT NULL PRIMARY KEY UNIQUE,
start_time TIMESTAMP WITHOUT TIME ZONE NOT NULL,
typ TEXT NOT NULL
);
CREATE TABLE sessions_finished
(
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
day DATE NOT NULL,
duration_ms INTEGER NOT NULL,
typ TEXT NOT NULL
);
CREATE TABLE sessions_unique_users
(
day DATE NOT NULL,
client_address TEXT NOT NULL,
PRIMARY KEY (day, client_address)
);
+13
View File
@@ -0,0 +1,13 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use thiserror::Error;
#[derive(Error, Debug)]
pub enum StatsStorageError {
#[error("Database experienced an internal error: {0}")]
InternalDatabaseError(#[from] sqlx::Error),
#[error("Failed to perform database migration: {0}")]
MigrationError(#[from] sqlx::migrate::MigrateError),
}
+195
View File
@@ -0,0 +1,195 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use error::StatsStorageError;
use models::{ActiveSession, FinishedSession, SessionType, StoredFinishedSession};
use nym_sphinx::DestinationAddressBytes;
use sessions::SessionManager;
use sqlx::ConnectOptions;
use std::path::Path;
use time::Date;
use tracing::{debug, error};
pub mod error;
pub mod models;
mod sessions;
// note that clone here is fine as upon cloning the same underlying pool will be used
#[derive(Clone)]
pub struct PersistentStatsStorage {
session_manager: SessionManager,
}
impl PersistentStatsStorage {
/// Initialises `PersistentStatsStorage` using the provided path.
///
/// # Arguments
///
/// * `database_path`: path to the database.
pub async fn init<P: AsRef<Path> + Send>(database_path: P) -> Result<Self, StatsStorageError> {
debug!(
"Attempting to connect to database {:?}",
database_path.as_ref().as_os_str()
);
// TODO: we can inject here more stuff based on our gateway global config
// struct. Maybe different pool size or timeout intervals?
let opts = sqlx::sqlite::SqliteConnectOptions::new()
.filename(database_path)
.create_if_missing(true)
.disable_statement_logging();
// TODO: do we want auto_vacuum ?
let connection_pool = match sqlx::SqlitePool::connect_with(opts).await {
Ok(db) => db,
Err(err) => {
error!("Failed to connect to SQLx database: {err}");
return Err(err.into());
}
};
if let Err(err) = sqlx::migrate!("./migrations").run(&connection_pool).await {
error!("Failed to perform migration on the SQLx database: {err}");
return Err(err.into());
}
// the cloning here are cheap as connection pool is stored behind an Arc
Ok(PersistentStatsStorage {
session_manager: sessions::SessionManager::new(connection_pool),
})
}
//Sessions fn
pub async fn insert_finished_session(
&self,
date: Date,
session: FinishedSession,
) -> Result<(), StatsStorageError> {
Ok(self
.session_manager
.insert_finished_session(
date,
session.duration.whole_milliseconds() as i64,
session.typ.to_string().into(),
)
.await?)
}
pub async fn get_finished_sessions(
&self,
date: Date,
) -> Result<Vec<StoredFinishedSession>, StatsStorageError> {
Ok(self.session_manager.get_finished_sessions(date).await?)
}
pub async fn delete_finished_sessions(
&self,
before_date: Date,
) -> Result<(), StatsStorageError> {
Ok(self
.session_manager
.delete_finished_sessions(before_date)
.await?)
}
pub async fn insert_unique_user(
&self,
date: Date,
client_address_bs58: String,
) -> Result<(), StatsStorageError> {
Ok(self
.session_manager
.insert_unique_user(date, client_address_bs58)
.await?)
}
pub async fn get_unique_users(&self, date: Date) -> Result<Vec<String>, StatsStorageError> {
Ok(self.session_manager.get_unique_users(date).await?)
}
pub async fn delete_unique_users(&self, before_date: Date) -> Result<(), StatsStorageError> {
Ok(self
.session_manager
.delete_unique_users(before_date)
.await?)
}
pub async fn insert_active_session(
&self,
client_address: DestinationAddressBytes,
session: ActiveSession,
) -> Result<(), StatsStorageError> {
Ok(self
.session_manager
.insert_active_session(
client_address.as_base58_string(),
session.start,
session.typ.to_string().into(),
)
.await?)
}
pub async fn update_active_session_type(
&self,
client_address: DestinationAddressBytes,
session_type: SessionType,
) -> Result<(), StatsStorageError> {
Ok(self
.session_manager
.update_active_session_type(
client_address.as_base58_string(),
session_type.to_string().into(),
)
.await?)
}
pub async fn get_active_session(
&self,
client_address: DestinationAddressBytes,
) -> Result<Option<ActiveSession>, StatsStorageError> {
Ok(self
.session_manager
.get_active_session(client_address.as_base58_string())
.await?
.map(Into::into))
}
pub async fn get_all_active_sessions(&self) -> Result<Vec<ActiveSession>, StatsStorageError> {
Ok(self
.session_manager
.get_all_active_sessions()
.await?
.into_iter()
.map(Into::into)
.collect())
}
pub async fn get_started_sessions_count(
&self,
start_date: Date,
) -> Result<i32, StatsStorageError> {
Ok(self
.session_manager
.get_started_sessions_count(start_date)
.await?)
}
pub async fn get_active_users(&self) -> Result<Vec<String>, StatsStorageError> {
Ok(self.session_manager.get_active_users().await?)
}
pub async fn delete_active_session(
&self,
client_address: DestinationAddressBytes,
) -> Result<(), StatsStorageError> {
Ok(self
.session_manager
.delete_active_session(client_address.as_base58_string())
.await?)
}
pub async fn cleanup_active_sessions(&self) -> Result<(), StatsStorageError> {
Ok(self.session_manager.cleanup_active_sessions().await?)
}
}
+109
View File
@@ -0,0 +1,109 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use nym_credentials_interface::TicketType;
use sqlx::prelude::FromRow;
use time::{Duration, OffsetDateTime};
#[derive(FromRow)]
pub struct StoredFinishedSession {
duration_ms: i64,
typ: String,
}
impl StoredFinishedSession {
pub fn serialize(&self) -> (u64, String) {
(
self.duration_ms as u64, //we are sure that it fits in a u64, see `fn end_at`
self.typ.clone(),
)
}
}
pub struct FinishedSession {
pub duration: Duration,
pub typ: SessionType,
}
#[derive(PartialEq)]
pub enum SessionType {
Vpn,
Mixnet,
Unknown,
}
impl SessionType {
pub fn to_string(&self) -> &str {
match self {
Self::Vpn => "vpn",
Self::Mixnet => "mixnet",
Self::Unknown => "unknown",
}
}
pub fn from_string(s: &str) -> Self {
match s {
"vpn" => Self::Vpn,
"mixnet" => Self::Mixnet,
_ => Self::Unknown,
}
}
}
impl From<TicketType> for SessionType {
fn from(value: TicketType) -> Self {
match value {
TicketType::V1MixnetEntry => Self::Mixnet,
TicketType::V1MixnetExit => Self::Mixnet,
TicketType::V1WireguardEntry => Self::Vpn,
TicketType::V1WireguardExit => Self::Vpn,
}
}
}
#[derive(FromRow)]
pub(crate) struct StoredActiveSession {
start_time: OffsetDateTime,
typ: String,
}
pub struct ActiveSession {
pub start: OffsetDateTime,
pub typ: SessionType,
}
impl ActiveSession {
pub fn new(start_time: OffsetDateTime) -> Self {
ActiveSession {
start: start_time,
typ: SessionType::Unknown,
}
}
pub fn set_type(&mut self, ticket_type: TicketType) {
self.typ = ticket_type.into();
}
pub fn end_at(self, stop_time: OffsetDateTime) -> Option<FinishedSession> {
let session_duration = stop_time - self.start;
//ensure duration is positive to fit in a u64
//u64::max milliseconds is 500k millenia so no overflow issue
if session_duration > Duration::ZERO {
Some(FinishedSession {
duration: session_duration,
typ: self.typ,
})
} else {
None
}
}
}
impl From<StoredActiveSession> for ActiveSession {
fn from(value: StoredActiveSession) -> Self {
ActiveSession {
start: value.start_time,
typ: SessionType::from_string(&value.typ),
}
}
}
@@ -0,0 +1,176 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use time::{Date, OffsetDateTime};
use crate::models::{StoredActiveSession, StoredFinishedSession};
pub(crate) type Result<T> = std::result::Result<T, sqlx::Error>;
#[derive(Clone)]
pub(crate) struct SessionManager {
connection_pool: sqlx::SqlitePool,
}
impl SessionManager {
/// Creates new instance of the `SessionsManager` with the provided sqlite connection pool.
///
/// # Arguments
///
/// * `connection_pool`: database connection pool to use.
pub(crate) fn new(connection_pool: sqlx::SqlitePool) -> Self {
SessionManager { connection_pool }
}
pub(crate) async fn insert_finished_session(
&self,
date: Date,
duration_ms: i64,
typ: String,
) -> Result<()> {
sqlx::query!(
"INSERT INTO sessions_finished (day, duration_ms, typ) VALUES (?, ?, ?)",
date,
duration_ms,
typ
)
.execute(&self.connection_pool)
.await?;
Ok(())
}
pub(crate) async fn get_finished_sessions(
&self,
date: Date,
) -> Result<Vec<StoredFinishedSession>> {
sqlx::query_as("SELECT duration_ms, typ FROM sessions_finished WHERE day = ?")
.bind(date)
.fetch_all(&self.connection_pool)
.await
}
pub(crate) async fn delete_finished_sessions(&self, before_date: Date) -> Result<()> {
sqlx::query!("DELETE FROM sessions_finished WHERE day <= ? ", before_date)
.execute(&self.connection_pool)
.await?;
Ok(())
}
pub(crate) async fn insert_unique_user(
&self,
date: Date,
client_address_b58: String,
) -> Result<()> {
sqlx::query!(
"INSERT OR IGNORE INTO sessions_unique_users (day, client_address) VALUES (?, ?)",
date,
client_address_b58,
)
.execute(&self.connection_pool)
.await?;
Ok(())
}
pub(crate) async fn get_unique_users(&self, date: Date) -> Result<Vec<String>> {
sqlx::query_scalar!(
"SELECT client_address as count FROM sessions_unique_users WHERE day = ?",
date
)
.fetch_all(&self.connection_pool)
.await
}
pub(crate) async fn delete_unique_users(&self, before_date: Date) -> Result<()> {
sqlx::query!(
"DELETE FROM sessions_unique_users WHERE day <= ? ",
before_date
)
.execute(&self.connection_pool)
.await?;
Ok(())
}
pub(crate) async fn insert_active_session(
&self,
client_address_b58: String,
start_time: OffsetDateTime,
typ: String,
) -> Result<()> {
sqlx::query!(
"INSERT INTO sessions_active (client_address, start_time, typ) VALUES (?, ?, ?)",
client_address_b58,
start_time,
typ
)
.execute(&self.connection_pool)
.await?;
Ok(())
}
pub(crate) async fn update_active_session_type(
&self,
client_address_b58: String,
typ: String,
) -> Result<()> {
sqlx::query!(
"UPDATE sessions_active SET typ = ? WHERE client_address = ?",
typ,
client_address_b58,
)
.execute(&self.connection_pool)
.await?;
Ok(())
}
pub(crate) async fn get_active_session(
&self,
client_address_b58: String,
) -> Result<Option<StoredActiveSession>> {
sqlx::query_as("SELECT start_time, typ FROM sessions_active WHERE client_address = ?")
.bind(client_address_b58)
.fetch_optional(&self.connection_pool)
.await
}
pub(crate) async fn get_all_active_sessions(&self) -> Result<Vec<StoredActiveSession>> {
sqlx::query_as("SELECT start_time, typ FROM sessions_active")
.fetch_all(&self.connection_pool)
.await
}
pub(crate) async fn get_started_sessions_count(&self, start_date: Date) -> Result<i32> {
Ok(sqlx::query!(
"SELECT COUNT(*) as count FROM sessions_active WHERE date(start_time) = ?",
start_date
)
.fetch_one(&self.connection_pool)
.await?
.count)
}
pub(crate) async fn get_active_users(&self) -> Result<Vec<String>> {
Ok(sqlx::query!("SELECT client_address from sessions_active")
.fetch_all(&self.connection_pool)
.await?
.into_iter()
.map(|record| record.client_address)
.collect())
}
pub(crate) async fn delete_active_session(&self, client_address_b58: String) -> Result<()> {
sqlx::query!(
"DELETE FROM sessions_active WHERE client_address = ?",
client_address_b58
)
.execute(&self.connection_pool)
.await?;
Ok(())
}
pub(crate) async fn cleanup_active_sessions(&self) -> Result<()> {
sqlx::query!("DELETE FROM sessions_active")
.execute(&self.connection_pool)
.await?;
Ok(())
}
}
+18 -5
View File
@@ -9,7 +9,7 @@ use serde::{Deserialize, Serialize};
use std::fmt::Display;
use std::time::Duration;
use thiserror::Error;
use tracing::warn;
use tracing::{instrument, warn};
use url::Url;
pub use reqwest::IntoUrl;
@@ -205,6 +205,7 @@ impl Client {
self.reqwest_client.get(url)
}
#[instrument(level = "debug", skip_all, fields(path=?path))]
async fn send_get_request<K, V, E>(
&self,
path: PathSegments<'_>,
@@ -215,6 +216,7 @@ impl Client {
V: AsRef<str>,
E: Display,
{
tracing::trace!("Sending GET request");
let url = sanitize_url(&self.base_url, path, params);
#[cfg(target_arch = "wasm32")]
@@ -280,6 +282,7 @@ impl Client {
}
}
#[instrument(level = "debug", skip_all)]
pub async fn get_json<T, K, V, E>(
&self,
path: PathSegments<'_>,
@@ -515,12 +518,14 @@ pub fn sanitize_url<K: AsRef<str>, V: AsRef<str>>(
url
}
#[tracing::instrument(level = "debug", skip_all)]
pub async fn parse_response<T, E>(res: Response, allow_empty: bool) -> Result<T, HttpClientError<E>>
where
T: DeserializeOwned,
E: DeserializeOwned + Display,
{
let status = res.status();
tracing::debug!("Status: {} (success: {})", &status, status.is_success());
if !allow_empty {
if let Some(0) = res.content_length() {
@@ -529,11 +534,19 @@ where
}
if res.status().is_success() {
let text = res.text().await?;
match serde_json::from_str(&text) {
Ok(res) => Ok(res),
Err(source) => Err(HttpClientError::ResponseDeserialisationFailure { source }),
#[cfg(debug_assertions)]
{
let text = res.text().await.inspect_err(|err| {
tracing::error!("Couldn't even get response text: {err}");
})?;
tracing::trace!("Result:\n{:#?}", text);
serde_json::from_str(&text)
.map_err(|err| HttpClientError::GenericRequestFailure(err.to_string()))
}
#[cfg(not(debug_assertions))]
Ok(res.json().await?)
} else if res.status() == StatusCode::NOT_FOUND {
Err(HttpClientError::NotFound)
} else {
+1 -1
View File
@@ -248,7 +248,7 @@ mod tests {
data: IpPacketRequestData::StaticConnect(
StaticConnectRequest {
request_id: 123,
ips: IpPair::new(Ipv4Addr::from_str("10.0.0.1").unwrap(), Ipv6Addr::from_str("2001:db8:a160::1").unwrap()),
ips: IpPair::new(Ipv4Addr::from_str("10.0.0.1").unwrap(), Ipv6Addr::from_str("fc00::1").unwrap()),
reply_to: Recipient::try_from_base58_string("D1rrpsysCGCYXy9saP8y3kmNpGtJZUXN9SvFoUcqAsM9.9Ssso1ea5NfkbMASdiseDSjTN1fSWda5SgEVjdSN4CvV@GJqd3ZxpXWSNxTfx7B1pPtswpetH4LnJdFeLeuY5KUuN").unwrap(),
reply_to_hops: None,
reply_to_avg_mix_delays: None,
+1 -1
View File
@@ -429,7 +429,7 @@ mod tests {
SignedStaticConnectRequest {
request: StaticConnectRequest {
request_id: 123,
ips: IpPair::new(Ipv4Addr::from_str("10.0.0.1").unwrap(), Ipv6Addr::from_str("2001:db8:a160::1").unwrap()),
ips: IpPair::new(Ipv4Addr::from_str("10.0.0.1").unwrap(), Ipv6Addr::from_str("fc00::1").unwrap()),
reply_to: Recipient::try_from_base58_string("D1rrpsysCGCYXy9saP8y3kmNpGtJZUXN9SvFoUcqAsM9.9Ssso1ea5NfkbMASdiseDSjTN1fSWda5SgEVjdSN4CvV@GJqd3ZxpXWSNxTfx7B1pPtswpetH4LnJdFeLeuY5KUuN").unwrap(),
reply_to_hops: None,
reply_to_avg_mix_delays: None,
+6 -15
View File
@@ -14,7 +14,6 @@ use nym_task::TaskClient;
use rand::seq::SliceRandom;
use rand::thread_rng;
use std::net::SocketAddr;
use std::net::ToSocketAddrs;
use std::sync::Arc;
use std::time::Duration;
use tokio::task::JoinHandle;
@@ -313,7 +312,7 @@ impl VerlocMeasurer {
info!("Starting verloc measurements");
// TODO: should we also measure gateways?
let all_mixes = match self.validator_client.get_cached_mixnodes().await {
let all_mixes = match self.validator_client.get_all_described_nodes().await {
Ok(nodes) => nodes,
Err(err) => {
error!(
@@ -332,22 +331,14 @@ impl VerlocMeasurer {
// we only care about address and identity
let tested_nodes = all_mixes
.into_iter()
.filter(|n| n.description.declared_role.mixnode)
.filter_map(|node| {
let mix_node = node.bond_information.mix_node;
// check if the node has sufficient version to be able to understand the packets
let node_version = parse_version(&mix_node.version).ok()?;
if node_version < self.config.minimum_compatible_node_version {
return None;
}
// try to parse the identity and host
let node_identity =
identity::PublicKey::from_base58_string(mix_node.identity_key).ok()?;
let node_identity = node.ed25519_identity_key();
let verloc_host = (&*mix_node.host, mix_node.verloc_port)
.to_socket_addrs()
.ok()?
.next()?;
let ip = node.description.host_information.ip_address.first()?;
let verloc_port = node.description.verloc_port();
let verloc_host = SocketAddr::new(*ip, verloc_port);
// TODO: possible problem in the future, this does name resolution and theoretically
// if a lot of nodes maliciously mis-configured themselves, it might take a while to resolve them all
+14
View File
@@ -0,0 +1,14 @@
[package]
name = "nym-common-models"
version = "0.1.0"
authors.workspace = true
repository.workspace = true
homepage.workspace = true
documentation.workspace = true
edition.workspace = true
license.workspace = true
rust-version.workspace = true
readme.workspace = true
[dependencies]
serde = { workspace = true, features = ["derive"] }
+1
View File
@@ -0,0 +1 @@
pub mod ns_api;
+7
View File
@@ -0,0 +1,7 @@
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct TestrunAssignment {
pub testrun_id: i64,
pub gateway_identity_key: String,
}
+4
View File
@@ -23,3 +23,7 @@ default = ["env", "network"]
env = ["dotenvy", "log"]
network = ["schemars", "serde", "url"]
utoipa = [ "dep:utoipa" ]
[build-dependencies]
regex = { workspace = true }
cargo_metadata = { version = "0.18" }
+85
View File
@@ -0,0 +1,85 @@
use cargo_metadata::MetadataCommand;
use regex::Regex;
use std::{collections::HashMap, fs, path::PathBuf};
/// Sync variable values defined in code with .env file
fn main() {
let source_of_truth = include_str!("src/mainnet.rs");
let mut output_path = workspace_root();
output_path.push("envs");
output_path.push("mainnet.env");
println!("{}", output_path.display());
let variables_to_track = [
"NETWORK_NAME",
"BECH32_PREFIX",
"MIXNET_CONTRACT_ADDRESS",
"VESTING_CONTRACT_ADDRESS",
"GROUP_CONTRACT_ADDRESS",
"ECASH_CONTRACT_ADDRESS",
"MULTISIG_CONTRACT_ADDRESS",
"COCONUT_DKG_CONTRACT_ADDRESS",
"REWARDING_VALIDATOR_ADDRESS",
"NYM_API",
"NYXD_WS",
"EXPLORER_API",
"NYM_VPN_API",
];
let mut replace_with = HashMap::new();
for var in variables_to_track {
// if script fails, debug with `cargo check -vv``
println!("Looking for {}", var);
// read pattern that looks like:
// <var>: &str = "<whatever is between quotes>"
let pattern = format!(r#"{}: &str\s*=\s*"([^"]*)""#, regex::escape(var));
let re = Regex::new(&pattern).unwrap();
let value = re
.captures(source_of_truth)
.and_then(|caps| caps.get(1).map(|match_| match_.as_str().to_string()))
.expect("Couldn't find var in source file");
println!("Storing {}={}", var, value);
replace_with.insert(var, value);
}
let mut contents = fs::read_to_string(&output_path).unwrap();
for (var, value) in replace_with {
// match a pattern that looks like:
// <var> = <value>
// where `<var>` is a variable name inserted into search pattern
let pattern = format!(r#"{}\s*=\s*([^\n]*)"#, regex::escape(var));
// replace matched pattern with
// <var>=<value>
let re = Regex::new(&pattern).unwrap();
contents = re
.replace(&contents, |_: &regex::Captures| {
format!(r#"{}={}"#, var, value)
})
.to_string();
}
println!("File contents to write:\n{}", contents);
if output_path.exists() {
fs::write(output_path, contents).unwrap();
} else {
panic!("{} doesn't exist", output_path.display());
}
}
fn workspace_root() -> PathBuf {
let metadata = MetadataCommand::new()
.exec()
.expect("Failed to get cargo metadata");
metadata
.workspace_root
.into_std_path_buf()
.canonicalize()
.expect("Failed to canonicalize path")
}
+7 -4
View File
@@ -45,13 +45,16 @@ pub mod nyx {
}
pub mod wireguard {
use std::net::{IpAddr, Ipv4Addr};
use std::net::{Ipv4Addr, Ipv6Addr};
pub const WG_PORT: u16 = 51822;
// The interface used to route traffic
pub const WG_TUN_BASE_NAME: &str = "nymwg";
pub const WG_TUN_BASE_NAME_V4: &str = "nymwg";
pub const WG_TUN_BASE_NAME_V6: &str = "nymwgv6";
pub const WG_TUN_DEVICE_ADDRESS: &str = "10.1.0.1";
pub const WG_TUN_DEVICE_IP_ADDRESS: IpAddr = IpAddr::V4(Ipv4Addr::new(10, 1, 0, 1));
pub const WG_TUN_DEVICE_NETMASK: &str = "255.255.255.0";
pub const WG_TUN_DEVICE_IP_ADDRESS_V4: Ipv4Addr = Ipv4Addr::new(10, 1, 0, 1);
pub const WG_TUN_DEVICE_NETMASK_V4: u8 = 16;
pub const WG_TUN_DEVICE_IP_ADDRESS_V6: Ipv6Addr = Ipv6Addr::new(0xfc01, 0, 0, 0, 0, 0, 0, 0x1); // fc01::1
pub const WG_TUN_DEVICE_NETMASK_V6: u8 = 112;
}
+1
View File
@@ -25,6 +25,7 @@ pub const NYXD_WEBSOCKET: &str = "NYXD_WS";
pub const EXPLORER_API: &str = "EXPLORER_API";
pub const EXIT_POLICY_URL: &str = "EXIT_POLICY";
pub const NYM_VPN_API: &str = "NYM_VPN_API";
pub const CLIENT_STATS_COLLECTION_PROVIDER: &str = "CLIENT_STATS_COLLECTION_PROVIDER";
pub const DKG_TIME_CONFIGURATION: &str = "DKG_TIME_CONFIGURATION";
@@ -21,7 +21,7 @@ nym-sphinx-types = { path = "../types" }
nym-topology = { path = "../../topology" }
[target."cfg(target_arch = \"wasm32\")".dependencies.wasm-bindgen]
version = "0.2.93"
version = "0.2.95"
[dev-dependencies]
rand_chacha = { workspace = true }
+6
View File
@@ -28,3 +28,9 @@ impl From<ConnectionError> for Socks5ClientCoreError {
}
}
}
impl nym_task::TaskStatusEvent for Socks5ClientCoreError {
fn as_any(&self) -> &dyn std::any::Any {
self
}
}
+1 -2
View File
@@ -23,8 +23,7 @@ use nym_client_core::init::types::GatewaySetup;
use nym_credential_storage::storage::Storage as CredentialStorage;
use nym_sphinx::addressing::clients::Recipient;
use nym_sphinx::params::PacketType;
use nym_task::manager::TaskStatus;
use nym_task::{TaskClient, TaskHandle};
use nym_task::{TaskClient, TaskHandle, TaskStatus};
use anyhow::anyhow;
use nym_validator_client::UserAgent;
+9
View File
@@ -11,7 +11,16 @@ license.workspace = true
[dependencies]
futures = { workspace = true }
log = { workspace = true }
sysinfo = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
thiserror = { workspace = true }
time = { workspace = true }
tokio = { workspace = true }
si-scale = { workspace = true }
nym-sphinx = { path = "../nymsphinx" }
nym-credentials-interface = { path = "../credentials-interface" }
nym-metrics = { path = "../nym-metrics" }
nym-task = { path = "../task" }
@@ -0,0 +1,83 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
//! # Gateway Connection statistics
//!
//! Metrics collected by the client while establishing and maintaining connections to the gateway.
use super::ClientStatsEvents;
use std::collections::VecDeque;
use nym_metrics::{inc, inc_by};
use serde::{Deserialize, Serialize};
#[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub(crate) struct GatewayStats {
// Sent
real_packets_sent: u64,
real_packets_sent_size: usize,
/// failed connection statistics
failures: VecDeque<()>, // TODO
}
impl GatewayStats {
fn handle(&mut self, event: GatewayStatsEvent) {
match event {
GatewayStatsEvent::RealPacketSent(packet_size) => {
self.real_packets_sent += 1;
self.real_packets_sent_size += packet_size;
inc!("real_packets_sent");
inc_by!("real_packets_sent_size", packet_size);
}
}
}
fn summary(&self) -> (String, String) {
(
format!("packets sent: {}", self.real_packets_sent),
"packets received: todo".to_owned(),
)
}
}
impl From<GatewayStatsEvent> for ClientStatsEvents {
fn from(event: GatewayStatsEvent) -> ClientStatsEvents {
ClientStatsEvents::GatewayConn(event)
}
}
/// Event space for Gateway Connection Events
#[derive(Debug)]
pub enum GatewayStatsEvent {
/// The real packets sent. Recall that acks are sent by the gateway, so it's not included here.
RealPacketSent(usize),
}
/// Gateway Statistics Tracking
#[derive(Default)]
pub struct GatewayStatsControl {
// Keep track of packet statistics over time
stats: GatewayStats,
}
impl GatewayStatsControl {
pub(crate) fn handle_event(&mut self, event: GatewayStatsEvent) {
self.stats.handle(event)
}
pub(crate) fn report(&self) -> GatewayStats {
self.stats.clone()
}
pub(crate) fn local_report(&self) {
self.report_counters();
}
fn report_counters(&self) {
log::trace!("packet statistics: {:?}", &self.stats);
let (summary_sent, summary_recv) = self.stats.summary();
log::debug!("{}", summary_sent);
log::debug!("{}", summary_recv);
}
}
+135
View File
@@ -0,0 +1,135 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::report::{ClientStatsReport, OsInformation};
use nym_task::TaskClient;
use time::{OffsetDateTime, Time};
use tokio::sync::mpsc::UnboundedSender;
/// Active gateway connection statistics.
pub mod gateway_conn_statistics;
/// Nym API connection statistics.
pub mod nym_api_statistics;
/// Packet count based statistics.
pub mod packet_statistics;
/// Channel receiving generic stats events to be used by a statistics aggregator.
pub type ClientStatsReceiver = tokio::sync::mpsc::UnboundedReceiver<ClientStatsEvents>;
/// Channel allowing generic statistics events to be reported to a stats event aggregator
#[derive(Clone)]
pub struct ClientStatsSender {
stats_tx: Option<UnboundedSender<ClientStatsEvents>>,
}
impl ClientStatsSender {
/// Create a new statistics Sender
pub fn new(stats_tx: Option<UnboundedSender<ClientStatsEvents>>) -> Self {
ClientStatsSender { stats_tx }
}
/// Report a statistics event using the sender.
pub fn report(&self, event: ClientStatsEvents) {
if let Some(tx) = &self.stats_tx {
if let Err(err) = tx.send(event) {
log::error!("Failed to send stats event: {:?}", err);
}
}
}
}
/// Client Statistics events (static for now)
pub enum ClientStatsEvents {
/// Packet count events
PacketStatistics(packet_statistics::PacketStatisticsEvent),
/// Gateway Connection events
GatewayConn(gateway_conn_statistics::GatewayStatsEvent),
/// Nym API connection events
NymApi(nym_api_statistics::NymApiStatsEvent),
}
/// Controls stats event handling and reporting
pub struct ClientStatsController {
//static infos
last_update_time: OffsetDateTime,
client_id: String,
client_type: String,
os_information: OsInformation,
// stats collection modules
packet_stats: packet_statistics::PacketStatisticsControl,
gateway_conn_stats: gateway_conn_statistics::GatewayStatsControl,
nym_api_stats: nym_api_statistics::NymApiStatsControl,
}
impl ClientStatsController {
/// Creates a ClientStatsController given a client_id
pub fn new(client_id: String, client_type: String) -> Self {
ClientStatsController {
last_update_time: ClientStatsController::get_update_time(),
client_id,
client_type,
os_information: OsInformation::new(),
packet_stats: Default::default(),
gateway_conn_stats: Default::default(),
nym_api_stats: Default::default(),
}
}
/// Returns a static ClientStatsReport that can be sent somewhere
pub fn build_report(&self) -> ClientStatsReport {
ClientStatsReport {
last_update_time: self.last_update_time,
client_id: self.client_id.clone(),
client_type: self.client_type.clone(),
os_information: self.os_information.clone(),
packet_stats: self.packet_stats.report(),
gateway_conn_stats: self.gateway_conn_stats.report(),
nym_api_stats: self.nym_api_stats.report(),
}
}
/// Handle and dispatch incoming stats event
pub fn handle_event(&mut self, stats_event: ClientStatsEvents) {
match stats_event {
ClientStatsEvents::PacketStatistics(event) => self.packet_stats.handle_event(event),
ClientStatsEvents::GatewayConn(event) => self.gateway_conn_stats.handle_event(event),
ClientStatsEvents::NymApi(event) => self.nym_api_stats.handle_event(event),
}
}
/// Reset the metrics to their initial state.
///
/// Used to periodically reset the metrics in accordance with periodic reporting strategy
pub fn reset(&mut self) {
self.nym_api_stats = Default::default();
self.gateway_conn_stats = Default::default();
//no periodic reset for packet stats
self.last_update_time = ClientStatsController::get_update_time();
}
/// snapshot the current state of the metrics for module that needs it
pub fn snapshot(&mut self) {
//no snapshot for gateway_conn_stats
//no snapshot for nym_api_stats
self.packet_stats.snapshot();
}
pub fn local_report(&mut self, task_client: &mut TaskClient) {
self.packet_stats.local_report(task_client);
self.gateway_conn_stats.local_report();
self.nym_api_stats.local_report();
}
fn get_update_time() -> OffsetDateTime {
let now = OffsetDateTime::now_utc();
#[allow(clippy::unwrap_used)]
//Safety : 0 is always a valid number of seconds, hours and minutes comes from a valid source
let new_time = Time::from_hms(now.hour(), now.minute(), 0).unwrap();
//allows a bigger anonymity by hiding exact sending time
now.replace_time(new_time)
}
}
@@ -0,0 +1,83 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
//! # API Connection statistics
//!
//! Metrics collected by the client while attempting to pull config from the API.
use super::ClientStatsEvents;
use std::collections::VecDeque;
use nym_metrics::{inc, inc_by};
use serde::{Deserialize, Serialize};
#[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub(crate) struct NymApiStats {
// Sent
real_packets_sent: u64,
real_packets_sent_size: usize,
/// API connection failure statistics
failures: VecDeque<()>, // TODO
}
impl NymApiStats {
fn handle(&mut self, event: NymApiStatsEvent) {
match event {
NymApiStatsEvent::RealPacketSent(packet_size) => {
self.real_packets_sent += 1;
self.real_packets_sent_size += packet_size;
inc!("real_packets_sent");
inc_by!("real_packets_sent_size", packet_size);
}
}
}
fn summary(&self) -> (String, String) {
(
format!("packets sent: {}", self.real_packets_sent,),
"packets received: todo".to_owned(),
)
}
}
/// Event space for Nym API statistics tracking
#[derive(Debug)]
pub enum NymApiStatsEvent {
/// The real packets sent. Recall that acks are sent by the Api, so it's not included here.
RealPacketSent(usize),
}
impl From<NymApiStatsEvent> for ClientStatsEvents {
fn from(event: NymApiStatsEvent) -> ClientStatsEvents {
ClientStatsEvents::NymApi(event)
}
}
/// Nym API statistics tracking object
#[derive(Default)]
pub struct NymApiStatsControl {
// Keep track of packet statistics over time
stats: NymApiStats,
}
impl NymApiStatsControl {
pub(crate) fn handle_event(&mut self, event: NymApiStatsEvent) {
self.stats.handle(event)
}
pub(crate) fn report(&self) -> NymApiStats {
self.stats.clone()
}
pub(crate) fn local_report(&self) {
self.report_counters();
}
fn report_counters(&self) {
log::trace!("packet statistics: {:?}", &self.stats);
let (summary_sent, summary_recv) = self.stats.summary();
log::debug!("{}", summary_sent);
log::debug!("{}", summary_recv);
}
}
@@ -1,48 +1,25 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use super::ClientStatsEvents;
use core::fmt;
use std::{
collections::VecDeque,
time::{Duration, Instant},
};
use nym_metrics::{inc, inc_by};
use serde::{Deserialize, Serialize};
use si_scale::helpers::bibytes2;
// Metrics server
use futures::future::{FusedFuture, OptionFuture};
use futures::FutureExt;
#[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))]
use http_body_util::Full;
#[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))]
use hyper::body::Bytes;
#[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))]
use hyper::server::conn::http1;
#[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))]
use hyper::service::service_fn;
#[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))]
use hyper::{Request, Response};
#[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))]
use hyper_util::rt::TokioIo;
#[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))]
use std::convert::Infallible;
#[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))]
#[cfg(feature = "metrics-server")]
use std::net::SocketAddr;
#[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))]
use tokio::net::TcpListener;
use crate::spawn_future;
// Time interval between reporting packet statistics
const PACKET_REPORT_INTERVAL_SECS: u64 = 2;
// Interval for taking snapshots of the packet statistics
const SNAPSHOT_INTERVAL_MS: u64 = 500;
// When computing rates, we include snapshots that are up to this old. We set it to some odd number
// a tad larger than an integer number of snapshot intervals, so that we don't have to worry about
// threshold effects.
// Also, set it larger than the packet report interval so that we don't miss notable singular events
const RECORDING_WINDOW_MS: u64 = 2300;
#[derive(Default, Debug, Clone)]
struct PacketStatistics {
#[derive(Default, Debug, Clone, Serialize, Deserialize)]
pub(crate) struct PacketStatistics {
// Sent
real_packets_sent: u64,
real_packets_sent_size: usize,
@@ -72,7 +49,7 @@ struct PacketStatistics {
}
impl PacketStatistics {
fn handle_event(&mut self, event: PacketStatisticsEvent) {
fn handle(&mut self, event: PacketStatisticsEvent) {
match event {
PacketStatisticsEvent::RealPacketSent(packet_size) => {
self.real_packets_sent += 1;
@@ -189,29 +166,64 @@ impl std::ops::Sub for PacketStatistics {
}
}
pub struct MixnetBandwidthStatisticsEvent {
pub rates: PacketRates,
}
impl MixnetBandwidthStatisticsEvent {
pub fn new(rates: PacketRates) -> Self {
Self { rates }
}
}
impl nym_task::TaskStatusEvent for MixnetBandwidthStatisticsEvent {
fn as_any(&self) -> &dyn std::any::Any {
self
}
}
impl fmt::Display for MixnetBandwidthStatisticsEvent {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.rates.summary())
}
}
#[derive(Debug, Clone)]
struct PacketRates {
real_packets_sent: f64,
real_packets_sent_size: f64,
cover_packets_sent: f64,
cover_packets_sent_size: f64,
pub struct PacketRates {
pub real_packets_sent: f64,
pub real_packets_sent_size: f64,
pub cover_packets_sent: f64,
pub cover_packets_sent_size: f64,
real_packets_received: f64,
real_packets_received_size: f64,
cover_packets_received: f64,
cover_packets_received_size: f64,
pub real_packets_received: f64,
pub real_packets_received_size: f64,
pub cover_packets_received: f64,
pub cover_packets_received_size: f64,
total_acks_received: f64,
total_acks_received_size: f64,
real_acks_received: f64,
real_acks_received_size: f64,
cover_acks_received: f64,
cover_acks_received_size: f64,
pub total_acks_received: f64,
pub total_acks_received_size: f64,
pub real_acks_received: f64,
pub real_acks_received_size: f64,
pub cover_acks_received: f64,
pub cover_acks_received_size: f64,
real_packets_queued: f64,
retransmissions_queued: f64,
reply_surbs_queued: f64,
additional_reply_surbs_queued: f64,
pub real_packets_queued: f64,
pub retransmissions_queued: f64,
pub reply_surbs_queued: f64,
pub additional_reply_surbs_queued: f64,
}
impl fmt::Display for PacketRates {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
f,
"down: {}/s, up: {}/s (cover down: {}/s, cover up: {}/s)",
bibytes2(self.real_packets_received_size),
bibytes2(self.real_packets_sent_size),
bibytes2(self.cover_packets_received_size),
bibytes2(self.cover_packets_sent_size),
)
}
}
impl From<PacketStatistics> for PacketRates {
@@ -330,56 +342,46 @@ impl PacketRates {
}
}
/// Event Space used for counting the Packet types used in a connection.
#[derive(Debug)]
pub(crate) enum PacketStatisticsEvent {
// The real packets sent. Recall that acks are sent by the gateway, so it's not included here.
pub enum PacketStatisticsEvent {
/// The real packets sent. Recall that acks are sent by the gateway, so it's not included here.
RealPacketSent(usize),
// The cover packets sent
/// The cover packets sent
CoverPacketSent(usize),
// Real packets received
/// Real packets received
RealPacketReceived(usize),
// Cover packets received
/// Cover packets received
CoverPacketReceived(usize),
// Ack of any type received. This is mostly used as a consistency check, and should be the sum
// of real and cover acks received.
/// Ack of any type received. This is mostly used as a consistency check, and should be the sum
/// of real and cover acks received.
AckReceived(usize),
// Out of the total acks received, this is the subset of those that were real
/// Out of the total acks received, this is the subset of those that were real
RealAckReceived(usize),
// Out of the total acks received, this is the subset of those that were for cover traffic
/// Out of the total acks received, this is the subset of those that were for cover traffic
CoverAckReceived(usize),
// Types of packets queued
/// Types of packets queued
RealPacketQueued,
/// Types of packets queued
RetransmissionQueued,
/// Types of packets queued
ReplySurbRequestQueued,
/// Types of packets queued
AdditionalReplySurbRequestQueued,
}
type PacketStatisticsReceiver = tokio::sync::mpsc::UnboundedReceiver<PacketStatisticsEvent>;
#[derive(Clone)]
pub(crate) struct PacketStatisticsReporter {
stats_tx: tokio::sync::mpsc::UnboundedSender<PacketStatisticsEvent>,
}
impl PacketStatisticsReporter {
pub(crate) fn new(stats_tx: tokio::sync::mpsc::UnboundedSender<PacketStatisticsEvent>) -> Self {
Self { stats_tx }
}
pub(crate) fn report(&self, event: PacketStatisticsEvent) {
self.stats_tx.send(event).unwrap_or_else(|err| {
log::error!("Failed to report packet stat: {:?}", err);
});
impl From<PacketStatisticsEvent> for ClientStatsEvents {
fn from(event: PacketStatisticsEvent) -> ClientStatsEvents {
ClientStatsEvents::PacketStatistics(event)
}
}
pub(crate) struct PacketStatisticsControl {
// Incoming packet stats events from other tasks
stats_rx: PacketStatisticsReceiver,
/// Statistics tracking for Packet based I/O
#[derive(Default)]
pub struct PacketStatisticsControl {
// Keep track of packet statistics over time
stats: PacketStatistics,
@@ -392,18 +394,28 @@ pub(crate) struct PacketStatisticsControl {
}
impl PacketStatisticsControl {
pub(crate) fn new() -> (Self, PacketStatisticsReporter) {
let (stats_tx, stats_rx) = tokio::sync::mpsc::unbounded_channel();
pub(crate) fn handle_event(&mut self, event: PacketStatisticsEvent) {
self.stats.handle(event)
}
(
Self {
stats_rx,
stats: PacketStatistics::default(),
history: VecDeque::new(),
rates: VecDeque::new(),
},
PacketStatisticsReporter::new(stats_tx),
)
pub(crate) fn snapshot(&mut self) {
self.update_history();
self.update_rates();
}
pub(crate) fn report(&self) -> PacketStatistics {
self.stats.clone()
}
pub(crate) fn local_report(&mut self, task_client: &mut nym_task::TaskClient) {
let rates = self.report_rates();
self.check_for_notable_events();
self.report_counters();
// Report our current bandwidth used to e.g a GUI client
if let Some(rates) = rates {
task_client.send_status_msg(Box::new(MixnetBandwidthStatisticsEvent::new(rates)));
}
}
// Add the current stats to the history, and remove old ones.
@@ -456,11 +468,13 @@ impl PacketStatisticsControl {
}
}
fn report_rates(&self) {
fn report_rates(&self) -> Option<PacketRates> {
if let Some((_, rates)) = self.rates.back() {
log::debug!("{}", rates.summary());
log::debug!("{}", rates.detailed_summary());
return Some(rates.clone());
}
None
}
fn report_counters(&self) {
@@ -498,124 +512,4 @@ impl PacketStatisticsControl {
// IDEA: if there is a burst of acks, that could indicate tokio task starvation.
}
pub(crate) async fn run_with_shutdown(&mut self, mut shutdown: nym_task::TaskClient) {
log::debug!("Started PacketStatisticsControl with graceful shutdown support");
let report_interval = Duration::from_secs(PACKET_REPORT_INTERVAL_SECS);
let mut report_interval = tokio::time::interval(report_interval);
let snapshot_interval = Duration::from_millis(SNAPSHOT_INTERVAL_MS);
let mut snapshot_interval = tokio::time::interval(snapshot_interval);
cfg_if::cfg_if! {
if #[cfg(all(target_arch = "wasm32", target_os = "unknown"))] {
log::warn!("Metrics server is not supported on wasm32-unknown-unknown");
let listener: Option<WasmEmpty> = None;
} else if #[cfg(feature = "metrics-server")] {
let mut metrics_port = 18000;
let listener: Option<TcpListener>;
loop {
let addr = SocketAddr::from(([0, 0, 0, 0], metrics_port));
match TcpListener::bind(addr).await {
Ok(l) => {
log::info!("###############################");
log::info!("Metrics endpoint is at: {:?}", l.local_addr());
log::info!("###############################");
listener = Some(l);
break;
},
Err(err) => {
log::warn!("Failed to bind metrics server: {:?}", err);
metrics_port += 1;
}
};
}
} else {
log::info!("Metrics server is disabled!");
let listener: Option<TcpListener> = None;
}
}
loop {
// it seems at some point tokio changed its select precondition evaluation,
// and it's no longer checked before the future is evaluated.
let accept_future: OptionFuture<_> = listener
.as_ref()
.map(|l| l.accept())
.map(FutureExt::fuse)
.into();
tokio::select! {
stats_event = self.stats_rx.recv() => match stats_event {
Some(stats_event) => {
log::trace!("PacketStatisticsControl: Received stats event");
self.stats.handle_event(stats_event);
},
None => {
log::trace!("PacketStatisticsControl: stopping since stats channel was closed");
break;
}
},
// conditional will disable the branch if we're in wasm32-unknown-unknown
// use `_` to calm down clippy when running for wasm
_result = accept_future, if !accept_future.is_terminated() => {
cfg_if::cfg_if! {
if #[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))] {
if let Some(Ok((stream, _))) = _result {
let io = TokioIo::new(stream);
tokio::task::spawn(async move {
if let Err(err) = http1::Builder::new()
.serve_connection(io, service_fn(serve_metrics))
.await
{
log::warn!("Error serving connection: {:?}", err);
}
});
} else {
log::warn!("Error accepting connection");
}
}
}
}
_ = snapshot_interval.tick() => {
self.update_history();
self.update_rates();
}
_ = report_interval.tick() => {
self.report_rates();
self.check_for_notable_events();
self.report_counters();
}
_ = shutdown.recv_with_delay() => {
log::trace!("PacketStatisticsControl: Received shutdown");
break;
},
}
}
log::debug!("PacketStatisticsControl: Exiting");
}
pub(crate) fn start_with_shutdown(mut self, task_client: nym_task::TaskClient) {
spawn_future(async move {
self.run_with_shutdown(task_client).await;
})
}
}
#[cfg(not(all(target_arch = "wasm32", target_os = "unknown")))]
async fn serve_metrics(
_: Request<hyper::body::Incoming>,
) -> Result<Response<Full<Bytes>>, Infallible> {
use nym_metrics::metrics;
Ok(Response::new(Full::new(Bytes::from(metrics!()))))
}
#[cfg(all(target_arch = "wasm32", target_os = "unknown"))]
struct WasmEmpty;
#[cfg(all(target_arch = "wasm32", target_os = "unknown"))]
impl WasmEmpty {
async fn accept(&self) {}
}
+15
View File
@@ -0,0 +1,15 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use thiserror::Error;
/// Error types occurring while processing statistics events and reporting.
#[derive(Debug, Error)]
#[allow(missing_docs)]
pub enum StatsError {
#[error("Failed to (de)serialize stats report : {0}")]
ReportJsonSerialization(#[from] serde_json::Error),
}
/// Result of a statistics operation.
pub type Result<T> = core::result::Result<T, StatsError>;
-54
View File
@@ -1,54 +0,0 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
use futures::channel::mpsc;
use nym_credentials_interface::TicketType;
use nym_sphinx::DestinationAddressBytes;
use time::OffsetDateTime;
pub type StatsEventSender = mpsc::UnboundedSender<StatsEvent>;
pub type StatsEventReceiver = mpsc::UnboundedReceiver<StatsEvent>;
pub enum StatsEvent {
SessionStatsEvent(SessionEvent),
}
impl StatsEvent {
pub fn new_session_start(client: DestinationAddressBytes) -> StatsEvent {
StatsEvent::SessionStatsEvent(SessionEvent::SessionStart {
start_time: OffsetDateTime::now_utc(),
client,
})
}
pub fn new_session_stop(client: DestinationAddressBytes) -> StatsEvent {
StatsEvent::SessionStatsEvent(SessionEvent::SessionStop {
stop_time: OffsetDateTime::now_utc(),
client,
})
}
pub fn new_ecash_ticket(
client: DestinationAddressBytes,
ticket_type: TicketType,
) -> StatsEvent {
StatsEvent::SessionStatsEvent(SessionEvent::EcashTicket {
ticket_type,
client,
})
}
}
pub enum SessionEvent {
SessionStart {
start_time: OffsetDateTime,
client: DestinationAddressBytes,
},
SessionStop {
stop_time: OffsetDateTime,
client: DestinationAddressBytes,
},
EcashTicket {
ticket_type: TicketType,
client: DestinationAddressBytes,
},
}
+89
View File
@@ -0,0 +1,89 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use nym_credentials_interface::TicketType;
use nym_sphinx::DestinationAddressBytes;
use time::OffsetDateTime;
/// Channel for receiving incoming Stats events
pub type GatewayStatsReceiver = tokio::sync::mpsc::UnboundedReceiver<GatewayStatsEvent>;
/// Channel allowing for generic statistics events to be reported to a stats event aggregator.
#[derive(Clone)]
pub struct GatewayStatsReporter {
stats_tx: tokio::sync::mpsc::UnboundedSender<GatewayStatsEvent>,
}
impl GatewayStatsReporter {
/// Construct a new gateway statistics event reporter
pub fn new(stats_tx: tokio::sync::mpsc::UnboundedSender<GatewayStatsEvent>) -> Self {
Self { stats_tx }
}
/// Report a gateway statistivs event using the reporter
pub fn report(&self, event: GatewayStatsEvent) {
self.stats_tx.send(event).unwrap_or_else(|err| {
log::error!("Failed to report gateway stat event : {err}");
});
}
}
/// Gateway Statistics events
pub enum GatewayStatsEvent {
/// Events in the lifecycle of an established client tunnel
SessionStatsEvent(SessionEvent),
}
impl GatewayStatsEvent {
/// A new session between this gateway and the client remote has successfully opened
pub fn new_session_start(client: DestinationAddressBytes) -> GatewayStatsEvent {
GatewayStatsEvent::SessionStatsEvent(SessionEvent::SessionStart {
start_time: OffsetDateTime::now_utc(),
client,
})
}
/// An existing session with the client remote has ended
pub fn new_session_stop(client: DestinationAddressBytes) -> GatewayStatsEvent {
GatewayStatsEvent::SessionStatsEvent(SessionEvent::SessionStop {
stop_time: OffsetDateTime::now_utc(),
client,
})
}
/// A new ecash ticket has been added / requested
pub fn new_ecash_ticket(
client: DestinationAddressBytes,
ticket_type: TicketType,
) -> GatewayStatsEvent {
GatewayStatsEvent::SessionStatsEvent(SessionEvent::EcashTicket {
ticket_type,
client,
})
}
}
/// Events in the lifecycle of an established client tunnel
pub enum SessionEvent {
/// A new session between this gateway and the client remote has successfully opened
SessionStart {
/// The timestamp of the session open event
start_time: OffsetDateTime,
/// Address of the remote client opening the connection
client: DestinationAddressBytes,
},
/// An existing session with the client remote has ended
SessionStop {
/// Timestamp of the session end event
stop_time: OffsetDateTime,
/// Address of the remote client opening the connection
client: DestinationAddressBytes,
},
/// A new ecash ticket has been added / requested
EcashTicket {
/// Type of ecash ticket that has been created as part of the session
ticket_type: TicketType,
/// Address of the remote client opening the connection
client: DestinationAddressBytes,
},
}
+19 -2
View File
@@ -1,4 +1,21 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: GPL-3.0-only
// SPDX-License-Identifier: Apache-2.0
pub mod events;
//! Nym Statistics
//!
//! This crate contains basic statistics utilities and abstractions to be re-used and
//! applied throughout both the client and gateway implementations.
#![warn(clippy::expect_used)]
#![warn(clippy::unwrap_used)]
#![warn(clippy::todo)]
#![warn(clippy::dbg_macro)]
/// Client specific statistics interfaces and events.
pub mod clients;
/// Statistics related errors.
pub mod error;
/// Gateway specific statistics interfaces and events.
pub mod gateways;
/// Statistics reporting abstractions and implementations.
pub mod report;
+64
View File
@@ -0,0 +1,64 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::clients::{
gateway_conn_statistics::GatewayStats, nym_api_statistics::NymApiStats,
packet_statistics::PacketStatistics,
};
use super::error::StatsError;
use serde::{Deserialize, Serialize};
use sysinfo::System;
use time::OffsetDateTime;
/// Report object containing both data to be reported and client / device context. We take extra care not to overcapture context information.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct ClientStatsReport {
pub(crate) last_update_time: OffsetDateTime,
pub(crate) client_id: String,
pub(crate) client_type: String,
pub(crate) os_information: OsInformation,
pub(crate) packet_stats: PacketStatistics,
pub(crate) gateway_conn_stats: GatewayStats,
pub(crate) nym_api_stats: NymApiStats,
}
impl From<ClientStatsReport> for Vec<u8> {
fn from(value: ClientStatsReport) -> Self {
// safety, no custom serialisation
#[allow(clippy::unwrap_used)]
let report_json = serde_json::to_string(&value).unwrap();
report_json.as_bytes().to_vec()
}
}
impl TryFrom<&[u8]> for ClientStatsReport {
type Error = StatsError;
fn try_from(value: &[u8]) -> Result<Self, Self::Error> {
Ok(serde_json::from_slice(value)?)
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct OsInformation {
pub(crate) os_type: String,
pub(crate) os_version: Option<String>,
pub(crate) os_arch: Option<String>,
}
impl OsInformation {
pub fn new() -> Self {
OsInformation {
os_type: System::distribution_id(),
os_version: System::long_os_version(),
os_arch: System::cpu_arch(),
}
}
}
impl Default for OsInformation {
fn default() -> Self {
Self::new()
}
}
+35
View File
@@ -0,0 +1,35 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use std::{any::Any, fmt};
pub type SentStatus = Box<dyn TaskStatusEvent>;
pub type StatusSender = futures::channel::mpsc::Sender<SentStatus>;
pub type StatusReceiver = futures::channel::mpsc::Receiver<SentStatus>;
pub trait TaskStatusEvent: Send + Sync + Any + fmt::Display {
fn as_any(&self) -> &dyn Any;
}
#[derive(Debug, PartialEq, Eq)]
pub enum TaskStatus {
Ready,
ReadyWithGateway(String),
}
impl fmt::Display for TaskStatus {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
TaskStatus::Ready => write!(f, "Ready"),
TaskStatus::ReadyWithGateway(gateway) => {
write!(f, "Ready and connected to gateway: {gateway}")
}
}
}
}
impl TaskStatusEvent for TaskStatus {
fn as_any(&self) -> &dyn Any {
self
}
}
+3 -1
View File
@@ -2,12 +2,14 @@
// SPDX-License-Identifier: Apache-2.0
pub mod connections;
pub mod event;
pub mod manager;
#[cfg(not(target_arch = "wasm32"))]
pub mod signal;
pub mod spawn;
pub use manager::{StatusReceiver, StatusSender, TaskClient, TaskHandle, TaskManager};
pub use event::{StatusReceiver, StatusSender, TaskStatus, TaskStatusEvent};
pub use manager::{TaskClient, TaskHandle, TaskManager};
#[cfg(not(target_arch = "wasm32"))]
pub use signal::wait_for_signal_and_error;
+8 -15
View File
@@ -1,15 +1,21 @@
// Copyright 2022 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use std::{
error::Error,
sync::atomic::{AtomicBool, Ordering},
time::Duration,
};
use futures::{future::pending, FutureExt, SinkExt, StreamExt};
use log::{log, Level};
use std::sync::atomic::{AtomicBool, Ordering};
use std::{error::Error, time::Duration};
use tokio::sync::{
mpsc,
watch::{self, error::SendError},
};
use crate::event::{SentStatus, StatusReceiver, StatusSender, TaskStatus};
#[cfg(not(target_arch = "wasm32"))]
use tokio::time::{sleep, timeout};
@@ -22,10 +28,6 @@ pub(crate) type SentError = Box<dyn Error + Send + Sync>;
type ErrorSender = mpsc::UnboundedSender<SentError>;
type ErrorReceiver = mpsc::UnboundedReceiver<SentError>;
pub type SentStatus = Box<dyn Error + Send + Sync>;
pub type StatusSender = futures::channel::mpsc::Sender<SentStatus>;
pub type StatusReceiver = futures::channel::mpsc::Receiver<SentStatus>;
fn try_recover_name(name: &Option<String>) -> String {
if let Some(name) = name {
name.clone()
@@ -40,15 +42,6 @@ enum TaskError {
UnexpectedHalt { shutdown_name: Option<String> },
}
// TODO: possibly we should create a `Status` trait instead of reusing `Error`
#[derive(thiserror::Error, Debug, PartialEq, Eq)]
pub enum TaskStatus {
#[error("Ready")]
Ready,
#[error("Ready and connected to gateway: {0}")]
ReadyWithGateway(String),
}
/// Listens to status and error messages from tasks, as well as notifying them to gracefully
/// shutdown. Keeps track of if task stop unexpectedly, such as in a panic.
#[derive(Debug)]
+51 -2
View File
@@ -17,7 +17,7 @@ pub use nym_client_core::config::{
Acknowledgements as ConfigAcknowledgements, Config as BaseClientConfig,
CoverTraffic as ConfigCoverTraffic, DebugConfig as ConfigDebug,
GatewayConnection as ConfigGatewayConnection, ReplySurbs as ConfigReplySurbs,
Topology as ConfigTopology, Traffic as ConfigTraffic,
StatsReporting as ConfigStatsReporting, Topology as ConfigTopology, Traffic as ConfigTraffic,
};
pub fn new_base_client_config(
@@ -86,7 +86,7 @@ pub fn no_cover_debug_obj() -> JsValue {
// just a helper structure to more easily pass through the JS boundary
#[wasm_bindgen(inspectable)]
#[derive(Debug, Copy, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct DebugWasm {
/// Defines all configuration options related to traffic streams.
@@ -106,6 +106,10 @@ pub struct DebugWasm {
/// Defines all configuration options related to reply SURBs.
pub reply_surbs: ReplySurbsWasm,
/// Defines all configuration options related to stats reporting.
#[wasm_bindgen(getter_with_clone)]
pub stats_reporting: StatsReportingWasm,
}
impl Default for DebugWasm {
@@ -123,6 +127,7 @@ impl From<DebugWasm> for ConfigDebug {
acknowledgements: debug.acknowledgements.into(),
topology: debug.topology.into(),
reply_surbs: debug.reply_surbs.into(),
stats_reporting: debug.stats_reporting.into(),
}
}
}
@@ -136,6 +141,7 @@ impl From<ConfigDebug> for DebugWasm {
acknowledgements: debug.acknowledgements.into(),
topology: debug.topology.into(),
reply_surbs: debug.reply_surbs.into(),
stats_reporting: debug.stats_reporting.into(),
}
}
}
@@ -505,3 +511,46 @@ impl From<ConfigReplySurbs> for ReplySurbsWasm {
}
}
}
#[wasm_bindgen(inspectable)]
#[derive(Debug, Clone, Deserialize, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct StatsReportingWasm {
/// Is stats reporting enabled
pub enabled: bool,
/// Address of the stats collector. If this is none, no reporting will happen, regardless of `enabled`
#[wasm_bindgen(getter_with_clone)]
pub provider_address: Option<String>,
/// With what frequence will statistics be sent
pub reporting_interval_ms: u32,
}
impl Default for StatsReportingWasm {
fn default() -> Self {
ConfigStatsReporting::default().into()
}
}
impl From<StatsReportingWasm> for ConfigStatsReporting {
fn from(stats_reporting: StatsReportingWasm) -> Self {
ConfigStatsReporting {
enabled: stats_reporting.enabled,
provider_address: stats_reporting
.provider_address
.map(|address| address.parse().expect("Invalid provider address")),
reporting_interval: Duration::from_millis(stats_reporting.reporting_interval_ms as u64),
}
}
}
impl From<ConfigStatsReporting> for StatsReportingWasm {
fn from(stats_reporting: ConfigStatsReporting) -> Self {
StatsReportingWasm {
enabled: stats_reporting.enabled,
provider_address: stats_reporting.provider_address.map(|r| r.to_string()),
reporting_interval_ms: stats_reporting.reporting_interval.as_millis() as u32,
}
}
}
+38 -2
View File
@@ -9,14 +9,14 @@
use super::{
AcknowledgementsWasm, CoverTrafficWasm, DebugWasm, GatewayConnectionWasm, ReplySurbsWasm,
TopologyWasm, TrafficWasm,
StatsReportingWasm, TopologyWasm, TrafficWasm,
};
use crate::config::ConfigDebug;
use serde::{Deserialize, Serialize};
use tsify::Tsify;
// just a helper structure to more easily pass through the JS boundary
#[derive(Tsify, Debug, Copy, Clone, Serialize, Deserialize)]
#[derive(Tsify, Debug, Clone, Serialize, Deserialize)]
#[tsify(into_wasm_abi, from_wasm_abi)]
#[serde(rename_all = "camelCase")]
pub struct DebugWasmOverride {
@@ -43,6 +43,10 @@ pub struct DebugWasmOverride {
/// Defines all configuration options related to reply SURBs.
#[tsify(optional)]
pub reply_surbs: Option<ReplySurbsWasmOverride>,
/// Defines all configuration options related to stats reporting.
#[tsify(optional)]
pub stats_reporting: Option<StatsReportingWasmOverride>,
}
impl From<DebugWasmOverride> for DebugWasm {
@@ -54,6 +58,7 @@ impl From<DebugWasmOverride> for DebugWasm {
acknowledgements: value.acknowledgements.map(Into::into).unwrap_or_default(),
topology: value.topology.map(Into::into).unwrap_or_default(),
reply_surbs: value.reply_surbs.map(Into::into).unwrap_or_default(),
stats_reporting: value.stats_reporting.map(Into::into).unwrap_or_default(),
}
}
}
@@ -366,3 +371,34 @@ impl From<ReplySurbsWasmOverride> for ReplySurbsWasm {
}
}
}
#[derive(Tsify, Debug, Clone, Serialize, Deserialize)]
#[tsify(into_wasm_abi, from_wasm_abi)]
#[serde(rename_all = "camelCase")]
pub struct StatsReportingWasmOverride {
/// Is stats reporting enabled
#[tsify(optional)]
pub enabled: Option<bool>,
/// Address of the stats collector. If this is none, no reporting will happen, regardless of `enabled`
#[tsify(optional)]
pub provider_address: Option<Option<String>>,
/// With what frequence will statistics be sent
#[tsify(optional)]
pub reporting_interval_ms: Option<u32>,
}
impl From<StatsReportingWasmOverride> for StatsReportingWasm {
fn from(value: StatsReportingWasmOverride) -> Self {
let def = StatsReportingWasm::default();
StatsReportingWasm {
enabled: value.enabled.unwrap_or(def.enabled),
provider_address: value.provider_address.unwrap_or(def.provider_address),
reporting_interval_ms: value
.reporting_interval_ms
.unwrap_or(def.reporting_interval_ms),
}
}
}
+14 -6
View File
@@ -1,7 +1,7 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use std::net::{IpAddr, SocketAddr};
use std::net::{Ipv4Addr, Ipv6Addr, SocketAddr};
#[derive(Copy, Clone, PartialEq, Eq, Hash, PartialOrd, Ord, Debug)]
pub struct Config {
@@ -9,15 +9,23 @@ pub struct Config {
/// default: `0.0.0.0:51822`
pub bind_address: SocketAddr,
/// Private IP address of the wireguard gateway.
/// Private IPv4 address of the wireguard gateway.
/// default: `10.1.0.1`
pub private_ip: IpAddr,
pub private_ipv4: Ipv4Addr,
/// Private IPv6 address of the wireguard gateway.
/// default: `fc01::1`
pub private_ipv6: Ipv6Addr,
/// Port announced to external clients wishing to connect to the wireguard interface.
/// Useful in the instances where the node is behind a proxy.
pub announced_port: u16,
/// The prefix denoting the maximum number of the clients that can be connected via Wireguard.
/// The maximum value for IPv4 is 32 and for IPv6 is 128
pub private_network_prefix: u8,
/// The prefix denoting the maximum number of the clients that can be connected via Wireguard using IPv4.
/// The maximum value for IPv4 is 32
pub private_network_prefix_v4: u8,
/// The prefix denoting the maximum number of the clients that can be connected via Wireguard using IPv6.
/// The maximum value for IPv6 is 128
pub private_network_prefix_v6: u8,
}
+3
View File
@@ -23,6 +23,9 @@ futures = { workspace = true }
x25519-dalek = { workspace = true }
ip_network = { workspace = true }
log.workspace = true
serde = { workspace = true, features = [
"derive",
] } # for config serialization/deserialization
thiserror = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "net", "io-util"] }
tokio-stream = { workspace = true }

Some files were not shown because too many files have changed in this diff Show More