Compare commits

...

128 Commits

Author SHA1 Message Date
Tommy Verrall edf10499f6 use docker to optimise contracts 2024-05-23 10:41:47 +02:00
Tommy Verrall 76953df4bb Merge pull request #4605 from nymtech/feature/nym-node-api-location
Feature/nym node api location
2024-05-21 17:28:53 +01:00
Tommy Verrall ce2449f86a Merge pull request #4599 from nymtech/feature/expose-all-api-reports
nym-api: make report/avg_uptime endpoints ignore blacklist
2024-05-21 17:28:07 +01:00
Jędrzej Stuczyński 598014bf30 fixed deserialisation of 'AuxiliaryDetails' api response 2024-05-21 17:06:15 +01:00
Jędrzej Stuczyński 73fe7618db removing needless clones 2024-05-21 17:06:15 +01:00
Jędrzej Stuczyński 7a416f8cf5 aggregate new data with nym-api 2024-05-21 17:06:14 +01:00
Jędrzej Stuczyński 3077c2ea8d create '/api/v1/auxiliary-details' that exposes nodes location 2024-05-21 17:06:13 +01:00
Tommy Verrall be6c63723c Merge pull request #4553 from nymtech/jon/autoinherit
Run cargo autoinherit on the main workspace
2024-05-21 16:33:38 +01:00
Jon Häggblad c5a3cb7707 Update to handle changes in develop 2024-05-21 17:15:38 +02:00
Jon Häggblad cce9f0b183 Merge remote-tracking branch 'origin/develop' into jon/autoinherit 2024-05-21 17:10:02 +02:00
Tommy Verrall 84b74703b2 Merge pull request #4604 from nymtech/jon/ipr-request-sign
Add optional signature to IPR request/response
2024-05-21 15:01:43 +01:00
Tommy Verrall 9bf3600e5b Merge pull request #4601 from nymtech/feature/unstable-tested-nodes-endpoint
Feature/unstable tested nodes endpoint
2024-05-21 14:51:46 +01:00
Jędrzej Stuczyński 9f20c8ed1f Merge pull request #4598 from nymtech/feature/remove-coconut-wait-buffer
removed blocking for coconut in the final epoch state
2024-05-21 10:19:38 +01:00
Jędrzej Stuczyński 84e66c34f2 Merge pull request #4584 from nymtech/feature/wasm-coconut
Feature/wasm coconut
2024-05-21 10:08:40 +01:00
Jędrzej Stuczyński e04df37988 Merge pull request #4595 from nymtech/feature/freepass-admin
allow using explicit admin address for issuing freepasses
2024-05-21 10:07:44 +01:00
import this 0eb6eb855b [DOCs/operators]: Create a Nym ISP sheet, write gateway-probe guide & clean FnF docs (#4590)
* initialise nym versin of good-bad isps by tor community

* syntax edit

* finalise csv2md

* initialise isp-list page

* add last update info

* add python3 plugis to ci/cd runnners

* remove argparse installation

* python3 syntax edit

* pip install attempt

* pip install attempt

* change python version in cmdrun command

* fix python modules installation

* correct command python version

* workflows python env update

* add probe guide, clean FnF pages

* fix typos
2024-05-20 12:27:18 +00:00
Jon Häggblad c91412f949 Restore exact same serde import for common contract crates 2024-05-20 13:45:18 +02:00
Jon Häggblad 0a89f31a29 Merge in new deps in the main list 2024-05-20 13:19:47 +02:00
Jon Häggblad 9badeac832 cargo autoinherit in root workspace 2024-05-20 13:16:25 +02:00
Jon Häggblad b59c41d9cd Fix test 2024-05-20 12:24:39 +02:00
Jon Häggblad 8f083ff91e Add signatures to dynamic connect and disconect, and remove from response 2024-05-20 12:18:28 +02:00
Jon Häggblad 0f44836025 Move signature one layer down 2024-05-20 12:04:05 +02:00
Jon Häggblad 68ee2d747d Add timestamps to requests 2024-05-20 11:53:00 +02:00
Jon Häggblad e29c76678d Add optional signature field to requests and responses 2024-05-20 10:58:10 +02:00
Jon Häggblad a4005c7d81 Add v7 request/responses 2024-05-20 10:57:56 +02:00
Jędrzej Stuczyński efe6d916e2 order results by timestamp desc 2024-05-20 09:29:09 +01:00
Jon Häggblad a06ae48e2f Add methods to MixnetClient to sign messages (#4602) 2024-05-20 10:22:28 +02:00
Jędrzej Stuczyński 910b6a1369 adding explicit 'unstable' path segment 2024-05-19 10:27:33 +01:00
Jędrzej Stuczyński 7818658ee8 flatten the response 2024-05-17 18:36:50 +01:00
Jędrzej Stuczyński 89e34b4fd3 queries for detailed node statuses with broken pagination 2024-05-17 17:02:29 +01:00
Jędrzej Stuczyński 2f5a00dbda removed blocking for coconut in the final epoch state 2024-05-17 11:04:19 +01:00
Jędrzej Stuczyński 7f87d42f9a nym-api: make report/avg_uptime endpoints ignore blacklist 2024-05-17 11:01:16 +01:00
Fouad e5f41731ae Explorer NextJS Rebuild (#4534)
* bootstrap next app + add overview page

* fix AssetList type

* fix up nav stuff

* Refactor Nav component and add network components pages

* Refactor WorldMap component and update TelegramIcon, GitHubIcon, NymVpnIcon, DiscordIcon, and TwitterIcon components

* add service providers page

* mixnodes page

* delegations page + use material react table for all tables

* nodes map page

* Refactor StyledLink component and remove unnecessary console.log statements

* Refactor ESLint configuration, remove unused dependencies, and update component imports

* update deps

* Refactor imports and update dependencies

* fix dark mode

* build single mixnode page

* build single gateway page

* Refactor handleOnDelegate function to use useCallback in mixnodes page.tsx

* Add defaults for constants

---------

Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
2024-05-16 16:12:06 +01:00
Jędrzej Stuczyński a6fda391ae Feature/rewarder voucher issuance (#4548)
* retrieve ed25519 identities of issuers

* signature verification on issued credentials

* wip

* persisting information about verified deposits, any failures and foul plays

* clippy
2024-05-16 10:15:24 +02:00
Tommy Verrall 1ded24dcfc Merge pull request #4586 from nymtech/bugfix/nym-api-noop-nobanner
[bugfix] noop flag for nym-api for nymvisor compatibility
2024-05-15 14:07:30 +01:00
Tommy Verrall 8c42640853 Merge pull request #4591 from nymtech/jon/described-offsettime-parsing
Use rfc3339 for last_polled in described nym-api endpoint
2024-05-15 14:03:42 +01:00
Tommy Verrall 38aabc7983 Merge pull request #4593 from nymtech/feature/extend-max-freepass-validity
change maximum validity of issuable freepass
2024-05-15 13:40:49 +01:00
Tommy Verrall 4324845d29 Merge pull request #4596 from nymtech/update-contract-addr
update mainnet contract addresses
2024-05-15 10:07:08 +01:00
Jędrzej Stuczyński 93b12bccca updated the lock file after the rebasing fixes 2024-05-14 17:47:43 +01:00
Jędrzej Stuczyński 89fb4ef03f fixing more build issues 2024-05-14 17:45:17 +01:00
Jędrzej Stuczyński b8ab187db0 fixed wrong version of gloo-net and removed reduntant imports 2024-05-14 17:45:17 +01:00
Jędrzej Stuczyński a9790c1f66 removed duplicate 'set_panic_hook' symbol in the browser extension code 2024-05-14 17:45:17 +01:00
Jędrzej Stuczyński b46634b8f7 fixed test import paths 2024-05-14 17:45:17 +01:00
Jędrzej Stuczyński 633e7ffb46 no idea what happened here 2024-05-14 17:45:16 +01:00
Jędrzej Stuczyński dd2077bf12 updated the makefile 2024-05-14 17:45:16 +01:00
Jędrzej Stuczyński 0323ba2bb9 ability to obtain bandwidth voucher from wasm via vpn-api 2024-05-14 17:45:14 +01:00
Jędrzej Stuczyński b9524a0f58 Chore/additional helpers (#4585)
* exposed additional helpers

* changes lost in rebasing
2024-05-14 18:33:56 +02:00
Sachin Kamath e7cd417894 update mainnet contract addresses 2024-05-14 21:30:31 +05:30
Jędrzej Stuczyński 07cc47a0ff allow using explicit admin address for issuing freepasses 2024-05-14 16:58:11 +01:00
benedetta davico ca25db845a Merge pull request #4573 from nymtech/feature/axum-upgrade
upgraded axum and related deps to the most recent version
2024-05-14 17:38:15 +02:00
benedetta davico 64a0ce31a8 Merge branch 'develop' into feature/axum-upgrade 2024-05-14 17:21:44 +02:00
Jędrzej Stuczyński a8fe8d9bfb change maximum validity of issuable freepass 2024-05-14 14:35:53 +01:00
Jędrzej Stuczyński c346f145d1 backwards compatibility to fallback to default in case of failures 2024-05-14 10:15:16 +01:00
Jon Häggblad 45dd6f2632 Fix typo for StorageError::ConstraintUnique (#4592) 2024-05-13 17:48:58 +02:00
Jon Häggblad 22d28759ab Explicitly use rfc3339 for last_polled 2024-05-13 15:55:32 +02:00
Jędrzej Stuczyński 890d0f7440 fixed incorrect dependency path for 'axum-extra' 2024-05-13 14:18:33 +01:00
Jędrzej Stuczyński b342eb870e removed explicit drops 2024-05-13 11:31:34 +01:00
Jędrzej Stuczyński fc71e0cafd fixed tests 2024-05-13 11:31:33 +01:00
Tommy Verrall 1ecb57fda0 Merge pull request #4588 from nymtech/jon/explicitly-handle-sqlite-constraint-violation
Explicitly handle constraint unique violation when importing credential
2024-05-13 11:30:31 +01:00
Tommy Verrall 3c1ec82289 Merge pull request #4588 from nymtech/jon/explicitly-handle-sqlite-constraint-violation
Explicitly handle constraint unique violation when importing credential
2024-05-13 09:57:42 +01:00
Tommy Verrall 089e403d87 Merge pull request #4589 from nymtech/master
merge latest master to develop
2024-05-13 09:56:34 +01:00
Tommy Verrall dd2b477cda Merge pull request #4587 from nymtech/jon/lock-files
Update stale lock files
2024-05-13 09:54:52 +01:00
benedetta davico 0902539332 Merge pull request #4583 from nymtech/release/2024.4-nutella
Release 2024.4-nutella to master
2024-05-13 10:27:08 +02:00
Jon Häggblad 0783c532de Explicitly handle constraint unique violation when importing credential 2024-05-13 10:19:01 +02:00
Jon Häggblad 8817ae7805 Update stale lock files 2024-05-13 08:57:49 +02:00
Jędrzej Stuczyński 6a900c3c42 fixed linter issue in nyxd-scraper 2024-05-10 15:05:37 +01:00
Jędrzej Stuczyński 0ba80c9a86 moved startup 'Starting nym api...' message from stdout to stderr 2024-05-10 11:16:10 +01:00
Jędrzej Stuczyński d712b65ec5 [bugfix] noop flag for nym-api for nymvisor compatibility 2024-05-10 11:02:21 +01:00
Tommy Verrall 383b2c1351 Merge pull request #4552 from nymtech/jon/validator-client-rustls
Add rustls-tls to reqwest in validator-client
2024-05-08 16:51:46 +01:00
benedetta davico fe7484f0f4 Merge pull request #4564 from nymtech/feature/nyxd-scraper-pruning
Feature/nyxd scraper pruning
2024-05-08 11:08:06 +02:00
benedettadavico f0a4350e83 pruning
version bump and changelog updates
2024-05-08 11:01:50 +02:00
Jędrzej Stuczyński b63d04b10c Merge pull request #4574 from nymtech/feature/coconut-unchecked-aggregation
[feature]: expose coconut methods for aggregation without verification
2024-05-08 09:03:43 +01:00
Jędrzej Stuczyński 5a35068c87 fixing clippy issues in the workspace 2024-05-08 08:44:09 +01:00
Jędrzej Stuczyński 4899773e61 fixed unblind call in tests 2024-05-08 08:43:14 +01:00
Jędrzej Stuczyński 996f4afaf7 [feature]: expose coconut methods for aggregation without verification 2024-05-08 08:43:13 +01:00
import this d5c2a01a34 [DOCs/operators]: 2024.4 nutella release changelog & mixnode ipv6 clarification (#4581)
* edit IPv6 mixnode info

* add mixnode ipv6 info and nuttela changelog

* syntax edit
2024-05-07 15:49:29 +00:00
benedetta davico b1c58b36fe Merge pull request #4578 from nymtech/update-sign-nym-node
updating sign commands to include nym-node
2024-05-07 14:50:51 +02:00
benedettadavico dfbcc781db extra space.. 2024-05-07 14:36:43 +02:00
benedettadavico 5026960169 linting 2024-05-07 14:10:57 +02:00
benedetta davico 7c2710b61a Merge pull request #4579 from nymtech/bugfix/exit-poisson
[fix] apply disable_poisson_rate from internal NR/IPR cfgs
2024-05-07 14:09:44 +02:00
Jędrzej Stuczyński 0af807ac92 fixed overflow subtraction 2024-05-07 12:14:20 +01:00
Jędrzej Stuczyński bf9fc2d537 external clippy 2024-05-07 11:49:25 +01:00
Jędrzej Stuczyński 4182af9199 [fix] apply disable_poisson_rate from internal NR/IPR cfgs 2024-05-07 11:41:57 +01:00
benedettadavico 408d803344 adding both options 2024-05-07 12:34:26 +02:00
benedettadavico c2a5d6c035 updating sign commands to nym-node 2024-05-07 12:20:55 +02:00
Tommy Verrall 1136901daf Merge pull request #4572 from nymtech/bugfix/change-redirects
changed nym-node redirects from 308 'Permanent Redirect' to 303: 'See Other'
2024-05-07 09:29:41 +01:00
Tommy Verrall 593a1da0ff Merge pull request #4565 from nymtech/bugfix/delegations
Bug fix: wallet delegations list is empty when RPC node doesn't hold block
2024-05-07 09:07:04 +01:00
Tommy Verrall 9c17b7c269 Merge pull request #4571 from nymtech/operators/ipv6-troubleshooting
[DOC/operators]: More troubleshooting for IPv6 & install dependencies guides
2024-05-07 08:50:55 +01:00
serinko df398dbe05 add mixnode IPv6 setup 2024-05-06 13:00:51 +02:00
Tommy Verrall effd03b2f5 Merge pull request #4567 from nymtech/dependabot/npm_and_yarn/ejs-3.1.10
Bump ejs from 3.1.9 to 3.1.10
2024-05-06 11:49:46 +01:00
Sachin Kamath e00db6adb9 docs: fix links to archive (#4576) 2024-05-06 09:08:21 +00:00
Jędrzej Stuczyński fd207d4699 changed nym-node redirects from 308 'Permanent Redirect' to 303: 'See Other' 2024-05-03 15:43:49 +01:00
serinko b9126dfc0e add troubleshooting for IPv6 & install dependencies 2024-05-03 13:19:59 +02:00
Jon Häggblad 7bbe153b8f Add AuthenticationFailureWithPreexistingSharedKey and a few log statements (#4568) 2024-05-03 09:53:53 +02:00
import this 36e1e73ed2 [DOCs]/operators: Create changelog page & add more nym-node troubleshooting (#4570)
* add note to binary download

* initialise changelog page

* finalise changelog draft

* add local ID rename guide

* remove old id

* syntax edit

* syntax edit

* syntax edit
2024-05-02 19:11:02 +02:00
Mark Sinclair 6e23322ac4 Update nym-wallet/src/components/Delegation/DelegationList.tsx 2024-05-02 14:57:00 +01:00
Mark Sinclair 729eedc960 Update publish-nym-wallet-win10.yml 2024-05-02 13:45:51 +01:00
Mark Sinclair 025cbf5231 Update publish-nym-wallet-ubuntu.yml 2024-05-02 13:45:46 +01:00
Mark Sinclair 3db3959a74 Update publish-nym-wallet-macos.yml 2024-05-02 13:45:39 +01:00
fmtabbara 3ba83795d4 add error dialog 2024-05-02 11:22:34 +01:00
dependabot[bot] 39b01d10bd Bump ejs from 3.1.9 to 3.1.10
Bumps [ejs](https://github.com/mde/ejs) from 3.1.9 to 3.1.10.
- [Release notes](https://github.com/mde/ejs/releases)
- [Commits](https://github.com/mde/ejs/compare/v3.1.9...v3.1.10)

---
updated-dependencies:
- dependency-name: ejs
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 07:02:37 +00:00
Tommy Verrall f99bedd7e8 Merge pull request #4566 from nymtech/dependabot/npm_and_yarn/nym-wallet/webdriver/ejs-3.1.10
Bump ejs from 3.1.7 to 3.1.10 in /nym-wallet/webdriver
2024-05-02 08:01:12 +01:00
dependabot[bot] 7717bf5cf9 Bump ejs from 3.1.7 to 3.1.10 in /nym-wallet/webdriver
Bumps [ejs](https://github.com/mde/ejs) from 3.1.7 to 3.1.10.
- [Release notes](https://github.com/mde/ejs/releases)
- [Commits](https://github.com/mde/ejs/compare/v3.1.7...v3.1.10)

---
updated-dependencies:
- dependency-name: ejs
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 00:47:12 +00:00
fmtabbara 6060ce5fb8 fix error tooltip overflow 2024-05-01 23:38:06 +01:00
Mark Sinclair 8fbad9cad8 Remove test errors 2024-05-01 19:33:10 +01:00
Mark Sinclair 650865e59a Wallet delegations: slow refresh from 1 min to 5 mins 2024-05-01 18:02:34 +01:00
Mark Sinclair 08e580ec8b Wallet delegations list - add an error to each row, and display as a tooltip if present 2024-05-01 17:59:46 +01:00
Mark Sinclair ad86ec9315 Wallet delegations: add more error information 2024-05-01 15:59:45 +01:00
Mark Sinclair 53ab4c8ec9 Wallet delegations: tap errors in requests per delegation. Return an error with strings of all sub errors. 2024-05-01 15:24:29 +01:00
Jędrzej Stuczyński f827eb4242 storage pruning implementation + additional logging 2024-04-30 15:53:45 +01:00
Jon Häggblad 6f4b00b5c2 Add default-features = true for tungstenite and non-wasm in client-core 2024-04-30 14:14:44 +02:00
Jon Häggblad d681ad20cf Keep default features off for tungstenite and wasm 2024-04-30 14:14:44 +02:00
Jon Häggblad 5818d58caf tweak feature args to tungstenite 2024-04-30 14:14:42 +02:00
Jon Häggblad da4eab8fdb Add rustls-tls to reqwest in validator-client 2024-04-30 14:14:00 +02:00
Jędrzej Stuczyński 9323ca9339 defined basic pruning types 2024-04-30 12:32:46 +01:00
import this f34c9d5d28 [DOCs]: Get nym-vpn guides ready for launch (#4563)
* update nymvpn cli guide

* update nymvpn landing page

* update nymvpn landing page

* syntax edit

* final version for review

* final version for review

* fix on feedback

* fix on feedback
2024-04-29 15:07:51 +00:00
import this b93afe7756 add useful feedback and edits (#4562) 2024-04-26 17:22:24 +00:00
benedetta davico 140cd7d940 Merge pull request #4560 from nymtech/master
Master to develop
2024-04-25 12:59:58 +02:00
benedetta davico 7d233a4a2f Merge pull request #4559 from nymtech/release/2024.3-eclipse
Merge release/2024.3 eclipse to master
2024-04-25 11:27:18 +02:00
import this 5f60344c2b HOTFIX ci-docs (#4558)
* HOTFIX ci-docs

* add a flag to bonding
2024-04-24 15:09:51 +00:00
import this c53b46ee1d [DOCs]: HOTFix CI/CD GH runners (#4554)
* build on master first

* rename pages to unique names

* rename pages to unique names

* module updates

* rm old cmrdun and fix modules

* syntax fix

* comment out cmdrun

* comment out cmdrun

* feedback fix

* TEMP hack to build books

* adding sleep to build script

* adding test to mdbook script

* TEMP hack to build books

* add cmdrun back

* add cmdrun back

* final commit - ready to go
2024-04-24 15:00:56 +02:00
benedetta davico 7fc9eca46f Update publish-nym-binaries.yml 2024-04-24 11:28:04 +02:00
Jon Häggblad 4e5c765a0d Quickfix for unused warnings after manually disabling legacy routes for release (#4557) 2024-04-23 16:24:59 +02:00
Jon Häggblad e1abbc0b5b Disable legacy endpoints in mixnode http client (#4556) 2024-04-23 13:46:02 +02:00
import this ce067db401 [DOCs]: Guides for NYM-NODE (#4541)
* initialise nym node pages

* new TOC flow and archive section

* vps setup page draft

* syntax fix

* syntax fix

* syntax fix

* add ulimit setup

* initialise nym-node page

* add ports

* syntax edit

* syntax edit

* add flags

* add setup, run and bonding pages

* add info

* syntax edit

* add configuration page

* syntax fix

* syntax fix

* add vps troubleshooting

* add vps troubleshooting

* create nyx configuration page

* add quickflow

* add quickflow

* add quickflow

* add quickflow

* add quickflow

* add quickflow

* syntax fix

* syntax fix

* move vps setup to configuration

* syntax fix

* syntax fix

* syntax fix

* edit points

* finish nym-node setup.md page

* new connnectivity configuration - page finished

* finish proxy and landing page guide

* finish nymvisor update

* finish performance testing upgdate

* finish faq pages update

* fix troubleshooting

* fix troubleshooting

* fix manual-upgrade

* finish introduction page

* update introduction

* update summary

* correct all links

* update graphs

* update cmdruns

* syntax edit and spellcheck

* updated mdbook plugins

* mdbook update

* update modules

* remove redundant

* fix version cmdrun

* removed smoosh-faw to archive

* syntax fix

* attempt to update mdbook admonish

* try dirty workaround hack

* try dirty workaround hack

* try dirty workaround hack

* try dirty workaround hack

* try dirty workaround hack

* try dirty workaround hack

* try dirty workaround hack

* try dirty workaround hack

* try dirty workaround hack

* address review comments

* address review comments

* address review comments

* PR ready to merge
2024-04-23 09:14:27 +00:00
Jon Häggblad 373cc54f3f cargo update -p rustls@0.21.10 (#4551) 2024-04-22 16:29:19 +02:00
benedettadavico a276608fd0 updating versions and changelog 2024-04-22 15:44:16 +02:00
Jędrzej Stuczyński b332a6b556 attach 'last_polled' metadata to node descriptions (#4550) 2024-04-22 09:54:01 +02:00
422 changed files with 64815 additions and 9680 deletions
+22 -3
View File
@@ -9,7 +9,11 @@ jobs:
steps:
- uses: actions/checkout@v3
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get install -y build-essential curl wget libssl-dev libudev-dev squashfs-tools protobuf-compiler
run: sudo apt-get update && sudo apt-get install -y build-essential curl wget libssl-dev libudev-dev squashfs-tools protobuf-compiler git python3 && sudo apt-get update --fix-missing
- name: Install pip3
run: sudo apt install -y python3-pip
- name: Install Python3 modules
run: sudo pip3 install pandas tabulate
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
@@ -30,9 +34,24 @@ jobs:
- name: Remove existing Nym config directory (`~/.nym/`)
run: cd documentation && ./remove_existing_config.sh
continue-on-error: false
- name: Build all projects in documentation/ & move to ~/dist/docs/
# This is the original flow
# - name: Build all projects in documentation/ & move to ~/dist/docs/
# run: cd documentation && ./build_all_to_dist.sh
# This is a workaround replacement which builds on the last working commit b332a6b55668f60988e36961f3f62a794ba82ddb and then on current branch
- name: Save current branch to ~/current_branch
run: git rev-parse --abbrev-ref HEAD > ~/current_branch
- name: Git pull & switch to b332a6b55668f60988e36961f3f62a794ba82ddb
run: git pull && git checkout b332a6b55668f60988e36961f3f62a794ba82ddb
- name: Build all projects in documentation/ & move to ~/dist/docs/ from b332a6b55668f60988e36961f3f62a794ba82ddb
run: cd documentation && ./build_all_to_dist.sh
continue-on-error: false
- name: Switch to current branch
run: git checkout $echo "$(cat ~/current_branch)"
- name: Build all projects in documentation/ & move to ~/dist/docs/ on current branch
run: cd documentation && ./build_all_to_dist.sh && rm ~/current_branch
# End of replacemet
- name: Post process
run: cd documentation && ./post_process.sh
@@ -35,7 +35,7 @@ jobs:
- name: Install Rust stable
uses: actions-rs/toolchain@v1
with:
toolchain: stable
toolchain: 1.77.0
target: wasm32-unknown-unknown
override: true
@@ -47,6 +47,17 @@ jobs:
- name: Build release contracts
run: make contracts
- name: Install CosmWasm optimizer
run: |
sudo apt-get update
sudo apt-get install -y docker.io
- name: Optimize WASM contracts
run: |
docker run --rm -v $(pwd)/contracts/target/wasm32-unknown-unknown/release:/code \
--mount type=volume,source=cosmwasm_cache,target=/code/cache \
cosmwasm/workspace-optimizer:0.12.9
- name: Prepare build output
shell: bash
env:
@@ -58,8 +69,6 @@ jobs:
cp contracts/target/wasm32-unknown-unknown/release/nym_coconut_dkg.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw3_flex_multisig.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/cw4_group.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_service_provider_directory.wasm $OUTPUT_DIR
cp contracts/target/wasm32-unknown-unknown/release/nym_name_service.wasm $OUTPUT_DIR
- name: Deploy branch to CI www
continue-on-error: true
+23 -3
View File
@@ -13,7 +13,11 @@ jobs:
steps:
- uses: actions/checkout@v3
- name: Install Dependencies (Linux)
run: sudo apt-get update && sudo apt-get install -y build-essential curl wget libssl-dev libudev-dev squashfs-tools protobuf-compiler
run: sudo apt-get update && sudo apt-get install -y build-essential curl wget libssl-dev libudev-dev squashfs-tools protobuf-compiler git python3 && sudo apt-get update --fix-missing
- name: Install pip3
run: sudo apt install -y python3-pip
- name: Install Python3 modules
run: sudo pip3 install pandas tabulate
- name: Install rsync
run: sudo apt-get install rsync
- uses: rlespinasse/github-slug-action@v3.x
@@ -34,9 +38,25 @@ jobs:
- name: Remove existing Nym config directory (`~/.nym/`)
run: cd documentation && ./remove_existing_config.sh
continue-on-error: false
- name: Build all projects in documentation/ & move to ~/dist/docs/
# This is the original flow
# - name: Build all projects in documentation/ & move to ~/dist/docs/
# run: cd documentation && ./build_all_to_dist.sh
# This is a workaround replacement which builds on the last working commit b332a6b55668f60988e36961f3f62a794ba82ddb and then on current branch
- name: Save current branch to ~/current_branch
run: git rev-parse --abbrev-ref HEAD > ~/current_branch
- name: Git pull & switch to b332a6b55668f60988e36961f3f62a794ba82ddb
run: git pull && git checkout b332a6b55668f60988e36961f3f62a794ba82ddb
- name: Build all projects in documentation/ & move to ~/dist/docs/ from b332a6b55668f60988e36961f3f62a794ba82ddb
run: cd documentation && ./build_all_to_dist.sh
continue-on-error: false
- name: Switch to current branch
run: git checkout $echo "$(cat ~/current_branch)"
- name: Build all projects in documentation/ & move to ~/dist/docs/ on current branch
run: cd documentation && ./build_all_to_dist.sh && rm ~/current_branch
# End of replacemet
- name: Deploy branch to CI www
continue-on-error: true
@@ -30,6 +30,7 @@ jobs:
mixnode_hash: ${{ steps.binary-hashes.outputs.mixnode_hash }}
gateway_hash: ${{ steps.binary-hashes.outputs.gateway_hash }}
nymvisor_hash: ${{ steps.binary-hashes.outputs.nymvisor_hash }}
nymnode_hash: ${{ steps.binary-hashes.outputs.nymnode_hash }}
socks5_hash: ${{ steps.binary-hashes.outputs.socks5_hash }}
netreq_hash: ${{ steps.binary-hashes.outputs.netreq_hash }}
cli_hash: ${{ steps.binary-hashes.outputs.cli_hash }}
@@ -38,6 +39,7 @@ jobs:
mixnode_version: ${{ steps.binary-versions.outputs.mixnode_version }}
gateway_version: ${{ steps.binary-versions.outputs.gateway_version }}
nymvisor_version: ${{ steps.binary-versions.outputs.nymvisor_version }}
nymnode_version: ${{ steps.binary-versions.outputs.nymnode_version }}
socks5_version: ${{ steps.binary-versions.outputs.socks5_version }}
netreq_version: ${{ steps.binary-versions.outputs.netreq_version }}
cli_version: ${{ steps.binary-versions.outputs.cli_version }}
@@ -81,6 +83,7 @@ jobs:
target/release/nym-network-statistics
target/release/nym-cli
target/release/nymvisor
target/release/nym-node
retention-days: 30
- id: create-release
@@ -99,6 +102,7 @@ jobs:
target/release/nym-network-statistics
target/release/nym-cli
target/release/nymvisor
target/release/nym-node
push-release-data-client:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-binaries-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
@@ -102,6 +102,18 @@ jobs:
nym-wallet/target/release/bundle/dmg/*.dmg
nym-wallet/target/release/bundle/macos/*.app.tar.gz*
- name: Deploy artifacts to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-avzr"
SOURCE: "nym-wallet/target/release/bundle/macos/nym-wallet.app.tar.gz"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/builds/${{ github.ref_name }}/nym-wallet
EXCLUDE: "/dist/, /node_modules/"
push-release-data:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
uses: ./.github/workflows/release-calculate-hash.yml
@@ -77,6 +77,18 @@ jobs:
nym-wallet/target/release/bundle/appimage/*.AppImage
nym-wallet/target/release/bundle/appimage/*.AppImage.tar.gz*
- name: Deploy artifacts to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-avzr"
SOURCE: "nym-wallet/target/release/bundle/appimage/nym-wallet*.AppImage.tar.gz"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/builds/${{ github.ref_name }}/nym-wallet
EXCLUDE: "/dist/, /node_modules/"
push-release-data:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
uses: ./.github/workflows/release-calculate-hash.yml
@@ -97,6 +97,18 @@ jobs:
nym-wallet/target/release/bundle/msi/*.msi
nym-wallet/target/release/bundle/msi/*.msi.zip*
- name: Deploy artifacts to CI www
continue-on-error: true
uses: easingthemes/ssh-deploy@main
env:
SSH_PRIVATE_KEY: ${{ secrets.CI_WWW_SSH_PRIVATE_KEY }}
ARGS: "-avzr"
SOURCE: "nym-wallet/target/release/bundle/msi/nym-wallet_1.*.msi"
REMOTE_HOST: ${{ secrets.CI_WWW_REMOTE_HOST }}
REMOTE_USER: ${{ secrets.CI_WWW_REMOTE_USER }}
TARGET: ${{ secrets.CI_WWW_REMOTE_TARGET }}/builds/${{ github.ref_name }}/nym-wallet
EXCLUDE: "/dist/, /node_modules/"
push-release-data:
if: ${{ (startsWith(github.ref, 'refs/tags/nym-wallet-') && github.event_name == 'release') || github.event_name == 'workflow_dispatch' }}
uses: ./.github/workflows/release-calculate-hash.yml
+24
View File
@@ -4,6 +4,30 @@ Post 1.0.0 release, the changelog format is based on [Keep a Changelog](https://
## [Unreleased]
## [2024.4-nutella] (2024-05-08)
- [fix] apply disable_poisson_rate from internal NR/IPR cfgs ([#4579])
- updating sign commands to include nym-node ([#4578])
- changed nym-node redirects from 308 'Permanent Redirect' to 303: 'See Other' ([#4572])
[#4579]: https://github.com/nymtech/nym/pull/4579
[#4578]: https://github.com/nymtech/nym/pull/4578
[#4572]: https://github.com/nymtech/nym/pull/4572
## [2024.3-eclipse] (2024-04-22)
- Initial release of the first iteration of the Nym Node
- Improvements to gateway functionality
- IPR development
- Removal of allow list in favour of implementing an exit policy
- Explorer delegation: enables direct delegation to nodes via the Nym Explorer
## [2024.2-fast-and-furious] (2024-03-25)
- Internal testing pre-release
## [2024.1-marabou] (2024-02-15)
**New Features:**
Generated
+1356 -3147
View File
File diff suppressed because it is too large Load Diff
+117 -17
View File
@@ -122,6 +122,7 @@ members = [
# "wasm/full-nym-wasm",
"wasm/mix-fetch",
"wasm/node-tester",
"wasm/zknym-lib",
]
default-members = [
@@ -158,44 +159,142 @@ edition = "2021"
license = "Apache-2.0"
[workspace.dependencies]
addr = "0.15.6"
aes = "0.8.1"
aes-gcm = "0.10.1"
anyhow = "1.0.71"
argon2 = "0.5.0"
async-trait = "0.1.68"
axum = "0.6.20"
axum = "0.7.5"
axum-extra = "0.9.3"
base64 = "0.21.4"
bs58 = "0.5.0"
bincode = "1.3.3"
bip39 = { version = "2.0.0", features = ["zeroize"] }
clap = "4.4.7"
bitvec = "1.0.0"
blake3 = "1.3.1"
bs58 = "0.5.0"
bytecodec = "0.4.15"
bytes = "1.5.0"
cargo_metadata = "0.18.1"
celes = "2.4.0"
cfg-if = "1.0.0"
chacha20 = "0.9.0"
chacha20poly1305 = "0.10.1"
chrono = "0.4.31"
cipher = "0.4.3"
clap = "4.4.7"
clap_complete = "4.0"
clap_complete_fig = "4.0"
colored = "2.0"
comfy-table = "6.0.0"
console-subscriber = "0.1.1"
console_error_panic_hook = "0.1"
const-str = "0.5.6"
const_format = "0.2.32"
criterion = "0.4"
csv = "1.3.0"
ctr = "0.9.1"
cupid = "0.6.1"
curve25519-dalek = "3.2"
dashmap = "5.5.3"
defguard_wireguard_rs = { git = "https://github.com/neacsu/wireguard-rs.git", rev = "c2cd0c1119f699f4bc43f5e6ffd6fc242caa42ed" }
doc-comment = "0.3"
dotenvy = "0.15.6"
ecdsa = "0.16"
ed25519-dalek = "1.0"
etherparse = "0.13.0"
eyre = "0.6.9"
flate2 = "1.0.28"
futures = "0.3.28"
generic-array = "0.14.7"
getrandom = "0.2.10"
getset = "0.1.1"
handlebars = "3.5.5"
headers = "0.4.0"
hex = "0.4.3"
hex-literal = "0.3.3"
hkdf = "0.12.3"
hmac = "0.12.1"
httpcodec = "0.2.3"
humantime = "2.1.0"
humantime-serde = "1.1.1"
hyper = "0.14.27"
http = "1"
hyper = "1.3.1"
indexed_db_futures = "0.3.0"
inquire = "0.6.2"
ip_network = "0.4.1"
isocountry = "0.3.2"
k256 = "0.13"
lazy_static = "1.4.0"
ledger-transport = "0.10.0"
ledger-transport-hid = "0.10.0"
log = "0.4"
maxminddb = "0.23.0"
mime = "0.3.17"
nix = "0.27.1"
notify = "5.1.0"
okapi = "0.7.0"
once_cell = "1.7.2"
opentelemetry = "0.19.0"
opentelemetry-jaeger = "0.18.0"
parking_lot = "0.12.1"
pem = "0.8"
pin-project = "1.0"
pretty_env_logger = "0.4.0"
publicsuffix = "2.2.3"
quote = "1"
rand = "0.8.5"
reqwest = { version = "0.11.22", default-features = false }
rand-07 = "0.7.3"
rand_chacha_02 = "0.2"
rand_core = "0.6.3"
rand_distr = "0.3"
rand_pcg = "0.3.1"
rand_seeder = "0.2.3"
rayon = "1.5.1"
regex = "1.8.4"
reqwest = { version = "0.12.4", default-features = false }
rocket = "0.5.0"
rocket_cors = "0.6.0"
rocket_okapi = "0.8.0"
safer-ffi = "0.1.4"
schemars = "0.8.1"
serde = "1.0.152"
serde_bytes = "0.11.6"
serde_derive = "1.0"
serde_json = "1.0.91"
serde_repr = "0.1"
serde_with = "3.4.0"
serde_yaml = "0.9.25"
si-scale = "0.2.2"
sphinx-packet = "0.1.0"
sqlx = "0.6.3"
strum = "0.25"
subtle-encoding = "0.5"
syn = "1"
tap = "1.0.1"
time = "0.3.30"
tar = "0.4.40"
tempfile = "3.5.0"
thiserror = "1.0.48"
time = "0.3.30"
tokio = "1.33.0"
tokio-util = "0.7.10"
tokio-stream = "0.1.14"
tokio-test = "0.4.2"
tokio-tungstenite = { version = "0.20.1" }
tokio-util = "0.7.10"
tower = "0.4.13"
tower-http = "0.5.2"
tracing = "0.1.37"
tungstenite = { version = "0.20.1", default-features = false }
tracing-opentelemetry = "0.19.0"
tracing-subscriber = "0.3.16"
tracing-tree = "0.2.2"
ts-rs = "7.0.0"
utoipa = "3.5.0"
utoipa-swagger-ui = "3.1.5"
tungstenite = { version = "0.20.1", default-features = false }
url = "2.4"
utoipa = "4.2.0"
utoipa-swagger-ui = "6.0.0"
vergen = { version = "=8.2.6", default-features = false }
walkdir = "2"
wasm-bindgen-test = "0.3.36"
zeroize = "1.6.0"
prometheus = { version = "0.13.0" }
@@ -207,7 +306,6 @@ bls12_381 = { git = "https://github.com/jstuczyn/bls12_381", branch = "feature/g
group = "0.13.0"
ff = "0.13.0"
# cosmwasm-related
cosmwasm-derive = "=1.4.3"
cosmwasm-schema = "=1.4.3"
@@ -235,14 +333,16 @@ tendermint-rpc = "0.34" # same version as used by cosmrs
prost = "0.12"
# wasm-related dependencies
gloo-utils = "0.1.7"
js-sys = "0.3.63"
serde-wasm-bindgen = "0.5.0"
gloo-utils = "0.2.0"
gloo-net = "0.5.0"
js-sys = "0.3.69"
serde-wasm-bindgen = "0.6.5"
tsify = "0.4.5"
wasm-bindgen = "0.2.86"
wasm-bindgen-futures = "0.4.37"
wasm-bindgen = "0.2.92"
wasm-bindgen-futures = "0.4.39"
wasmtimer = "0.2.0"
web-sys = "0.3.63"
web-sys = "0.3.69"
# Profile settings for individual crates
+2 -1
View File
@@ -105,6 +105,7 @@ sdk-wasm-build:
$(MAKE) -C wasm/client
$(MAKE) -C wasm/node-tester
$(MAKE) -C wasm/mix-fetch
$(MAKE) -C wasm/zknym-lib
#$(MAKE) -C wasm/full-nym-wasm
# run this from npm/yarn to ensure tools are in the path, e.g. yarn build:sdk from root of repo
@@ -115,7 +116,7 @@ sdk-typescript-build:
yarn --cwd sdk/typescript/codegen/contract-clients build
# NOTE: These targets are part of the main workspace (but not as wasm32-unknown-unknown)
WASM_CRATES = extension-storage nym-client-wasm nym-node-tester-wasm
WASM_CRATES = extension-storage nym-client-wasm nym-node-tester-wasm zknym-lib
sdk-wasm-test:
#cargo test $(addprefix -p , $(WASM_CRATES)) --target wasm32-unknown-unknown -- -Dwarnings
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-client"
version = "1.1.33"
version = "1.1.34"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
description = "Implementation of the Nym Client"
edition = "2021"
@@ -29,7 +29,7 @@ rand = { version = "0.7.3", features = ["wasm-bindgen"] } # rng-related traits +
serde = { workspace = true, features = ["derive"] } # for config serialization/deserialization
serde_json = { workspace = true }
thiserror = { workspace = true }
tap = "1.0.1"
tap = { workspace = true }
time = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "net", "signal"] } # async runtime
tokio-tungstenite = { workspace = true }
+3 -3
View File
@@ -1,6 +1,6 @@
[package]
name = "nym-socks5-client"
version = "1.1.33"
version = "1.1.34"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
description = "A SOCKS5 localhost proxy that converts incoming messages to Sphinx and sends them to a Nym address"
edition = "2021"
@@ -13,9 +13,9 @@ clap = { workspace = true, features = ["cargo", "derive"] }
log = { workspace = true }
serde = { workspace = true, features = ["derive"] } # for config serialization/deserialization
serde_json = { workspace = true }
tap = "1.0.1"
tap = { workspace = true }
thiserror = { workspace = true }
tokio = { version = "1.24.1", features = ["rt-multi-thread", "net", "signal"] }
tokio = { workspace = true, features = ["rt-multi-thread", "net", "signal"] }
rand = "0.7.3"
time = { workspace = true }
url = { workspace = true }
+2 -2
View File
@@ -8,6 +8,6 @@ license.workspace = true
[dependencies]
futures = { workspace = true }
log = "0.4"
notify = "5.1.0"
log = { workspace = true }
notify = { workspace = true }
tokio = { workspace = true, features = ["time"] }
+10 -22
View File
@@ -8,40 +8,28 @@ license = { workspace = true }
repository = { workspace = true }
[dependencies]
const-str = "0.5.6"
const-str = { workspace = true }
clap = { workspace = true, features = ["derive"] }
clap_complete = "4.0"
clap_complete_fig = "4.0"
clap_complete = { workspace = true }
clap_complete_fig = { workspace = true }
log = { workspace = true }
pretty_env_logger = "0.4.0"
pretty_env_logger = { workspace = true }
semver = "0.11"
schemars = { workspace = true, features = ["preserve_order"], optional = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true, optional = true }
## tracing
tracing-subscriber = { version = "0.3.16", features = [
"env-filter",
], optional = true }
tracing-tree = { version = "0.2.2", optional = true }
opentelemetry-jaeger = { version = "0.18.0", optional = true, features = [
"rt-tokio",
"collector_client",
"isahc_collector_client",
] }
tracing-opentelemetry = { version = "0.19.0", optional = true }
tracing-subscriber = { workspace = true, features = ["env-filter"], optional = true }
tracing-tree = { workspace = true, optional = true }
opentelemetry-jaeger = { workspace = true, features = ["rt-tokio", "collector_client", "isahc_collector_client"], optional = true }
tracing-opentelemetry = { workspace = true, optional = true }
utoipa = { workspace = true, optional = true }
opentelemetry = { version = "0.19.0", optional = true, features = ["rt-tokio"] }
opentelemetry = { workspace = true, features = ["rt-tokio"], optional = true }
[build-dependencies]
vergen = { version = "=8.2.6", default-features = false, features = [
"build",
"git",
"gitcl",
"rustc",
"cargo",
] }
vergen = { workspace = true, features = ["build", "git", "gitcl", "rustc", "cargo"] }
[features]
default = []
+17 -9
View File
@@ -3,7 +3,7 @@ name = "nym-client-core"
version = "1.1.15"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
edition = "2021"
rust-version = "1.66"
rust-version = "1.70"
license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -12,7 +12,7 @@ license.workspace = true
async-trait = { workspace = true }
base64 = "0.21.2"
bs58 = { workspace = true }
cfg-if = "1.0.0"
cfg-if = { workspace = true }
clap = { workspace = true, optional = true }
futures = { workspace = true }
humantime-serde = { workspace = true }
@@ -21,11 +21,10 @@ rand = { version = "0.7.3", features = ["wasm-bindgen"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
sha2 = "0.10.6"
si-scale = "0.2.2"
tap = "1.0.1"
si-scale = { workspace = true }
tap = { workspace = true }
thiserror = { workspace = true }
url = { workspace = true, features = ["serde"] }
tungstenite = { workspace = true, default-features = false }
tokio = { workspace = true, features = ["macros"] }
time = { workspace = true }
zeroize = { workspace = true }
@@ -48,7 +47,7 @@ nym-validator-client = { path = "../client-libs/validator-client", default-featu
nym-task = { path = "../task" }
nym-credential-storage = { path = "../credential-storage" }
nym-network-defaults = { path = "../network-defaults" }
nym-client-core-config-types = { path = "./config-types", features = ["disk-persistence"]}
nym-client-core-config-types = { path = "./config-types", features = ["disk-persistence"] }
nym-client-core-surb-storage = { path = "./surb-storage" }
nym-client-core-gateways-storage = { path = "./gateways-storage" }
@@ -74,8 +73,17 @@ workspace = true
features = ["time"]
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio-tungstenite]
version = "0.20.1"
features = ["rustls-tls-native-roots"]
workspace = true
features = ["rustls-tls-webpki-roots"]
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tungstenite]
workspace = true
default-features = true
features = ["rustls-tls-webpki-roots"]
[target."cfg(target_arch = \"wasm32\")".dependencies.tungstenite]
workspace = true
default-features = false
[target."cfg(target_arch = \"wasm32\")".dependencies.wasm-bindgen-futures]
workspace = true
@@ -100,7 +108,7 @@ version = "0.3.17"
features = ["wasm-bindgen"]
[dev-dependencies]
tempfile = "3.1.0"
tempfile = { workspace = true }
[features]
default = []
@@ -8,3 +8,12 @@ use thiserror::Error;
pub struct ConfigUpgradeFailure {
pub current_version: String,
}
#[derive(Error, Debug)]
pub enum InvalidTrafficModeFailure {
#[error("attempted to set medium toggle traffic mode with fast mode flag")]
MediumToggleWithFastMode,
#[error("attempted to set medium toggle traffic mode with no cover flag")]
MediumToggleWithNoCover,
}
@@ -56,6 +56,7 @@ const DEFAULT_MAXIMUM_REPLY_SURB_AGE: Duration = Duration::from_secs(12 * 60 * 6
// 24 hours
const DEFAULT_MAXIMUM_REPLY_KEY_AGE: Duration = Duration::from_secs(24 * 60 * 60);
use crate::error::InvalidTrafficModeFailure;
pub use nym_country_group::CountryGroup;
#[derive(Debug, Clone, Deserialize, PartialEq, Serialize)]
@@ -127,6 +128,56 @@ impl Config {
self
}
// TODO: this should be refactored properly
// as of 12.09.23 the below is true (not sure how this comment will rot in the future)
// medium_toggle:
// - sets secondary packet size to 16kb
// - disables poisson distribution of the main traffic stream
// - sets the cover traffic stream to 1 packet / 5s (on average)
// - disables per hop delay
//
// fastmode (to be renamed to `fast-poisson`):
// - sets average per hop delay to 10ms
// - sets the cover traffic stream to 1 packet / 2000s (on average); for all intents and purposes it disables the stream
// - sets the poisson distribution of the main traffic stream to 4ms, i.e. 250 packets / s on average
//
// no_cover:
// - disables poisson distribution of the main traffic stream
// - disables the secondary cover traffic stream
#[doc(hidden)]
pub fn try_apply_traffic_modes(
&mut self,
disable_poisson_process: bool,
medium_toggle: bool,
fast_mode: bool,
no_cover: bool,
) -> Result<(), InvalidTrafficModeFailure> {
if disable_poisson_process {
self.set_no_poisson_process()
}
if medium_toggle {
if fast_mode {
return Err(InvalidTrafficModeFailure::MediumToggleWithFastMode);
}
if no_cover {
return Err(InvalidTrafficModeFailure::MediumToggleWithNoCover);
}
self.set_experimental_medium_toggle();
}
if fast_mode {
self.set_high_default_traffic_volume()
}
if no_cover {
self.set_no_cover_traffic();
}
Ok(())
}
pub fn set_high_default_traffic_volume(&mut self) {
self.debug.traffic.average_packet_delay = Duration::from_millis(10);
// basically don't really send cover messages
@@ -136,6 +187,15 @@ impl Config {
self.debug.traffic.message_sending_average_delay = Duration::from_millis(4);
}
/// Enable medium mixnet traffic, for experiments only.
/// This includes things like disabling cover traffic, no per hop delays, etc.
#[doc(hidden)]
pub fn set_experimental_medium_toggle(&mut self) {
self.set_no_cover_traffic_with_keepalive();
self.set_no_per_hop_delays();
self.debug.traffic.secondary_packet_size = Some(PacketSize::ExtendedPacket16);
}
pub fn with_disabled_poisson_process(mut self, disabled: bool) -> Self {
if disabled {
self.set_no_poisson_process()
@@ -39,7 +39,7 @@ use log::{debug, error, info, warn};
use nym_bandwidth_controller::BandwidthController;
use nym_client_core_gateways_storage::{GatewayDetails, GatewaysDetailsStore};
use nym_credential_storage::storage::Storage as CredentialStorage;
use nym_crypto::asymmetric::encryption;
use nym_crypto::asymmetric::{encryption, identity};
use nym_gateway_client::{
AcknowledgementReceiver, GatewayClient, GatewayConfig, MixnetMessageReceiver, PacketRouter,
};
@@ -670,6 +670,7 @@ where
let self_address = Self::mix_address(&init_res);
let ack_key = init_res.client_keys.ack_key();
let encryption_keys = init_res.client_keys.encryption_keypair();
let identity_keys = init_res.client_keys.identity_keypair();
// the components are started in very specific order. Unless you know what you are doing,
// do not change that.
@@ -792,6 +793,7 @@ where
Ok(BaseClient {
address: self_address,
identity_keys,
client_input: ClientInputStatus::AwaitingProducer {
client_input: ClientInput {
connection_command_sender: client_connection_tx,
@@ -816,6 +818,7 @@ where
pub struct BaseClient {
pub address: Recipient,
pub identity_keys: Arc<identity::KeyPair>,
pub client_input: ClientInputStatus,
pub client_output: ClientOutputStatus,
pub client_state: ClientState,
+3 -6
View File
@@ -15,8 +15,8 @@ log = { workspace = true }
thiserror = { workspace = true }
url = { workspace = true }
rand = { version = "0.7.3", features = ["wasm-bindgen"] }
tokio = { version = "1.24.1", features = ["macros"] }
si-scale = "0.2.2"
tokio = { workspace = true, features = ["macros"] }
si-scale = { workspace = true }
time.workspace = true
# internal
@@ -48,10 +48,7 @@ features = ["net", "sync", "time"]
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio-tungstenite]
workspace = true
# the choice of this particular tls feature was arbitrary;
# if you reckon a different one would be more appropriate, feel free to change it
# features = ["native-tls"]
features = ["rustls-tls-native-roots"]
features = ["rustls-tls-webpki-roots"]
# wasm-only dependencies
[target."cfg(target_arch = \"wasm32\")".dependencies.wasm-bindgen]
@@ -442,7 +442,7 @@ impl<C, St> GatewayClient<C, St> {
}
debug_assert!(self.connection.is_available());
log::trace!("Registering gateway");
log::debug!("Registering gateway");
// it's fine to instantiate it here as it's only used once (during authentication or registration)
// and putting it into the GatewayClient struct would be a hassle
@@ -494,6 +494,7 @@ impl<C, St> GatewayClient<C, St> {
if !self.connection.is_established() {
return Err(GatewayClientError::ConnectionNotEstablished);
}
log::debug!("Authenticating with gateway");
// it's fine to instantiate it here as it's only used once (during authentication or registration)
// and putting it into the GatewayClient struct would be a hassle
@@ -529,6 +530,7 @@ impl<C, St> GatewayClient<C, St> {
self.authenticated = status;
self.bandwidth_remaining = bandwidth_remaining;
self.negotiated_protocol = protocol_version;
log::debug!("authenticated: {status}, bandwidth remaining: {bandwidth_remaining}");
Ok(())
}
ServerResponse::Error { message } => Err(GatewayClientError::GatewayError(message)),
@@ -541,10 +543,11 @@ impl<C, St> GatewayClient<C, St> {
&mut self,
) -> Result<Arc<SharedKeys>, GatewayClientError> {
if self.authenticated {
debug!("Already authenticated");
return if let Some(shared_key) = &self.shared_key {
Ok(Arc::clone(shared_key))
} else {
Err(GatewayClientError::AuthenticationFailure)
Err(GatewayClientError::AuthenticationFailureWithPreexistingSharedKey)
};
}
@@ -71,6 +71,9 @@ pub enum GatewayClientError {
#[error("Authentication failure")]
AuthenticationFailure,
#[error("Authentication failure with preexisting shared key")]
AuthenticationFailureWithPreexistingSharedKey,
#[error("Timed out")]
Timeout,
+11 -4
View File
@@ -10,7 +10,7 @@ license.workspace = true
[dependencies]
base64 = "0.13"
colored = "2.0"
colored = { workspace = true }
nym-coconut-dkg-common = { path = "../../cosmwasm-smart-contracts/coconut-dkg" }
nym-contracts-common = { path = "../../cosmwasm-smart-contracts/contracts-common" }
@@ -24,7 +24,6 @@ nym-group-contract-common = { path = "../../cosmwasm-smart-contracts/group-contr
nym-service-provider-directory-common = { path = "../../cosmwasm-smart-contracts/service-provider-directory" }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
reqwest = { workspace = true, features = ["json"] }
nym-http-api-client = { path = "../../../common/http-api-client"}
thiserror = { workspace = true }
log = { workspace = true }
@@ -49,14 +48,14 @@ tendermint-rpc = { workspace = true }
# of cargo's feature unification we'd get `bip32/std` meaning we'd get `std::error::Error` for the re-exported (via cosmrs) bip32::Error type
bip32 = { workspace = true, default-features = false, features = ["std"] }
eyre = { version = "0.6" }
eyre = { workspace = true }
cw-utils = { workspace = true }
cw2 = { workspace = true }
cw3 = { workspace = true }
cw4 = { workspace = true }
cw-controllers = { workspace = true }
prost = { workspace = true, default-features = false }
flate2 = { version = "1.0.20" }
flate2 = { workspace = true }
sha2 = { version = "0.9.5" }
itertools = { version = "0.10" }
zeroize = { workspace = true, features = ["zeroize_derive"] }
@@ -67,6 +66,14 @@ cosmwasm-std = { workspace = true }
workspace = true
features = ["tokio"]
[target."cfg(target_arch = \"wasm32\")".dependencies.reqwest]
workspace = true
features = ["json"]
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.reqwest]
workspace = true
features = ["json", "rustls-tls"]
[dev-dependencies]
bip39 = { workspace = true }
cosmrs = { workspace = true, features = ["bip32"] }
+7 -7
View File
@@ -10,26 +10,26 @@ anyhow = { workspace = true }
base64 = "0.13.0"
bip39 = { workspace = true }
bs58 = { workspace = true }
comfy-table = "6.0.0"
cfg-if = "1.0.0"
comfy-table = { workspace = true }
cfg-if = { workspace = true }
clap = { workspace = true, features = ["derive"] }
csv = "1.3.0"
csv = { workspace = true }
cw-utils = { workspace = true }
futures = { workspace = true }
handlebars = "3.0.1"
handlebars = { workspace = true }
humantime-serde = { workspace = true }
inquire = "0.6.2"
inquire = { workspace = true }
k256 = { workspace = true, features = ["ecdsa", "sha256"] }
log = { workspace = true }
rand = {version = "0.6", features = ["std"] }
serde = { version = "1.0", features = ["derive"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
thiserror = { workspace = true }
time = { workspace = true, features = ["parsing", "formatting"] }
tokio = { workspace = true, features = ["sync"]}
toml = "0.5.6"
url = { workspace = true }
tap = "1"
tap = { workspace = true }
zeroize = { workspace = true }
cosmrs = { workspace = true }
@@ -157,7 +157,7 @@ async fn fetch_delegation_data(
// If a pending undelegate tx is found, remove it from delegation map
Undelegate { owner, mix_id, .. } => {
if owner == address.as_ref()
&& existing_delegation_map.get(&mix_id.to_string()).is_some()
&& existing_delegation_map.contains_key(&mix_id.to_string())
{
existing_delegation_map.remove(&mix_id.to_string());
}
+1 -1
View File
@@ -9,7 +9,7 @@ license.workspace = true
[dependencies]
dirs = { version = "5.0.1", optional = true }
handlebars = "3.5.5"
handlebars = { workspace = true }
log = { workspace = true }
serde = { workspace = true, features = ["derive"] }
toml = "0.7.4"
@@ -328,4 +328,8 @@ impl EpochState {
pub fn is_dealing_exchange(&self) -> bool {
matches!(self, EpochState::DealingExchange { .. })
}
pub fn is_waiting_initialisation(&self) -> bool {
matches!(self, EpochState::WaitingInitialisation)
}
}
@@ -11,7 +11,7 @@ repository = { workspace = true }
bs58 = { workspace = true }
cosmwasm-std = { workspace = true }
cosmwasm-schema = { workspace = true }
schemars = "0.8"
schemars = { workspace = true }
serde = { workspace = true, features = ["derive"] }
thiserror = { workspace = true }
@@ -10,5 +10,5 @@ license.workspace = true
cosmwasm-schema = { workspace = true }
cw4 = { workspace = true }
cw-controllers = { workspace = true }
schemars = "0.8"
schemars = { workspace = true }
serde = { version = "1.0.103", default-features = false, features = ["derive"] }
@@ -14,14 +14,14 @@ cosmwasm-std = { workspace = true }
cosmwasm-schema = { workspace = true }
cw2 = { workspace = true, optional = true }
serde = { workspace = true, features = ["derive"] }
serde_repr = "0.1"
serde_repr = { workspace = true }
# we still have to preserve that import for `JsonSchema` for `Layer` type (since we can't use cw_serde macro due to custom serde impl)
schemars = "0.8"
schemars = { workspace = true }
thiserror = { workspace = true }
contracts-common = { path = "../contracts-common", package = "nym-contracts-common", version = "0.5.0" }
serde-json-wasm = { workspace = true }
humantime-serde = "1.1.1"
humantime-serde = { workspace = true }
# TO CHECK WHETHER STILL NEEDED:
log = { workspace = true }
@@ -13,6 +13,6 @@ cw4 = { workspace = true }
cw-storage-plus = { workspace = true }
cosmwasm-schema = { workspace = true }
cosmwasm-std = { workspace = true }
schemars = "0.8"
schemars = { workspace = true }
serde = { version = "1.0.103", default-features = false, features = ["derive"] }
thiserror = { workspace = true }
@@ -13,7 +13,7 @@ cosmwasm-schema = { workspace = true }
cw2 = { workspace = true, optional = true }
mixnet-contract-common = { path = "../mixnet-contract", package = "nym-mixnet-contract-common", version = "0.6.0" }
contracts-common = { path = "../contracts-common", package = "nym-contracts-common", version = "0.5.0" }
serde = { version = "1.0", features = ["derive"] }
serde = { workspace = true, features = ["derive"] }
thiserror = { workspace = true }
ts-rs = { workspace = true, optional = true}
+1 -1
View File
@@ -26,4 +26,4 @@ features = [ "rt-multi-thread", "net", "signal", "fs" ]
[build-dependencies]
sqlx = { workspace = true, features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate"] }
tokio = { version = "1.24.1", features = ["rt-multi-thread", "macros"] }
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
+3
View File
@@ -18,4 +18,7 @@ pub enum StorageError {
#[error("No unused credential in database. You need to buy at least one")]
NoCredential,
#[error("Database unique constraint violation. Is the credential already imported?")]
ConstraintUnique,
}
@@ -69,9 +69,21 @@ impl Storage for PersistentStorage {
bandwidth_credential.credential_data,
bandwidth_credential.epoch_id,
)
.await?;
Ok(())
.await
.map_err(|err| {
// There is one error we want to handle specifically.
// Check if database_error is `SqliteError` with code 2067 which
// means UNIQUE constraint violation
if let Some(db_error) = err.as_database_error() {
if db_error.code().map_or(false, |code| code == "2067") {
StorageError::ConstraintUnique
} else {
err.into()
}
} else {
err.into()
}
})
}
async fn get_next_unspent_credential(
-10
View File
@@ -11,11 +11,8 @@ use nym_validator_client::nyxd::contract_traits::{
};
use nym_validator_client::nyxd::Coin;
use std::path::PathBuf;
use std::process::exit;
use std::time::{Duration, SystemTime};
const SAFETY_BUFFER_SECS: u64 = 60; // 1 minute
pub async fn issue_credential<C>(
client: &C,
amount: Coin,
@@ -92,13 +89,6 @@ where
.as_secs();
if epoch.state.is_final() {
if let Some(finish_timestamp) = epoch.deadline {
if current_timestamp_secs + SAFETY_BUFFER_SECS >= finish_timestamp.seconds() {
info!("In the next {} minute(s), a transition will take place in the coconut system. Deposits should be halted in this time for safety reasons.", SAFETY_BUFFER_SECS / 60);
exit(0);
}
}
break;
} else if let Some(final_timestamp) = epoch.final_timestamp_secs() {
// Use 1 additional second to not start the next iteration immediately and spam get_current_epoch queries
+5 -5
View File
@@ -8,11 +8,11 @@ use std::str::FromStr;
use thiserror::Error;
pub use nym_coconut::{
aggregate_signature_shares, aggregate_verification_keys, blind_sign, hash_to_scalar, keygen,
prepare_blind_sign, prove_bandwidth_credential, verify_credential, Attribute, Base58,
BlindSignRequest, BlindedSerialNumber, BlindedSignature, Bytable, CoconutError, KeyPair,
Parameters, PrivateAttribute, PublicAttribute, SecretKey, Signature, SignatureShare,
VerificationKey, VerifyCredentialRequest,
aggregate_signature_shares, aggregate_signature_shares_and_verify, aggregate_verification_keys,
blind_sign, hash_to_scalar, keygen, prepare_blind_sign, prove_bandwidth_credential,
verify_credential, Attribute, Base58, BlindSignRequest, BlindedSerialNumber, BlindedSignature,
Bytable, CoconutError, KeyPair, Parameters, PrivateAttribute, PublicAttribute, SecretKey,
Signature, SignatureShare, VerificationKey, VerifyCredentialRequest,
};
pub const VOUCHER_INFO_TYPE: &str = "BandwidthVoucher";
+1 -1
View File
@@ -8,7 +8,7 @@ license.workspace = true
[dependencies]
bls12_381 = { workspace = true, default-features = false, features = ["pairings", "alloc", "experimental"] }
bincode = "1.3.3"
bincode = { workspace = true }
cosmrs = { workspace = true }
thiserror = { workspace = true }
log = { workspace = true }
@@ -12,7 +12,8 @@ use serde::{Deserialize, Serialize};
use time::{Duration, OffsetDateTime, Time};
use zeroize::{Zeroize, ZeroizeOnDrop};
pub const MAX_FREE_PASS_VALIDITY: Duration = Duration::WEEK; // 1 week
pub const DEFAULT_FREE_PASS_VALIDITY: Duration = Duration::WEEK; // 1 week
pub const MAX_FREE_PASS_VALIDITY: Duration = Duration::weeks(12); // 12 weeks
#[derive(Debug, Zeroize, ZeroizeOnDrop, Serialize, Deserialize)]
pub struct FreePassIssuedData {
@@ -77,9 +78,9 @@ impl FreePassIssuanceData {
}
pub fn default_expiry_date() -> OffsetDateTime {
// set it to furthest midnight in the future such as it's no more than a week away,
// set it to the furthest midnight in the future such as it's no more than a week away,
// i.e. if it's currently for example 9:43 on 2nd March 2024, it will set it to 0:00 on 9th March 2024
(OffsetDateTime::now_utc() + MAX_FREE_PASS_VALIDITY).replace_time(Time::MIDNIGHT)
(OffsetDateTime::now_utc() + DEFAULT_FREE_PASS_VALIDITY).replace_time(Time::MIDNIGHT)
}
pub fn expiry_date_attribute(&self) -> &Attribute {
@@ -10,18 +10,19 @@ use crate::coconut::bandwidth::{
use crate::coconut::utils::scalar_serde_helper;
use crate::error::Error;
use nym_credentials_interface::{
aggregate_signature_shares, hash_to_scalar, prepare_blind_sign, Attribute, BlindedSerialNumber,
BlindedSignature, Parameters, PrivateAttribute, PublicAttribute, Signature, SignatureShare,
VerificationKey,
aggregate_signature_shares, aggregate_signature_shares_and_verify, hash_to_scalar,
prepare_blind_sign, Attribute, BlindedSerialNumber, BlindedSignature, Parameters,
PrivateAttribute, PublicAttribute, Signature, SignatureShare, VerificationKey,
};
use nym_crypto::asymmetric::{encryption, identity};
use nym_validator_client::nym_api::EpochId;
use nym_validator_client::nyxd::{Coin, Hash};
use nym_validator_client::signing::AccountData;
use serde::{Deserialize, Serialize};
use time::OffsetDateTime;
use zeroize::{Zeroize, ZeroizeOnDrop};
pub use nym_validator_client::nyxd::{Coin, Hash};
#[derive(Zeroize, ZeroizeOnDrop, Serialize, Deserialize)]
pub enum BandwidthCredentialIssuanceDataVariant {
Voucher(BandwidthVoucherIssuanceData),
@@ -265,6 +266,13 @@ impl IssuanceBandwidthCredential {
self.unblind_signature(validator_vk, &signing_data, blinded_signature)
}
pub fn unchecked_aggregate_signature_shares(
&self,
shares: &[SignatureShare],
) -> Result<Signature, Error> {
aggregate_signature_shares(shares).map_err(Error::SignatureAggregationError)
}
pub fn aggregate_signature_shares(
&self,
verification_key: &VerificationKey,
@@ -279,7 +287,7 @@ impl IssuanceBandwidthCredential {
attributes.extend_from_slice(&private_attributes);
attributes.extend_from_slice(&public_attributes);
aggregate_signature_shares(params, verification_key, &attributes, shares)
aggregate_signature_shares_and_verify(params, verification_key, &attributes, shares)
.map_err(Error::SignatureAggregationError)
}
@@ -155,11 +155,6 @@ impl IssuedBandwidthCredential {
})
}
pub fn randomise_signature(&mut self) {
let signature_prime = self.signature.randomise(bandwidth_credential_params());
self.signature = signature_prime.0
}
pub fn default_parameters() -> Parameters {
IssuanceBandwidthCredential::default_parameters()
}
@@ -6,7 +6,7 @@ use crate::coconut::utils::scalar_serde_helper;
use crate::error::Error;
use nym_api_requests::coconut::BlindSignRequestBody;
use nym_credentials_interface::{
hash_to_scalar, Attribute, BlindSignRequest, BlindedSignature, PublicAttribute,
hash_to_scalar, Attribute, BlindSignRequest, BlindedSignature, CredentialType, PublicAttribute,
};
use nym_crypto::asymmetric::{encryption, identity};
use nym_validator_client::nyxd::{Coin, Hash};
@@ -30,6 +30,10 @@ impl<'a> From<&'a BandwidthVoucherIssuanceData> for BandwidthVoucherIssuedData {
}
impl BandwidthVoucherIssuedData {
pub fn new(value: Coin) -> Self {
BandwidthVoucherIssuedData { value }
}
pub fn value(&self) -> &Coin {
&self.value
}
@@ -123,6 +127,10 @@ impl BandwidthVoucherIssuanceData {
&self.value_prehashed
}
pub fn typ() -> CredentialType {
CredentialType::Voucher
}
pub fn tx_hash(&self) -> Hash {
self.deposit_tx_hash
}
+9 -9
View File
@@ -8,21 +8,21 @@ license = { workspace = true }
repository = { workspace = true }
[dependencies]
aes = { version = "0.8.1", optional = true }
aes = { workspace = true, optional = true }
bs58 = { workspace = true }
blake3 = { version = "1.3.1", features = ["traits-preview"], optional = true }
ctr = { version = "0.9.1", optional = true }
blake3 = { workspace = true, features = ["traits-preview"], optional = true }
ctr = { workspace = true, optional = true }
digest = { version = "0.10.3", optional = true }
generic-array = { workspace = true, optional = true }
hkdf = { version = "0.12.3", optional = true }
hmac = { version = "0.12.1", optional = true }
cipher = { version = "0.4.3", optional = true }
hkdf = { workspace = true, optional = true }
hmac = { workspace = true, optional = true }
cipher = { workspace = true, optional = true }
x25519-dalek = { version = "1.1", optional = true }
ed25519-dalek = { version = "1.0", optional = true }
ed25519-dalek = { workspace = true, optional = true }
rand = { version = "0.7.3", features = ["wasm-bindgen"], optional = true }
serde_bytes = { version = "0.11.6", optional = true }
serde_bytes = { workspace = true, optional = true }
serde_crate = { version = "1.0", optional = true, default_features = false, features = ["derive"], package = "serde" }
subtle-encoding = { version = "0.5", features = ["bech32-preview"]}
subtle-encoding = { workspace = true, features = ["bech32-preview"] }
thiserror = { workspace = true }
zeroize = { workspace = true, optional = true, features = ["zeroize_derive"] }
@@ -200,6 +200,14 @@ impl<'a> From<&'a PrivateKey> for PublicKey {
}
}
impl FromStr for PrivateKey {
type Err = KeyRecoveryError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
PrivateKey::from_base58_string(s)
}
}
impl PrivateKey {
#[cfg(feature = "rand")]
pub fn new<R: RngCore + CryptoRng>(rng: &mut R) -> Self {
@@ -227,6 +227,14 @@ impl<'a> From<&'a PrivateKey> for PublicKey {
}
}
impl FromStr for PrivateKey {
type Err = Ed25519RecoveryError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
PrivateKey::from_base58_string(s)
}
}
impl PrivateKey {
#[cfg(feature = "rand")]
pub fn new<R: RngCore + CryptoRng>(rng: &mut R) -> Self {
+5 -5
View File
@@ -8,7 +8,7 @@ license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
bitvec = "1.0.0"
bitvec = { workspace = true }
# unfortunately until https://github.com/zkcrypto/bls12_381/issues/10 is resolved, we have to rely on the fork
# as we need to be able to serialize Gt so that we could create the lookup table for baby-step-giant-step algorithm
@@ -17,13 +17,13 @@ nym-contracts-common = { path = "../cosmwasm-smart-contracts/contracts-common",
bs58 = { workspace = true }
lazy_static = "1.4.0"
lazy_static = { workspace = true }
rand = { version = "0.8.5", default-features = false}
rand_chacha = "0.3"
rand_core = "0.6.3"
rand_core = { workspace = true }
sha2 = "0.9"
serde = { workspace = true }
serde_derive = "1.0"
serde_derive = { workspace = true }
thiserror = { workspace = true }
zeroize = { workspace = true, features = ["zeroize_derive"] }
@@ -38,7 +38,7 @@ workspace = true
default-features = false
[dev-dependencies]
criterion = "0.4"
criterion = { workspace = true }
[[bench]]
name = "benchmarks"
+2 -2
View File
@@ -8,5 +8,5 @@ license.workspace = true
proc-macro = true
[dependencies]
syn = { version = "1", features = ["full"] }
quote = "1"
syn = { workspace = true, features = ["full"] }
quote = { workspace = true }
+1
View File
@@ -13,6 +13,7 @@ license.workspace = true
[dependencies]
async-trait = { workspace = true }
reqwest = { workspace = true, features = ["json"] }
http.workspace = true
url = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
+130 -31
View File
@@ -2,7 +2,8 @@
// SPDX-License-Identifier: Apache-2.0
use async_trait::async_trait;
use reqwest::{IntoUrl, Response, StatusCode};
use reqwest::header::HeaderValue;
use reqwest::{RequestBuilder, Response, StatusCode};
use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize};
use std::fmt::Display;
@@ -11,6 +12,8 @@ use thiserror::Error;
use tracing::warn;
use url::Url;
pub use reqwest::IntoUrl;
pub const DEFAULT_TIMEOUT: Duration = Duration::from_secs(10);
pub type PathSegments<'a> = &'a [&'a str];
@@ -52,6 +55,88 @@ pub enum HttpClientError<E: Display = String> {
RequestTimeout,
}
pub struct ClientBuilder {
url: Url,
timeout: Option<Duration>,
custom_user_agent: bool,
reqwest_client_builder: reqwest::ClientBuilder,
}
impl ClientBuilder {
pub fn new<U, E>(url: U) -> Result<Self, HttpClientError<E>>
where
U: IntoUrl,
E: Display,
{
// a naive check: if the provided URL does not start with http(s), add that scheme
let str_url = url.as_str();
if !str_url.starts_with("http") {
let alt = format!("http://{str_url}");
warn!("the provided url ('{str_url}') does not contain scheme information. Changing it to '{alt}' ...");
// TODO: or should we maybe default to https?
Self::new(alt)
} else {
Ok(ClientBuilder {
url: url.into_url()?,
timeout: None,
custom_user_agent: false,
reqwest_client_builder: reqwest::ClientBuilder::new(),
})
}
}
pub fn with_timeout(mut self, timeout: Duration) -> Self {
self.timeout = Some(timeout);
self
}
pub fn with_reqwest_builder(mut self, reqwest_builder: reqwest::ClientBuilder) -> Self {
self.reqwest_client_builder = reqwest_builder;
self
}
pub fn with_user_agent<V>(mut self, value: V) -> Self
where
V: TryInto<HeaderValue>,
V::Error: Into<http::Error>,
{
self.custom_user_agent = true;
self.reqwest_client_builder = self.reqwest_client_builder.user_agent(value);
self
}
pub fn build<E>(self) -> Result<Client, HttpClientError<E>>
where
E: Display,
{
#[cfg(target_arch = "wasm32")]
let reqwest_client = self.reqwest_client_builder.build()?;
// TODO: we should probably be propagating the error rather than panicking,
// but that'd break bunch of things due to type changes
#[cfg(not(target_arch = "wasm32"))]
let reqwest_client = {
let mut builder = self
.reqwest_client_builder
.timeout(self.timeout.unwrap_or(DEFAULT_TIMEOUT));
if !self.custom_user_agent {
builder =
builder.user_agent(format!("nym-http-api-client/{}", env!("CARGO_PKG_VERSION")))
}
builder.build()?
};
Ok(Client {
base_url: self.url,
reqwest_client,
#[cfg(target_arch = "wasm32")]
request_timeout: self.timeout.unwrap_or(DEFAULT_TIMEOUT),
})
}
}
/// A simple extendable client wrapper for http request with extra url sanitization.
#[derive(Debug, Clone)]
pub struct Client {
@@ -65,25 +150,9 @@ pub struct Client {
impl Client {
// no timeout until https://github.com/seanmonstar/reqwest/issues/1135 is fixed
pub fn new(base_url: Url, timeout: Option<Duration>) -> Self {
#[cfg(target_arch = "wasm32")]
let reqwest_client = reqwest::Client::new();
// TODO: we should probably be propagating the error rather than panicking,
// but that'd break bunch of things due to type changes
#[cfg(not(target_arch = "wasm32"))]
let reqwest_client = reqwest::ClientBuilder::new()
.timeout(timeout.unwrap_or(DEFAULT_TIMEOUT))
.user_agent(format!("nym-http-api-client/{}", env!("CARGO_PKG_VERSION")))
.build()
.expect("Client::new()");
Client {
base_url,
reqwest_client,
#[cfg(target_arch = "wasm32")]
request_timeout: timeout.unwrap_or(DEFAULT_TIMEOUT),
}
Self::new_url::<_, String>(base_url, timeout).expect(
"we provided valid url and we were unwrapping previous construction errors anyway",
)
}
pub fn new_url<U, E>(url: U, timeout: Option<Duration>) -> Result<Self, HttpClientError<E>>
@@ -91,19 +160,21 @@ impl Client {
U: IntoUrl,
E: Display,
{
// a naive check: if the provided URL does not start with http(s), add that scheme
let str_url = url.as_str();
if !str_url.starts_with("http") {
let alt = format!("http://{str_url}");
warn!("the provided url ('{str_url}') does not contain scheme information. Changing it to '{alt}' ...");
// TODO: or should we maybe default to https?
Self::new_url(alt, timeout)
} else {
Ok(Self::new(url.into_url()?, timeout))
let builder = Self::builder(url)?;
match timeout {
Some(timeout) => builder.with_timeout(timeout).build(),
None => builder.build(),
}
}
pub fn builder<U, E>(url: U) -> Result<ClientBuilder, HttpClientError<E>>
where
U: IntoUrl,
E: Display,
{
ClientBuilder::new(url)
}
pub fn change_base_url(&mut self, new_url: Url) {
self.base_url = new_url
}
@@ -112,6 +183,19 @@ impl Client {
&self.base_url
}
pub fn create_get_request<K, V>(
&self,
path: PathSegments<'_>,
params: Params<'_, K, V>,
) -> RequestBuilder
where
K: AsRef<str>,
V: AsRef<str>,
{
let url = sanitize_url(&self.base_url, path, params);
self.reqwest_client.get(url)
}
async fn send_get_request<K, V, E>(
&self,
path: PathSegments<'_>,
@@ -142,6 +226,21 @@ impl Client {
}
}
pub fn create_post_request<B, K, V>(
&self,
path: PathSegments<'_>,
params: Params<'_, K, V>,
json_body: &B,
) -> RequestBuilder
where
B: Serialize + ?Sized,
K: AsRef<str>,
V: AsRef<str>,
{
let url = sanitize_url(&self.base_url, path, params);
self.reqwest_client.post(url).json(json_body)
}
async fn send_post_request<B, K, V, E>(
&self,
path: PathSegments<'_>,
@@ -407,7 +506,7 @@ pub fn sanitize_url<K: AsRef<str>, V: AsRef<str>>(
url
}
async fn parse_response<T, E>(res: Response, allow_empty: bool) -> Result<T, HttpClientError<E>>
pub async fn parse_response<T, E>(res: Response, allow_empty: bool) -> Result<T, HttpClientError<E>>
where
T: DeserializeOwned,
E: DeserializeOwned + Display,
+3 -3
View File
@@ -12,9 +12,9 @@ license.workspace = true
[dependencies]
axum.workspace = true
bytes = "1.5.0"
mime = "0.3.17"
bytes = { workspace = true }
mime = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
serde_yaml = "0.9.25"
serde_yaml = { workspace = true }
utoipa = { workspace = true }
+3 -2
View File
@@ -9,12 +9,13 @@ edition.workspace = true
license.workspace = true
[dependencies]
bincode = "1.3.3"
bytes = "1.5.0"
bincode = { workspace = true }
bytes = { workspace = true }
nym-bin-common = { path = "../bin-common" }
nym-sphinx = { path = "../nymsphinx" }
rand = "0.8.5"
serde = { workspace = true, features = ["derive"] }
thiserror = { workspace = true }
time = { workspace = true }
tokio = { workspace = true, features = ["time"] }
tokio-util = { workspace = true, features = ["codec"] }
+2
View File
@@ -12,11 +12,13 @@ pub use v6::response;
pub mod codec;
pub mod v6;
pub mod v7;
// version 3: initial version
// version 4: IPv6 support
// version 5: Add severity level to info response
// version 6: Increase the available IPs
// version 7: Add signature support (for the future)
pub const CURRENT_VERSION: u8 = 6;
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
+2
View File
@@ -0,0 +1,2 @@
pub mod request;
pub mod response;
+395
View File
@@ -0,0 +1,395 @@
use nym_sphinx::addressing::clients::Recipient;
use serde::{Deserialize, Serialize};
use time::OffsetDateTime;
use crate::{make_bincode_serializer, IpPair, CURRENT_VERSION};
fn generate_random() -> u64 {
use rand::RngCore;
let mut rng = rand::rngs::OsRng;
rng.next_u64()
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct IpPacketRequest {
pub version: u8,
pub data: IpPacketRequestData,
}
impl IpPacketRequest {
pub fn new_static_connect_request(
ips: IpPair,
reply_to: Recipient,
reply_to_hops: Option<u8>,
reply_to_avg_mix_delays: Option<f64>,
buffer_timeout: Option<u64>,
) -> (Self, u64) {
let request_id = generate_random();
(
Self {
version: CURRENT_VERSION,
data: IpPacketRequestData::StaticConnect(SignedStaticConnectRequest {
request: StaticConnectRequest {
request_id,
ips,
reply_to,
reply_to_hops,
reply_to_avg_mix_delays,
buffer_timeout,
timestamp: OffsetDateTime::now_utc(),
},
signature: None,
}),
},
request_id,
)
}
pub fn new_dynamic_connect_request(
reply_to: Recipient,
reply_to_hops: Option<u8>,
reply_to_avg_mix_delays: Option<f64>,
buffer_timeout: Option<u64>,
) -> (Self, u64) {
let request_id = generate_random();
(
Self {
version: CURRENT_VERSION,
data: IpPacketRequestData::DynamicConnect(SignedDynamicConnectRequest {
request: DynamicConnectRequest {
request_id,
reply_to,
reply_to_hops,
reply_to_avg_mix_delays,
buffer_timeout,
timestamp: OffsetDateTime::now_utc(),
},
signature: None,
}),
},
request_id,
)
}
pub fn new_disconnect_request(reply_to: Recipient) -> (Self, u64) {
let request_id = generate_random();
(
Self {
version: CURRENT_VERSION,
data: IpPacketRequestData::Disconnect(SignedDisconnectRequest {
request: DisconnectRequest {
request_id,
reply_to,
timestamp: OffsetDateTime::now_utc(),
},
signature: None,
}),
},
request_id,
)
}
pub fn new_data_request(ip_packets: bytes::Bytes) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketRequestData::Data(DataRequest { ip_packets }),
}
}
pub fn new_ping(reply_to: Recipient) -> (Self, u64) {
let request_id = generate_random();
(
Self {
version: CURRENT_VERSION,
data: IpPacketRequestData::Ping(PingRequest {
request_id,
reply_to,
timestamp: OffsetDateTime::now_utc(),
}),
},
request_id,
)
}
pub fn new_health_request(reply_to: Recipient) -> (Self, u64) {
let request_id = generate_random();
(
Self {
version: CURRENT_VERSION,
data: IpPacketRequestData::Health(HealthRequest {
request_id,
reply_to,
timestamp: OffsetDateTime::now_utc(),
}),
},
request_id,
)
}
pub fn id(&self) -> Option<u64> {
match &self.data {
IpPacketRequestData::StaticConnect(request) => Some(request.request.request_id),
IpPacketRequestData::DynamicConnect(request) => Some(request.request.request_id),
IpPacketRequestData::Disconnect(request) => Some(request.request.request_id),
IpPacketRequestData::Data(_) => None,
IpPacketRequestData::Ping(request) => Some(request.request_id),
IpPacketRequestData::Health(request) => Some(request.request_id),
}
}
pub fn recipient(&self) -> Option<&Recipient> {
match &self.data {
IpPacketRequestData::StaticConnect(request) => Some(&request.request.reply_to),
IpPacketRequestData::DynamicConnect(request) => Some(&request.request.reply_to),
IpPacketRequestData::Disconnect(request) => Some(&request.request.reply_to),
IpPacketRequestData::Data(_) => None,
IpPacketRequestData::Ping(request) => Some(&request.reply_to),
IpPacketRequestData::Health(request) => Some(&request.reply_to),
}
}
pub fn to_bytes(&self) -> Result<Vec<u8>, bincode::Error> {
use bincode::Options;
make_bincode_serializer().serialize(self)
}
pub fn from_reconstructed_message(
message: &nym_sphinx::receiver::ReconstructedMessage,
) -> Result<Self, bincode::Error> {
use bincode::Options;
make_bincode_serializer().deserialize(&message.message)
}
}
#[allow(clippy::large_enum_variant)]
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub enum IpPacketRequestData {
StaticConnect(SignedStaticConnectRequest),
DynamicConnect(SignedDynamicConnectRequest),
Disconnect(SignedDisconnectRequest),
Data(DataRequest),
Ping(PingRequest),
Health(HealthRequest),
}
impl IpPacketRequestData {
pub fn add_signature(&mut self, signature: Vec<u8>) -> Option<Vec<u8>> {
match self {
IpPacketRequestData::StaticConnect(request) => {
request.signature = Some(signature);
request.signature.clone()
}
IpPacketRequestData::DynamicConnect(request) => {
request.signature = Some(signature);
request.signature.clone()
}
IpPacketRequestData::Disconnect(request) => {
request.signature = Some(signature);
request.signature.clone()
}
IpPacketRequestData::Data(_)
| IpPacketRequestData::Ping(_)
| IpPacketRequestData::Health(_) => None,
}
}
}
// A static connect request is when the client provides the internal IP address it will use on the
// ip packet router.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct StaticConnectRequest {
pub request_id: u64,
pub ips: IpPair,
// The nym-address the response should be sent back to
pub reply_to: Recipient,
// The number of mix node hops that responses should take, in addition to the entry and exit
// node. Zero means only client -> entry -> exit -> client.
pub reply_to_hops: Option<u8>,
// The average delay at each mix node, in milliseconds. Currently this is not supported by the
// ip packet router.
pub reply_to_avg_mix_delays: Option<f64>,
// The maximum time in milliseconds the IPR should wait when filling up a mix packet
// with ip packets.
pub buffer_timeout: Option<u64>,
// Timestamp of when the request was sent by the client.
pub timestamp: OffsetDateTime,
}
impl StaticConnectRequest {
pub fn to_bytes(&self) -> Result<Vec<u8>, bincode::Error> {
use bincode::Options;
make_bincode_serializer().serialize(self)
}
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct SignedStaticConnectRequest {
pub request: StaticConnectRequest,
pub signature: Option<Vec<u8>>,
}
// A dynamic connect request is when the client does not provide the internal IP address it will use
// on the ip packet router, and instead requests one to be assigned to it.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct DynamicConnectRequest {
pub request_id: u64,
// The nym-address the response should be sent back to
pub reply_to: Recipient,
// The number of mix node hops that responses should take, in addition to the entry and exit
// node. Zero means only client -> entry -> exit -> client.
pub reply_to_hops: Option<u8>,
// The average delay at each mix node, in milliseconds. Currently this is not supported by the
// ip packet router.
pub reply_to_avg_mix_delays: Option<f64>,
// The maximum time in milliseconds the IPR should wait when filling up a mix packet
// with ip packets.
pub buffer_timeout: Option<u64>,
// Timestamp of when the request was sent by the client.
pub timestamp: OffsetDateTime,
}
impl DynamicConnectRequest {
pub fn to_bytes(&self) -> Result<Vec<u8>, bincode::Error> {
use bincode::Options;
make_bincode_serializer().serialize(self)
}
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct SignedDynamicConnectRequest {
pub request: DynamicConnectRequest,
pub signature: Option<Vec<u8>>,
}
// A disconnect request is when the client wants to disconnect from the ip packet router and free
// up the allocated IP address.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct DisconnectRequest {
pub request_id: u64,
// The nym-address the response should be sent back to
pub reply_to: Recipient,
// Timestamp of when the request was sent by the client.
pub timestamp: OffsetDateTime,
}
impl DisconnectRequest {
pub fn to_bytes(&self) -> Result<Vec<u8>, bincode::Error> {
use bincode::Options;
make_bincode_serializer().serialize(self)
}
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct SignedDisconnectRequest {
pub request: DisconnectRequest,
pub signature: Option<Vec<u8>>,
}
// A data request is when the client wants to send an IP packet to a destination.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct DataRequest {
pub ip_packets: bytes::Bytes,
}
// A ping request is when the client wants to check if the ip packet router is still alive.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct PingRequest {
pub request_id: u64,
// The nym-address the response should be sent back to
pub reply_to: Recipient,
// Timestamp of when the request was sent by the client.
pub timestamp: OffsetDateTime,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct HealthRequest {
pub request_id: u64,
// The nym-address the response should be sent back to
pub reply_to: Recipient,
// Timestamp of when the request was sent by the client.
pub timestamp: OffsetDateTime,
}
#[cfg(test)]
mod tests {
use super::*;
use std::net::{Ipv4Addr, Ipv6Addr};
use std::str::FromStr;
#[test]
fn check_size_of_request() {
let connect = IpPacketRequest {
version: 4,
data: IpPacketRequestData::StaticConnect(
SignedStaticConnectRequest {
request: StaticConnectRequest {
request_id: 123,
ips: IpPair::new(Ipv4Addr::from_str("10.0.0.1").unwrap(), Ipv6Addr::from_str("2001:db8:a160::1").unwrap()),
reply_to: Recipient::try_from_base58_string("D1rrpsysCGCYXy9saP8y3kmNpGtJZUXN9SvFoUcqAsM9.9Ssso1ea5NfkbMASdiseDSjTN1fSWda5SgEVjdSN4CvV@GJqd3ZxpXWSNxTfx7B1pPtswpetH4LnJdFeLeuY5KUuN").unwrap(),
reply_to_hops: None,
reply_to_avg_mix_delays: None,
buffer_timeout: None,
timestamp: OffsetDateTime::now_utc(),
},
signature: None,
}
),
};
assert_eq!(connect.to_bytes().unwrap().len(), 139);
}
#[test]
fn check_size_of_data() {
let data = IpPacketRequest {
version: 4,
data: IpPacketRequestData::Data(DataRequest {
ip_packets: bytes::Bytes::from(vec![1u8; 32]),
}),
};
assert_eq!(data.to_bytes().unwrap().len(), 35);
}
#[test]
fn serialize_and_deserialize_data_request() {
let data = IpPacketRequest {
version: 4,
data: IpPacketRequestData::Data(DataRequest {
ip_packets: bytes::Bytes::from(vec![1, 2, 4, 2, 5]),
}),
};
let serialized = data.to_bytes().unwrap();
let deserialized = IpPacketRequest::from_reconstructed_message(
&nym_sphinx::receiver::ReconstructedMessage {
message: serialized,
sender_tag: None,
},
)
.unwrap();
assert_eq!(deserialized.version, 4);
assert_eq!(
deserialized.data,
IpPacketRequestData::Data(DataRequest {
ip_packets: bytes::Bytes::from(vec![1, 2, 4, 2, 5]),
})
);
}
}
@@ -0,0 +1,410 @@
use nym_sphinx::addressing::clients::Recipient;
use serde::{Deserialize, Serialize};
use crate::{make_bincode_serializer, IpPair, CURRENT_VERSION};
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct IpPacketResponse {
pub version: u8,
pub data: IpPacketResponseData,
}
impl IpPacketResponse {
pub fn new_static_connect_success(request_id: u64, reply_to: Recipient) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::StaticConnect(StaticConnectResponse {
request_id,
reply_to,
reply: StaticConnectResponseReply::Success,
}),
}
}
pub fn new_static_connect_failure(
request_id: u64,
reply_to: Recipient,
reason: StaticConnectFailureReason,
) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::StaticConnect(StaticConnectResponse {
request_id,
reply_to,
reply: StaticConnectResponseReply::Failure(reason),
}),
}
}
pub fn new_dynamic_connect_success(request_id: u64, reply_to: Recipient, ips: IpPair) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::DynamicConnect(DynamicConnectResponse {
request_id,
reply_to,
reply: DynamicConnectResponseReply::Success(DynamicConnectSuccess { ips }),
}),
}
}
pub fn new_dynamic_connect_failure(
request_id: u64,
reply_to: Recipient,
reason: DynamicConnectFailureReason,
) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::DynamicConnect(DynamicConnectResponse {
request_id,
reply_to,
reply: DynamicConnectResponseReply::Failure(reason),
}),
}
}
pub fn new_disconnect_success(request_id: u64, reply_to: Recipient) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::Disconnect(DisconnectResponse {
request_id,
reply_to,
reply: DisconnectResponseReply::Success,
}),
}
}
pub fn new_disconnect_failure(
request_id: u64,
reply_to: Recipient,
reason: DisconnectFailureReason,
) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::Disconnect(DisconnectResponse {
request_id,
reply_to,
reply: DisconnectResponseReply::Failure(reason),
}),
}
}
pub fn new_unrequested_disconnect(
reply_to: Recipient,
reason: UnrequestedDisconnectReason,
) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::UnrequestedDisconnect(UnrequestedDisconnect {
reply_to,
reason,
}),
}
}
pub fn new_ip_packet(ip_packet: bytes::Bytes) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::Data(DataResponse { ip_packet }),
}
}
pub fn new_version_mismatch(
request_id: u64,
reply_to: Recipient,
request_version: u8,
our_version: u8,
) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::Info(InfoResponse {
request_id,
reply_to,
reply: InfoResponseReply::VersionMismatch {
request_version,
response_version: our_version,
},
level: InfoLevel::Error,
}),
}
}
pub fn new_data_info_response(
reply_to: Recipient,
reply: InfoResponseReply,
level: InfoLevel,
) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::Info(InfoResponse {
request_id: 0,
reply_to,
reply,
level,
}),
}
}
pub fn new_pong(request_id: u64, reply_to: Recipient) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::Pong(PongResponse {
request_id,
reply_to,
}),
}
}
pub fn new_health_response(
request_id: u64,
reply_to: Recipient,
build_info: nym_bin_common::build_information::BinaryBuildInformationOwned,
routable: Option<bool>,
) -> Self {
Self {
version: CURRENT_VERSION,
data: IpPacketResponseData::Health(HealthResponse {
request_id,
reply_to,
reply: HealthResponseReply {
build_info,
routable,
},
}),
}
}
pub fn id(&self) -> Option<u64> {
match &self.data {
IpPacketResponseData::StaticConnect(response) => Some(response.request_id),
IpPacketResponseData::DynamicConnect(response) => Some(response.request_id),
IpPacketResponseData::Disconnect(response) => Some(response.request_id),
IpPacketResponseData::UnrequestedDisconnect(_) => None,
IpPacketResponseData::Data(_) => None,
IpPacketResponseData::Pong(response) => Some(response.request_id),
IpPacketResponseData::Health(response) => Some(response.request_id),
IpPacketResponseData::Info(response) => Some(response.request_id),
}
}
pub fn recipient(&self) -> Option<&Recipient> {
match &self.data {
IpPacketResponseData::StaticConnect(response) => Some(&response.reply_to),
IpPacketResponseData::DynamicConnect(response) => Some(&response.reply_to),
IpPacketResponseData::Disconnect(response) => Some(&response.reply_to),
IpPacketResponseData::UnrequestedDisconnect(response) => Some(&response.reply_to),
IpPacketResponseData::Data(_) => None,
IpPacketResponseData::Pong(response) => Some(&response.reply_to),
IpPacketResponseData::Health(response) => Some(&response.reply_to),
IpPacketResponseData::Info(response) => Some(&response.reply_to),
}
}
pub fn to_bytes(&self) -> Result<Vec<u8>, bincode::Error> {
use bincode::Options;
make_bincode_serializer().serialize(self)
}
pub fn from_reconstructed_message(
message: &nym_sphinx::receiver::ReconstructedMessage,
) -> Result<Self, bincode::Error> {
use bincode::Options;
make_bincode_serializer().deserialize(&message.message)
}
}
#[allow(clippy::large_enum_variant)]
#[derive(Clone, Debug, Serialize, Deserialize)]
pub enum IpPacketResponseData {
// Response for a static connect request
StaticConnect(StaticConnectResponse),
// Response for a dynamic connect request
DynamicConnect(DynamicConnectResponse),
// Response for a disconnect initiqated by the client
Disconnect(DisconnectResponse),
// Message from the server that the client got disconnected without the client initiating it
UnrequestedDisconnect(UnrequestedDisconnect),
// Response to a data request
Data(DataResponse),
// Response to ping request
Pong(PongResponse),
// Response for a health request
Health(HealthResponse),
// Info response. This can be anything from informative messages to errors
Info(InfoResponse),
}
impl IpPacketResponseData {
pub fn to_bytes(&self) -> Result<Vec<u8>, bincode::Error> {
use bincode::Options;
make_bincode_serializer().serialize(self)
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct StaticConnectResponse {
pub request_id: u64,
pub reply_to: Recipient,
pub reply: StaticConnectResponseReply,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub enum StaticConnectResponseReply {
Success,
Failure(StaticConnectFailureReason),
}
impl StaticConnectResponseReply {
pub fn is_success(&self) -> bool {
match self {
StaticConnectResponseReply::Success => true,
StaticConnectResponseReply::Failure(_) => false,
}
}
}
#[derive(Clone, Debug, Serialize, Deserialize, thiserror::Error)]
pub enum StaticConnectFailureReason {
#[error("requested ip address is already in use")]
RequestedIpAlreadyInUse,
#[error("requested nym-address is already in use")]
RequestedNymAddressAlreadyInUse,
#[error("{0}")]
Other(String),
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct DynamicConnectResponse {
pub request_id: u64,
pub reply_to: Recipient,
pub reply: DynamicConnectResponseReply,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub enum DynamicConnectResponseReply {
Success(DynamicConnectSuccess),
Failure(DynamicConnectFailureReason),
}
impl DynamicConnectResponseReply {
pub fn is_success(&self) -> bool {
match self {
DynamicConnectResponseReply::Success(_) => true,
DynamicConnectResponseReply::Failure(_) => false,
}
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct DynamicConnectSuccess {
pub ips: IpPair,
}
#[derive(Clone, Debug, Serialize, Deserialize, thiserror::Error)]
pub enum DynamicConnectFailureReason {
#[error("requested nym-address is already in use")]
RequestedNymAddressAlreadyInUse,
#[error("no available ip address")]
NoAvailableIp,
#[error("{0}")]
Other(String),
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct DisconnectResponse {
pub request_id: u64,
pub reply_to: Recipient,
pub reply: DisconnectResponseReply,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub enum DisconnectResponseReply {
Success,
Failure(DisconnectFailureReason),
}
#[derive(Clone, Debug, Serialize, Deserialize, thiserror::Error)]
pub enum DisconnectFailureReason {
#[error("requested nym-address is not currently connected")]
RequestedNymAddressNotConnected,
#[error("{0}")]
Other(String),
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct UnrequestedDisconnect {
pub reply_to: Recipient,
pub reason: UnrequestedDisconnectReason,
}
#[derive(Clone, Debug, Serialize, Deserialize, thiserror::Error)]
pub enum UnrequestedDisconnectReason {
#[error("client mixnet traffic timeout")]
ClientMixnetTrafficTimeout,
#[error("client tun traffic timeout")]
ClientTunTrafficTimeout,
#[error("{0}")]
Other(String),
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct DataResponse {
pub ip_packet: bytes::Bytes,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct PongResponse {
pub request_id: u64,
pub reply_to: Recipient,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct HealthResponse {
pub request_id: u64,
pub reply_to: Recipient,
pub reply: HealthResponseReply,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct HealthResponseReply {
// Return the binary build information of the IPR
pub build_info: nym_bin_common::build_information::BinaryBuildInformationOwned,
// Return if the IPR has performed a successful routing test.
pub routable: Option<bool>,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct InfoResponse {
pub request_id: u64,
pub reply_to: Recipient,
pub reply: InfoResponseReply,
pub level: InfoLevel,
}
#[derive(Clone, Debug, Serialize, Deserialize, thiserror::Error)]
pub enum InfoResponseReply {
#[error("{msg}")]
Generic { msg: String },
#[error(
"version mismatch: response is v{request_version} and response is v{response_version}"
)]
VersionMismatch {
request_version: u8,
response_version: u8,
},
#[error("destination failed exit policy filter check: {dst}")]
ExitPolicyFilterCheckFailed { dst: String },
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub enum InfoLevel {
Info,
Warn,
Error,
}
+3 -3
View File
@@ -7,8 +7,8 @@ license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
bip32 = "0.5.1"
bip32 = { workspace = true }
k256 = { workspace = true }
ledger-transport = "0.10.0"
ledger-transport-hid = "0.10.0"
ledger-transport = { workspace = true }
ledger-transport-hid = { workspace = true }
thiserror = { workspace = true }
+2 -2
View File
@@ -8,7 +8,7 @@ license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
bytes = "1.0"
bytes = { workspace = true }
futures = { workspace = true }
humantime-serde = { workspace = true }
log = { workspace = true }
@@ -38,4 +38,4 @@ nym-task = { path = "../task" }
nym-validator-client = { path = "../client-libs/validator-client" }
nym-bin-common = { path = "../bin-common" }
nym-metrics = { path = "../nym-metrics" }
nym-node-http-api = { path = "../../nym-node/nym-node-http-api" }
nym-node-http-api = { path = "../../nym-node/nym-node-http-api" }
+1 -1
View File
@@ -10,7 +10,7 @@ repository.workspace = true
[dependencies]
cfg-if = { workspace = true }
dotenvy = { workspace = true }
hex-literal = "0.3.3"
hex-literal = { workspace = true }
log = { workspace = true }
once_cell = { workspace = true }
schemars = { workspace = true, features = ["preserve_order"] }
+6 -3
View File
@@ -18,9 +18,12 @@ pub const VESTING_CONTRACT_ADDRESS: &str =
"n1nc5tatafv6eyq7llkr2gv50ff9e22mnf70qgjlv737ktmt4eswrq73f2nw";
pub const COCONUT_BANDWIDTH_CONTRACT_ADDRESS: &str = "";
pub const GROUP_CONTRACT_ADDRESS: &str = "";
pub const MULTISIG_CONTRACT_ADDRESS: &str = "";
pub const COCONUT_DKG_CONTRACT_ADDRESS: &str = "";
pub const GROUP_CONTRACT_ADDRESS: &str =
"n1e2zq4886zzewpvpucmlw8v9p7zv692f6yck4zjzxh699dkcmlrfqk2knsr";
pub const MULTISIG_CONTRACT_ADDRESS: &str =
"n1txayqfz5g9qww3rlflpg025xd26m9payz96u54x4fe3s2ktz39xqk67gzx";
pub const COCONUT_DKG_CONTRACT_ADDRESS: &str =
"n19604yflqggs9mk2z26mqygq43q2kr3n932egxx630svywd5mpxjsztfpvx";
pub const EPHEMERA_CONTRACT_ADDRESS: &str = "";
pub const REWARDING_VALIDATOR_ADDRESS: &str = "n10yyd98e2tuwu0f7ypz9dy3hhjw7v772q6287gy";
+1 -1
View File
@@ -8,7 +8,7 @@ license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
tokio-stream = "0.1.11" # this one seems to be a thing until `Stream` trait is stabilised in stdlib
tokio-stream = { workspace = true } # this one seems to be a thing until `Stream` trait is stabilised in stdlib
[target."cfg(not(target_arch = \"wasm32\"))".dependencies.tokio]
workspace = true
+1 -1
View File
@@ -14,4 +14,4 @@ license.workspace = true
prometheus = { workspace = true }
log = { workspace = true }
dashmap = { workspace = true }
lazy_static = "1.4"
lazy_static = { workspace = true }
+3 -3
View File
@@ -14,7 +14,7 @@ digest = "0.9"
rand = "0.8"
thiserror = { workspace = true }
serde = { workspace = true }
serde_derive = "1.0"
serde_derive = { workspace = true }
bs58 = { workspace = true }
sha2 = "0.9"
zeroize = { workspace = true, optional = true }
@@ -31,8 +31,8 @@ workspace = true
default-features = false
[dev-dependencies]
criterion = { version="0.4", features=["html_reports"] }
doc-comment = "0.3"
criterion = { workspace = true, features = ["html_reports"] }
doc-comment = { workspace = true }
rand_chacha = "0.3"
[[bench]]
+5 -5
View File
@@ -6,10 +6,10 @@ use criterion::{criterion_group, criterion_main, Criterion};
use ff::Field;
use group::{Curve, Group};
use nym_coconut::{
aggregate_signature_shares, aggregate_verification_keys, blind_sign, prepare_blind_sign,
prove_bandwidth_credential, random_scalars_refs, setup, ttp_keygen, verify_credential,
verify_partial_blind_signature, Attribute, BlindedSignature, Parameters, Signature,
SignatureShare, VerificationKey,
aggregate_signature_shares_and_verify, aggregate_verification_keys, blind_sign,
prepare_blind_sign, prove_bandwidth_credential, random_scalars_refs, setup, ttp_keygen,
verify_credential, verify_partial_blind_signature, Attribute, BlindedSignature, Parameters,
Signature, SignatureShare, VerificationKey,
};
use rand::seq::SliceRandom;
use std::ops::Neg;
@@ -99,7 +99,7 @@ fn unblind_and_aggregate(
let mut attributes = vec![];
attributes.extend_from_slice(private_attributes);
attributes.extend_from_slice(public_attributes);
aggregate_signature_shares(
aggregate_signature_shares_and_verify(
params,
verification_key,
&attributes,
+7
View File
@@ -4,14 +4,18 @@
#![warn(clippy::expect_used)]
#![warn(clippy::unwrap_used)]
pub use bls12_381::Scalar;
pub use elgamal::elgamal_keygen;
pub use elgamal::ElGamalKeyPair;
pub use elgamal::PublicKey;
pub use error::CoconutError;
pub use scheme::aggregation::aggregate_key_shares;
pub use scheme::aggregation::aggregate_signature_shares;
pub use scheme::aggregation::aggregate_signature_shares_and_verify;
pub use scheme::aggregation::aggregate_verification_keys;
pub use scheme::issuance::blind_sign;
pub use scheme::issuance::prepare_blind_sign;
pub use scheme::issuance::sign;
pub use scheme::issuance::verify_partial_blind_signature;
pub use scheme::issuance::BlindSignRequest;
pub use scheme::keygen::keygen;
@@ -19,16 +23,19 @@ pub use scheme::keygen::ttp_keygen;
pub use scheme::keygen::KeyPair;
pub use scheme::keygen::SecretKey;
pub use scheme::keygen::VerificationKey;
pub use scheme::keygen::VerificationKeyShare;
pub use scheme::setup::setup;
pub use scheme::setup::Parameters;
pub use scheme::verification::check_vk_pairing;
pub use scheme::verification::prove_bandwidth_credential;
pub use scheme::verification::verify;
pub use scheme::verification::verify_credential;
pub use scheme::verification::BlindedSerialNumber;
pub use scheme::verification::VerifyCredentialRequest;
pub use scheme::BlindedSignature;
pub use scheme::Signature;
pub use scheme::SignatureShare;
pub use scheme::SignerIndex;
pub use traits::Base58;
pub use traits::Bytable;
pub use utils::hash_to_scalar;
+55 -25
View File
@@ -12,7 +12,7 @@ use crate::error::{CoconutError, Result};
use crate::scheme::verification::check_bilinear_pairing;
use crate::scheme::{PartialSignature, Signature, SignatureShare, SignerIndex, VerificationKey};
use crate::utils::perform_lagrangian_interpolation_at_origin;
use crate::{Attribute, Parameters};
use crate::{Attribute, Parameters, VerificationKeyShare};
pub(crate) trait Aggregatable: Sized {
fn aggregate(aggregatable: &[Self], indices: Option<&[SignerIndex]>) -> Result<Self>;
@@ -80,7 +80,23 @@ pub fn aggregate_verification_keys(
Aggregatable::aggregate(keys, indices)
}
pub fn aggregate_key_shares(shares: &[VerificationKeyShare]) -> Result<VerificationKey> {
let (keys, indices): (Vec<_>, Vec<_>) = shares
.iter()
.map(|share| (share.key.clone(), share.index))
.unzip();
aggregate_verification_keys(&keys, Some(&indices))
}
pub fn aggregate_signatures(
signatures: &[PartialSignature],
indices: Option<&[SignerIndex]>,
) -> Result<Signature> {
Aggregatable::aggregate(signatures, indices)
}
pub fn aggregate_signatures_and_verify(
params: &Parameters,
verification_key: &VerificationKey,
attributes: &[&Attribute],
@@ -88,11 +104,7 @@ pub fn aggregate_signatures(
indices: Option<&[SignerIndex]>,
) -> Result<Signature> {
// aggregate the signature
let signature = match Aggregatable::aggregate(signatures, indices) {
Ok(res) => res,
Err(err) => return Err(err),
};
let signature = aggregate_signatures(signatures, indices)?;
// Verify the signature
let alpha = verification_key.alpha;
@@ -116,7 +128,16 @@ pub fn aggregate_signatures(
Ok(signature)
}
pub fn aggregate_signature_shares(
pub fn aggregate_signature_shares(shares: &[SignatureShare]) -> Result<Signature> {
let (signatures, indices): (Vec<_>, Vec<_>) = shares
.iter()
.map(|share| (*share.signature(), share.index()))
.unzip();
aggregate_signatures(&signatures, Some(&indices))
}
pub fn aggregate_signature_shares_and_verify(
params: &Parameters,
verification_key: &VerificationKey,
attributes: &[&Attribute],
@@ -127,7 +148,7 @@ pub fn aggregate_signature_shares(
.map(|share| (*share.signature(), share.index()))
.unzip();
aggregate_signatures(
aggregate_signatures_and_verify(
params,
verification_key,
attributes,
@@ -210,7 +231,7 @@ mod tests {
#[test]
fn signature_aggregation_works_for_any_subset_of_signatures() {
let mut params = Parameters::new(2).unwrap();
let params = Parameters::new(2).unwrap();
random_scalars_refs!(attributes, params, 2);
let keypairs = ttp_keygen(&params, 3, 5).unwrap();
@@ -227,12 +248,12 @@ mod tests {
let sigs = sks
.iter()
.map(|sk| sign(&mut params, sk, &attributes).unwrap())
.map(|sk| sign(&params, sk, &attributes).unwrap())
.collect::<Vec<_>>();
// aggregating (any) threshold works
let aggr_vk_1 = aggregate_verification_keys(&vks[..3], Some(&[1, 2, 3])).unwrap();
let aggr_sig1 = aggregate_signatures(
let aggr_sig1 = aggregate_signatures_and_verify(
&params,
&aggr_vk_1,
&attributes,
@@ -242,7 +263,7 @@ mod tests {
.unwrap();
let aggr_vk_2 = aggregate_verification_keys(&vks[2..], Some(&[3, 4, 5])).unwrap();
let aggr_sig2 = aggregate_signatures(
let aggr_sig2 = aggregate_signatures_and_verify(
&params,
&aggr_vk_1,
&attributes,
@@ -258,7 +279,7 @@ mod tests {
// aggregating threshold+1 works
let aggr_vk_more = aggregate_verification_keys(&vks[1..], Some(&[2, 3, 4, 5])).unwrap();
let aggr_more = aggregate_signatures(
let aggr_more = aggregate_signatures_and_verify(
&params,
&aggr_vk_more,
&attributes,
@@ -270,7 +291,7 @@ mod tests {
// aggregating all
let aggr_vk_all = aggregate_verification_keys(&vks, Some(&[1, 2, 3, 4, 5])).unwrap();
let aggr_all = aggregate_signatures(
let aggr_all = aggregate_signatures_and_verify(
&params,
&aggr_vk_all,
&attributes,
@@ -282,7 +303,7 @@ mod tests {
// not taking enough points (threshold was 3) should fail
let aggr_vk_not_enough = aggregate_verification_keys(&vks[..2], Some(&[1, 2])).unwrap();
let aggr_not_enough = aggregate_signatures(
let aggr_not_enough = aggregate_signatures_and_verify(
&params,
&aggr_vk_not_enough,
&attributes,
@@ -294,7 +315,7 @@ mod tests {
// taking wrong index should fail
let aggr_vk_bad = aggregate_verification_keys(&vks[2..], Some(&[1, 2, 3])).unwrap();
assert!(aggregate_signatures(
assert!(aggregate_signatures_and_verify(
&params,
&aggr_vk_bad,
&attributes,
@@ -330,9 +351,14 @@ mod tests {
.unzip();
let aggr_vk_all = aggregate_verification_keys(&vks, None).unwrap();
assert!(
aggregate_signatures(&params, &aggr_vk_all, &attributes, &signatures, None).is_err()
);
assert!(aggregate_signatures_and_verify(
&params,
&aggr_vk_all,
&attributes,
&signatures,
None
)
.is_err());
}
#[test]
@@ -352,11 +378,15 @@ mod tests {
.unzip();
let aggr_vk_all = aggregate_verification_keys(&vks, None).unwrap();
assert!(
aggregate_signatures(&params, &aggr_vk_all, &attributes, &signatures, Some(&[]))
.is_err()
);
assert!(aggregate_signatures(
assert!(aggregate_signatures_and_verify(
&params,
&aggr_vk_all,
&attributes,
&signatures,
Some(&[])
)
.is_err());
assert!(aggregate_signatures_and_verify(
&params,
&aggr_vk_all,
&attributes,
@@ -383,7 +413,7 @@ mod tests {
.unzip();
let aggr_vk_all = aggregate_verification_keys(&vks, None).unwrap();
assert!(aggregate_signatures(
assert!(aggregate_signatures_and_verify(
&params,
&aggr_vk_all,
&attributes,
+7 -4
View File
@@ -13,9 +13,8 @@ use crate::scheme::setup::Parameters;
use crate::scheme::BlindedSignature;
use crate::scheme::SecretKey;
use crate::Attribute;
/// Creates a Coconut Signature under a given secret key on a set of public attributes only.
#[cfg(test)]
use crate::Signature;
// TODO: possibly completely remove those two functions.
// They only exist to have a simpler and smaller code snippets to test
// basic functionalities.
@@ -158,6 +157,10 @@ impl BlindSignRequest {
)
}
pub fn verify_commitment_hash(&self, public_attributes: &[&Attribute]) -> bool {
self.commitment_hash == compute_hash(self.commitment, public_attributes)
}
pub fn get_commitment_hash(&self) -> G1Projective {
self.commitment_hash
}
@@ -426,9 +429,9 @@ pub fn verify_partial_blind_signature(
.into()
}
#[cfg(test)]
/// Creates a Coconut Signature under a given secret key on a set of public attributes only.
pub fn sign(
params: &mut Parameters,
params: &Parameters,
secret_key: &SecretKey,
public_attributes: &[&Attribute],
) -> Result<Signature> {
+28 -8
View File
@@ -151,10 +151,6 @@ impl Base58 for SecretKey {}
// TODO: perhaps change points to affine representation
// to make verification slightly more efficient?
#[derive(Debug, PartialEq, Eq, Clone)]
#[cfg_attr(
feature = "key-zeroize",
derive(zeroize::Zeroize, zeroize::ZeroizeOnDrop)
)]
pub struct VerificationKey {
// TODO add gen2 as per the paper or imply it from the fact library is using bls381?
pub(crate) alpha: G2Projective,
@@ -411,12 +407,23 @@ impl Bytable for VerificationKey {
impl Base58 for VerificationKey {}
#[derive(Debug, Clone)]
pub struct VerificationKeyShare {
pub key: VerificationKey,
pub index: SignerIndex,
}
impl From<(VerificationKey, SignerIndex)> for VerificationKeyShare {
fn from(value: (VerificationKey, SignerIndex)) -> Self {
VerificationKeyShare {
key: value.0,
index: value.1,
}
}
}
#[derive(Debug, Serialize, Deserialize)]
#[cfg_attr(test, derive(PartialEq, Eq, Clone))]
#[cfg_attr(
feature = "key-zeroize",
derive(zeroize::Zeroize, zeroize::ZeroizeOnDrop)
)]
pub struct KeyPair {
secret_key: SecretKey,
verification_key: VerificationKey,
@@ -425,6 +432,12 @@ pub struct KeyPair {
pub index: Option<SignerIndex>,
}
impl From<KeyPair> for (SecretKey, VerificationKey) {
fn from(value: KeyPair) -> Self {
(value.secret_key, value.verification_key)
}
}
impl PemStorableKeyPair for KeyPair {
type PrivatePemKey = SecretKey;
type PublicPemKey = VerificationKey;
@@ -461,6 +474,13 @@ impl KeyPair {
&self.verification_key
}
pub fn to_verification_key_share(&self) -> Option<VerificationKeyShare> {
self.index.map(|index| VerificationKeyShare {
key: self.verification_key.clone(),
index,
})
}
pub fn to_bytes(&self) -> Vec<u8> {
// Schema is coconutkeypair[14]|secret_key_len[8]|secret_key[secret_key_len]|verification_key_len[8]|verification_key[verification_key_len]|signer_index[8] - optional
self.to_byte_vec()
+31 -13
View File
@@ -70,6 +70,11 @@ impl Signature {
&self.1
}
pub fn randomise_simple(&self, params: &Parameters) -> Signature {
let r = params.random_scalar();
Signature(self.0 * r, self.1 * r)
}
pub fn randomise(&self, params: &Parameters) -> (Signature, Scalar) {
let r = params.random_scalar();
let r_prime = params.random_scalar();
@@ -191,7 +196,7 @@ impl BlindedSignature {
&self,
partial_verification_key: &VerificationKey,
pedersen_commitments_openings: &[Scalar],
) -> Result<Signature> {
) -> Signature {
// parse the signature
let h = &self.0;
let c = &self.1;
@@ -204,7 +209,7 @@ impl BlindedSignature {
let unblinded_c = c - blinding_removers;
Ok(Signature(*h, unblinded_c))
Signature(*h, unblinded_c)
}
pub fn unblind_and_verify(
@@ -216,7 +221,7 @@ impl BlindedSignature {
commitment_hash: &G1Projective,
pedersen_commitments_openings: &[Scalar],
) -> Result<Signature> {
let unblinded = self.unblind(partial_verification_key, pedersen_commitments_openings)?;
let unblinded = self.unblind(partial_verification_key, pedersen_commitments_openings);
unblinded.verify(
params,
partial_verification_key,
@@ -240,6 +245,7 @@ impl BlindedSignature {
}
// perhaps this should take signature by reference? we'll see how it goes
#[derive(Clone, Copy)]
pub struct SignatureShare {
signature: Signature,
index: SignerIndex,
@@ -276,7 +282,9 @@ impl SignatureShare {
mod tests {
use super::*;
use crate::hash_to_scalar;
use crate::scheme::aggregation::{aggregate_signatures, aggregate_verification_keys};
use crate::scheme::aggregation::{
aggregate_signatures_and_verify, aggregate_verification_keys,
};
use crate::scheme::issuance::{blind_sign, compute_hash, prepare_blind_sign, sign};
use crate::scheme::keygen::{keygen, ttp_keygen};
use crate::scheme::verification::{prove_bandwidth_credential, verify, verify_credential};
@@ -418,13 +426,13 @@ mod tests {
#[test]
fn verification_on_two_public_attributes() {
let mut params = Parameters::new(2).unwrap();
let params = Parameters::new(2).unwrap();
random_scalars_refs!(attributes, params, 2);
let keypair1 = keygen(&params);
let keypair2 = keygen(&params);
let sig1 = sign(&mut params, keypair1.secret_key(), &attributes).unwrap();
let sig2 = sign(&mut params, keypair2.secret_key(), &attributes).unwrap();
let sig1 = sign(&params, keypair1.secret_key(), &attributes).unwrap();
let sig2 = sign(&params, keypair2.secret_key(), &attributes).unwrap();
assert!(verify(
&params,
@@ -568,9 +576,14 @@ mod tests {
attributes.extend_from_slice(&public_attributes);
let aggr_vk = aggregate_verification_keys(&vks[..2], Some(&[1, 2])).unwrap();
let aggr_sig =
aggregate_signatures(&params, &aggr_vk, &attributes, &sigs[..2], Some(&[1, 2]))
.unwrap();
let aggr_sig = aggregate_signatures_and_verify(
&params,
&aggr_vk,
&attributes,
&sigs[..2],
Some(&[1, 2]),
)
.unwrap();
let theta = prove_bandwidth_credential(
&params,
@@ -590,9 +603,14 @@ mod tests {
// taking different subset of keys and credentials
let aggr_vk = aggregate_verification_keys(&vks[1..], Some(&[2, 3])).unwrap();
let aggr_sig =
aggregate_signatures(&params, &aggr_vk, &attributes, &sigs[1..], Some(&[2, 3]))
.unwrap();
let aggr_sig = aggregate_signatures_and_verify(
&params,
&aggr_vk,
&attributes,
&sigs[1..],
Some(&[2, 3]),
)
.unwrap();
let theta = prove_bandwidth_credential(
&params,
+1
View File
@@ -10,6 +10,7 @@ use crate::error::{CoconutError, Result};
use crate::utils::hash_g1;
/// System-wide parameters used for the protocol
#[derive(Clone)]
pub struct Parameters {
/// Generator of the G1 group
g1: G1Affine,
@@ -288,7 +288,6 @@ pub fn verify_credential(
}
// Used in tests only
#[cfg(test)]
pub fn verify(
params: &Parameters,
verification_key: &VerificationKey,
+6 -2
View File
@@ -75,8 +75,12 @@ pub fn theta_from_keys_and_attributes(
attributes.extend_from_slice(public_attributes);
// Randomize credentials and generate any cryptographic material to verify them
let signature =
aggregate_signature_shares(params, &verification_key, &attributes, &signature_shares)?;
let signature = aggregate_signature_shares_and_verify(
params,
&verification_key,
&attributes,
&signature_shares,
)?;
// Generate cryptographic material to verify them
let theta = prove_bandwidth_credential(
+1 -1
View File
@@ -10,7 +10,7 @@ repository = { workspace = true }
[dependencies]
log = { workspace = true }
rand = { version = "0.7.3", features = ["wasm-bindgen"] }
rand_distr = "0.3"
rand_distr = { workspace = true }
thiserror = { workspace = true }
nym-sphinx-acknowledgements = { path = "acknowledgements" }
+2 -2
View File
@@ -10,9 +10,9 @@ repository = { workspace = true }
[dependencies]
nym-crypto = { path = "../../crypto", features = ["asymmetric"] } # all addresses are expressed in terms on their crypto keys
nym-sphinx-types = { path = "../types", features = ["sphinx"] } # we need to be able to refer to some types defined inside sphinx crate
serde = "1.0" # implementing serialization/deserialization for some types, like `Recipient`
serde = { workspace = true } # implementing serialization/deserialization for some types, like `Recipient`
thiserror = { workspace = true }
[dev-dependencies]
rand = "0.7"
nym-crypto = { path = "../../crypto", features = ["rand"] }
nym-crypto = { path = "../../crypto", features = ["rand"] }
+1 -1
View File
@@ -8,7 +8,7 @@ license = { workspace = true }
repository = { workspace = true }
[dependencies]
bytes = "1.0"
bytes = { workspace = true }
tokio-util = { workspace = true, features = ["codec"] }
thiserror = { workspace = true }
+1 -1
View File
@@ -8,7 +8,7 @@ license = { workspace = true }
repository = { workspace = true }
[dependencies]
sphinx-packet = { version = "0.1.0", optional = true }
sphinx-packet = { workspace = true, optional = true }
nym-outfox = { path = "../../../nym-outfox", optional = true }
thiserror = { workspace = true }
+7 -5
View File
@@ -12,25 +12,27 @@ license.workspace = true
[dependencies]
async-trait.workspace = true
const_format = "0.2.32"
const_format = { workspace = true }
cosmrs.workspace = true
eyre = "0.6.9"
eyre = { workspace = true }
futures.workspace = true
humantime = { workspace = true }
sha2 = "0.10.8"
serde = { workspace = true, features = ["derive"] }
sqlx = { workspace = true, features = ["runtime-tokio-rustls", "sqlite", "macros", "migrate", "time"] }
tendermint.workspace = true
tendermint-rpc = { workspace = true, features = ["websocket-client", "http-client"] }
thiserror.workspace = true
time = { workspace = true }
tokio = { workspace = true, features = ["full"] }
tokio-stream = "0.1.14"
tokio-util = { version = "0.7.10", features = ["rt"]}
tokio-stream = { workspace = true }
tokio-util = { workspace = true, features = ["rt"] }
tracing.workspace = true
url.workspace = true
# TEMP
nym-bin-common = { path = "../bin-common", features = ["basic_tracing"]}
#nym-bin-common = { path = "../bin-common", features = ["basic_tracing"]}
[build-dependencies]
+23
View File
@@ -0,0 +1,23 @@
# Nyxd Scraper
## Pruning
Similarly to cosmos-sdk, we incorporate pruning into our (scraped) chain data. We attempt to follow their strategies as
closely as possible for convenience's sake. Therefore, the following are available:
### Strategies
The strategies are configured in `config.toml`, with the format `pruning = "<strategy>"` where the options are:
* `default`: only the last 362,880 states(approximately 3.5 weeks worth of state) are kept; pruning at 10 block
intervals
* `nothing`: all historic states will be saved, nothing will be deleted (i.e. archiving node)
* `everything`: 2 latest states will be kept; pruning at 10 block intervals.
* `custom`: allow pruning options to be manually specified through `pruning.keep_recent`, and `pruning.interval`
### Custom Pruning
These are applied if and only if the pruning strategy is `custom`:
* `pruning.keep_recent`: N means to keep all of the last N blocks
* `pruning.interval`: N means to delete old block data from disk every Nth block.
+75 -2
View File
@@ -8,6 +8,7 @@ use crate::error::ScraperError;
use crate::modules::{BlockModule, MsgModule, TxModule};
use crate::rpc_client::RpcClient;
use crate::storage::{persist_block, ScraperStorage};
use crate::PruningOptions;
use futures::StreamExt;
use std::collections::{BTreeMap, HashSet, VecDeque};
use std::ops::{Add, Range};
@@ -18,9 +19,10 @@ use tokio::sync::Notify;
use tokio::time::{interval_at, Instant};
use tokio_stream::wrappers::UnboundedReceiverStream;
use tokio_util::sync::CancellationToken;
use tracing::{debug, error, info, warn};
use tracing::{debug, error, info, instrument, trace, warn};
mod helpers;
pub(crate) mod pruning;
pub(crate) mod types;
const MISSING_BLOCKS_CHECK_INTERVAL: Duration = Duration::from_secs(30);
@@ -40,9 +42,11 @@ impl PendingSync {
}
pub struct BlockProcessor {
pruning_options: PruningOptions,
cancel: CancellationToken,
synced: Arc<Notify>,
last_processed_height: u32,
last_pruned_height: u32,
last_processed_at: Instant,
pending_sync: PendingSync,
queued_blocks: BTreeMap<u32, BlockToProcess>,
@@ -62,6 +66,7 @@ pub struct BlockProcessor {
impl BlockProcessor {
pub async fn new(
pruning_options: PruningOptions,
cancel: CancellationToken,
synced: Arc<Notify>,
incoming: UnboundedReceiver<BlockToProcess>,
@@ -70,11 +75,17 @@ impl BlockProcessor {
rpc_client: RpcClient,
) -> Result<Self, ScraperError> {
let last_processed = storage.get_last_processed_height().await?;
let last_processed_height = last_processed.try_into().unwrap_or_default();
let last_pruned = storage.get_pruned_height().await?;
let last_pruned_height = last_pruned.try_into().unwrap_or_default();
Ok(BlockProcessor {
pruning_options,
cancel,
synced,
last_processed_height: last_processed.try_into().unwrap_or_default(),
last_processed_height,
last_pruned_height,
last_processed_at: Instant::now(),
pending_sync: Default::default(),
queued_blocks: Default::default(),
@@ -131,12 +142,17 @@ impl BlockProcessor {
}
}
let commit_start = Instant::now();
tx.commit()
.await
.map_err(|source| ScraperError::StorageTxCommitFailure { source })?;
crate::storage::log_db_operation_time("committing processing tx", commit_start);
self.last_processed_height = full_info.block.header.height.value() as u32;
self.last_processed_at = Instant::now();
if let Err(err) = self.maybe_prune_storage().await {
error!("failed to prune the storage: {err}");
}
Ok(())
}
@@ -210,6 +226,61 @@ impl BlockProcessor {
Ok(())
}
#[instrument(skip(self))]
async fn prune_storage(&mut self) -> Result<(), ScraperError> {
let keep_recent = self.pruning_options.strategy_keep_recent();
let last_to_keep = self.last_processed_height - keep_recent;
info!(
keep_recent,
oldest_to_keep = last_to_keep,
"pruning the storage"
);
let lowest: u32 = self
.storage
.lowest_block_height()
.await?
.unwrap_or_default()
.try_into()
.unwrap_or_default();
let to_prune = last_to_keep.saturating_sub(lowest);
match to_prune {
v if v > 1000 => warn!("approximately {v} blocks worth of data will be pruned"),
v if v > 100 => info!("approximately {v} blocks worth of data will be pruned"),
0 => trace!("no blocks to prune"),
v => debug!("approximately {v} blocks worth of data will be pruned"),
}
if to_prune == 0 {
return Ok(());
}
self.storage
.prune_storage(last_to_keep, self.last_processed_height)
.await?;
self.last_pruned_height = self.last_processed_height;
Ok(())
}
async fn maybe_prune_storage(&mut self) -> Result<(), ScraperError> {
debug!("checking for storage pruning");
if self.pruning_options.strategy.is_nothing() {
trace!("the current pruning strategy is 'nothing'");
return Ok(());
}
let interval = self.pruning_options.strategy_interval();
if self.last_pruned_height + interval <= self.last_processed_height {
self.prune_storage().await?;
}
Ok(())
}
async fn next_incoming(&mut self, block: BlockToProcess) {
let height = block.height;
@@ -279,6 +350,8 @@ impl BlockProcessor {
async fn startup_resync(&mut self) -> Result<(), ScraperError> {
assert!(self.pending_sync.is_empty());
self.maybe_prune_storage().await?;
let latest_block = self.rpc_client.current_block_height().await? as u32;
if latest_block > self.last_processed_height && self.last_processed_height != 0 {
let request_range = self.last_processed_height + 1..latest_block + 1;
@@ -0,0 +1,122 @@
// Copyright 2024 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::error::ScraperError;
use serde::{Deserialize, Serialize};
pub const DEFAULT_PRUNING_KEEP_RECENT: u32 = 362880;
pub const DEFAULT_PRUNING_INTERVAL: u32 = 10;
pub const EVERYTHING_PRUNING_KEEP_RECENT: u32 = 2;
pub const EVERYTHING_PRUNING_INTERVAL: u32 = 10;
/// We follow cosmos-sdk pruning strategies for conveniences sake.
#[derive(Debug, Default, Clone, Copy, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum PruningStrategy {
/// 'Default' strategy defines a pruning strategy where the last 362880 heights are
/// kept where to-be pruned heights are pruned at every 10th height.
/// The last 362880 heights are kept(approximately 3.5 weeks worth of state) assuming the typical
/// block time is 6s. If these values do not match the applications' requirements, use the "custom" option.
#[default]
Default,
/// 'Everything' strategy defines a pruning strategy where all committed heights are
/// deleted, storing only the current height and last 2 states. To-be pruned heights are
/// pruned at every 10th height.
Everything,
/// 'Nothing' strategy defines a pruning strategy where all heights are kept on disk.
Nothing,
/// 'Custom' strategy defines a pruning strategy where the user specifies the pruning.
Custom,
}
impl PruningStrategy {
pub fn is_custom(&self) -> bool {
matches!(self, PruningStrategy::Custom)
}
pub fn is_nothing(&self) -> bool {
matches!(self, PruningStrategy::Nothing)
}
pub fn is_everything(&self) -> bool {
matches!(self, PruningStrategy::Everything)
}
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize)]
pub struct PruningOptions {
/// keep_recent defines how many recent heights to keep on disk.
pub keep_recent: u32,
/// interval defines the frequency of removing the pruned heights from the disk.
pub interval: u32,
/// strategy defines the currently used kind of [PruningStrategy].
pub strategy: PruningStrategy,
}
impl PruningOptions {
pub fn validate(&self) -> Result<(), ScraperError> {
// if strategy is not set to custom, other options are meaningless since they won't be applied
if !self.strategy.is_custom() {
return Ok(());
}
if self.interval == 0 {
return Err(ScraperError::ZeroPruningInterval);
}
if self.interval < EVERYTHING_PRUNING_INTERVAL {
return Err(ScraperError::TooSmallPruningInterval {
interval: self.interval,
});
}
if self.keep_recent < EVERYTHING_PRUNING_KEEP_RECENT {
return Err(ScraperError::TooSmallKeepRecent {
keep_recent: self.keep_recent,
});
}
Ok(())
}
pub fn nothing() -> Self {
PruningOptions {
keep_recent: 0,
interval: 0,
strategy: PruningStrategy::Nothing,
}
}
pub fn strategy_interval(&self) -> u32 {
match self.strategy {
PruningStrategy::Default => DEFAULT_PRUNING_INTERVAL,
PruningStrategy::Everything => EVERYTHING_PRUNING_INTERVAL,
PruningStrategy::Nothing => 0,
PruningStrategy::Custom => self.interval,
}
}
pub fn strategy_keep_recent(&self) -> u32 {
match self.strategy {
PruningStrategy::Default => DEFAULT_PRUNING_KEEP_RECENT,
PruningStrategy::Everything => EVERYTHING_PRUNING_KEEP_RECENT,
PruningStrategy::Nothing => 0,
PruningStrategy::Custom => self.keep_recent,
}
}
}
impl Default for PruningOptions {
fn default() -> Self {
PruningOptions {
keep_recent: DEFAULT_PRUNING_KEEP_RECENT,
interval: DEFAULT_PRUNING_INTERVAL,
strategy: Default::default(),
}
}
}
+12
View File
@@ -1,6 +1,9 @@
// Copyright 2023 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::block_processor::pruning::{
EVERYTHING_PRUNING_INTERVAL, EVERYTHING_PRUNING_KEEP_RECENT,
};
use tendermint::Hash;
use thiserror::Error;
use tokio::sync::mpsc::error::SendError;
@@ -122,6 +125,15 @@ pub enum ScraperError {
"could not find validator information for {address}; the validator has signed a commit"
)]
MissingValidatorInfoCommitted { address: String },
#[error("pruning.interval must not be set to 0. If you want to disable pruning, select pruning.strategy = \"nothing\"")]
ZeroPruningInterval,
#[error("pruning.interval must not be smaller than {}. got: {interval}. for most aggressive pruning, select pruning.strategy = \"everything\"", EVERYTHING_PRUNING_INTERVAL)]
TooSmallPruningInterval { interval: u32 },
#[error("pruning.keep_recent must not be smaller than {}. got: {keep_recent}. for most aggressive pruning, select pruning.strategy = \"everything\"", EVERYTHING_PRUNING_KEEP_RECENT)]
TooSmallKeepRecent { keep_recent: u32 },
}
impl<T> From<SendError<T>> for ScraperError {
+1
View File
@@ -14,6 +14,7 @@ pub(crate) mod rpc_client;
pub(crate) mod scraper;
pub mod storage;
pub use block_processor::pruning::{PruningOptions, PruningStrategy};
pub use modules::{BlockModule, MsgModule, TxModule};
pub use scraper::{Config, NyxdScraper};
pub use storage::models;
+6
View File
@@ -8,6 +8,7 @@ use crate::modules::{BlockModule, MsgModule, TxModule};
use crate::rpc_client::RpcClient;
use crate::scraper::subscriber::ChainSubscriber;
use crate::storage::ScraperStorage;
use crate::PruningOptions;
use std::path::PathBuf;
use std::sync::Arc;
use tokio::sync::mpsc::{channel, unbounded_channel};
@@ -27,6 +28,8 @@ pub struct Config {
pub rpc_url: Url,
pub database_path: PathBuf,
pub pruning_options: PruningOptions,
}
pub struct NyxdScraperBuilder {
@@ -54,6 +57,7 @@ impl NyxdScraperBuilder {
processing_tx.clone(),
);
let mut block_processor = BlockProcessor::new(
scraper.config.pruning_options,
scraper.cancel_token.clone(),
scraper.startup_sync.clone(),
processing_rx,
@@ -119,6 +123,7 @@ impl NyxdScraper {
}
pub async fn new(config: Config) -> Result<Self, ScraperError> {
config.pruning_options.validate()?;
let storage = ScraperStorage::init(&config.database_path).await?;
Ok(NyxdScraper {
@@ -160,6 +165,7 @@ impl NyxdScraper {
processing_tx.clone(),
);
let block_processor = BlockProcessor::new(
self.config.pruning_options,
self.cancel_token.clone(),
self.startup_sync.clone(),
processing_rx,
+191 -11
View File
@@ -1,9 +1,11 @@
// Copyright 2023 - Nym Technologies SA <contact@nymtech.net>
// SPDX-License-Identifier: Apache-2.0
use crate::storage::log_db_operation_time;
use crate::storage::models::{CommitSignature, Validator};
use sqlx::types::time::OffsetDateTime;
use sqlx::{Executor, Sqlite};
use tokio::time::Instant;
use tracing::{instrument, trace};
#[derive(Clone)]
@@ -25,10 +27,36 @@ impl StorageManager {
Ok(())
}
pub(crate) async fn get_lowest_block(&self) -> Result<Option<i64>, sqlx::Error> {
trace!("get_lowest_block");
let start = Instant::now();
let maybe_record = sqlx::query!(
r#"
SELECT height
FROM block
ORDER BY height ASC
LIMIT 1
"#,
)
.fetch_optional(&self.connection_pool)
.await?;
log_db_operation_time("get_lowest_block", start);
if let Some(row) = maybe_record {
Ok(row.height)
} else {
Ok(None)
}
}
pub(crate) async fn get_first_block_height_after(
&self,
time: OffsetDateTime,
) -> Result<Option<i64>, sqlx::Error> {
trace!("get_first_block_height_after");
let start = Instant::now();
let maybe_record = sqlx::query!(
r#"
SELECT height
@@ -41,6 +69,7 @@ impl StorageManager {
)
.fetch_optional(&self.connection_pool)
.await?;
log_db_operation_time("get_first_block_height_after", start);
if let Some(row) = maybe_record {
Ok(row.height)
@@ -53,6 +82,9 @@ impl StorageManager {
&self,
time: OffsetDateTime,
) -> Result<Option<i64>, sqlx::Error> {
trace!("get_last_block_height_before");
let start = Instant::now();
let maybe_record = sqlx::query!(
r#"
SELECT height
@@ -65,6 +97,7 @@ impl StorageManager {
)
.fetch_optional(&self.connection_pool)
.await?;
log_db_operation_time("get_last_block_height_before", start);
if let Some(row) = maybe_record {
Ok(row.height)
@@ -79,6 +112,9 @@ impl StorageManager {
start_height: i64,
end_height: i64,
) -> Result<i32, sqlx::Error> {
trace!("get_signed_between");
let start = Instant::now();
let count = sqlx::query!(
r#"
SELECT COUNT(*) as count FROM pre_commit
@@ -94,6 +130,7 @@ impl StorageManager {
.fetch_one(&self.connection_pool)
.await?
.count;
log_db_operation_time("get_signed_between", start);
Ok(count)
}
@@ -103,7 +140,10 @@ impl StorageManager {
consensus_address: &str,
height: i64,
) -> Result<Option<CommitSignature>, sqlx::Error> {
sqlx::query_as(
trace!("get_precommit");
let start = Instant::now();
let res = sqlx::query_as(
r#"
SELECT * FROM pre_commit
WHERE validator_address = ?
@@ -113,14 +153,20 @@ impl StorageManager {
.bind(consensus_address)
.bind(height)
.fetch_optional(&self.connection_pool)
.await
.await?;
log_db_operation_time("get_precommit", start);
Ok(res)
}
pub(crate) async fn get_block_validators(
&self,
height: i64,
) -> Result<Vec<Validator>, sqlx::Error> {
sqlx::query_as!(
trace!("get_block_validators");
let start = Instant::now();
let res = sqlx::query_as!(
Validator,
r#"
SELECT * FROM validator
@@ -133,16 +179,28 @@ impl StorageManager {
height
)
.fetch_all(&self.connection_pool)
.await
.await?;
log_db_operation_time("get_block_validators", start);
Ok(res)
}
pub(crate) async fn get_validators(&self) -> Result<Vec<Validator>, sqlx::Error> {
sqlx::query_as("SELECT * FROM validator")
trace!("get_validators");
let start = Instant::now();
let res = sqlx::query_as("SELECT * FROM validator")
.fetch_all(&self.connection_pool)
.await
.await?;
log_db_operation_time("get_validators", start);
Ok(res)
}
pub(crate) async fn get_last_processed_height(&self) -> Result<i64, sqlx::Error> {
trace!("get_last_processed_height");
let start = Instant::now();
let maybe_record = sqlx::query!(
r#"
SELECT last_processed_height FROM metadata
@@ -150,6 +208,7 @@ impl StorageManager {
)
.fetch_optional(&self.connection_pool)
.await?;
log_db_operation_time("get_last_processed_height", start);
if let Some(row) = maybe_record {
Ok(row.last_processed_height)
@@ -157,6 +216,27 @@ impl StorageManager {
Ok(-1)
}
}
pub(crate) async fn get_pruned_height(&self) -> Result<i64, sqlx::Error> {
trace!("get_pruned_height");
let start = Instant::now();
let maybe_record = sqlx::query!(
r#"
SELECT last_pruned_height FROM pruning
"#
)
.fetch_optional(&self.connection_pool)
.await?;
log_db_operation_time("get_pruned_height", start);
if let Some(row) = maybe_record {
Ok(row.last_pruned_height)
} else {
Ok(-1)
}
}
}
// make those generic over executor so that they could be performed over connection pool and a tx
@@ -170,7 +250,8 @@ pub(crate) async fn insert_validator<'a, E>(
where
E: Executor<'a, Database = Sqlite>,
{
trace!("insert validator");
trace!("insert_validator");
let start = Instant::now();
sqlx::query!(
r#"
@@ -183,6 +264,7 @@ where
)
.execute(executor)
.await?;
log_db_operation_time("insert_validator", start);
Ok(())
}
@@ -200,7 +282,8 @@ pub(crate) async fn insert_block<'a, E>(
where
E: Executor<'a, Database = Sqlite>,
{
trace!("insert block");
trace!("insert_block");
let start = Instant::now();
sqlx::query!(
r#"
@@ -217,6 +300,7 @@ where
)
.execute(executor)
.await?;
log_db_operation_time("insert_block", start);
Ok(())
}
@@ -233,7 +317,8 @@ pub(crate) async fn insert_precommit<'a, E>(
where
E: Executor<'a, Database = Sqlite>,
{
trace!("insert precommit");
trace!("insert_precommit");
let start = Instant::now();
sqlx::query!(
r#"
@@ -249,6 +334,7 @@ where
)
.execute(executor)
.await?;
log_db_operation_time("insert_precommit", start);
Ok(())
}
@@ -270,7 +356,8 @@ pub(crate) async fn insert_transaction<'a, E>(
where
E: Executor<'a, Database = Sqlite>,
{
trace!("insert transaction");
trace!("insert_transaction");
let start = Instant::now();
sqlx::query!(
r#"
@@ -298,6 +385,7 @@ where
)
.execute(executor)
.await?;
log_db_operation_time("insert_transaction", start);
Ok(())
}
@@ -313,7 +401,8 @@ pub(crate) async fn insert_message<'a, E>(
where
E: Executor<'a, Database = Sqlite>,
{
trace!("insert message");
trace!("insert_message");
let start = Instant::now();
sqlx::query!(
r#"
@@ -330,6 +419,7 @@ where
)
.execute(executor)
.await?;
log_db_operation_time("insert_message", start);
Ok(())
}
@@ -343,10 +433,100 @@ where
E: Executor<'a, Database = Sqlite>,
{
trace!("update_last_processed");
let start = Instant::now();
sqlx::query!("UPDATE metadata SET last_processed_height = ?", height)
.execute(executor)
.await?;
log_db_operation_time("update_last_processed", start);
Ok(())
}
#[instrument(skip(executor))]
pub(crate) async fn update_last_pruned<'a, E>(height: i64, executor: E) -> Result<(), sqlx::Error>
where
E: Executor<'a, Database = Sqlite>,
{
trace!("update_last_pruned");
let start = Instant::now();
sqlx::query!("UPDATE pruning SET last_pruned_height = ?", height)
.execute(executor)
.await?;
log_db_operation_time("update_last_pruned", start);
Ok(())
}
pub(crate) async fn prune_blocks<'a, E>(oldest_to_keep: i64, executor: E) -> Result<(), sqlx::Error>
where
E: Executor<'a, Database = Sqlite>,
{
trace!("prune_blocks");
let start = Instant::now();
sqlx::query!("DELETE FROM block WHERE height < ?", oldest_to_keep)
.execute(executor)
.await?;
log_db_operation_time("prune_blocks", start);
Ok(())
}
pub(crate) async fn prune_pre_commits<'a, E>(
oldest_to_keep: i64,
executor: E,
) -> Result<(), sqlx::Error>
where
E: Executor<'a, Database = Sqlite>,
{
trace!("prune_pre_commits");
let start = Instant::now();
sqlx::query!("DELETE FROM pre_commit WHERE height < ?", oldest_to_keep)
.execute(executor)
.await?;
log_db_operation_time("prune_pre_commits", start);
Ok(())
}
pub(crate) async fn prune_transactions<'a, E>(
oldest_to_keep: i64,
executor: E,
) -> Result<(), sqlx::Error>
where
E: Executor<'a, Database = Sqlite>,
{
trace!("prune_transactions");
let start = Instant::now();
sqlx::query!(
"DELETE FROM \"transaction\" WHERE height < ?",
oldest_to_keep
)
.execute(executor)
.await?;
log_db_operation_time("prune_transactions", start);
Ok(())
}
pub(crate) async fn prune_messages<'a, E>(
oldest_to_keep: i64,
executor: E,
) -> Result<(), sqlx::Error>
where
E: Executor<'a, Database = Sqlite>,
{
trace!("prune_messages");
let start = Instant::now();
sqlx::query!("DELETE FROM message WHERE height < ?", oldest_to_keep)
.execute(executor)
.await?;
log_db_operation_time("prune_messages", start);
Ok(())
}
+50 -1
View File
@@ -5,7 +5,8 @@ use crate::block_processor::types::{FullBlockInformation, ParsedTransactionRespo
use crate::error::ScraperError;
use crate::storage::manager::{
insert_block, insert_message, insert_precommit, insert_transaction, insert_validator,
update_last_processed, StorageManager,
prune_blocks, prune_messages, prune_pre_commits, prune_transactions, update_last_processed,
update_last_pruned, StorageManager,
};
use crate::storage::models::{CommitSignature, Validator};
use sqlx::types::time::OffsetDateTime;
@@ -15,6 +16,7 @@ use std::path::Path;
use tendermint::block::{Commit, CommitSig};
use tendermint::Block;
use tendermint_rpc::endpoint::validators;
use tokio::time::Instant;
use tracing::{debug, error, info, instrument, trace, warn};
mod helpers;
@@ -28,6 +30,19 @@ pub struct ScraperStorage {
pub(crate) manager: StorageManager,
}
pub(crate) fn log_db_operation_time(op_name: &str, start_time: Instant) {
let elapsed = start_time.elapsed();
let formatted = humantime::format_duration(elapsed);
match elapsed.as_millis() {
v if v > 10000 => error!("{op_name} took {formatted} to execute"),
v if v > 1000 => warn!("{op_name} took {formatted} to execute"),
v if v > 100 => info!("{op_name} took {formatted} to execute"),
v if v > 10 => debug!("{op_name} took {formatted} to execute"),
_ => trace!("{op_name} took {formatted} to execute"),
}
}
impl ScraperStorage {
#[instrument]
pub async fn init<P: AsRef<Path> + Debug>(database_path: P) -> Result<Self, ScraperError> {
@@ -65,6 +80,32 @@ impl ScraperStorage {
Ok(storage)
}
#[instrument(skip(self))]
pub async fn prune_storage(
&self,
oldest_to_keep: u32,
current_height: u32,
) -> Result<(), ScraperError> {
let start = Instant::now();
let mut tx = self.begin_processing_tx().await?;
prune_messages(oldest_to_keep.into(), &mut tx).await?;
prune_transactions(oldest_to_keep.into(), &mut tx).await?;
prune_pre_commits(oldest_to_keep.into(), &mut tx).await?;
prune_blocks(oldest_to_keep.into(), &mut tx).await?;
update_last_pruned(current_height.into(), &mut tx).await?;
let commit_start = Instant::now();
tx.commit()
.await
.map_err(|source| ScraperError::StorageTxCommitFailure { source })?;
log_db_operation_time("committing pruning tx", commit_start);
log_db_operation_time("pruning storage", start);
Ok(())
}
#[instrument(skip_all)]
pub async fn begin_processing_tx(&self) -> Result<StorageTransaction, ScraperError> {
debug!("starting storage tx");
@@ -75,6 +116,10 @@ impl ScraperStorage {
.map_err(|source| ScraperError::StorageTxBeginFailure { source })
}
pub async fn lowest_block_height(&self) -> Result<Option<i64>, ScraperError> {
Ok(self.manager.get_lowest_block().await?)
}
pub async fn get_first_block_height_after(
&self,
time: OffsetDateTime,
@@ -155,6 +200,10 @@ impl ScraperStorage {
pub async fn get_last_processed_height(&self) -> Result<i64, ScraperError> {
Ok(self.manager.get_last_processed_height().await?)
}
pub async fn get_pruned_height(&self) -> Result<i64, ScraperError> {
Ok(self.manager.get_pruned_height().await?)
}
}
pub async fn persist_block(
+1 -1
View File
@@ -8,4 +8,4 @@ license = { workspace = true }
repository = { workspace = true }
[dependencies]
pem = "0.8"
pem = { workspace = true }
+3 -3
View File
@@ -11,14 +11,14 @@ anyhow = { workspace = true }
dirs = "4.0"
futures = { workspace = true }
log = { workspace = true }
pin-project = "1.0"
pin-project = { workspace = true }
rand = { version = "0.7.3", features = ["wasm-bindgen"] }
reqwest = { workspace = true }
schemars = { workspace = true, features = ["preserve_order"] }
serde = { workspace = true, features = ["derive"] } # for config serialization/deserialization
tap = "1.0.1"
tap = { workspace = true }
thiserror = { workspace = true }
tokio = { version = "1.24.1", features = ["rt-multi-thread", "net", "signal"] }
tokio = { workspace = true, features = ["rt-multi-thread", "net", "signal"] }
url = { workspace = true }
nym-bandwidth-controller = { path = "../../common/bandwidth-controller" }
+2 -2
View File
@@ -8,7 +8,7 @@ license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
bytes = "1.0"
bytes = { workspace = true }
tokio = { workspace = true, features = [ "net", "io-util", "sync", "macros", "time", "rt-multi-thread" ] }
tokio-util = { workspace = true, features = [ "io" ] } # reason for getting this guy is to to able to port to tokio 1.X more quickly by being able to use
# their `read_buf` [from the util crate] replacement rather than having to rethink/reimplement `AvailableReader` with the new AsyncRead trait definition.
@@ -22,4 +22,4 @@ nym-socks5-requests = { path = "../requests" }
nym-task = { path = "../../task" }
[dev-dependencies]
tokio-test = "0.4.2"
tokio-test = { workspace = true }
+1 -1
View File
@@ -8,7 +8,7 @@ license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
bincode = "1.3.3"
bincode = { workspace = true }
log = { workspace = true }
nym-exit-policy = { path = "../../../common/exit-policy"}
nym-service-providers-common = { path = "../../../service-providers/common" }
+3 -3
View File
@@ -13,8 +13,8 @@ license.workspace = true
async-trait = { workspace = true }
log = { workspace = true }
reqwest = { workspace = true, features = ["json"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1"
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
sqlx = { workspace = true, features = ["runtime-tokio-rustls", "chrono"]}
thiserror = { workspace = true }
tokio = { version = "1.24.1", features = [ "time" ] }
tokio = { workspace = true, features = ["time"] }
+3 -3
View File
@@ -7,14 +7,14 @@ license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
aes-gcm = { version = "0.10.1" }
argon2 = { version = "0.5.0" }
aes-gcm = { workspace = true }
argon2 = { workspace = true }
generic-array = { workspace = true, features = ["zeroize"] }
rand = "0.8.5"
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true, optional = true }
thiserror = { workspace = true }
zeroize = { version = "1.6.0", features = ["zeroize_derive"] }
zeroize = { workspace = true, features = ["zeroize_derive"] }
[target.'cfg(target_env = "wasm32-unknown-unknown")'.dependencies]
getrandom = { version = "0.2", features = ["js"] }
+1 -1
View File
@@ -13,7 +13,7 @@ license.workspace = true
[dependencies]
thiserror.workspace = true
tokio = { workspace = true, features = ["rt-multi-thread", "net", "io-util", "time", "sync", "macros"] }
etherparse = "0.13.0"
etherparse = { workspace = true }
log.workspace = true
nym-wireguard-types = { path = "../wireguard-types", optional = true }
+1 -1
View File
@@ -159,7 +159,7 @@ impl TunDevice {
"add",
&format!("{}/{}", ipv6, netmaskv6),
"dev",
&tun.name(),
(tun.name()),
])
.output()?;
Ok(tun)
+6 -6
View File
@@ -9,16 +9,16 @@ license.workspace = true
[dependencies]
base64 = "0.21.4"
eyre = "0.6.5"
hmac = "0.12.1"
eyre = { workspace = true }
hmac = { workspace = true }
itertools = "0.11"
log = { workspace = true }
reqwest = { workspace = true }
schemars = "0.8"
serde = { version = "1.0", features = ["derive"] }
schemars = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
sha2 = "0.10.8"
strum = { version = "0.25", features = ["derive"] }
strum = { workspace = true, features = ["derive"] }
thiserror = { workspace = true }
ts-rs = { workspace = true }
url = { workspace = true }
@@ -34,7 +34,7 @@ nym-config = { path = "../../common/config" }
nym-crypto = { path = "../../common/crypto", features = ["asymmetric"] }
[dev-dependencies]
tempfile = "3.3.0"
tempfile = { workspace = true }
[features]
default = []
+3 -1
View File
@@ -50,7 +50,7 @@ pub struct DelegationWithEverything {
pub accumulated_by_delegates: Option<DecCoin>,
pub accumulated_by_operator: Option<DecCoin>,
pub block_height: u64,
pub delegated_on_iso_datetime: String,
pub delegated_on_iso_datetime: Option<String>,
pub cost_params: Option<MixNodeCostParams>,
pub avg_uptime_percent: Option<u8>,
@@ -60,6 +60,8 @@ pub struct DelegationWithEverything {
pub uses_vesting_contract_tokens: bool,
pub unclaimed_rewards: Option<DecCoin>,
pub errors: Option<String>,
// DEPRECATED, IF POSSIBLE TRY TO DISCONTINUE USE OF IT!
pub pending_events: Vec<DelegationEvent>,
pub mixnode_is_unbonding: Option<bool>,
+2 -2
View File
@@ -41,7 +41,7 @@ wasm-storage = { path = "../storage" }
# logging them with `console.error`. This is great for development, but requires
# all the `std::fmt` and `std::panicking` infrastructure, so isn't great for
# code size when deploying.
console_error_panic_hook = { version = "0.1", optional = true }
console_error_panic_hook = { workspace = true, optional = true }
[features]
default = ["console_error_panic_hook"]
default = ["console_error_panic_hook"]
-1
View File
@@ -3,7 +3,6 @@
use crate::storage::wasm_client_traits::WasmClientStorageError;
use crate::topology::WasmTopologyError;
use js_sys::Promise;
use nym_client_core::client::base_client::storage::gateways_storage::BadGateway;
use nym_client_core::error::ClientCoreError;
use nym_crypto::asymmetric::identity::Ed25519RecoveryError;
+1 -14
View File
@@ -32,17 +32,4 @@ pub use nym_validator_client::{DirectSigningReqwestRpcNyxdClient, QueryReqwestRp
pub use nym_validator_client::client::IdentityKey;
#[cfg(target_arch = "wasm32")]
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
#[cfg(target_arch = "wasm32")]
pub fn set_panic_hook() {
// When the `console_error_panic_hook` feature is enabled, we can call the
// `set_panic_hook` function at least once during initialization, and then
// we will get better error messages if our code ever panics.
//
// For more details see
// https://github.com/rustwasm/console_error_panic_hook#readme
#[cfg(feature = "console_error_panic_hook")]
console_error_panic_hook::set_once();
}
pub use wasm_utils::set_panic_hook;
+1 -1
View File
@@ -14,7 +14,7 @@ js-sys = { workspace = true }
wasm-bindgen = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde-wasm-bindgen = { workspace = true }
indexed_db_futures = { version = " 0.3.0"}
indexed_db_futures = { workspace = true }
thiserror = { workspace = true }
nym-store-cipher = { path = "../../store-cipher", features = ["json"] }
+2 -2
View File
@@ -12,9 +12,9 @@ futures = { workspace = true }
js-sys = { workspace = true }
wasm-bindgen = { workspace = true }
wasm-bindgen-futures = { workspace = true }
getrandom = { workspace = true, features=["js"], optional = true }
getrandom = { workspace = true, features = ["js"], optional = true }
gloo-utils = { workspace = true }
gloo-net = { version = "0.3.1", features = ["websocket"], optional = true }
gloo-net = { workspace = true, features = ["websocket"], optional = true }
#gloo-net = { path = "../../../../gloo/crates/net", features = ["websocket"], optional = true }
# we don't want entire tokio-tungstenite, tungstenite itself is just fine - we just want message and error enums
+1 -1
View File
@@ -21,7 +21,7 @@ macro_rules! wasm_error {
impl From<$struct> for js_sys::Promise {
fn from(value: $struct) -> Self {
Promise::reject(&value.into())
js_sys::Promise::reject(&value.into())
}
}
};

Some files were not shown because too many files have changed in this diff Show More