88 Commits

Author SHA1 Message Date
mfahampshire 924d7d1ccc Enforce ordering of [package] fields in cargo.toml files 2026-04-17 07:49:50 +01:00
benedetta davico 46b9d5374b Merge pull request #6271 from nymtech/bugfix/data-observatory
Fix migrations in the Data Observatory
2026-02-11 16:02:43 +01:00
mfahampshire cf3fd00350 Max/crates io prep v2 (#6270)
* - standardise versions for all nym-sdk workspace dependencies
- prepend sqlx-pool-guard with 'nym-'

* Test remove nym-api from deps

* Add oneliner to client_pool doc comments

* Add note to commented out docs.rs link in sdk

* remove nym-api from script

* add publishing file

* bring non-binary / contract / tools into workspace version

* added more info to publishing.md

* make deps workspace version

* remove uploaded sphinx-types crate from script

* remove erroueously included ignore-defaults

* add zeroise to feature

* chore: Release

* add topology to batch

* more cargo versioning

* more cargo versioning - wasm utils

* more cargo versioning - wasm utils

* Add publish=false to manifest for cargo workspaces / crates.io
publishing exclusion

* remove script now switched to manifest based exclusion

* rename import based on rename of contracts-common dep

* Making workspace versions for publication + removing unnecessary crates
from publication

* Remove OOD info from publishing sdk guide

* rename contract imports + remove package

* temp commit: continuing with removal of path from cargo manifest and
replacing with workspace version import for publication

* continuing with cargo.toml updates

* dryrun only erroring on known version problem crates

* remove old published-crates file

* Minor comment change

* remove default features warning

* Additional info on workspace dep comment re publish list

* Add missing description to cargo.toml

* Fix missing feature flags

* Add missing descriptions

* Fix remaining path import

* Add workspace repo / homepage / documentation links to cargo.toml files

* remove workspace version from excluded crate

* Remove todo descriptions

* Minor comment change

* add homepage etc

* move from bls git import to nym_bls_fork crate

* Modify rest of imports from path to workspace import, excluding binaries

* add directory/homepage info

* fix cargo fmt

* add notes to gitignore

* better solution to contracts/ experiment

* wasm -> nym_wasm crate renaming

* fix fatfinger

* add metadata to ecash cargo.toml

* stub publishing guide

* fix misrevolved netlink- version

* Fixes and block publication of rebase re: LP

* first pass @ workflows
2026-01-19 13:19:45 +00:00
Mark Sinclair 18e72c90df run the migrations from the data observatory and not the base chain scraper 2025-12-04 16:06:56 +00:00
Mark Sinclair f2091cc9d6 Data Observatory (#6172)
* rename nyxd-scraper to sqlite

wip: made storage mostly generic minus modules

changed error types to make modules dyn compatible

implemented traits for sqlite instance

using sqlite instance for rewarder and chain watcher

psql scaffolding

initial postgres support - missing some proto -> json parsing

use postgres in chain scraper

added message registry to block processor

message content parsing in psql

involved addresses

adding null value for logs

Revert "use postgres in chain scraper"

This reverts commit 83c84bfd2d.

using SignerInfo proto definitions for db serialisation

added ibc messages to MessageRegistry

* add the data observatory

* add the data observatory

* move message parsing and change webhook

* handle wasm messages in a module

* formatting and clippy

* copy shared migrations and add comments to ignore file to explain

* update offline queries

* change to clap args and use url::Url to parse args

* tidy up README, startup info, typos

* tidy up validator rewarder

* lock file

* change webhook module from msg to tx handler

* ignore profiler output

* add missing things and make clippy happy

* updated cosmrs version used by the nym wallet

* add glob to workspace dependencies

* rename migration files

* remove copying from shared migrations

* duplicate shared migrations to keep things simple

* add check for manual migration sync that will fail on `cargo build` in CI

* build.rs checks data observatory migrations have content of all shared scraper migrations and errors on changes or new files

* update runner

* add reset target to make file

* process events and logs

* migrations - remove unnecessary columns

* update offline queries

* chore: run cargo fmt

* fix up: inpsect_err instead of map_err

---------

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
Co-authored-by: benedettadavico <benedetta.davico@gmail.com>
2025-12-02 21:27:22 +00:00
Jędrzej Stuczyński 827c13b69e moved nym-gateway-probe to monorepo and updated rust-edition to 2024 (#6094)
dont build netstack in CI

additional rust 2024 fixes

fixes

removed temp.rs

first round of cleanup

removed duplicated NS types

moved gateway probe to the monorepo
2025-10-08 16:17:43 +01:00
Simon Wicky 6a028417ad [chore] Clippy fix (#6060)
* clippy multiple of fix

* removed dead code?

* huh?

* ci fixes
2025-09-26 11:58:59 +02:00
Drazen Urch 90e9e3cff8 Domain fronting integration (#5974)
* feat: unify HTTP client creation and enable domain fronting

Enhanced the base nym_http_api_client to reduce fragmentation and enable domain fronting:

- Added SerializationFormat enum for explicit JSON/bincode choice (no auto-detection)
- Added from_network() method to create clients from NymNetworkDetails with domain fronting
- Added with_bincode() builder method for explicit serialization configuration
- Set Accept header based on serialization preference
- Added deprecation paths for NymApiClient wrapper and nym_api::Client re-export
- Enabled domain fronting support via network defaults feature

This is part of a broader effort to consolidate HTTP client implementations across the codebase,
reducing ~500 lines of wrapper code and providing automatic domain fronting for censorship resistance.

* feat: migrate NymApiClient usage to unified HTTP client

- Wire up domain fronting configuration in NymNetworkDetails
- Implement NymApiClientExt trait for base nym_http_api_client::Client
- Migrate direct NymApiClient usage in multiple components:
  - nym-network-monitor
  - verloc measurements
  - connection tester
  - coconut/ecash client
  - validator rewarder
- Add Copy derive to ApiUrlConst to enable iteration
- Update error handling and Display implementations

This enables automatic domain fronting for all Nym API calls via the configured CDN front hosts.

* fix: resolve all compilation errors after NymApiClient migration

- Add missing nym-http-api-client dependencies to multiple crates
- Add NymApiClientExt trait imports where needed
- Fix type mismatches from NymApiClient to unified Client
- Add error conversions for NymAPIError in various error enums
- Implement missing trait methods (get_current_rewarded_set, get_all_basic_nodes_with_metadata, get_all_described_nodes)
- Fix type conversions for RewardedSetResponse in network monitor
- Update all API client instantiation to use new unified HTTP client

* feat: complete migration to unified HTTP client and fix all compilation errors

- Added missing NymApiClientExt trait methods (get_all_expanded_nodes, change_base_urls)
- Fixed all compilation errors across the workspace
- Updated nym-node to use unified client instead of deprecated NymApiClient
- Fixed type conversions for RewardedSetResponse → EpochRewardedSet
- Added nym-http-api-client dependency where needed
- Updated all examples and documentation to use new client API

* fix: provide all API URLs for automatic failover in endpoint rotation

Previously, when rotating API endpoints, only a single URL was provided to the
HTTP client, defeating the purpose of having multiple URLs for resilience.

Changes:
- NymApiTopologyProvider now provides all URLs in rotated order when switching endpoints
- NymApisClient similarly provides all URLs starting from the working endpoint
- Added clarifying comments for broadcast/exhaustive query methods where single URLs are intentionally used
- This enables the HTTP client's built-in failover mechanism while maintaining endpoint rotation behavior

The fix ensures that if the primary endpoint fails, the client can automatically
failover to alternative endpoints without manual intervention, improving overall
network resilience.

* Update common/client-core/src/client/base_client/mod.rs

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>

* Remove error generics, address PR comments

* Explicit warning on missing fronting configuration

* Assorted CI fixes

* Registry proc-macro

* Rename macro

* Syn workspace version

* Where do we need to put inventory

* Ergonomics and call sites, incept the builder

* fix: Address critical issues in client configuration registry implementation

- Fixed HeaderMapInit parsing bug that would cause compilation errors
- Added comprehensive documentation with usage examples and DSL reference
- Improved error handling with better error messages for invalid headers
- Added test coverage for both macro and registry functionality
- Added debug inspection capabilities for registered configurations
- Fixed module name conflicts in tests by using separate modules

All tests now passing:
- 7 macro tests validating DSL parsing and code generation
- 4 registry tests verifying configuration collection and application

* Use default value for the ports until api is deployed

* Feature/improved http error (#6025)

* use display impl for urls

* feat: attempt to add more details to reqwest errors

* temporarily restored GenericRequestFailure variant

* another restoration

* cleanup

* Some debug tooling, and default timeout fix

* Fix user-agent override

* Fix various wasm things

---------

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
Co-authored-by: Bogdan-Ștefan Neacşu <bogdan@nymtech.net>
2025-09-15 14:32:15 +02:00
Jędrzej Stuczyński 0ee387d983 Feature/cancellation migration (#6014)
* squashing work on using cancellation in nym crates

making nym-task wasm compilable

removed sending of status messages

replaced TaskManager with ShutdownManager in the validator rewarder

additional helpers for ShutdownManager

simplified ShutdownToken by removing the name field

TaskClient => ShutdownToken within all client tasks

wip: remove TaskHandle

* track all long-living client tasks

* add task tracking for most top level tasks within nym-node

* improved default builder

* split up cancellation module

* module documentation and unit tests

* nym node fixes and naming consistency

* wasm fixes

* assert_eq => assert

* wasm fixes and made 'run_until_shutdown' take reference instead of ownership

* linux-specific fixes to IpPacketRouter

* post rebasing fixes for signing monitor

* add ShutdownManager constructor to build it from an external token

* applying PR review suggestions
2025-09-10 13:56:39 +01:00
Jędrzej Stuczyński 911b365609 chore: purge temp databases on build (#5984)
* purge any temp databases on build

* updated min rust version

* fixed clippy::manual_abs_diff' in verloc due to updated msrv

* wasm
2025-08-29 11:41:08 +01:00
Jędrzej Stuczyński dc59149a5d squashed feature/ecash-liveness-check (#5890) (#5890)
delay to gruyere

chore: delay to Feta

added threshold information to the response

nym api test clippy

bugfixes and endpoint improvements

expose results on api endpoints

wip: making nym api monitor network signers

added fallback legacy queries to get basic support idea

refactored the code to expose bool-only methods for status

ecash-signer-check lib for obtaining basic ecash signer information
2025-08-05 12:28:42 +01:00
Jędrzej Stuczyński 833114372a bugfix: key-rotation + reply SURBs (#5876)
* wip: changes to surb logic + stronger db typing

* surb invalidation logic

* chore: remove unused deps

* resolving todos

* a lot of additional bugfixes

* 1.88 clippy

* wasm fixes

* wasm clippy

* wallet clippy

* wait for epoch end when setting up new network

* split ReplyController into Sender and Receiver for easier reasoning

* additional reply surbs improvements

includes, but is not limited to: unconditionally reseting sender tag on restart, limiting number of surb re-requests, resetting stale surbs on load

* fixed calculation of number of removed surbs

* add additional calculated field to key rotation info

* DBG: 'request_reply_surbs_for_queue_clearing' temp logs

* fixes for silly mistakes

* conditionally reduce log severity
2025-07-04 16:29:03 +01:00
Andrej Mihajlov 62520c9308 Update sqlx cache 2025-05-30 09:28:48 +02:00
Andrej Mihajlov e65d455c91 Switch counters to i64 since sqlx started giving it back 2025-05-30 09:28:48 +02:00
Jędrzej Stuczyński 5df76ea2a9 Feature/paginated ticketbooks challenge (#5619)
* change ticketbook data request to allow for pagination

* implemented api endpoints on nym-api side

* auxiliary nym-api queries for number of issued ticketbooks

* ensure that challenged issuers support new queries

* addeed persistent identity to the rewarder

* clippy

* stupid chrono feature workaround

* clippy

* debugging issuance verification

* remove redundant closure

* added a minimum issuance threshold
2025-03-21 13:44:25 +00:00
Jędrzej Stuczyński 33992542b1 feature: upgrade cosmwasm to 2.2 (#5479)
* updated contracts to cosmwasm2.2 and fixed build issues

* removed old coconut contract code + additional dkg fixes

* replace deprecated to_binary and from_binary functions

* mixnet contract tests compiling

some are failing due to incorrect addresses

* made other contract tests compile

* fixed remaining tests

* allow usage of manually dispatching contract replies

* nym-api test fixes

* removed old toolchain from contracts CI

* linter fixes

* regenerated contract schema

* fixed easy_addr

* further license fixes

* post rebase fixes + update to 2.2.2

* change ci runner

* minor CI adjustments

* change wallet CI to use node 20

* more CI changes...

* run cosmwasm-check against release contracts

* test ci changes

* wip...
2025-03-21 13:43:35 +00:00
Fran Arbanas b709d3ba0b Fix/pull from harbor (#5521)
* fix: pull from harbor instead of dockerhub

* add remaining

* add comments saying that these changes will only work with VPN
2025-02-28 14:01:33 +01:00
Jędrzej Stuczyński a6c586a33b chore :update version of chain watcher and validator rewarder (#5394) 2025-01-27 15:47:37 +00:00
dynco-nym 41fb17a31b Extend swagger docs (#5235)
* WIP adding derive(ToSchema)

* Derive ToSchema for more types

* ContractBuildInformation on /nym_contracts_detailed

* rustfmt

* Add cfg_attr

* A bunch of annotations

* Compiles with utoipa 5.2

* WIP

* Post rebase fixes

* Gitattributes to ignore .sqlx diffs

* generate Sqlx schema files

* Improvements

* Move ecash schema out of ecash crate

* Move redocly config to nym-api/

* Move redocly config to nym-api/

* Remove ErrorResponse

* Move generated openapi spec to .gitignore

* Include BSL licence

* Remove utoipa from ecash toml file

* Remove placeholder annotations

* Chain-watcher rebase changes

* Update licence info

* Treat Scalar as String in OpenAPI
2024-12-20 12:18:45 +01:00
Sachin Kamath 6e66cc2467 validator-rewarder: fix errors 2024-12-19 14:10:28 +00:00
Jędrzej Stuczyński d8a6ca48c1 implemented starting block logic inside the chain scraper itself 2024-12-19 14:10:28 +00:00
Mark Sinclair 572875058d Add config, overrides and CLI 2024-12-19 14:10:28 +00:00
Sachin Kamath a884aee1e9 fix review comments 2024-12-19 14:10:28 +00:00
Sachin Kamath c99a240ed4 nyxd-scraper: add config to make pre-commit storage optional 2024-12-19 14:10:28 +00:00
dynco-nym b4f51baf94 Change sqlite journal mode to WAL (#5213)
* Change sqlite journal mode to WAL

* Synchronous mode & auto vacuum

* Bump probe git ref to 1.1.0
2024-12-16 16:40:02 +01:00
Fran Arbanas 01e6a77cf1 feat: add whitelist overrides to rewarder validator (#5138)
* feat: add whitelist overrides to rewarder validator

* cleanup by using account id instead of string
2024-11-15 13:18:40 +01:00
Jędrzej Stuczyński a348ff43b0 feature: rewarding for ticketbook issuance (#5112)
* fixed pagination for querying for validators

* wip: decoupling block signing from ticketbook issuance

* added ecash contract query for latest deposit

* parking the branch: wrappers for merkle tree for issued ticketbooks

* make nym-api store merkle trees of issued ticketbooks

* nym-api route for returning all deposits alongside merkle root

* return index alongside deposit id

* persisting merkle index alongside issued ticketbook details

* wip

* responses for issued deposit challenges

* nym-api cleanup

* verification of issued partial ticketbooks

* cleanup of rewarder code

* make the rest of codebase compile

* updated config file

* improved logging

* fixed division by zero if there were no ticketbooks issued in a day

* using correct budget when rewarding operators

* fixed routes for issued data

* fixed ecash test fixture

* fixed incorrect deserialisation of expiration_date param

* additional bugfixes for ticketbook issuance

* more fixes and updated tests

* fixed formatting after rebasing

* updated schema

* fixed edge case unit test
2024-11-14 16:55:02 +00:00
Fran Arbanas 261caae7f6 feat: add Dockerfile and add env vars for clap arguments 2024-11-12 11:59:09 +01:00
Dinko Zdravac 4e65617f08 Bump sqlx to 0.7.4 (#4959)
* Upgrade to v0.7

More goodies

Revert cargo.toml

Fix cargo toml

* Generate offline sqlx schemas

Fix data-observatory

Fix credential-storage

Fix gateway-storage

Fix client-core/-gateways-storage

Generate offline schemas for nym-api as well

* Update nym-api after rebase

---------

Co-authored-by: Andrej Mihajlov <andrej@nymtech.net>
2024-10-18 12:51:41 +02:00
Jędrzej Stuczyński 4ad52accc0 fixed stabilised clippy issue 2024-09-27 18:10:59 +01:00
Jędrzej Stuczyński 8aa5711bee fixed query for historical validator data 2024-09-27 17:48:56 +01:00
Jędrzej Stuczyński 07022314fc fixed typos and formatting 2024-09-27 16:51:45 +01:00
Jędrzej Stuczyński 76c3081470 introduced rewarding resync alongside recovery instructions 2024-09-27 16:41:07 +01:00
Jędrzej Stuczyński d399161d31 adjust 'process_until' command to allow empty stop height 2024-09-27 16:35:53 +01:00
Jędrzej Stuczyński 27fb4ae0cc log error when rewarding fails due to missing blocks 2024-09-27 16:21:27 +01:00
Jędrzej Stuczyński 457c478a03 introduced cli command to process a block range 2024-09-27 15:15:13 +01:00
Jędrzej Stuczyński 5e95992427 introduced cli command to process an individual block 2024-09-27 14:57:39 +01:00
Bogdan-Ștefan Neacşu 18891e5f20 Use ecash credential type for bandwidth value (#4840)
* Use ecash credential type for bandwidth value

* Fill explicit default value for args

* Use up-to-date values for tickets

* Fix sdk example default value

* Another default

* Fix sdk test

* Fix TicketTypeRepr default
2024-09-05 13:19:19 +02:00
Jędrzej Stuczyński 4c10cebf1b propagated new ticket type through the whole stack 2024-07-30 11:31:49 +01:00
Simon Wicky fc2eedfc66 Another Grand Ecash Squasheroo
add offline ecash library

minor changes in coconut benchmarks

add ecash smart contract

change contract traits from coconut to ecash

first wave of andrew's suggestion

first wave of andrew's suggestion

second wave of andrew's suggestion for ecash lib

andrew's suggestion for ecash contract

licensing commit

safety comments for most unwraps

more unwrap handling

change chrono crate for time

latest cargo lock

error revamp

small visibility fix

small fix

remove indexedmap from contract + some tweaks

add cw2 version in ecash contract

remove envryption key from contract

change types from coconut to ecash types

adapt api model for credential issuance

adapt issued credential storage on API

add signatures cache on API

change API routes for new blind signing

modify issued_credential table

add issuance logic client-side

credential and signature storage client side

utils for credential issuance

first wave of fix

some of andrew's suggestions

remove encryption key from deposit

freepass issuance client side

freepass issuance API side

andrew's suggested fixes

other suggested fix

adapt change from PR below

allow offline verification flag

credential spending models

credential spending models for client

credential preperation for the client

credential preperation for the client

credential storage for spending on client

bloom filter for API

spent credential storage on validators

API route for spending online and offline ecash

API routes in the client lib

credential storage on gateway

ecash verifier to replace coconut verifier

accept credentials on gateway

bandwidth expiration for gateways

client ask for more bandwidth if it runs out

credential import

adapt nym validator rewarder and sdk

fix tests api tests and add constants

cargo fmt and lock and small test fix

cargo fmt and lock and small test fix

cargo lock

move stuff where they belong in ecash and static parameters

move some constants, error handling and phase out time crate

error revamp part 2

secret key by ref instead of clone

change l in wallet and v visibility

rework payinfo

rework monster tuples

fix expiration date signature cloning

minor fixes

final bits and bobs fixes

final bits and bobs fixes

rename l accessor to tickets_spent

wave of fixes

second wave of fixes

change hash domain value

removed benchmark flag

remove useless stringification in storage

nuke Bandwidth voucher

change timestamps to offsetdatetime

key name change

post-rebase fixes

update nym-connect 'time' dep due to broken semver

upload ecash contract to the build server

make wasm zknym-lib compile

but it won't work properly just yet

make wasm zknym-lib compile

but it won't work properly just yet

fix typo in ecash contract deps

make sure to use 0.1.0 sphinx packet

optimise pairings in 'check_vk_pairing'

derive serde for ecash types

simplified g1 tuple byte conversion

further optimise the pairing

unified signature type + renamed nym-api coconut module to ecash

using bincode serialiser for more complex binary types

using multimiller loop instead of rayon for verifying coin indices signatures

batching signature verification wherever possible

feature-locked rayon

clippy

refactor ecash contract a bit + introduce deposit storage

reworked find_proposal_id

various minor fixed

add offline_zk_nyms to nym-node everywhere

add missing #query

change test value to fit new serialization

optimised deposits storage

removed duplicate decompression code

using deposit_id instead of transaction hash

removed freepasses

split up ecash handling

unified shared state

fixed deposit_id parsing

log recovered deposit id

removed online verification

add detailed build info to ecash contract

fixed deserialisation of deposit amount received from nyxd queries

changed deposit to only persist attached pubkey

first iteration of split of verification and redemption

basic tool for setting up new network

expanded the tool with the option to bypass DKG

rename + init network without DKG

setting up locally running apis

ecash key migration

more local functionalities

wip fixing sql schemas

gateway immediately submitting redemption proposal

and getting it passed if valid

most of the gateway logic for split redemption with error recovery

fixed gateway not persisting ecash signers

simplify creation of compatible client

create properly serialised ecash key from the beginning

rebuild missing tickets and proposals on startup

stop ticket issuance during DKG transition

fixing build issues

split out ecash storage on nym-api side

master-verification-key route

caching all the signatures and keys

implemented aggregated routes for nym-apis

swagger UI for ecash endpoints

added explicit annotation for index and expiration signatures

revamped client ticketbook storage

save all recovery information in the same underlying storage

wrapper for bloomfilter

being more aggressive with marking tickets as used

ensure client has correct signatures before making deposit

fix deserialisation of AggregatedExpirationDateSignatureResponse + add ticketbook table

split nym-api ecash routes handlers into multiple files

fixed deserialisation of encoded expiration date

add tt_gamma1 to challenge and change naming for paper consistency

rotating double spending bloomfilter

nym-api test fixes + make sure to insert initial BF params

fixed ecash benchmark code

updated contract schema

updated CI to not upload gateway/mixnode binaries

ticket bandwidth revocation

added default deserialisation for zk nym config

post-rebase fixes
2024-07-30 11:27:21 +01:00
Jon Häggblad 8acd3a0975 Move a few more deps partially to workspace 2024-06-27 10:52:15 +02:00
Jon Häggblad 9badeac832 cargo autoinherit in root workspace 2024-05-20 13:16:25 +02:00
Jędrzej Stuczyński a6fda391ae Feature/rewarder voucher issuance (#4548)
* retrieve ed25519 identities of issuers

* signature verification on issued credentials

* wip

* persisting information about verified deposits, any failures and foul plays

* clippy
2024-05-16 10:15:24 +02:00
Jędrzej Stuczyński f827eb4242 storage pruning implementation + additional logging 2024-04-30 15:53:45 +01:00
Jędrzej Stuczyński 9323ca9339 defined basic pruning types 2024-04-30 12:32:46 +01:00
Jędrzej Stuczyński 4c5351ba60 Feature/one binary to rule them all (#4500) 2024-04-05 19:34:31 +02:00
Jędrzej Stuczyński 7e6feb0cd2 fixed beta clippy: lints + updated cosmwasm to 1.4 2024-03-27 16:04:51 +00:00
Jędrzej Stuczyński 6f3dd9f778 wip 2024-02-19 11:39:53 +00:00
Jędrzej Stuczyński b764fcc756 revamped BandwidthVoucher to allow for different kinds of bandwidth credentials 2024-02-19 11:39:53 +00:00
Jędrzej Stuczyński 0b46e5b753 improved startup log regarding the epoch 2024-02-14 16:36:37 +00:00