84 Commits

Author SHA1 Message Date
mfahampshire a70e68c7bd Max/smolmix docs (#6716)
* Smolmix documentation

* Add smolmix docs: landing page, tutorials, and developer page links

* Add Exit Gateway services page (NR vs IPR) and link from existing docs

* Update auto-generated command and API outputs

* Reorg of tutorials and architecture pages

* License information + remove TODO from docs.rs visibile comment + reorg
readme

* Add versions file for doc-wide versioning

* Relative -> absolute links

* Relative -> absolute links

* Update license + add old tutorial code as examples

* Streamline smolmix docs

* Clippy

* Clean up doc comments

* Last pass

* Add larger file download to list

* set new versions

* Clippy

* Remove blake pin from docs + add version range to root Cargo.toml

* Format example logging

* Remove crate blocked component

* Loose whitespace

* Add doc verification script for inline mdx

* Formatting

* Components regen

* Reorg + tighten text

* Voicing cohesion pass + remove bloated examples

* Voicing cont.

* Reduce max download size

* Small suggested clarifications

* Max/docs voicing consistency (#6769)

* Reduce max download size

* voicing consistency across docs

* New landing order w smolmix

* Tweaks

* Final tweaks
2026-05-13 11:19:44 +00:00
mfahampshire 9db748e8dd Max/sdk docrs (#6566)
* Improve SDK rustdoc and add ARCHITECTURE.md files

- Rewrite lib.rs module docs with quick-start example and module overview
- Add stream example and include_str! ARCHITECTURE.md to mixnet module
- Add ARCHITECTURE.md for mixnet, client_pool, and stream modules
- Add rustdoc to MixnetClientBuilder, MixnetClientSender, MixnetMessageSender
- Add cancel safety and drop behavior annotations to async methods
- Add TcpProxy deprecation notice pointing to stream module

* Fix rustdoc errors and add stepwise comments to remaining examples

Rustdoc fixes:
- Add missing .unwrap() on connect_new example
- Replace broken turbofish intra-doc link in MixnetClientBuilder
- Fix NymProxyServer::new args in tcp_proxy example
- Wrap BandwidthImporter example in scoped block to fix borrow-then-move
- Change misleading "5-hop routing" to "multi-hop routing"
- Fix copy-paste "forget me" in send_remember_me error message
- Fix wrong cargo run command in stream_simple_read_write
- Fix DecayWrapper description

* Cut down doc comment length

* Trimmed down SDK ARCHITECTURE files

* Slim Rust SDK docs and rename opener to dialer

- Merge tour page into SDK landing page, delete tour.mdx
- Trim all three tutorials: cut boilerplate, duplicated code, and misplaced content
- Make FFI page evergreen with Go and C++ snippets, link to repo examples
- Rename "opener" to "dialer" in stream docs, source ARCHITECTURE.md, and rustdoc
- Add reply-to-open arrow in stream mermaid diagram
- Replace remaining Unicode dashes in mermaid flowchart

* - elevate streams in rustdoc: examples on lib.rs, MixnetClient, open_stream, listener
- add stream quick reference to mixnet ARCHITECTURE.md
- add stream types to key types list in ARCHITECTURE.md
- add docs.rs links for AsyncRead/AsyncWrite and stream submodule
- tcp_proxy: replace bold deprecation with warning box

* - replace individual example doc pages with GitHub-linked tables
- add step-by-step inline comments to all SDK example source files
- add doc comments to examples missing them (simple, surb_reply, builder, etc.)
- expand mixnet tutorial with persistent identity and split_sender sections
- add tcpproxy tutorial
- rename "API Reference" to "TypeDoc Reference" in TS SDK sidebar
- rename "Misc" to "Extras" in developer sidebar, move VPN CLI up
- remove echo server from tools
- update message-queue callout to reference actual modules
- fix mixnet/examples redirect collision

* Add missing mut to example code

* Update ARCHITECTURE.md with LP Framing + stream examples with sequencing

* Update doc comment in utils.rs

* Standardise commenting style across Rust SDK examples

* Fix inline doc examples and trim re-export boilerplate

* Update sdk/rust/nym-sdk/examples/bandwidth.rs

Co-authored-by: Simon Wicky <simon@nymtech.net>

* Fix review comments

---------

Co-authored-by: Simon Wicky <simon@nymtech.net>
2026-04-10 10:51:38 +00:00
mfahampshire f648349e82 Max/docs-diataxis-ify (#6494)
* Diatixisify!

* First pass at Typedoc generation for TS SDK

* Remove overview pages

* Fix typos and remove codebase references from docs

Fix typos across network and developer docs: Quorum, available,
cryptosystem, transaction, proportional, Standalone. Remove TODO
placeholder from dVPN protocol page. Strip GitHub source links
from network docs to decouple documentation from repo structure.

* Expand thin landing pages across network and developer docs

- Add intro content to network overview, infrastructure, and reference landing pages
- Expand developer index with "where to start" guide
- Add usage instructions and explanations to all five TS playground pages
- Expand WebSocket client page with setup and message format examples

* Restructure Rust SDK developer docs

- Delete redundant mixnet example, message-helpers, and message-types subpages
- Delete client-pool architecture and example subpages (content folded into landing)
- Delete tcpproxy troubleshooting (folded into landing page)
- Add deprecation notices to TcpProxy pages, pointing to Stream module
- Add stream module docs: landing page, architecture, tutorial, and 4 example pages
- Add mixnet and client-pool tutorials
- Add SDK tour page
- Update navigation and landing pages with docs.rs links

* Restructure TS SDK developer docs

- Merge overview, installation, and getting started into TS SDK landing page
- Fold FAQ content into bundling/troubleshooting section
- Delete redundant overview, installation, start, and FAQ pages
- Update internal links in browsers.mdx and native.mdx
- Update navigation and example page imports

* Flatten and expand APIs section

- Collapse nested API subpages into single pages with inline Redoc embeds
- Rewrite introduction as landing page with decision table
- Add endpoint categories, quick curl examples to each API page
- Mark Explorer API as deprecated
- Move NS API deployment guide to operators/performance-and-testing
- Fix dangling /apis/nym-api/mainnet link in network-components
- Remove sandbox endpoints from all API pages

* Add redirects for moved and deleted pages

- Add 25 redirects covering TS SDK, Rust SDK, APIs, and network sections
- Fix dangling /developers/typescript/start link in operators changelog

* Replace individual example doc pages with GitHub-linked tables, expand tutorials

- replace individual example doc pages with GitHub-linked tables
- expand mixnet tutorial with persistent identity and split_sender sections
- add tcpproxy tutorial
- rename "API Reference" to "TypeDoc Reference" in TS SDK sidebar
- rename "Misc" to "Extras" in developer sidebar, move VPN CLI up
- remove echo server from tools
- update message-queue callout to reference actual modules
- fix mixnet/examples redirect collision

* Add SEO frontmatter, validate encryption standards, clean up URLs

- add title/description/schemaType/section/lastUpdated frontmatter to 48
  pages across developers, network, and APIs sections
- remove network/.archive/ directory (compare against develop instead)
- update nymtech.net → nym.com for website/blog links (keep infra URLs)
- add native proxy "in progress" callout for Rust/C/Go

* API-scraper update (#6598)

* read nodes and locations

* update python-prebuild.sh

* Address PR #6494 review feedback
- Use "mode" consistently instead of "role" on nym-nodes page
- Replace "staking" with "bonding" for NYM token collateral
- Wire up auto-scraped node counts via TimeNow + nodes-count.json
- Fix broken licensing images: download CC icons locally, replace inline HTML
- Fix 9 stale redirects pointing through deleted /network/architecture path

* Fix linkcheck errors
- Fix stale cross-links: /network/concepts/ → /network/mixnet-mode/
- Replace README.md references with globals.md in TypeDoc output
- Add entryFileName: globals to typedoc.json configs to prevent recurrence

* Fix remaining stale /network/architecture links
- zk-nym-overview: architecture/nyx#nym-api → /network/infrastructure/nyx#nym-api
- setup: network/architecture → /network/overview

* Remove accidentally re-included architecture.md file from rebase

* Standardize tutorials, document examples, add llms.txt, apply tone fixes

- Expand Rust SDK tutorials with step-by-step structure; document all SDK examples across mixnet, client-pool, and tcpproxy pages
- Add llms.txt generation script, wire into build and CI workflows
- Apply tone/style fixes: deduplicate callouts, vary sentence structure, standardize voice consistency across changed pages

* Consolidate redundant network overview docs

* Trim dev docs: git-first imports, stream notice, collapse TcpProxy

* Update tutorial

* Refresh auto-generated API and command outputs

* Update network section docs

* Update developer and API docs: reusable components, stream protocol, conventions, tutorial fixes

* Fix Rust SDK tutorial bugs: setup_env, port conflicts, logging,
open_stream race condition

* Update stream.mdx

* Remove docs.rs link from Stream overview for the moment

* add llms.txt and llms-full.txt note to readme

---------

Co-authored-by: import this <97586125+serinko@users.noreply.github.com>
2026-04-09 15:25:31 +00:00
dynco-nym 89de989ad1 Optimize GW probe in NS agent (#6636)
* WIP

* NS agent calls probe as lib

* Clippy: ns agent

* Fix submit_v2 on API

* Adjust dockerfile, deployment details for the new flow

* Bump package versions

* PR feedback

* Fix CI

* Final version
2026-04-07 16:35:50 +02:00
mfahampshire c07ef0253d Max/sdk stream wrapper (#6320)
* Replace MixnetStream with LP framing
- Replace custom header with LpFrameHeader
- Added sequence number for message ordering

* IPR: support LP Stream-framed client connections
- Detect and route LP Stream frames in mixnet_listener
- Wrap inline responses in LP Stream frames
- Thread stream_id to ConnectedClientHandler for TUN responses

* sdk: add ipr_wrapper module with IpMixStream
- IpMixStream wraps MixnetStream for IPR tunnel over mixnet
- LP Stream framing handled automatically by MixnetStream
- Gateway discovery, connect handshake, IP packet send/receive

* sdk: remove superseded stream_wrapper module

* Trim obvious comments, add architecture.md stub

* sdk: add missing deps and fix warnings

* Cut down architecture diagram until finished with rest of the code, leaving stubs

* sdk: refactor IpMixStream, extract shared helpers

- Extract gateway discovery and connect response parsing
- Add recv() to MixnetStream, remove 64KB read buffer
- Simplify IpMixStream constructor

* Fix SphinxStream renames missed during rebase

* Add IpPacketResponse::from_bytes() for stream-based deserialization

* Clean up ip_packet_client: delete stale connect.rs, take raw bytes not ReconstructedMessage

* Clippy

* Delete unused ip_packet_client modules

- Remove helpers.rs (ICMP utilities moved to example)
- Remove error.rs (errors consolidated into sdk/error.rs)
- Remove README.md
- Update module root to only export discovery + listener

* Simplify listener, IpMixStream, and network_env

- Collapse IprListener struct into standalone handle_ipr_response()
- Move check_ipr_message_version() into listener.rs
- Remove IpMixStream test module (moved to example)
- Remove parse_network() and commented-out Sandbox arms
- Return Result from find_workspace_root() instead of panicking
- Add IprTunnelDisconnected and WorkspaceRootNotFound error variants

* Refactor IPR stream handling and document seq conventions
- Inline stream_id tracking (remove current_stream_id field)
- Re-export encode_stream_frame from clients module
- Document seq=0 reservation for inline control responses
- Document data-path counter starting at 1 with skip-on-wrap

* Add ipr_tunnel example for integration testing
- ICMP ping through IPR with --gateway flag for targeting specific exits
- Move pnet_packet from dependencies to dev-dependencies

* Add message reordering to stream router
- Buffer out-of-order messages per-stream using BTreeMap
- Drain contiguous sequences individually to preserve message boundaries
- Drop duplicate/old sequence numbers with a warning
- Remove dead_code allow on StreamFrame::sequence_num

* Clean up comments and fill architecture.md
- Remove separator line comments
- Update stale comments about ordering not being implemented
- Remove collapsible_if allows, use let-else instead
- Fill in architecture.md data flow and connection lifecycle

* Simplify ipr_tunnel example to minimal smoke test
- Single ping instead of multi-ping loop
- Remove identifier and PING_COUNT
- Collapse ICMP helpers into single build_icmp_ping function

* Add dual-stack IPv6 ping and rename gateway → ipr
- Rename --gateway flag to --ipr and new_with_gateway() to new_with_ipr()
- Add ICMPv6 ping to ipr_tunnel example for dual-stack smoke test
- Tighten echo reply validation (protocol field check, diagnostic output)
- Document IP allocation (subnets, static vs dynamic, client keying) in architecture.md
- Promote LP Stream Open handshake log to INFO

* Tweak subnet comment in docs

* Don't stop IPR listener on decode failure
- Change break to continue so garbage packets can't kill the listener
- Remaining valid packets in the bundle are still processed

* Fix license headers and use workspace dep for pnet_packet
- Switch GPL-3.0 to Apache-2.0 on all SDK library files
- Add missing license headers to 7 files
- Use workspace version for pnet_packet dependency

* Document IP pool isolation from WG/LP dVPN pool
- IPR uses 10.0.0.0/16 on nymtun, WG uses 10.1.0.0/16 on nymwg
- Reference constants.rs as source of truth

* Remove network_env.rs and simplify IpMixStream API
  - Default to mainnet via setup_env(None) instead of requiring env param
  - Remove NetworkEnvironment enum and workspace root detection
  - Remove WorkspaceRootNotFound error variant
  - Update ipr_tunnel example to match new signatures

* Use weighted random selection for IPR gateway discovery
  - Replace max_by_key with choose_weighted biased by performance score
  - Prevents all clients converging on a single highest-performing IPR

* Cap stream reorder buffer to prevent unbounded memory growth
- Add MAX_REORDER_BUFFER (256) to limit per-stream pending messages:
	- buffer overflows = skip ahead to lowest buffered seq and drain
	- protects against malicious senders that deliberately skip sequence numbers

* Extract shared IPR response helpers into nym-ip-packet-requests
  - Add response_helpers module with version check, connect response
    parsing, and control response dispatch
  - SDK ip_packet_client now delegates to shared module
  - Monorepo nym-ip-packet-client uses shared version check and
    connect response parsing
  - Fix doc comment attributing fork to nym-vpn-client

* Extract ICMP test helpers into nym-ip-packet-requests
  - Add icmp_utils module behind test-utils feature flag
  - Move build_icmp_ping, build_icmpv6_ping, is_echo_reply_v4/v6 from
    example
  - Update ipr_tunnel example to use shared helpers

* Add protocol v9 LP-framed transport marker

- Add v9 module (re-exports v8, VERSION=9)
- Accept v9 requests and responses in IPR
- Switch SDK IpMixStream to send v9

* Log protocol version in dynamic connect requests

* Remove KCP from IPR and fix unwrap_or_default in SDK
- Remove all KCP session management from ip-packet-router (replaced by
  LP Stream framing)
- Drop nym-kcp dependency and KcpError variant from IPR
- Replace unwrap_or_default with ok_or(Error::NoNymAPIUrl) in
  IpMixStream::new()

* Add v9 protocol wrapper constructors and enforce version/transport
consistency
- Add v9::new_connect_request(), new_data_request(),
  new_ip_packet_response() to centralise version stamping
- Replace manual protocol.version overrides in SDK and IPR with v9
  wrapper calls
- Bump nym-ip-packet-client current re-export from v8 to v9
- Enforce LP Stream frames must carry v9+ payloads, non-stream must be
  v8 or lower

* Filter IPR exit nodes by minimum v9-compatible release version
- Define MIN_RELEASE_VERSION (1.30.0) in ip-packet-requests/v9 alongside protocol constants
- Add semver-based filtering in SDK gateway discovery to skip nodes below v9 threshold
- Add semver dependency to ip-packet-requests and nym-sdk

* Use numeric version comparison for transport/version enforcement
- Compare version as u8 instead of enum equality so future v10+ is handled correctly
- Remove unused `use super::*` import left over from KCP test removal
2026-03-27 20:35:26 +00:00
mfahampshire b231eb4f04 Max/asyncread asyncwrite nym client (#6318)
* Remove AsyncRead/Write traits from native client - moving them to
stream/

* Substream model first push

* Update / add examples

* Update lockfile

* Clippy

* clippy examples

* remove codecs

* Remove unused bincode setup

* Revert a lot of changes when SDK client itself implemented
AsyncRead/Write

* Remove unnecessary mut

* Use local PollSender in MixnetStream instead of client_input.input_sender

Now that client-core's input_sender is back to mpsc::Sender (reverted
PollSender migration), MixnetStream creates its own PollSender wrapper
for the AsyncWrite impl's poll_ready/start_send calls.

* Remove now-unnecessary parameter

* Clippy

* Cleanup more stragglers from previous setup (Async traits on
MixnetClient)

* Rename files (remove module inception)

* - Shrink StreamId from 16 bytes to u64, add version byte to wire format
  - Introduce MixStreamHeader/MixStreamFrame structs for decode
  - Replace StreamMap type alias with struct using tokio::sync::Mutex
  - Add StreamMap helper methods, eliminate lock().expect() panics
  - Rename stream.rs -> mixnet_stream.rs to avoid module inception
  - Document irrevocable stream mode, add LP integration TODO

* - Remove dummy channel
- Add err variant for reciever alredy taken
- Remove panics

* add timeout to stream

* clippy
2026-03-13 09:40:45 +00:00
mfahampshire cfcf804b47 Max/crates publishing tweaks (#6343)
* add semver validator action

* update runner

* update runner with sed for old version in CI

* Add no commit to publish for the moment

* fix version bump command

* configure git bot

* error check

* make dryrun less opaque

* Reintroduce error check - keep logging dryrun in for debug (commented
out)

* fix grep check

* bring non-dry-run to parity

* add node for npx semver check to action

* updated sed command

* revert erroneous version bump

* added semver check to publish workflow

* allow from other branches

* allow from other branches again

* publishing guide

* update publication runner

* Release 1.20.3

nym-api-requests@1.20.3
nym-async-file-watcher@1.20.3
nym-authenticator-requests@1.20.3
nym-bandwidth-controller@1.20.3
nym-bin-common@1.20.3
nym-cache@1.20.3
nym-cli-commands@1.20.3
nym-client-core@1.20.3
nym-client-core-config-types@1.20.3
nym-client-core-gateways-storage@1.20.3
nym-client-core-surb-storage@1.20.3
nym-client-websocket-requests@1.20.3
nym-coconut-dkg-common@1.20.3
nym-common@1.20.3
nym-compact-ecash@1.20.3
nym-config@1.20.3
nym-contracts-common@1.20.3
nym-contracts-common-testing@1.20.3
nym-cpp-ffi@1.20.3
nym-credential-proxy-lib@1.20.3
nym-credential-proxy-requests@1.20.3
nym-credential-storage@1.20.3
nym-credential-utils@1.20.3
nym-credential-verification@1.20.3
nym-credentials@1.20.3
nym-credentials-interface@1.20.3
nym-crypto@1.20.3
nym-dkg@1.20.3
nym-ecash-contract-common@1.20.3
nym-ecash-signer-check@1.20.3
nym-ecash-signer-check-types@1.20.3
nym-ecash-time@1.20.3
nym-exit-policy@1.20.3
nym-ffi-shared@1.20.3
nym-gateway-client@1.20.3
nym-gateway-requests@1.20.3
nym-gateway-stats-storage@1.20.3
nym-gateway-storage@1.20.3
nym-go-ffi@1.20.3
nym-group-contract-common@1.20.3
nym-http-api-client@1.20.3
nym-http-api-client-macro@1.20.3
nym-http-api-common@1.20.3
nym-id@1.20.3
nym-inclusion-probability@1.20.3
nym-ip-packet-client@1.20.3
nym-ip-packet-requests@1.20.3
nym-metrics@1.20.3
nym-mixnet-client@1.20.3
nym-mixnet-contract-common@1.20.3
nym-mixnode-common@1.20.3
nym-multisig-contract-common@1.20.3
nym-network-defaults@1.20.3
nym-node-metrics@1.20.3
nym-node-requests@1.20.3
nym-node-tester-utils@1.20.3
nym-noise@1.20.3
nym-noise-keys@1.20.3
nym-nonexhaustive-delayqueue@1.20.3
nym-ordered-buffer@1.20.3
nym-outfox@1.20.3
nym-pemstore@1.20.3
nym-performance-contract-common@1.20.3
nym-pool-contract-common@1.20.3
nym-registration-common@1.20.3
nym-sdk@1.20.3
nym-serde-helpers@1.20.3
nym-service-provider-requests-common@1.20.3
nym-service-providers-common@1.20.3
nym-socks5-client-core@1.20.3
nym-socks5-proxy-helpers@1.20.3
nym-socks5-requests@1.20.3
nym-sphinx@1.20.3
nym-sphinx-acknowledgements@1.20.3
nym-sphinx-addressing@1.20.3
nym-sphinx-anonymous-replies@1.20.3
nym-sphinx-chunking@1.20.3
nym-sphinx-cover@1.20.3
nym-sphinx-forwarding@1.20.3
nym-sphinx-framing@1.20.3
nym-sphinx-params@1.20.3
nym-sphinx-routing@1.20.3
nym-sphinx-types@1.20.3
nym-sqlx-pool-guard@1.20.3
nym-statistics-common@1.20.3
nym-store-cipher@1.20.3
nym-task@1.20.3
nym-test-utils@1.20.3
nym-ticketbooks-merkle@1.20.3
nym-topology@1.20.3
nym-tun@1.20.3
nym-types@1.20.3
nym-upgrade-mode-check@1.20.3
nym-validator-client@1.20.3
nym-verloc@1.20.3
nym-vesting-contract-common@1.20.3
nym-wasm-client-core@1.20.3
nym-wasm-storage@1.20.3
nym-wasm-utils@1.20.3
nym-wireguard@1.20.3
nym-wireguard-private-metadata-client@1.20.3
nym-wireguard-private-metadata-server@1.20.3
nym-wireguard-private-metadata-shared@1.20.3
nym-wireguard-private-metadata-tests@1.20.3
nym-wireguard-types@1.20.3
nyxd-scraper-shared@1.20.3

Generated by cargo-workspaces

* remove --allow-branch ; no commit, doesn't need branch restrictions

* remove another clashing flag

* again

* exclude build.rs from crate for crates.io

* various in process scripts to pick up deployment where it left off

* rename workflows

* Version bump fix from borked publish run

* add publishing doc + updated publish-resume ci

* move example from service-providers to sdk examples/ to remove circular dev dependency for cargo publication

* remove wildcard version import

* Workflows and documentation for publication

* add contracts/ patch + imports

* Reintroduce missing kkt dep from rebase

* fix borked rebase cargo lock

---------

Co-authored-by: Nym bot <nym-bot@users.noreply.github.com>
2026-02-03 11:32:38 +00:00
Simon Wicky 46268edf9c [Feature] Fallback gateway listener and remove legacy key support (#6249)
* one commit to rule them all

* remove too aggressive copy pasting

* update details when outdated

* typo and serde alias

* no hostname option and fixes

* fix wasm client?

* non fallback fixed

* improve gateway details update

* better ws addresses

* PR review fixes

* improve type safety on update_gateway_published_data

* fix client gateway storage migration
2025-12-03 09:49:23 +01:00
Drazen Urch 90e9e3cff8 Domain fronting integration (#5974)
* feat: unify HTTP client creation and enable domain fronting

Enhanced the base nym_http_api_client to reduce fragmentation and enable domain fronting:

- Added SerializationFormat enum for explicit JSON/bincode choice (no auto-detection)
- Added from_network() method to create clients from NymNetworkDetails with domain fronting
- Added with_bincode() builder method for explicit serialization configuration
- Set Accept header based on serialization preference
- Added deprecation paths for NymApiClient wrapper and nym_api::Client re-export
- Enabled domain fronting support via network defaults feature

This is part of a broader effort to consolidate HTTP client implementations across the codebase,
reducing ~500 lines of wrapper code and providing automatic domain fronting for censorship resistance.

* feat: migrate NymApiClient usage to unified HTTP client

- Wire up domain fronting configuration in NymNetworkDetails
- Implement NymApiClientExt trait for base nym_http_api_client::Client
- Migrate direct NymApiClient usage in multiple components:
  - nym-network-monitor
  - verloc measurements
  - connection tester
  - coconut/ecash client
  - validator rewarder
- Add Copy derive to ApiUrlConst to enable iteration
- Update error handling and Display implementations

This enables automatic domain fronting for all Nym API calls via the configured CDN front hosts.

* fix: resolve all compilation errors after NymApiClient migration

- Add missing nym-http-api-client dependencies to multiple crates
- Add NymApiClientExt trait imports where needed
- Fix type mismatches from NymApiClient to unified Client
- Add error conversions for NymAPIError in various error enums
- Implement missing trait methods (get_current_rewarded_set, get_all_basic_nodes_with_metadata, get_all_described_nodes)
- Fix type conversions for RewardedSetResponse in network monitor
- Update all API client instantiation to use new unified HTTP client

* feat: complete migration to unified HTTP client and fix all compilation errors

- Added missing NymApiClientExt trait methods (get_all_expanded_nodes, change_base_urls)
- Fixed all compilation errors across the workspace
- Updated nym-node to use unified client instead of deprecated NymApiClient
- Fixed type conversions for RewardedSetResponse → EpochRewardedSet
- Added nym-http-api-client dependency where needed
- Updated all examples and documentation to use new client API

* fix: provide all API URLs for automatic failover in endpoint rotation

Previously, when rotating API endpoints, only a single URL was provided to the
HTTP client, defeating the purpose of having multiple URLs for resilience.

Changes:
- NymApiTopologyProvider now provides all URLs in rotated order when switching endpoints
- NymApisClient similarly provides all URLs starting from the working endpoint
- Added clarifying comments for broadcast/exhaustive query methods where single URLs are intentionally used
- This enables the HTTP client's built-in failover mechanism while maintaining endpoint rotation behavior

The fix ensures that if the primary endpoint fails, the client can automatically
failover to alternative endpoints without manual intervention, improving overall
network resilience.

* Update common/client-core/src/client/base_client/mod.rs

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>

* Remove error generics, address PR comments

* Explicit warning on missing fronting configuration

* Assorted CI fixes

* Registry proc-macro

* Rename macro

* Syn workspace version

* Where do we need to put inventory

* Ergonomics and call sites, incept the builder

* fix: Address critical issues in client configuration registry implementation

- Fixed HeaderMapInit parsing bug that would cause compilation errors
- Added comprehensive documentation with usage examples and DSL reference
- Improved error handling with better error messages for invalid headers
- Added test coverage for both macro and registry functionality
- Added debug inspection capabilities for registered configurations
- Fixed module name conflicts in tests by using separate modules

All tests now passing:
- 7 macro tests validating DSL parsing and code generation
- 4 registry tests verifying configuration collection and application

* Use default value for the ports until api is deployed

* Feature/improved http error (#6025)

* use display impl for urls

* feat: attempt to add more details to reqwest errors

* temporarily restored GenericRequestFailure variant

* another restoration

* cleanup

* Some debug tooling, and default timeout fix

* Fix user-agent override

* Fix various wasm things

---------

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
Co-authored-by: Bogdan-Ștefan Neacşu <bogdan@nymtech.net>
2025-09-15 14:32:15 +02:00
Jędrzej Stuczyński 833114372a bugfix: key-rotation + reply SURBs (#5876)
* wip: changes to surb logic + stronger db typing

* surb invalidation logic

* chore: remove unused deps

* resolving todos

* a lot of additional bugfixes

* 1.88 clippy

* wasm fixes

* wasm clippy

* wallet clippy

* wait for epoch end when setting up new network

* split ReplyController into Sender and Receiver for easier reasoning

* additional reply surbs improvements

includes, but is not limited to: unconditionally reseting sender tag on restart, limiting number of surb re-requests, resetting stale surbs on load

* fixed calculation of number of removed surbs

* add additional calculated field to key rotation info

* DBG: 'request_reply_surbs_for_queue_clearing' temp logs

* fixes for silly mistakes

* conditionally reduce log severity
2025-07-04 16:29:03 +01:00
Jędrzej Stuczyński 6d47046a38 fixed client route for obtaining v2 list of gateways (#5859) 2025-06-16 14:32:46 +01:00
Jędrzej Stuczyński d8c84cc4d6 feat: key rotation (#5777)
* wip

* wip: wrap node's sphinx key with a manager

* wip: choosing correct key for packet processing

* further propagation of key rotation information

* attaching key rotation information to reply surbs

* added basic key rotation information to mixnet contract

* wip: introducing cached queries for key rotation info from nym api

* unified nym-api contract cache refreshing

* finish packet decoding

* multi api client + retrieving rotation id

* rotating sphinx key files

* logic for migrating config file

* wip: putting new sphinx keys to self described endpoints

* processing loop of KeyRotationController

* fixed sphinx key loading

* rotating bloomfilters

* wired up KeyRotationController

* flushing bloomfilters to disk and loading

* most of nym-node changes

* post rebase fixes

* fixes due to backwards compatible hostkeys

* split http state.rs file

* dont use deprecated fields

* fixed backwards compatible deserialisation of host information

* split up node describe cache

* added a dedicated CacheRefresher listener to perform full refresh outside the set interval

* controlling announced sphinx keys within nym-api

* retrieving rotation id when pulling topology

* split nym-nodes http handlers

* v2 nym-api endpoints to retrieve nodes with additional metadata information

* bug fixes...

* additional bugfixes and guards against stuck epoch

* testnet manager: set first nym-api as the rewarder

* fixed host information deserialisation

* fixed panic during first key rotation

* post rebase fixes

* clippy

* more guards against stuck epochs

* added helper method to reset node's sphinx key

* instantiate mixnet contract with custom key rotation validity

* additional bugfixes and debugging nym-api deadlock

* passing shutdown to nym apis client

* remove dead test

* post rebasing fixes

* missing MixnetQueryClient variants

* remove usage of deprecated methods in sdk example

* fix: incorrect method signature

* post rebasing fixes

* attempt to retrieve key rotation id before doing any config migration work

* ignore tests relying on networking behaviour

* allow networking failures in certain tests
2025-06-03 11:22:51 +01:00
Jędrzej Stuczyński 4a1ce8154a chore: resolve 1.87 clippy warnings (#5802)
* Clippy in wallet & sdk

* Clippy in wallet

* Pin rust to 1.86 in builder

* apply changes from b7da75a18c

* missing nym-node features

* Box all the things

* additional boxes in the wallet

* post rebasing clippy

---------

Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
2025-05-27 11:08:36 +01:00
mfahampshire 9b6b2117dd Max/general abstraction updates (#5560)
- new instance of echo server with lib / cli split 
- echo server docs update 
- tcpproxy and echosever now listen for kill signal 
- ffi bindings of tcpproxy functions updated
2025-05-14 15:51:18 +00:00
Jon Häggblad 419e16eb31 Remove pretty_env_logger and switch remaining crates to use tracing (#5749)
* Remove pretty_env_logger dependency

* Switch remaining instances of pretty_env_logger to tracing
2025-05-08 15:05:08 +02:00
mfahampshire d45d1eb313 change hardcoded file to tempdir 2025-03-06 17:37:19 +01:00
Bogdan-Ștefan Neacşu 8670693952 Uncouple storage reference for bandwidth client (#5372) 2025-01-22 12:12:06 +01:00
mfahampshire e454d71b78 Max/client pool (#5188)
* tcp conn tracker

* make default decay const

* first pass connpool

* err handling conpool start

* added notes for next features

* first version working

* first pass spin out client_pool

* cancel token

* logging change

* bump default decay time

* bugfix: make sure to apply gateway score filtering when choosing initial node

* add duplicate packets received to troubleshooting

* client_pool.rs mod

* client pool example

* clippy

* client pool example done

* added disconnect to client pool

* update mod file

* add cancel token disconnect fn

* comments

* comments

* add clone

* added disconnect thread

* update example files tcpproxy

* client pool docs

* remove comments for future ffi push + lower default pool size from 4 to 2

* comment on ffi

* update command help

* clone impl

* remove clone

* fix clippy

* fix clippy again

* fix test

* tweaked text grammar

* updated comment in example

* future is now

* cherry

* cherry

* fix borked rebase

* fix fmt

* wasm fix

---------

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
2025-01-14 16:11:47 +00:00
Jędrzej Stuczyński a2322d6cdf feature: nym topology revamp (#5271)
* revamped NymTopology

* wip

* working e2e client

* updated nym-api

* updated nym-node

* updated rest of non-test code

* updated the rest of the codebase

* additional tweaks

* linux clippy fixes + adding additional dummy ipr types for better linting on non-linux targets
2024-12-19 10:44:34 +00:00
Jędrzej Stuczyński 675e5a0305 removed semver filtering (#5224) 2024-12-06 17:21:21 +00:00
Jędrzej Stuczyński 753a21f8ca bugfix/feature: added NymApiClient method to get all skimmed nodes (#5062)
* bugfix/feature: added NymApiClient method to get all skimmed nodes

* wasm

* helper: utility method for getting ed25519 identity directly from node description
2024-10-30 12:21:27 +00:00
Jędrzej Stuczyński d04331a5df updated clients to use 'new' endpoints 2024-10-14 18:12:56 +01:00
Jędrzej Stuczyński 45f4eeeff2 fixed linter issues and outdated tests 2024-10-10 15:38:00 +01:00
Jędrzej Stuczyński 51b511b27e Rebased the branch one more time
WIP; rebasing

Another branch squash

Squashing the v3 branch

changing min pledge amounts

logic for adding new nymnode into the contract

converting mixnode/gateway bonding into nym-node bonding

logic for migrating gateways into nymnodes

ibid for mixnodes

further nym-node work + fixed most existing unit tests

forbid nymnode migration with pending cost params changes

preassign nodeid for gateways

changing role assignment and epoch progression

changing role assignment and epoch progression

optional custom http port

logic for unbonding a nym-node

updating Delegation struct

logic for increasing pledge of either mixnode or nymnode

logic for decreasing pledge of either mixnode or a nym node

logic for changing cost params of either mixnode or a nym node

wip

initialise nymnodes storage

fixing transaction tests

fixed naive family tests

reward-compatibility related works

resolving delegation events

introduced rewarded set metadata

another iteration of restoring old tests

updated rewarding part of nym-api

parking the branch

unparking the branch

wip

purged families

added 'ExitGateway' role

passing explicit work factor for rewarding function

remove legacy layers storage

wip: node description queries

added announced ports to self-described api

step1 in gruelling journey of adding node_id to gateways

ensure epoch work never goes above 1.0

changed active set to contain role distribution

[theoretically] sending rewarding messages for the new rewarded set

[theoretically] assigning new rewarded set

reimplementing more nym-api features

remove legacy types

re-implement legacy network monitor

restoring further routes + minor refactor of NodeStatusCache

skimmed routes now return legacy nodes alongside nym-nodes

seemingly restored all functionalities in nym-api

removing more legacy things from the contract

initial contract cleanup

added nym-api endpoints to return generic annotations regardless of type

updated simulator to use new rewarding parameters

more contract cleanup

made existing mixnet contract tests compile

extra validation of nym-node bonding parameters

fixed additional compilation issues

fixed nym-api v3 database migration failure

added additional nym-node contract queries

updated the schema

made additional delegation/rewards queries compatible with both legacy mixnodes and nym-nodes

fixing existing unit tests in mixnet contract

wip

resolved first batch of 500 compiler errors

re-deprecating routes

making wallet's rust backend compile

fixed non-determinism in contract + nym-api build

fixes to the build

populating cotracts-cache with nym-nodes data

more missing nymnodes queries

temp mixnet contract methods + restored result submission in nym-api

allow deprecated routes

submitting correct results for mixnode results

removed deprecated re-export of AxumAppState and removed smurf naming

moved axum modules into support::http

cleaning up nym-api warnings

determine entry gateways before exits

exposed transaction to update nym-node config

missing memo for updating node config

 new routes

added routes to swagger and fixed relative paths

fixed some macro derivations

added nym-node commands to nym-cli
2024-10-10 13:27:05 +01:00
mx 60fa5cfeb8 Max/rust sdk stream abstraction (#4743)
* add TcpProxyClient and TcpProxyServer abstractions to SDK 
* add single connection example
* add multi-connection example 
* add simple echo server to `tools/`: used for multi-connection example 
* update FFI toml files: switched to local imports 
* add proxy bindings to `ffi/shared`
* add proxy bindings and example to `ffi/go` 
* add note to `ffi/cpp` about lack of Proxy bindings for the moment
2024-09-24 09:29:46 +00:00
Jędrzej Stuczyński 7c0235ab26 fixed wasm build and trait impl 2024-09-19 10:06:59 +01:00
Bogdan-Ștefan Neacşu 18891e5f20 Use ecash credential type for bandwidth value (#4840)
* Use ecash credential type for bandwidth value

* Fill explicit default value for args

* Use up-to-date values for tickets

* Fix sdk example default value

* Another default

* Fix sdk test

* Fix TicketTypeRepr default
2024-09-05 13:19:19 +02:00
Jędrzej Stuczyński 4c10cebf1b propagated new ticket type through the whole stack 2024-07-30 11:31:49 +01:00
Simon Wicky fc2eedfc66 Another Grand Ecash Squasheroo
add offline ecash library

minor changes in coconut benchmarks

add ecash smart contract

change contract traits from coconut to ecash

first wave of andrew's suggestion

first wave of andrew's suggestion

second wave of andrew's suggestion for ecash lib

andrew's suggestion for ecash contract

licensing commit

safety comments for most unwraps

more unwrap handling

change chrono crate for time

latest cargo lock

error revamp

small visibility fix

small fix

remove indexedmap from contract + some tweaks

add cw2 version in ecash contract

remove envryption key from contract

change types from coconut to ecash types

adapt api model for credential issuance

adapt issued credential storage on API

add signatures cache on API

change API routes for new blind signing

modify issued_credential table

add issuance logic client-side

credential and signature storage client side

utils for credential issuance

first wave of fix

some of andrew's suggestions

remove encryption key from deposit

freepass issuance client side

freepass issuance API side

andrew's suggested fixes

other suggested fix

adapt change from PR below

allow offline verification flag

credential spending models

credential spending models for client

credential preperation for the client

credential preperation for the client

credential storage for spending on client

bloom filter for API

spent credential storage on validators

API route for spending online and offline ecash

API routes in the client lib

credential storage on gateway

ecash verifier to replace coconut verifier

accept credentials on gateway

bandwidth expiration for gateways

client ask for more bandwidth if it runs out

credential import

adapt nym validator rewarder and sdk

fix tests api tests and add constants

cargo fmt and lock and small test fix

cargo fmt and lock and small test fix

cargo lock

move stuff where they belong in ecash and static parameters

move some constants, error handling and phase out time crate

error revamp part 2

secret key by ref instead of clone

change l in wallet and v visibility

rework payinfo

rework monster tuples

fix expiration date signature cloning

minor fixes

final bits and bobs fixes

final bits and bobs fixes

rename l accessor to tickets_spent

wave of fixes

second wave of fixes

change hash domain value

removed benchmark flag

remove useless stringification in storage

nuke Bandwidth voucher

change timestamps to offsetdatetime

key name change

post-rebase fixes

update nym-connect 'time' dep due to broken semver

upload ecash contract to the build server

make wasm zknym-lib compile

but it won't work properly just yet

make wasm zknym-lib compile

but it won't work properly just yet

fix typo in ecash contract deps

make sure to use 0.1.0 sphinx packet

optimise pairings in 'check_vk_pairing'

derive serde for ecash types

simplified g1 tuple byte conversion

further optimise the pairing

unified signature type + renamed nym-api coconut module to ecash

using bincode serialiser for more complex binary types

using multimiller loop instead of rayon for verifying coin indices signatures

batching signature verification wherever possible

feature-locked rayon

clippy

refactor ecash contract a bit + introduce deposit storage

reworked find_proposal_id

various minor fixed

add offline_zk_nyms to nym-node everywhere

add missing #query

change test value to fit new serialization

optimised deposits storage

removed duplicate decompression code

using deposit_id instead of transaction hash

removed freepasses

split up ecash handling

unified shared state

fixed deposit_id parsing

log recovered deposit id

removed online verification

add detailed build info to ecash contract

fixed deserialisation of deposit amount received from nyxd queries

changed deposit to only persist attached pubkey

first iteration of split of verification and redemption

basic tool for setting up new network

expanded the tool with the option to bypass DKG

rename + init network without DKG

setting up locally running apis

ecash key migration

more local functionalities

wip fixing sql schemas

gateway immediately submitting redemption proposal

and getting it passed if valid

most of the gateway logic for split redemption with error recovery

fixed gateway not persisting ecash signers

simplify creation of compatible client

create properly serialised ecash key from the beginning

rebuild missing tickets and proposals on startup

stop ticket issuance during DKG transition

fixing build issues

split out ecash storage on nym-api side

master-verification-key route

caching all the signatures and keys

implemented aggregated routes for nym-apis

swagger UI for ecash endpoints

added explicit annotation for index and expiration signatures

revamped client ticketbook storage

save all recovery information in the same underlying storage

wrapper for bloomfilter

being more aggressive with marking tickets as used

ensure client has correct signatures before making deposit

fix deserialisation of AggregatedExpirationDateSignatureResponse + add ticketbook table

split nym-api ecash routes handlers into multiple files

fixed deserialisation of encoded expiration date

add tt_gamma1 to challenge and change naming for paper consistency

rotating double spending bloomfilter

nym-api test fixes + make sure to insert initial BF params

fixed ecash benchmark code

updated contract schema

updated CI to not upload gateway/mixnode binaries

ticket bandwidth revocation

added default deserialisation for zk nym config

post-rebase fixes
2024-07-30 11:27:21 +01:00
Jędrzej Stuczyński 3cbcd001f4 clippy + test build issues 2024-06-10 11:36:57 +01:00
Jędrzej Stuczyński 5cc3ee32e9 fix test 2024-06-10 11:17:44 +01:00
Tommy Verrall 1ecb57fda0 Merge pull request #4588 from nymtech/jon/explicitly-handle-sqlite-constraint-violation
Explicitly handle constraint unique violation when importing credential
2024-05-13 11:30:31 +01:00
Jędrzej Stuczyński a5a9b5128f Merge pull request #4508 from nymtech/chore/beta-CI
fixed beta clippy: lints + updated cosmwasm to 1.4
2024-03-28 10:12:33 +00:00
Jędrzej Stuczyński a4e5b2af93 further wasm fixes 2024-03-27 17:20:03 +00:00
Jędrzej Stuczyński 765ac715c1 fixing builds of x86-based binaries 2024-03-27 17:20:03 +00:00
Jędrzej Stuczyński 5bb96bf42c additional fixes to key creation 2024-03-27 17:20:01 +00:00
Jędrzej Stuczyński c80c0b88bf starting to weave in new storage trait 2024-03-27 17:20:00 +00:00
Jędrzej Stuczyński 7e6feb0cd2 fixed beta clippy: lints + updated cosmwasm to 1.4 2024-03-27 16:04:51 +00:00
Jon Häggblad fb5d775857 Fix warnings for rustc 1.77 2024-03-21 22:20:43 +01:00
Jędrzej Stuczyński 2d9e34cc81 clippy 2024-01-02 10:27:43 +00:00
mfahampshire 9549d3b681 revert change of file name 2023-11-07 15:12:33 +01:00
Jon Häggblad fb2b7d3480 Fix rust-analyzer warnings (#3974)
* Fix rust-analyzer warnings

Fix some warnings that rust-analyzer emits due to enabling all features.
These are annoying when you try to list all warnings in the entire
workspace.

* Revert change to signing client

* Instead add nested feature
2023-10-10 12:02:00 +02:00
Jon Häggblad da404b636d Add sdk example for geo-aware topology provider (#3941)
* Add sdk example for geo-aware topology provider

* Re-export geo topology types
2023-10-02 14:21:53 +02:00
Jędrzej Stuczyński 114d92f93f Feature/gateway inbuilt nr (#3877)
* changed NymConfigTemplate trait to call 'template' by reference

* network requester lib

* introduced generic parameter to 'MixTrafficController' to allow non-remote gateways

* allowing for custom gateway sender

* types cleanup

* minor gateway cmds refactor + initial NR work

* wip

* running a NR inside gateway

note: this NR isnt tied to the gateway yet

* rebase fixes

* propagating same shutdown handle

* wip

* starting NR with appropriate local transceiver

* fixed premature shutdown

* wiring up PacketRouter

* both ends wired together

* actually working

so much cleanup to do now

* started removing dead code

* wip

* temp: hardcode gateway

* further cleanup

* fixed build of other binaries

* setup-network-requester subcmd

* overriding NR config in gateway init/run

* wip making it wasm-compatible [again]

* refactored 'GatewaySetup'

* clippy and friends

* removed debug code

* rust 1.72 lints

* ensuring local gateway is available + some comments

* correctly putting network requester data in the same underlying details struct

* improved gateway errors

* changed 'network_requester_config' deserialization

* missing clap annotation for 'enabled' flag in 'setup-network-requester' command

* saving config file after 'setup-network-requester'

* removed dead code

* review comments

* make embedded NR wait for gateway to come online (for at most 70min)

* fixed shutdown on successful gateway wait

* updated NR config override
2023-09-20 15:47:05 +01:00
Jędrzej Stuczyński 4890c528bc feat: mixFetch - the final countdown (#3737)
* mixFetch

* clippy

* removed redundant Arc over 'WasmStorage' in the 'ClientStorage'

---------

Co-authored-by: Fouad <fmtabbara@hotmail.co.uk>
2023-09-07 12:45:03 +01:00
Fran Arbanas 2f6617daac Feature/issue credentials (#3691)
* Move the functionality to issue credentials from the credential binary and connect it with nym-cli

* finished CLI part, trying to fit SDK part

* finished Rust SDK

* fix: cleanup

* linting

* linting

* linting

* remove one layer of coconut in nym-cli

* linting

* Fixes based on PR comments

* formatting

* fixes based on PR comments

* formatting

* fixing clippy errors

* fixed post-rebasing issues and converted the lib into shared dep for other binaries

* removed credentials client in favour of moving the functionality to nym-cli

* removed redundant 'issue_credential' example (it did the same thing as 'bandwdith')

* removed credentials client from build server

* made the coconut cli also accept nym-api configs

* fixed support for socks5 and NR

---------

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
2023-08-25 17:13:52 +02:00
Jędrzej Stuczyński b429c64168 [demo] libp2p example with nym-sdk (#3763)
* imported libp2p to monorepo

* fixed vanilla ping example

* added libp2p client to workspace

* naively replaced dockerised client with the sdk

* moved libp2p code to sdk examples

* reduced number of dependencies required for libp2p example

* updated readmes

* added protobuf compiler to build dependencies

* added protoc dependency to readme for chat and ping examples

---------

Co-authored-by: mfahampshire <maxhampshire@pm.me>
2023-08-22 13:12:13 +02:00
Jędrzej Stuczyński 2993e85c7a removed redundant notify and instead awaiting the task futures 2023-08-17 12:21:15 +01:00
Jędrzej Stuczyński de6c1ba1f0 made the Stream return a single message at each call 2023-08-09 14:39:57 +01:00
Jędrzej Stuczyński e705b76b36 unified sending interface + example of split sending 2023-08-09 14:27:17 +01:00