Commit Graph

202 Commits

Author SHA1 Message Date
mfahampshire 296f243433 Remove serde and bincode from ReconstructedMessage - simplify
encoding/decoding
2026-02-02 15:13:06 +00:00
mfahampshire fa2cbb5d21 Move bincode out of nym-sphinx 2026-02-02 10:31:41 +00:00
mfahampshire 933bbbb67d Wrapper for API stability with LP client 2026-02-02 09:55:10 +00:00
mfahampshire e32783bced Remove serde requirement for MixPacket 2026-01-30 16:01:10 +00:00
mfahampshire 8b8c583ac5 Suppress clippy deprecated warnings for generic-array 2026-01-30 15:59:24 +00:00
mfahampshire f060d63f2e Add serde derives and codecs for nymsphinx message types 2026-01-30 15:53:36 +00:00
benedettadavico 727d39ad72 Fix conflicts 2026-01-23 17:45:47 +01:00
benedettadavico b5d1e6a93f ack fix 2026-01-23 17:24:48 +01:00
mfahampshire cf3fd00350 Max/crates io prep v2 (#6270)
* - standardise versions for all nym-sdk workspace dependencies
- prepend sqlx-pool-guard with 'nym-'

* Test remove nym-api from deps

* Add oneliner to client_pool doc comments

* Add note to commented out docs.rs link in sdk

* remove nym-api from script

* add publishing file

* bring non-binary / contract / tools into workspace version

* added more info to publishing.md

* make deps workspace version

* remove uploaded sphinx-types crate from script

* remove erroueously included ignore-defaults

* add zeroise to feature

* chore: Release

* add topology to batch

* more cargo versioning

* more cargo versioning - wasm utils

* more cargo versioning - wasm utils

* Add publish=false to manifest for cargo workspaces / crates.io
publishing exclusion

* remove script now switched to manifest based exclusion

* rename import based on rename of contracts-common dep

* Making workspace versions for publication + removing unnecessary crates
from publication

* Remove OOD info from publishing sdk guide

* rename contract imports + remove package

* temp commit: continuing with removal of path from cargo manifest and
replacing with workspace version import for publication

* continuing with cargo.toml updates

* dryrun only erroring on known version problem crates

* remove old published-crates file

* Minor comment change

* remove default features warning

* Additional info on workspace dep comment re publish list

* Add missing description to cargo.toml

* Fix missing feature flags

* Add missing descriptions

* Fix remaining path import

* Add workspace repo / homepage / documentation links to cargo.toml files

* remove workspace version from excluded crate

* Remove todo descriptions

* Minor comment change

* add homepage etc

* move from bls git import to nym_bls_fork crate

* Modify rest of imports from path to workspace import, excluding binaries

* add directory/homepage info

* fix cargo fmt

* add notes to gitignore

* better solution to contracts/ experiment

* wasm -> nym_wasm crate renaming

* fix fatfinger

* add metadata to ecash cargo.toml

* stub publishing guide

* fix misrevolved netlink- version

* Fixes and block publication of rebase re: LP

* first pass @ workflows
2026-01-19 13:19:45 +00:00
Drazen Urch 8a00ed6071 LP Registration + Telescoping + Gateway Probe Localnet Mode (#6286)
* Add KKT cryptographic primitives

Post-quantum Key Encapsulation Mechanism (KEM) Key Transfer protocol.
Enables efficient distribution of post-quantum KEM public keys.

Squashed from georgio/noise-psq branch.

* Implement LP registration protocol with KKT/PSQ integration

Initial implementation of the Lewes Protocol (LP) for gateway registration:
- Add nym-lp crate with Noise protocol handshake
- Add LP listener to gateway for handling registrations
- Add LP client for registration flow
- Integrate KKT for post-quantum KEM key exchange
- Integrate PSQ for post-quantum PSK derivation
- Add Ed25519 authentication throughout
- Add docker/localnet support for testing

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>

* Add LP telescoping with nested sessions and subsession support

Extends LP protocol with telescoping architecture for nested sessions:
- Add nested session support with KKpsk0 rekeying
- Add subsession support with collision detection
- Implement unified packet format with outer header
- Refactor gateway handlers for single-packet forwarding
- Add TTL-based state cleanup for stale sessions
- Add outer AEAD encryption layer
- Refactor registration client for packet-per-connection model

* Add gateway-probe localnet mode with WireGuard tunnel support

Adds localnet testing mode to gateway-probe for LP development:
- Add TestMode enum for different probe configurations
- Add --gateway-ip flag for direct gateway testing
- Implement two-hop WireGuard tunnel for localnet
- Add mock ecash support for testing without real credentials
- Add netstack Go bindings for userspace networking
- Restructure probe with mode and common modules
- Update README with localnet mode documentation

* Increase KCP fragment limit from u8 to u16

- Change frg field from u8 to u16 in packet header (25 bytes total)
- Update encode/decode to use get_u16_le/put_u16_le
- Update Segment struct frg field to u16
- Remove truncating cast in session.rs
- Max message size now ~91MB (65,535 fragments × MTU)
- Internal protocol only, no interop concerns

Nym uses KCP for reliability and multiplexing, not standard real-time
use cases. The u8 limit (255 fragments, ~355KB) was insufficient.

Addresses: nym-yih9

* Zeroize Ed25519 key material in to_x25519 conversion

Wrap hash and x25519_bytes in zeroize::Zeroizing to ensure private
key material is cleared from memory after use.

Closes: nym-k55g

* Return Result from KCP session input() for error detection

Change KcpSession::input() to return Result<(), KcpError> so callers
can detect invalid packets instead of silently ignoring them.

- Add ConvMismatch error variant for conversation ID mismatches
- Update driver to propagate errors from session.input()
- Update all test and example callers

Closes: nym-n0kk

* Fix Zeroizing deref in ed25519 to_x25519 conversion

The from_bytes() function expects &[u8], need to deref the Zeroizing
wrapper to get the inner array.

* Add semaphore-based connection limiting for LP packet forwarding

Limits concurrent outbound connections when forwarding LP packets to
prevent file descriptor exhaustion under high load.

Key changes:
- Add max_concurrent_forwards config (default 1000)
- Add forward_semaphore to LpHandlerState
- Acquire semaphore permit before connecting in handle_forward_packet
- Return "Gateway at forward capacity" error when at limit

This provides load signaling so clients can choose another gateway
when the current one is overloaded.

Design note: Connection pooling was considered but provides minimal
benefit since telescope setup is one-time and targets are distributed
across many different gateways. See AIDEV-NOTE in LpHandlerState for
full analysis.

Closes: nym-xi3m

* Return error on session unavailable in handle_subsession_packet

Replace .session().ok() with proper error handling to fail fast when
session is Closed or Processing after state machine processing.

Previously, the code silently continued with outer_key = None, which
could cause protocol errors downstream.

Closes: nym-8de0

* Use explicit bincode Options helper in nested_session

Add bincode_options() helper that returns DefaultOptions with explicit
big_endian and varint_encoding configuration. This future-proofs against
bincode 1.x/2.x default changes and makes serialization format explicit.

Updated all 4 bincode usages in nested_session.rs to use the helper.

* Deduplicate outer_key lookup pattern in nested_session.rs

Extract common state_machine.session().ok().and_then(...) pattern into
two helper methods:
- get_send_key() for encryption (outer_aead_key_for_sending)
- get_recv_key() for decryption (outer_aead_key)

Updated 6 call sites to use the helpers, reducing verbosity.

* Add LpConfig struct and AIDEV-NOTE documentation for KKT+PSQ

- Create config.rs with LpConfig struct (kem_algorithm, psk_ttl, enable_kkt)
- Export LpConfig from lib.rs
- Add AIDEV-NOTE to psk.rs explaining:
  - Why PSQ is embedded in Noise (single round-trip, PSK binding)
  - KEM migration path (X25519 → MlKem768 → XWing)
- Add AIDEV-NOTE to state_machine.rs explaining protocol flow:
  - KKTExchange → Handshaking → Transport state transitions
  - PSK derivation formula (ECDH || PSQ || salt)

* Add forward_timeout to LP client config

Add forward_timeout (30s default) to LpConfig and wrap send_forward_packet's
connect_send_receive call with tokio::time::timeout, matching the pattern
used by register() with registration_timeout.

This prevents indefinite hangs when forwarding packets through entry gateway.

* Add negotiated_version field to LpSession

Add AtomicU8 field to store the protocol version from handshake packet
headers. Includes getter and setter methods for future version negotiation
and compatibility checks.

- negotiated_version() returns current version (defaults to 1)
- set_negotiated_version() allows setting during handshake
- Subsessions inherit version 1 (can be enhanced to inherit parent's)

* Change MessageType from u16 to u32

Breaking wire protocol change: MessageType field increased from 2 bytes
to 4 bytes in LP packets. This future-proofs the message type space and
aligns with other u32 fields.

Changes:
- message.rs: #[repr(u32)], from_u32(), to_u32()
- error.rs: InvalidMessageType(u32)
- codec.rs: All serialization/deserialization updated to 4-byte msg_type
  - Cleartext parsing: inner_bytes[4..8], content at [8..]
  - AEAD parsing: decrypted[4..8], content at [8..]
  - Serialization: 4 bytes for message type

* Various smaller fixes

* Refactor LP to stream-oriented TCP processing

Gateway (handler.rs):
- Add bound_receiver_idx field for session-affine connections
- Convert handle() from single-packet to loop with EOF detection
- Add validate_or_set_binding() for receiver_idx validation
- Set binding in handle_client_hello after collision check
- Centralize emit_lifecycle_metrics in main loop only
- Add is_connection_closed() helper for graceful EOF

Client (client.rs):
- Add stream field for persistent TCP connection
- Add ensure_connected(), send_packet(), receive_packet(), close() methods
- Modify perform_handshake_inner() to use persistent stream
- Modify register_with_credential() to use persistent stream
- Modify send_forward_packet() to use persistent stream
- Keep connect_send_receive() for reference (marked dead_code)

This reduces handshake overhead from ~5 TCP connections to 1.

Drive-by: Fix log::info! -> info! in wireguard peer_controller.rs

* Add persistent exit stream for entry→exit forwarding

Entry gateway now maintains a persistent TCP connection to the exit
gateway per client session, reusing it for all forward requests from
that client. This reduces TCP handshake overhead significantly.

Key changes:
- Add exit_stream: Option<(TcpStream, SocketAddr)> to LpConnectionHandler
- Modify handle_forward_packet() to open on first forward, reuse after
- Clear exit_stream on connection errors (auto-reconnect on next forward)
- Semaphore only acquired for connection opens, not reuse (sequential access)

* Fix code review issues for stream-oriented LP

- Add 30s timeout to exit stream I/O operations (nym-df31)
  Prevents handler from hanging on unresponsive exit gateway

- Return error on forward target address mismatch (nym-zegu)
  Previously warned and proceeded, which could mask bugs

- Close client stream on handshake error paths (nym-scvm)
  Prevents state machine inconsistency on timeout or failure

* Add LP registration idempotency and retry logic

Make LP registration resilient to network failures that could waste
credentials. When registration succeeds on the gateway but the response
is lost (e.g., network drop), clients can retry with the same WG key
and get the cached result instead of spending another credential.

Gateway-side:
- Add check_existing_registration() helper that looks up WG peer and
  returns cached GatewayData if already registered
- Add idempotency check in process_registration() dVPN branch
- Only return cached response if bandwidth > 0 (ensures registration
  was actually completed, not just peer created)
- Track idempotent registrations with lp_registration_dvpn_idempotent metric

Client-side:
- Add register_with_retry() to LpRegistrationClient that acquires
  credential once and retries handshake+registration on failure
- Add handshake_and_register_with_retry() to NestedLpSession for
  exit gateway registration via forwarding
- Add exponential backoff with jitter between retry attempts
- Verify outer session validity before nested session retry

Both retry methods clear state machine before retry to ensure fresh
handshake, and reuse the same credential across all attempts.

* Add no-mix-acks feature flag to nym-sphinx-framing

When enabled, mix nodes skip ack extraction and forwarding entirely.
The full payload (including ack portion) is returned as the message.

Closes: nym-3wrr

* Create nym-lp-speedtest crate scaffold

- Created tools/nym-lp-speedtest/ with Cargo.toml
- Added main.rs with CLI argument parsing
- Created stub modules: client.rs, speedtest.rs, topology.rs
- Added to workspace members
- Verified compilation with cargo check

* Implement topology fetching for nym-lp-speedtest

- Add topology.rs with NymTopology integration
- Fetch mix nodes and gateways from nym-api
- Build GatewayInfo with LP addresses (port 41264)
- Provide random_route_to_gateway() for Sphinx routing
- Add required Cargo.toml dependencies

* Implement LP+Sphinx+KCP client with SURB support

- Add send_data() and send_data_with_surbs() methods for mixnet data
- Integrate KCP reliable delivery with Sphinx packet construction
- Add x25519 encryption keypair for SURB reply mechanism
- Wire up main.rs to test LP handshake and data path
- Add NymRouteProvider support in topology for SURB construction
- Refactor send_data() to delegate to send_data_with_surbs(0) (DRY)

The client can now:
- Perform LP handshake with gateways
- Send data through the mixnet wrapped in KCP + Sphinx packets
- Attach SURBs for bidirectional communication
- Return encryption keys for decrypting replies

* Rename nym-lp-speedtest to nym-lp-client and fix KCP bug

- Rename crate from nym-lp-speedtest to nym-lp-client
- Fix KCP bug: add driver.update() call before fetch_outgoing()
  Without update(), KCP never moves segments from snd_queue to snd_buf
- Update CLI name, about string, and user agent to match new name

* Add LP mixnet mode registration with nym address return

- Extend RegistrationMode::Mixnet to include client_ed25519_pubkey
  and client_x25519_pubkey for nym address construction
- Add LpGatewayData struct containing gateway_identity and
  gateway_sphinx_key for SURB reply routing
- Add lp_gateway_data field to LpRegistrationResponse for mixnet mode
- Implement success_mixnet() constructor for mixnet registrations
- Update gateway registration to insert clients into ActiveClientsStore
  for SURB reply delivery, matching the websocket flow

* Implement LP data handler on UDP:51264

- Add LpDataHandler for UDP data plane (port 51264)
- Decrypt LP layer and forward Sphinx packets to mixnet
- Add outbound_mix_sender to LpHandlerState
- Integrate data handler spawn into LpListener::run()
- Add metrics for data packets received/forwarded/errors

Implements nym-yzzm

* Fix replay protection vulnerability in LP data handler

Use state machine process_input() instead of manual decryption to ensure
proper replay protection:
- Counter check against receiving window
- Counter marking after successful decryption

Also handle subsession actions gracefully (SendPacket ignored on UDP,
clients should use TCP control plane for rekeying).

Security fix for nym-yzzm implementation.

* feat(ipr): add KcpSessionManager for LP client KCP handling

- Add fetch_incoming() and recv() methods to KcpDriver for retrieving
  reassembled messages
- Create KcpSessionManager in ip-packet-router that manages KCP sessions
  keyed by conv_id (first 4 bytes of KCP packet header)
- Store ReplySurbs per session for sending anonymous replies
- Implement session timeout (5 min) and max sessions limit (10000)
- Add comprehensive tests for session lifecycle and KCP roundtrip

* feat(ipr): integrate KcpSessionManager into MixnetListener

- Add KcpSessionManager field to MixnetListener struct
- Add is_kcp_message() helper to detect KCP-wrapped payloads
- Add on_kcp_message() to process LP client KCP messages
- Refactor on_reconstructed_message() to route KCP vs regular IPR
- Add KCP tick timer (100ms) for session updates and cleanup
- Initialize KcpSessionManager in IpPacketRouter::run_service_provider()

KCP messages are detected by checking byte 4 for valid KCP commands
(81-84), which doesn't conflict with IPR protocol version bytes (6-8)
at position 0.

Closes: nym-96zl

* fix(ipr): prevent KCP detection false positives on IPR messages

Add secondary check in is_kcp_message() to exclude messages that match
IPR protocol header pattern (version 6-8 at byte 0, ServiceProviderType
0-2 at byte 1). This prevents false positives where IPR messages with
byte 4 in range 81-84 would be incorrectly routed to KCP processing.

Added 4 unit tests to validate the detection logic.

Closes: nym-6f3x

* fix(ipr): wrap KCP client responses in KCP before SURB reply

- Modify on_kcp_message to handle responses directly instead of returning them
- Add handle_kcp_response method that wraps response in KCP and sends via mixnet
- Ensures KCP clients receive KCP-wrapped responses for proper reassembly

Closes: nym-7oh2

* fix(ipr): send KCP protocol packets in tick instead of just logging

- Add get_sender_tag() and fetch_outgoing_for_conv() to KcpSessionManager
- Change handle_kcp_tick() to actually send ACKs/retransmissions via mixnet
- Reduce KCP tick interval from 100ms to 10ms for better responsiveness

This fixes the KCP reliability protocol which was broken because
protocol packets (ACKs, retransmissions) were generated but never sent.

* feat(lp-client): wrap payload in IpPacketRequest before KCP

- Add nym-ip-packet-requests and bytes dependencies
- Wrap payload in IpPacketRequest::new_data_request() before sending to KCP
- Add LP_DATA_PORT constant (51264) and lp_data_address field to GatewayInfo

This ensures IPR can properly parse incoming messages as DataRequest.
LP framing (wrapping Sphinx in LP before sending) is a separate task.

* feat(lp-client): add LP session management and UDP data plane support

- Add wrap_data() and session_id() to LpRegistrationClient for LP packet
  creation after handshake
- Add init_lp_session() and close_lp_session() to SpeedtestClient for
  managing LP sessions
- Extract prepare_sphinx_fragments() helper to reduce code duplication
  between send_data_with_surbs() and send_data_via_lp()
- Add send_data_via_lp() for sending Sphinx packets through LP's UDP
  data plane (port 51264)

The LP session is kept alive after TCP handshake closes, allowing
subsequent wrap_data() calls for UDP transmission without re-handshaking.

* random formatting

* replaced all instances of bincode::serialize and bincode::deserialize with explicit lp_bincode_serialiser() within the LP

* additional formatting

* removed source of possible panic from nym-kkt

invalid KEM mapping will now return an Err rather than panicking

* integration test for LP entry registration

This includes creation of mocks of various gateway-related components, such as the PeerController

* changed ClientHelloData serialisation

the old variant using bincode did not produce constant-length output in some cases

* Fixed generation of receiver index

removes the possible clash with the boostrap id

* Integration test for nested LP registration

- move `LpTransport` trait definition to shared `nym-lp-transport` crate
- make transport layer within `LpConnectionHandler` generic with respect to the forwarding target. it must, however, use the same type as the incoming client connection
- extracted explicit `LpConnectionHandler::establish_exit_stream` to more easily modify it in the future to fully protect the channel and disallow using untrusted egress points
- fix additional log-string interpolation nits

* resolved clippy issues pointed out by clippy 1.91

* added LP discovery into self-described endpoint:

- removed changes to the node bonding within the contract
- introduced '/api/v1/lewes-protocol' route within nym-node http api
- added 'lewes_protocol' field to 'NymNodeData' inside of NymNodeDescription
- refactored LpConfig to allow separate bind and announce addresses and used more strict typing

* chore: allow unwrap/expect within kkt benchmarking code

* chore: downgraded sha2 dep for cosmwasm compatibility

* clippy

* marking simd calls as unsafe

* fixed calls to '_mm_testz_si128'

* additional clippy fixes

---------

Co-authored-by: Georgio Nicolas <me@georgio.xyz>
Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
2026-01-14 09:06:02 +00:00
Simon Wicky 892341fa59 [chore] clippy fixes and use fixed rust version from REQUIRED_RUSTC_VERSION (#6295)
* clippy fix part 1

* use REQUIRED_RUSTC_VERSION instead of stable

* workflow fix

* forgot latest
2025-12-16 13:53:45 +01:00
Jędrzej Stuczyński 827c13b69e moved nym-gateway-probe to monorepo and updated rust-edition to 2024 (#6094)
dont build netstack in CI

additional rust 2024 fixes

fixes

removed temp.rs

first round of cleanup

removed duplicated NS types

moved gateway probe to the monorepo
2025-10-08 16:17:43 +01:00
mfahampshire 83a598907f Max/fix wasm client + build commands (#6043)
* Debug logging 

* Yield based logging

* Reintroduce non-dummy task manager, try add counting for
BatchMessageSender, a couple of compiler target introductions on use
statements.

* Fixed time runtime err

* Uncomment forgetme/rememberme

* remove diffs from debug

* missed commented out forgetme

* yet more forgetme comments

* * Added missing clientreqestsender clone to wasm client to stop
  premature drop & busyloop
* Removed hacky mem::forget fix

* Remove debug panic_hook

* Conditional import + use of wasm_utils::console_log

* add wasm_util dep

* Commenting out or removing debug logging

* Remove missed comment

* cleanup gitignore

* clippy

* update go version in ci

* removed unused deps

* add clippy ignore

* remove mixfetch from ci build

* add minifetch fix

* comment out unused ts builds

* stop contract clients killing ci for the moment

* wasm target locking for imports

* Either remove console_log! macro or introduce cfg(debug_assertions)

* downgrade netlink

* debug assertions for console_log import

* modify config logging (debug -> normal)

* remove clone for client_request_sender + grab directly in struct
  creation

* reintroduce debug print for config in debug mode

* remove ood / unused custom topology from worker example file

* clippy

* clippy - ignore todo() tests

* modified humantime test in line with new parsing rules
2025-10-07 09:55:41 +00:00
Simon Wicky 6a028417ad [chore] Clippy fix (#6060)
* clippy multiple of fix

* removed dead code?

* huh?

* ci fixes
2025-09-26 11:58:59 +02:00
Jędrzej Stuczyński d97be2d8ef bugfix: Recipient deserialisation for deserialisers missing bytes specialisation (#5991)
* bugfix: Recipient deserialisation for deserialisers missing bytes specialisation

for example toml or json will just default to visit_seq ignoring bytes related optimisations

* clippy
2025-09-01 11:30:35 +01:00
Jędrzej Stuczyński 615e98b166 Merge branch 'develop' into merge/release/2025.12-dolcelatte 2025-07-09 15:37:41 +01:00
Jędrzej Stuczyński 833114372a bugfix: key-rotation + reply SURBs (#5876)
* wip: changes to surb logic + stronger db typing

* surb invalidation logic

* chore: remove unused deps

* resolving todos

* a lot of additional bugfixes

* 1.88 clippy

* wasm fixes

* wasm clippy

* wallet clippy

* wait for epoch end when setting up new network

* split ReplyController into Sender and Receiver for easier reasoning

* additional reply surbs improvements

includes, but is not limited to: unconditionally reseting sender tag on restart, limiting number of surb re-requests, resetting stale surbs on load

* fixed calculation of number of removed surbs

* add additional calculated field to key rotation info

* DBG: 'request_reply_surbs_for_queue_clearing' temp logs

* fixes for silly mistakes

* conditionally reduce log severity
2025-07-04 16:29:03 +01:00
Jack Wampler a7b57d7e58 Make Mix hops optional for Mixnet Client SURBs (#5861)
* allow SURBs to be configured without mix hops

* gateways require consistency in surb format so if disabling mixnhops - use updated format
2025-07-03 09:21:50 -06:00
Jędrzej Stuczyński fa1d47e941 Bugfix/backwards compat (#5865)
* lowered log severity

* make nodes use legacy encoding for forwarding packets

* note regarding localnet noise
2025-06-19 09:57:46 +01:00
Jędrzej Stuczyński d8c84cc4d6 feat: key rotation (#5777)
* wip

* wip: wrap node's sphinx key with a manager

* wip: choosing correct key for packet processing

* further propagation of key rotation information

* attaching key rotation information to reply surbs

* added basic key rotation information to mixnet contract

* wip: introducing cached queries for key rotation info from nym api

* unified nym-api contract cache refreshing

* finish packet decoding

* multi api client + retrieving rotation id

* rotating sphinx key files

* logic for migrating config file

* wip: putting new sphinx keys to self described endpoints

* processing loop of KeyRotationController

* fixed sphinx key loading

* rotating bloomfilters

* wired up KeyRotationController

* flushing bloomfilters to disk and loading

* most of nym-node changes

* post rebase fixes

* fixes due to backwards compatible hostkeys

* split http state.rs file

* dont use deprecated fields

* fixed backwards compatible deserialisation of host information

* split up node describe cache

* added a dedicated CacheRefresher listener to perform full refresh outside the set interval

* controlling announced sphinx keys within nym-api

* retrieving rotation id when pulling topology

* split nym-nodes http handlers

* v2 nym-api endpoints to retrieve nodes with additional metadata information

* bug fixes...

* additional bugfixes and guards against stuck epoch

* testnet manager: set first nym-api as the rewarder

* fixed host information deserialisation

* fixed panic during first key rotation

* post rebase fixes

* clippy

* more guards against stuck epochs

* added helper method to reset node's sphinx key

* instantiate mixnet contract with custom key rotation validity

* additional bugfixes and debugging nym-api deadlock

* passing shutdown to nym apis client

* remove dead test

* post rebasing fixes

* missing MixnetQueryClient variants

* remove usage of deprecated methods in sdk example

* fix: incorrect method signature

* post rebasing fixes

* attempt to retrieve key rotation id before doing any config migration work

* ignore tests relying on networking behaviour

* allow networking failures in certain tests
2025-06-03 11:22:51 +01:00
Jędrzej Stuczyński b04d3ba376 add reserved byte to reply surb serialisation (#5731) 2025-04-25 10:05:38 +01:00
Jack Wampler 96239a7812 allow client to specify whether to include mix hops or not in MixnetClient Debug Config (#5696) 2025-04-15 10:49:11 -06:00
Jędrzej Stuczyński 84db9f6bcd chore: rename 'identity' module to 'ed25519' and 'encryption' to 'x25519' (#5707) 2025-04-13 11:58:25 +01:00
Jędrzej Stuczyński af2c4f50b6 Feature/updated sphinx payload keys (#5698)
* removed support for legacy packet types from NymCodec

I think nodes had plenty of time to upgrade given versioned variant was introduced in 2022

* temp: use local sphinx packet for development

* introduce new messages that use more efficient reply surbs encoding

* checks for incorrect encoding

* generate correct message depending on config value

* fixed current packet version

* made packet type selection configurable

* updated sphinx packet crate to the published version

* fixed wasm build

* fixes in outfox due to sphinx api changes

* additional tests

* clippy

* fixed log/tracing import
2025-04-10 13:43:29 +01:00
Jędrzej Stuczyński 0e38126fc5 Feature/replay protection (#5682)
* remove old packettype + fix: apply routing filter BEFORE delaying

* updated sphinx crate for allow usage of reply tags

* full pipeline for placeholder checking of packet replay

* replay protection with batched insertion

* running background task for clearing/flushing the BF

* allow disabling the replay detection + cleanup

* allow unwrap in bench code
2025-04-08 09:50:25 +01:00
Jędrzej Stuczyński 12026305d5 chore: clippy for 1.86 (#5685)
* chore: clippy for 1.86

* clippy inside wallet
2025-04-04 10:43:21 +01:00
Jędrzej Stuczyński d062524d32 mix throughput tester (#5661)
* wip: sending with single client

* tag packets to measure latency

* constantly logging rates

* concurrency

* adjusting some values

* write results to files upon completion
2025-03-31 15:57:24 +01:00
Jon Häggblad 3c432ac073 Clean stale partially received buffers (#5536)
* Clean stale partially received buffers

* Tweak timeout

* Do cleanup after handling new messages instead of in the select loop

* Debug logging and remove unreachable

* Downgrade log

* Tweak logs

* tweak whitespace

* Only run the stale check every 10 sec
2025-03-20 10:01:42 +01:00
Jędrzej Stuczyński f62d8813e0 chore: start sending v2 sphinx packets (#5554)
* chore: start sending v2 sphinx packets

* updated surb construction to use current format
2025-03-12 12:01:58 +00:00
Jędrzej Stuczyński 63002e784a Merge branch 'develop' into merge/release/2025.4-dorina-patched 2025-03-11 09:53:56 +00:00
Jędrzej Stuczyński 01c052e9a4 use legacy crypto for constructing SURB headers (#5579) 2025-03-06 20:13:16 +00:00
Jędrzej Stuczyński 27dc9c8024 Merge branch 'develop' into merge/release/2025.4-dorina 2025-03-04 11:00:24 +00:00
Jędrzej Stuczyński 94ff8a79ee feature: disallow routing mix packets to nodes not present in the topology (#5526)
* new NymNodeTopologyProvider to also keep track of ips of all nodes

* added nym-api endpoint for nodes existence by ip

* change behaviour of updating allowed nodes alongside the topology

* clippy

* license fix

* fix default filtering limit
2025-03-03 18:03:47 +00:00
Jędrzej Stuczyński 65175fee09 merge #5512 again after reverting due to incorrect rebase (#5520)
* setup workspace global lints to prevent needless panics

* removed sources of panic in nym-crypto, nym-node and nym-api

* adjusted test code
2025-02-26 10:52:09 +00:00
Jędrzej Stuczyński 8ba5322997 bugfix: bound check when recovering a reply SURB (#5502) 2025-02-26 09:48:21 +00:00
Jędrzej Stuczyński 8f5457e698 feature: allow nym-nodes to understand future version of sphinx packets (#5496) (#5518)
* use updated sphinx crate

* updated outfox usage of keygen in tests

* use x25519 in outfox

* remove redundant constructor

* adjusted key convertion traits
2025-02-26 09:47:57 +00:00
Jędrzej Stuczyński 6e5d0dac1b feature: allow nym-nodes to understand future version of sphinx packets (#5496)
* use updated sphinx crate

* updated outfox usage of keygen in tests

* use x25519 in outfox

* remove redundant constructor

* adjusted key convertion traits
2025-02-21 11:06:07 +00:00
Jędrzej Stuczyński a2322d6cdf feature: nym topology revamp (#5271)
* revamped NymTopology

* wip

* working e2e client

* updated nym-api

* updated nym-node

* updated rest of non-test code

* updated the rest of the codebase

* additional tweaks

* linux clippy fixes + adding additional dummy ipr types for better linting on non-linux targets
2024-12-19 10:44:34 +00:00
Jędrzej Stuczyński 5a07b73375 feature: hopefully final steps of the smoosh™️ (#5201)
* removed mnemonic from gateway config struct

scaffolding for common mixnet listener

running verloc unconditionally in a nym-node

remove filtering by mixnode

extracted verloc to separate crate

integrated nym-node-http-server more tightly with the binary

most logic for handling forward packets

running all mixnode-related tasks natively inside nymnode

removed gateway storage trait in favour of the only concrete implementation

most logic for handling final hop packets

using nym-node owned socket listener for gateways

utility for sending plain message through mixnet + gateway fix

using common packet forwarding in both modes

nifying nym-node metrics

reproduce behaviour of the console logger

cleaned up cli args

redesigned gateway tasks startup procedure

removing dead code

scaffolding for old config v6

config migration

implemented MixnetMetricsCleaner

* clippy

* require entry/exit for wireguard

* removed dead code in migration code

* updated config template

* use custom user agent for verloc queries

* fixed premature shutdown of gateway tasks

* hidden nym-api flag to allow illegal node ips

* experiment: final hop handing with wireguard

* added additional startup logs

* typo

* fixed legacy stats endpoint data

* additional logs

* apply review comments

* fixed local testnet manager
2024-12-05 17:21:36 +00:00
Jędrzej Stuczyński cfebd14655 Merge branch 'release/2024.14-crunch' into merge/release/2024.14-crunch 2024-12-02 11:21:09 +00:00
dynco-nym a9e62889c3 Remove explorer dependency (#5190)
* Move monitor code to a struct
- to store state in a struct

* explorer deprecation wip

* Replace explorer with ipinfo calls

* PR feedback

* Fix clippy

* Bump package version

* Remove ipinfo crate due to openssl dep

* Add remaining bandwidth log
2024-11-29 16:45:55 +01:00
Jędrzej Stuczyński 56206433e6 chore: apply 1.84 linter suggestions (#5192) 2024-11-29 09:20:45 +00:00
Jędrzej Stuczyński 0e5bd966dd Merge branch 'release/2024.13-magura-patched' into develop 2024-11-22 15:53:59 +00:00
Jędrzej Stuczyński 119c36b0bb added 'deterministic_route_selection' for sphinx packet route selection (#5157)
* added 'deterministic_route_selection' for sphinx packet route selection

* clippy + wasm

* Switch to ChaCha8Rng for deterministic route generation

---------

Co-authored-by: durch <durch@users.noreply.github.com>
2024-11-21 18:21:01 +00:00
dependabot[bot] 7abe1f505c Bump the patch-updates group across 1 directory with 10 updates
Bumps the patch-updates group with 9 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [anyhow](https://github.com/dtolnay/anyhow) | `1.0.89` | `1.0.90` |
| [clap](https://github.com/clap-rs/clap) | `4.5.18` | `4.5.20` |
| [clap_complete](https://github.com/clap-rs/clap) | `4.5.29` | `4.5.33` |
| [pin-project](https://github.com/taiki-e/pin-project) | `1.1.5` | `1.1.6` |
| [serde](https://github.com/serde-rs/serde) | `1.0.210` | `1.0.211` |
| [serde_json](https://github.com/serde-rs/json) | `1.0.128` | `1.0.132` |
| [wasm-bindgen](https://github.com/rustwasm/wasm-bindgen) | `0.2.93` | `0.2.95` |
| [wasm-bindgen-futures](https://github.com/rustwasm/wasm-bindgen) | `0.4.43` | `0.4.45` |
| [web-sys](https://github.com/rustwasm/wasm-bindgen) | `0.3.70` | `0.3.72` |



Updates `anyhow` from 1.0.89 to 1.0.90
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.89...1.0.90)

Updates `clap` from 4.5.18 to 4.5.20
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.18...clap_complete-v4.5.20)

Updates `clap_complete` from 4.5.29 to 4.5.33
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.29...clap_complete-v4.5.33)

Updates `pin-project` from 1.1.5 to 1.1.6
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.1.5...v1.1.6)

Updates `serde` from 1.0.210 to 1.0.211
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.210...v1.0.211)

Updates `serde_derive` from 1.0.210 to 1.0.211
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.210...v1.0.211)

Updates `serde_json` from 1.0.128 to 1.0.132
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/1.0.128...1.0.132)

Updates `wasm-bindgen` from 0.2.93 to 0.2.95
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/compare/0.2.93...0.2.95)

Updates `wasm-bindgen-futures` from 0.4.43 to 0.4.45
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/commits)

Updates `web-sys` from 0.3.70 to 0.3.72
- [Release notes](https://github.com/rustwasm/wasm-bindgen/releases)
- [Changelog](https://github.com/rustwasm/wasm-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustwasm/wasm-bindgen/commits)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: clap_complete
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: pin-project
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: serde_derive
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: wasm-bindgen
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: wasm-bindgen-futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
- dependency-name: web-sys
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: patch-updates
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-10-22 14:56:39 +00:00
Jędrzej Stuczyński d36ea20366 bugfix: dont store sent/received fragments unless explicitly enabled (#4991) 2024-10-21 09:27:48 +01:00
Jędrzej Stuczyński 45f4eeeff2 fixed linter issues and outdated tests 2024-10-10 15:38:00 +01:00
Jędrzej Stuczyński 51b511b27e Rebased the branch one more time
WIP; rebasing

Another branch squash

Squashing the v3 branch

changing min pledge amounts

logic for adding new nymnode into the contract

converting mixnode/gateway bonding into nym-node bonding

logic for migrating gateways into nymnodes

ibid for mixnodes

further nym-node work + fixed most existing unit tests

forbid nymnode migration with pending cost params changes

preassign nodeid for gateways

changing role assignment and epoch progression

changing role assignment and epoch progression

optional custom http port

logic for unbonding a nym-node

updating Delegation struct

logic for increasing pledge of either mixnode or nymnode

logic for decreasing pledge of either mixnode or a nym node

logic for changing cost params of either mixnode or a nym node

wip

initialise nymnodes storage

fixing transaction tests

fixed naive family tests

reward-compatibility related works

resolving delegation events

introduced rewarded set metadata

another iteration of restoring old tests

updated rewarding part of nym-api

parking the branch

unparking the branch

wip

purged families

added 'ExitGateway' role

passing explicit work factor for rewarding function

remove legacy layers storage

wip: node description queries

added announced ports to self-described api

step1 in gruelling journey of adding node_id to gateways

ensure epoch work never goes above 1.0

changed active set to contain role distribution

[theoretically] sending rewarding messages for the new rewarded set

[theoretically] assigning new rewarded set

reimplementing more nym-api features

remove legacy types

re-implement legacy network monitor

restoring further routes + minor refactor of NodeStatusCache

skimmed routes now return legacy nodes alongside nym-nodes

seemingly restored all functionalities in nym-api

removing more legacy things from the contract

initial contract cleanup

added nym-api endpoints to return generic annotations regardless of type

updated simulator to use new rewarding parameters

more contract cleanup

made existing mixnet contract tests compile

extra validation of nym-node bonding parameters

fixed additional compilation issues

fixed nym-api v3 database migration failure

added additional nym-node contract queries

updated the schema

made additional delegation/rewards queries compatible with both legacy mixnodes and nym-nodes

fixing existing unit tests in mixnet contract

wip

resolved first batch of 500 compiler errors

re-deprecating routes

making wallet's rust backend compile

fixed non-determinism in contract + nym-api build

fixes to the build

populating cotracts-cache with nym-nodes data

more missing nymnodes queries

temp mixnet contract methods + restored result submission in nym-api

allow deprecated routes

submitting correct results for mixnode results

removed deprecated re-export of AxumAppState and removed smurf naming

moved axum modules into support::http

cleaning up nym-api warnings

determine entry gateways before exits

exposed transaction to update nym-node config

missing memo for updating node config

 new routes

added routes to swagger and fixed relative paths

fixed some macro derivations

added nym-node commands to nym-cli
2024-10-10 13:27:05 +01:00
Drazen Urch e2d1806e49 Extract packet processing from mixnode-common (#4949)
* Extract packet processing from mixnode-common

* Cleanup
2024-10-07 12:00:36 +02:00
Jędrzej Stuczyński 0d2418ef6a Merge pull request #4885 from nymtech/feature/updated-gateway-registration
Feature/updated gateway registration
2024-09-20 09:09:28 +01:00