Use state machine process_input() instead of manual decryption to ensure
proper replay protection:
- Counter check against receiving window
- Counter marking after successful decryption
Also handle subsession actions gracefully (SendPacket ignored on UDP,
clients should use TCP control plane for rekeying).
Security fix for nym-yzzm implementation.
- Add LpDataHandler for UDP data plane (port 51264)
- Decrypt LP layer and forward Sphinx packets to mixnet
- Add outbound_mix_sender to LpHandlerState
- Integrate data handler spawn into LpListener::run()
- Add metrics for data packets received/forwarded/errors
Implements nym-yzzm
- Extend RegistrationMode::Mixnet to include client_ed25519_pubkey
and client_x25519_pubkey for nym address construction
- Add LpGatewayData struct containing gateway_identity and
gateway_sphinx_key for SURB reply routing
- Add lp_gateway_data field to LpRegistrationResponse for mixnet mode
- Implement success_mixnet() constructor for mixnet registrations
- Update gateway registration to insert clients into ActiveClientsStore
for SURB reply delivery, matching the websocket flow
Make LP registration resilient to network failures that could waste
credentials. When registration succeeds on the gateway but the response
is lost (e.g., network drop), clients can retry with the same WG key
and get the cached result instead of spending another credential.
Gateway-side:
- Add check_existing_registration() helper that looks up WG peer and
returns cached GatewayData if already registered
- Add idempotency check in process_registration() dVPN branch
- Only return cached response if bandwidth > 0 (ensures registration
was actually completed, not just peer created)
- Track idempotent registrations with lp_registration_dvpn_idempotent metric
Client-side:
- Add register_with_retry() to LpRegistrationClient that acquires
credential once and retries handshake+registration on failure
- Add handshake_and_register_with_retry() to NestedLpSession for
exit gateway registration via forwarding
- Add exponential backoff with jitter between retry attempts
- Verify outer session validity before nested session retry
Both retry methods clear state machine before retry to ensure fresh
handshake, and reuse the same credential across all attempts.
- Add 30s timeout to exit stream I/O operations (nym-df31)
Prevents handler from hanging on unresponsive exit gateway
- Return error on forward target address mismatch (nym-zegu)
Previously warned and proceeded, which could mask bugs
- Close client stream on handshake error paths (nym-scvm)
Prevents state machine inconsistency on timeout or failure
Entry gateway now maintains a persistent TCP connection to the exit
gateway per client session, reusing it for all forward requests from
that client. This reduces TCP handshake overhead significantly.
Key changes:
- Add exit_stream: Option<(TcpStream, SocketAddr)> to LpConnectionHandler
- Modify handle_forward_packet() to open on first forward, reuse after
- Clear exit_stream on connection errors (auto-reconnect on next forward)
- Semaphore only acquired for connection opens, not reuse (sequential access)
Gateway (handler.rs):
- Add bound_receiver_idx field for session-affine connections
- Convert handle() from single-packet to loop with EOF detection
- Add validate_or_set_binding() for receiver_idx validation
- Set binding in handle_client_hello after collision check
- Centralize emit_lifecycle_metrics in main loop only
- Add is_connection_closed() helper for graceful EOF
Client (client.rs):
- Add stream field for persistent TCP connection
- Add ensure_connected(), send_packet(), receive_packet(), close() methods
- Modify perform_handshake_inner() to use persistent stream
- Modify register_with_credential() to use persistent stream
- Modify send_forward_packet() to use persistent stream
- Keep connect_send_receive() for reference (marked dead_code)
This reduces handshake overhead from ~5 TCP connections to 1.
Drive-by: Fix log::info! -> info! in wireguard peer_controller.rs
Breaking wire protocol change: MessageType field increased from 2 bytes
to 4 bytes in LP packets. This future-proofs the message type space and
aligns with other u32 fields.
Changes:
- message.rs: #[repr(u32)], from_u32(), to_u32()
- error.rs: InvalidMessageType(u32)
- codec.rs: All serialization/deserialization updated to 4-byte msg_type
- Cleartext parsing: inner_bytes[4..8], content at [8..]
- AEAD parsing: decrypted[4..8], content at [8..]
- Serialization: 4 bytes for message type
Replace .session().ok() with proper error handling to fail fast when
session is Closed or Processing after state machine processing.
Previously, the code silently continued with outer_key = None, which
could cause protocol errors downstream.
Closes: nym-8de0
Limits concurrent outbound connections when forwarding LP packets to
prevent file descriptor exhaustion under high load.
Key changes:
- Add max_concurrent_forwards config (default 1000)
- Add forward_semaphore to LpHandlerState
- Acquire semaphore permit before connecting in handle_forward_packet
- Return "Gateway at forward capacity" error when at limit
This provides load signaling so clients can choose another gateway
when the current one is overloaded.
Design note: Connection pooling was considered but provides minimal
benefit since telescope setup is one-time and targets are distributed
across many different gateways. See AIDEV-NOTE in LpHandlerState for
full analysis.
Closes: nym-xi3m
Adds localnet testing mode to gateway-probe for LP development:
- Add TestMode enum for different probe configurations
- Add --gateway-ip flag for direct gateway testing
- Implement two-hop WireGuard tunnel for localnet
- Add mock ecash support for testing without real credentials
- Add netstack Go bindings for userspace networking
- Restructure probe with mode and common modules
- Update README with localnet mode documentation
Extends LP protocol with telescoping architecture for nested sessions:
- Add nested session support with KKpsk0 rekeying
- Add subsession support with collision detection
- Implement unified packet format with outer header
- Refactor gateway handlers for single-packet forwarding
- Add TTL-based state cleanup for stale sessions
- Add outer AEAD encryption layer
- Refactor registration client for packet-per-connection model
* one commit to rule them all
* remove too aggressive copy pasting
* update details when outdated
* typo and serde alias
* no hostname option and fixes
* fix wasm client?
* non fallback fixed
* improve gateway details update
* better ws addresses
* PR review fixes
* improve type safety on update_gateway_published_data
* fix client gateway storage migration
* placeholder handling of wg registration with upgrade mode token
* include upgrade mode credentials as part of credential storage
* introduce helper for decoding JWT payload
* expose methods for removing emergency credentials from the storage
* don't allow duplicate emergency credentials with the same content
* added authenticator ClientMessage for upgrade mode check
* retrieve credentials with longest expiration first
* post rebasing fixes
* fixed gateway config
* feat: allow specifying minimum node performance for client init
* nym-node UM improvements
* fixed upgrade mode bandwidth on initial authentication
* fix: logs and thresholds
* expose attestation information from nym-node http api
* additional logs
* post rebasing fixes
* make @simonwicky happy by removing empty lines in emergency_credential table definition
* chore: remove '_' prefix for internal counters within in-mem ecash storage
* improved import of 'UpgradeModeState' within the nym-node
* use explicit time dependency within credential-storage
* re-order imports within the gateway-client
* moved 'AvailableBandwidth' definition to the monorepo
* squashing feat: merge intermediate upgrade mode changes #6174 to more easily resolve merge conflicts during rebasing
added additional v2 query for metadata endpoint for requesting upgrade mode recheck
added additional message to v6 authenticator to request explicit upgrade mode recheck
clippy
test fixes due to updated keys
updated assertion for upgrading v1 top up request to v2
compare attester public key against the expected value within the credential proxy
use pre-generated attestation public keys within nym-nodes
remove version deprecation
bugfix: default bandwidth response for authenticator
expose upgrade mode information in authenticator responses
adding tests for new v2 server
passing upgrade mode information in metadata endpoint
v2 wireguard private metadata
bugfix: make sure to immediately poll for attestation after spawning task
fix gateway probe and remove code duplication for finalizing registration
squashing before rebasing
post rebasing fixes
AuthenticatorVersion helpers
additional nits
allow unwraps in mocks
fixed linux build
clippy
integrating upgrade mode into authenticator
fixed build after adding wrappers to response types
conditionally updating peer handle bandwidth
cleanup
negotiate initial protocol during registration
change auth to use highest protocol
handler for JWT message
dont meter client bandwidth in upgrade mode
handling recheck requests
sending information about upgrade_mode on client messages
gateway watching for upgrade mode attestation
wip: gateways to disable bandwidth metering on upgrade mode
* fixed ServerResponse deserialisation
* fixed incorrect swagger path for upgrade mode check endpoint
* moved upgrade mode endpoint out of bandwidth routes
* chore: remove unused error variant
* removed re-export of UpgradeModeAttestation from credentials-interface
* chore: define single source of truth for minimum bandwidth threshold value
* moved type definitions out of traits.rs
* updated v6 versioning to point to niolo release instead
* fixed incorrect error mapping
* squashing work on using cancellation in nym crates
making nym-task wasm compilable
removed sending of status messages
replaced TaskManager with ShutdownManager in the validator rewarder
additional helpers for ShutdownManager
simplified ShutdownToken by removing the name field
TaskClient => ShutdownToken within all client tasks
wip: remove TaskHandle
* track all long-living client tasks
* add task tracking for most top level tasks within nym-node
* improved default builder
* split up cancellation module
* module documentation and unit tests
* nym node fixes and naming consistency
* wasm fixes
* assert_eq => assert
* wasm fixes and made 'run_until_shutdown' take reference instead of ownership
* linux-specific fixes to IpPacketRouter
* post rebasing fixes for signing monitor
* add ShutdownManager constructor to build it from an external token
* applying PR review suggestions
* Set cached storage counters to 0 (#5812)
* Set cached storage counters to 0
* u64 to i64 log possible error
* Check addition too
Debug commit
Remove more data from wg storage peer
Put actual ticket type in storage
Simplify add peer
Finish rebase
Pass defguard Peer
Cache less data for consumption
GatewayStorage traits
Wg API trait
Mock test structures
Unit test for peer controller
EcashManager trait
Init test of Authenticator
Remove peer test
* Fix windows different API
* Use make_bincode_serializer like in other places
* Add log_slow_statements to gateway storage
* Use correct LevelFilter
* Fix clippy
* More win fix
* Win clippy
* Use two error variants more
* Use only one Arc<RwLock<T>> instead of many more
* Remove commented test
* Specific trait import
* wip
* wip: wrap node's sphinx key with a manager
* wip: choosing correct key for packet processing
* further propagation of key rotation information
* attaching key rotation information to reply surbs
* added basic key rotation information to mixnet contract
* wip: introducing cached queries for key rotation info from nym api
* unified nym-api contract cache refreshing
* finish packet decoding
* multi api client + retrieving rotation id
* rotating sphinx key files
* logic for migrating config file
* wip: putting new sphinx keys to self described endpoints
* processing loop of KeyRotationController
* fixed sphinx key loading
* rotating bloomfilters
* wired up KeyRotationController
* flushing bloomfilters to disk and loading
* most of nym-node changes
* post rebase fixes
* fixes due to backwards compatible hostkeys
* split http state.rs file
* dont use deprecated fields
* fixed backwards compatible deserialisation of host information
* split up node describe cache
* added a dedicated CacheRefresher listener to perform full refresh outside the set interval
* controlling announced sphinx keys within nym-api
* retrieving rotation id when pulling topology
* split nym-nodes http handlers
* v2 nym-api endpoints to retrieve nodes with additional metadata information
* bug fixes...
* additional bugfixes and guards against stuck epoch
* testnet manager: set first nym-api as the rewarder
* fixed host information deserialisation
* fixed panic during first key rotation
* post rebase fixes
* clippy
* more guards against stuck epochs
* added helper method to reset node's sphinx key
* instantiate mixnet contract with custom key rotation validity
* additional bugfixes and debugging nym-api deadlock
* passing shutdown to nym apis client
* remove dead test
* post rebasing fixes
* missing MixnetQueryClient variants
* remove usage of deprecated methods in sdk example
* fix: incorrect method signature
* post rebasing fixes
* attempt to retrieve key rotation id before doing any config migration work
* ignore tests relying on networking behaviour
* allow networking failures in certain tests
* Clippy in wallet & sdk
* Clippy in wallet
* Pin rust to 1.86 in builder
* apply changes from b7da75a18c
* missing nym-node features
* Box all the things
* additional boxes in the wallet
* post rebasing clippy
---------
Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
* move SessionType into statsitcis common crate
* add RememberMe to clients config
* change stats collection logic to handle remember me
* set up sdk client to send remember me message
* bump NS API version
* introduced v2 authentication request between clients and gateways
* client to send v2 auth when possible
* added persistence to last used authentication timestamp
* added clients identity to signed plaintext
* Upgrade to thiserror 2.0
* Remove line macros in vesting contract error type
* Name positional arguments in GatewayRequestsError
* Named positional argument
* Revert "Remove line macros in vesting contract error type"
This reverts commit 49f937da3f.
* Use positional arguments for line
* add timestamp to stored client messages
* removed dead code
* starting node task to remove old messages
* added log for number of removed messages
* debug log on task finishing
* fixed bearer auth for prometheus route
* basic prometheus metrics
* added rates on global values
* improved structure on the prometheus metrics
* added additional metrics for ingress websockets and egress mixnet connections
* some channel business metrics
* fixed metrics registration and added additional variants
* added counter for number of disk persisted packets
* counter for pending egress packets
* counter for pending egress forward packets
* clippy
* removed mnemonic from gateway config struct
scaffolding for common mixnet listener
running verloc unconditionally in a nym-node
remove filtering by mixnode
extracted verloc to separate crate
integrated nym-node-http-server more tightly with the binary
most logic for handling forward packets
running all mixnode-related tasks natively inside nymnode
removed gateway storage trait in favour of the only concrete implementation
most logic for handling final hop packets
using nym-node owned socket listener for gateways
utility for sending plain message through mixnet + gateway fix
using common packet forwarding in both modes
nifying nym-node metrics
reproduce behaviour of the console logger
cleaned up cli args
redesigned gateway tasks startup procedure
removing dead code
scaffolding for old config v6
config migration
implemented MixnetMetricsCleaner
* clippy
* require entry/exit for wireguard
* removed dead code in migration code
* updated config template
* use custom user agent for verloc queries
* fixed premature shutdown of gateway tasks
* hidden nym-api flag to allow illegal node ips
* experiment: final hop handing with wireguard
* added additional startup logs
* typo
* fixed legacy stats endpoint data
* additional logs
* apply review comments
* fixed local testnet manager
durch