Limits concurrent outbound connections when forwarding LP packets to
prevent file descriptor exhaustion under high load.
Key changes:
- Add max_concurrent_forwards config (default 1000)
- Add forward_semaphore to LpHandlerState
- Acquire semaphore permit before connecting in handle_forward_packet
- Return "Gateway at forward capacity" error when at limit
This provides load signaling so clients can choose another gateway
when the current one is overloaded.
Design note: Connection pooling was considered but provides minimal
benefit since telescope setup is one-time and targets are distributed
across many different gateways. See AIDEV-NOTE in LpHandlerState for
full analysis.
Closes: nym-xi3m
Extends LP protocol with telescoping architecture for nested sessions:
- Add nested session support with KKpsk0 rekeying
- Add subsession support with collision detection
- Implement unified packet format with outer header
- Refactor gateway handlers for single-packet forwarding
- Add TTL-based state cleanup for stale sessions
- Add outer AEAD encryption layer
- Refactor registration client for packet-per-connection model
* placeholder handling of wg registration with upgrade mode token
* include upgrade mode credentials as part of credential storage
* introduce helper for decoding JWT payload
* expose methods for removing emergency credentials from the storage
* don't allow duplicate emergency credentials with the same content
* added authenticator ClientMessage for upgrade mode check
* retrieve credentials with longest expiration first
* post rebasing fixes
* fixed gateway config
* feat: allow specifying minimum node performance for client init
* nym-node UM improvements
* fixed upgrade mode bandwidth on initial authentication
* fix: logs and thresholds
* expose attestation information from nym-node http api
* additional logs
* post rebasing fixes
* make @simonwicky happy by removing empty lines in emergency_credential table definition
* chore: remove '_' prefix for internal counters within in-mem ecash storage
* improved import of 'UpgradeModeState' within the nym-node
* use explicit time dependency within credential-storage
* re-order imports within the gateway-client
* moved 'AvailableBandwidth' definition to the monorepo
* squashing feat: merge intermediate upgrade mode changes #6174 to more easily resolve merge conflicts during rebasing
added additional v2 query for metadata endpoint for requesting upgrade mode recheck
added additional message to v6 authenticator to request explicit upgrade mode recheck
clippy
test fixes due to updated keys
updated assertion for upgrading v1 top up request to v2
compare attester public key against the expected value within the credential proxy
use pre-generated attestation public keys within nym-nodes
remove version deprecation
bugfix: default bandwidth response for authenticator
expose upgrade mode information in authenticator responses
adding tests for new v2 server
passing upgrade mode information in metadata endpoint
v2 wireguard private metadata
bugfix: make sure to immediately poll for attestation after spawning task
fix gateway probe and remove code duplication for finalizing registration
squashing before rebasing
post rebasing fixes
AuthenticatorVersion helpers
additional nits
allow unwraps in mocks
fixed linux build
clippy
integrating upgrade mode into authenticator
fixed build after adding wrappers to response types
conditionally updating peer handle bandwidth
cleanup
negotiate initial protocol during registration
change auth to use highest protocol
handler for JWT message
dont meter client bandwidth in upgrade mode
handling recheck requests
sending information about upgrade_mode on client messages
gateway watching for upgrade mode attestation
wip: gateways to disable bandwidth metering on upgrade mode
* fixed ServerResponse deserialisation
* fixed incorrect swagger path for upgrade mode check endpoint
* moved upgrade mode endpoint out of bandwidth routes
* chore: remove unused error variant
* removed re-export of UpgradeModeAttestation from credentials-interface
* chore: define single source of truth for minimum bandwidth threshold value
* moved type definitions out of traits.rs
* updated v6 versioning to point to niolo release instead
* fixed incorrect error mapping
* squashing work on using cancellation in nym crates
making nym-task wasm compilable
removed sending of status messages
replaced TaskManager with ShutdownManager in the validator rewarder
additional helpers for ShutdownManager
simplified ShutdownToken by removing the name field
TaskClient => ShutdownToken within all client tasks
wip: remove TaskHandle
* track all long-living client tasks
* add task tracking for most top level tasks within nym-node
* improved default builder
* split up cancellation module
* module documentation and unit tests
* nym node fixes and naming consistency
* wasm fixes
* assert_eq => assert
* wasm fixes and made 'run_until_shutdown' take reference instead of ownership
* linux-specific fixes to IpPacketRouter
* post rebasing fixes for signing monitor
* add ShutdownManager constructor to build it from an external token
* applying PR review suggestions
* Set cached storage counters to 0 (#5812)
* Set cached storage counters to 0
* u64 to i64 log possible error
* Check addition too
Debug commit
Remove more data from wg storage peer
Put actual ticket type in storage
Simplify add peer
Finish rebase
Pass defguard Peer
Cache less data for consumption
GatewayStorage traits
Wg API trait
Mock test structures
Unit test for peer controller
EcashManager trait
Init test of Authenticator
Remove peer test
* Fix windows different API
* Use make_bincode_serializer like in other places
* Add log_slow_statements to gateway storage
* Use correct LevelFilter
* Fix clippy
* More win fix
* Win clippy
* Use two error variants more
* Use only one Arc<RwLock<T>> instead of many more
* Remove commented test
* Specific trait import
* introduced v2 authentication request between clients and gateways
* client to send v2 auth when possible
* added persistence to last used authentication timestamp
* added clients identity to signed plaintext
* add timestamp to stored client messages
* removed dead code
* starting node task to remove old messages
* added log for number of removed messages
* debug log on task finishing
* fixed bearer auth for prometheus route
* basic prometheus metrics
* added rates on global values
* improved structure on the prometheus metrics
* added additional metrics for ingress websockets and egress mixnet connections
* some channel business metrics
* fixed metrics registration and added additional variants
* added counter for number of disk persisted packets
* counter for pending egress packets
* counter for pending egress forward packets
* clippy
* removed mnemonic from gateway config struct
scaffolding for common mixnet listener
running verloc unconditionally in a nym-node
remove filtering by mixnode
extracted verloc to separate crate
integrated nym-node-http-server more tightly with the binary
most logic for handling forward packets
running all mixnode-related tasks natively inside nymnode
removed gateway storage trait in favour of the only concrete implementation
most logic for handling final hop packets
using nym-node owned socket listener for gateways
utility for sending plain message through mixnet + gateway fix
using common packet forwarding in both modes
nifying nym-node metrics
reproduce behaviour of the console logger
cleaned up cli args
redesigned gateway tasks startup procedure
removing dead code
scaffolding for old config v6
config migration
implemented MixnetMetricsCleaner
* clippy
* require entry/exit for wireguard
* removed dead code in migration code
* updated config template
* use custom user agent for verloc queries
* fixed premature shutdown of gateway tasks
* hidden nym-api flag to allow illegal node ips
* experiment: final hop handing with wireguard
* added additional startup logs
* typo
* fixed legacy stats endpoint data
* additional logs
* apply review comments
* fixed local testnet manager
* draft of client data collection
* refactor gateway stats collection to fit client stats collection in same common crate
* moved client stats event and reporter to common crate
* basic os reporting
* add stats reporting address in sdk
* integrate stats scaffolding changes
* remove tokio spawn to potentially accomodate wasm32
* fmt
* fix typo
* add client_stats_id
* unify stats reporting
* avoid shutdown handle drop
* add client_type to stats reporting
* better way to build statsReportingconfig
* disarm shutdown on sink
* remove sink reporter and env dev-dependency
* cherrypick from jon/send-packet-stats
* uncoditionally start controller + licensing
* improve ClientStatsReport serialization
* better time handling
* reintroduce proper local reporting
* Let task wait for shutdown when exiting
* Log tweak
---------
Co-authored-by: jmwample <jmwample@users.noreply.github.com>
Co-authored-by: Jon Häggblad <jon.haggblad@gmail.com>
* add stats storage to gateways
* config fix
* add stats storage model and logic
* adapt stats collection to new storage
* stats cleanup on start
* change to linux only code
* tweaks
* modified stats cleanup + change session started
* change wrong table name
* store crashed session as 0 duration
* adapt for sqlx 0.7
* remove unused dependencies
* revert changes from gateway config, as it is broken anyway
* copyright and misc stuff
---------
Co-authored-by: Simon Wicky <simon@linode2-2.net>
* added explicit SP suffix to started tasks
* added 'GatewayTopologyProvider' that always injects itself into the network
* use the new topology provider to bypass described bootstrapping problem
* add session type based on ecash ticket collection
* avoid setting session type if already set
* change duration type to duration
* add a cap for finished sessions
* add stats model
* add stats collection
* add stats route
* propagate stuff and run stuff
* cargo stuff
* sqlx unused what?
* add sessions started stat
* session durations in miliseconds
* apply Jon's comments
* [Product Data] Second step in gateway usage data collection (#4964)
* turn stats collection into event based
* move events into a common crate for future use elsewhere
* apply Jon's comments
* Create credential verifier in authenticator
* Add new version of peer storage with client id
* Fix v1 to what it was before
* Compact storage into ecash verifier
* Fix non-linux build
* Less overlapping conditions
* Remove moved code
* Use handler thread for each peer
* Re-spawn stored handles at startup
* Keep new function without async & Result
* Put query peer in function too
* Query bandwidth
* Fix clippy
* Replace tap with inspect_err
* Fix copyright year
* Handle version 2 on the reqeust deser
* Add protocol type in req/resp messages
* Store wireguard peers in db
* Add update to nym-node
* Move gateway-requests and gateway-storage to common
* Carry storage to PeerController field
* Double kernel modifications with storage ones
* Take storage peers at boot
* Link storage query for registration flow
* Move authenticator peer comms in peer manager
* Modify template too
* Remove unused
* Fix clippy
* Fix clippy non-linux
* Keep storage data up-to-date on every check
* Check for staleness in storage timestamps
* Remove potential for panic in unwrap
* Fix clippy
* Fmt
* Clippy after rebase
* Remove in memory test structure
add offline ecash library
minor changes in coconut benchmarks
add ecash smart contract
change contract traits from coconut to ecash
first wave of andrew's suggestion
first wave of andrew's suggestion
second wave of andrew's suggestion for ecash lib
andrew's suggestion for ecash contract
licensing commit
safety comments for most unwraps
more unwrap handling
change chrono crate for time
latest cargo lock
error revamp
small visibility fix
small fix
remove indexedmap from contract + some tweaks
add cw2 version in ecash contract
remove envryption key from contract
change types from coconut to ecash types
adapt api model for credential issuance
adapt issued credential storage on API
add signatures cache on API
change API routes for new blind signing
modify issued_credential table
add issuance logic client-side
credential and signature storage client side
utils for credential issuance
first wave of fix
some of andrew's suggestions
remove encryption key from deposit
freepass issuance client side
freepass issuance API side
andrew's suggested fixes
other suggested fix
adapt change from PR below
allow offline verification flag
credential spending models
credential spending models for client
credential preperation for the client
credential preperation for the client
credential storage for spending on client
bloom filter for API
spent credential storage on validators
API route for spending online and offline ecash
API routes in the client lib
credential storage on gateway
ecash verifier to replace coconut verifier
accept credentials on gateway
bandwidth expiration for gateways
client ask for more bandwidth if it runs out
credential import
adapt nym validator rewarder and sdk
fix tests api tests and add constants
cargo fmt and lock and small test fix
cargo fmt and lock and small test fix
cargo lock
move stuff where they belong in ecash and static parameters
move some constants, error handling and phase out time crate
error revamp part 2
secret key by ref instead of clone
change l in wallet and v visibility
rework payinfo
rework monster tuples
fix expiration date signature cloning
minor fixes
final bits and bobs fixes
final bits and bobs fixes
rename l accessor to tickets_spent
wave of fixes
second wave of fixes
change hash domain value
removed benchmark flag
remove useless stringification in storage
nuke Bandwidth voucher
change timestamps to offsetdatetime
key name change
post-rebase fixes
update nym-connect 'time' dep due to broken semver
upload ecash contract to the build server
make wasm zknym-lib compile
but it won't work properly just yet
make wasm zknym-lib compile
but it won't work properly just yet
fix typo in ecash contract deps
make sure to use 0.1.0 sphinx packet
optimise pairings in 'check_vk_pairing'
derive serde for ecash types
simplified g1 tuple byte conversion
further optimise the pairing
unified signature type + renamed nym-api coconut module to ecash
using bincode serialiser for more complex binary types
using multimiller loop instead of rayon for verifying coin indices signatures
batching signature verification wherever possible
feature-locked rayon
clippy
refactor ecash contract a bit + introduce deposit storage
reworked find_proposal_id
various minor fixed
add offline_zk_nyms to nym-node everywhere
add missing #query
change test value to fit new serialization
optimised deposits storage
removed duplicate decompression code
using deposit_id instead of transaction hash
removed freepasses
split up ecash handling
unified shared state
fixed deposit_id parsing
log recovered deposit id
removed online verification
add detailed build info to ecash contract
fixed deserialisation of deposit amount received from nyxd queries
changed deposit to only persist attached pubkey
first iteration of split of verification and redemption
basic tool for setting up new network
expanded the tool with the option to bypass DKG
rename + init network without DKG
setting up locally running apis
ecash key migration
more local functionalities
wip fixing sql schemas
gateway immediately submitting redemption proposal
and getting it passed if valid
most of the gateway logic for split redemption with error recovery
fixed gateway not persisting ecash signers
simplify creation of compatible client
create properly serialised ecash key from the beginning
rebuild missing tickets and proposals on startup
stop ticket issuance during DKG transition
fixing build issues
split out ecash storage on nym-api side
master-verification-key route
caching all the signatures and keys
implemented aggregated routes for nym-apis
swagger UI for ecash endpoints
added explicit annotation for index and expiration signatures
revamped client ticketbook storage
save all recovery information in the same underlying storage
wrapper for bloomfilter
being more aggressive with marking tickets as used
ensure client has correct signatures before making deposit
fix deserialisation of AggregatedExpirationDateSignatureResponse + add ticketbook table
split nym-api ecash routes handlers into multiple files
fixed deserialisation of encoded expiration date
add tt_gamma1 to challenge and change naming for paper consistency
rotating double spending bloomfilter
nym-api test fixes + make sure to insert initial BF params
fixed ecash benchmark code
updated contract schema
updated CI to not upload gateway/mixnode binaries
ticket bandwidth revocation
added default deserialisation for zk nym config
post-rebase fixes
* Add check for 1GB/day/user and remove stale check
* Use saturated_sub
* Remove from wg peers
* Use 10 seconds instead of 1
* Query bandwidth message
* Ad client query message too
* Keep stale check
* Make bandwidth cap value public
* Fix consumed vs available bug
* Don't overwrite existing registrations
* Use self pub key instead of peer's
* Remove nym-network-statistics from workspace
* Remove nym-network-statistics
* Cargo.lock
* Update ci workflow
* Remove code that refers to removed nym-network-statistics
* Remove more
* Remove nym-statistics-common
* Delete commented out modules
* Remove commented out code
* Remove more commented out code
* Remove more commented out stuff
* Remove unused function
durch