* wip
* batch processing of forward packets
* tmp: additional metrics for remote node
* fixed incorrect prometheus metric registration
* unified runtime metrics
* unify mixnet client metrics
* packet forwarding cleanup
* add batching for emptying the delay queue
* cleanup client io loop
* feat(nym-node): reap idle mixnet connections (ingress + egress)
Close mixnet connections that sit with no traffic past a configurable idle period (mixnet.debug.connection_idle_timeout, default 5min, 0 disables) to bound lingering tokio tasks/sockets.
Ingress handle_stream is read-only, so a silently-gone peer (NAT drop, crash without FIN, half-open) never triggers FIN/RST and the task would block on .next() forever; a new idle select arm closes it (the post-loop replay flush still runs, so nothing is stranded). Egress run_io_loop gets the symmetric arm keyed on last_send; on close EvictOnDrop clears the cache entry and the next packet transparently reconnects.
Adds a cumulative nym_node_network_idle_closed_ingress_mixnet_connections counter; egress reaping is observed via the existing active-egress gauge plus an exit_reason=idle_timeout log.
* downgrade sysinfo
* refactor(nym-node): split PacketForwarder into router + delay-queue tasks
Split the single PacketForwarder task into two concurrently-scheduled tasks connected by a bounded handoff channel, so intake and delayed-release no longer block each other.
PacketRouter (router.rs) is the intake task: sole consumer of the ingress channel, it applies the routing filter and either forwards zero/already-elapsed-delay packets directly or hands delayed ones to the delay task. Its per-packet work is sub-µs, so new packets no longer wait behind delayed-release processing (collapses the ForwarderQueue tail).
DelayForwarder (delay.rs) owns the NonExhaustiveDelayQueue exclusively (it can't be shared by reference). Its run loop services BOTH branches on every wakeup - draining pending inserts first to bring the queue current, then flushing everything now due - so the biased select can't let releases and inserts starve each other, and a freshly-arrived-but-already-due packet releases in the same pass (marginally improving DelayQueueOverrun).
The mixnet client is shared as Arc<C>; handoff-channel overflow is dropped as an egress drop rather than blocking, keeping intake decoupled from release.
* feat(nym-node): bound egress flush with a write timeout
Cap how long a single egress batch flush may block on a congested peer socket (mixnet.debug.connection_write_timeout, default 500ms, 0 disables), so a slow peer can no longer back this connection's egress queue up into the multi-second range - the root of the EgressQueue and SocketWrite tails.
A single timeout is treated as transient congestion: the un-fed tail of the batch is abandoned but the connection is retained. This is sound because NoiseStream::poll_write encrypts and buffers each frame synchronously, so a cancelled flush leaves the noise transport nonce-consistent and a later flush resumes the byte stream in order - so a momentary spike costs no re-handshake. Only MAX_CONSECUTIVE_WRITE_TIMEOUTS (3) timeouts in a row, i.e. a persistently congested peer, tears the connection down (it reconnects on the next packet); a successful flush resets the counter.
Buffer-size tuning (maximum_connection_buffer_size) deliberately left for live data.
* revert PacketForwarder split in favour of a single task that clears both channels on wake
* squashing localnet-v2 commits (again)
cargo fmt
fixes to localnet purge
provide path in the error message
output args
log failed exec
print based on tty
check-prerequisites cmd
checked iptables update
basic kernel features check
enable ipv6 rules
add forwarding rules
squashing localnet-v2 commits
additional changes
propagate custom-dns flag to all run containers
remove is_mock from EcashManager
another localnet squash
unused import
chore: remove redundant testnet manager
missing impl
additional linux fixes
command to rebuild container image
wait for at least 2 blocks
additional node startup fixes
added --custom-dns flag to nym node setup
add gateway probe + wait for DKG magic file
fixed localnet down on linux
container ls
re-enable state resync
additional feature locking
macos adjustments
working nyxd startup on linux
wip linux box
wip
separating network inspect betweewn macos and linux
initial linux feature locking
moved all container commands into a single location
finally working initial node performance
squashing orchestrator commits
cleanup
fixed condition for naive rearrangement
added cache of cosmwasm contracts for speed up on subsequent runs
'down' command
refreshing described cache after nodes are bonded
nym nodes setup + wip on nym api refresh
nodes setup WIP
first pass cleanup
placeholder for nym-node setup
bypassing the dkg
further progress on nym-api setup
wip: api setup
up/down/purge placeholders
persisting contract setup data
fix contract upload by forcing amd64 container platform
wip: contracts setup4
wip: contracts setup3
wip: contracts setup2
wip: contracts setup
include network setup
init and spawn nyxd
build nyxd image in dedicated orchestrator
build nyxd image
squashed cherry-picked lp changes
Bits and bobs to make everything work
Title
MacOS setup instructions
Docker/Container localnet
* clippy
* fixes on non-unix targets
---------
Co-authored-by: durch <durch@users.noreply.github.com>
* feat: introduce /v3/unstable/nym-nodes/semi-skimmed to aggregate LP information
nym-nodes will require this information to establish shared PSQ
* reorganised imports
* announce KEM key hashes and use generated value within LpStateMachine
* added digest of remote KEM key into LpSession
* changed constructor to LpSession to take explicit key materials for local and remote
this makes it easier to change keys required by each party without having to change all the interfaces everywhere again
* extended the changes to LpStateMachine constructor
* modify the interface to LpRegistrationHandler and LpListener
* gateway probe fixes
* temp nym-lp-client fixes
* review nits
* remove network test
* introduced v2/nym-nodes/described endpoint for returning nodes description alongside LP data
* missed V1 -> V2 description replacements
* removed deprecated call within mix-fetch
* use old v1 call in network stats
* - standardise versions for all nym-sdk workspace dependencies
- prepend sqlx-pool-guard with 'nym-'
* Test remove nym-api from deps
* Add oneliner to client_pool doc comments
* Add note to commented out docs.rs link in sdk
* remove nym-api from script
* add publishing file
* bring non-binary / contract / tools into workspace version
* added more info to publishing.md
* make deps workspace version
* remove uploaded sphinx-types crate from script
* remove erroueously included ignore-defaults
* add zeroise to feature
* chore: Release
* add topology to batch
* more cargo versioning
* more cargo versioning - wasm utils
* more cargo versioning - wasm utils
* Add publish=false to manifest for cargo workspaces / crates.io
publishing exclusion
* remove script now switched to manifest based exclusion
* rename import based on rename of contracts-common dep
* Making workspace versions for publication + removing unnecessary crates
from publication
* Remove OOD info from publishing sdk guide
* rename contract imports + remove package
* temp commit: continuing with removal of path from cargo manifest and
replacing with workspace version import for publication
* continuing with cargo.toml updates
* dryrun only erroring on known version problem crates
* remove old published-crates file
* Minor comment change
* remove default features warning
* Additional info on workspace dep comment re publish list
* Add missing description to cargo.toml
* Fix missing feature flags
* Add missing descriptions
* Fix remaining path import
* Add workspace repo / homepage / documentation links to cargo.toml files
* remove workspace version from excluded crate
* Remove todo descriptions
* Minor comment change
* add homepage etc
* move from bls git import to nym_bls_fork crate
* Modify rest of imports from path to workspace import, excluding binaries
* add directory/homepage info
* fix cargo fmt
* add notes to gitignore
* better solution to contracts/ experiment
* wasm -> nym_wasm crate renaming
* fix fatfinger
* add metadata to ecash cargo.toml
* stub publishing guide
* fix misrevolved netlink- version
* Fixes and block publication of rebase re: LP
* first pass @ workflows
* one commit to rule them all
* remove too aggressive copy pasting
* update details when outdated
* typo and serde alias
* no hostname option and fixes
* fix wasm client?
* non fallback fixed
* improve gateway details update
* better ws addresses
* PR review fixes
* improve type safety on update_gateway_published_data
* fix client gateway storage migration
dont build netstack in CI
additional rust 2024 fixes
fixes
removed temp.rs
first round of cleanup
removed duplicated NS types
moved gateway probe to the monorepo
* wip: changes to surb logic + stronger db typing
* surb invalidation logic
* chore: remove unused deps
* resolving todos
* a lot of additional bugfixes
* 1.88 clippy
* wasm fixes
* wasm clippy
* wallet clippy
* wait for epoch end when setting up new network
* split ReplyController into Sender and Receiver for easier reasoning
* additional reply surbs improvements
includes, but is not limited to: unconditionally reseting sender tag on restart, limiting number of surb re-requests, resetting stale surbs on load
* fixed calculation of number of removed surbs
* add additional calculated field to key rotation info
* DBG: 'request_reply_surbs_for_queue_clearing' temp logs
* fixes for silly mistakes
* conditionally reduce log severity
* wip
* wip: wrap node's sphinx key with a manager
* wip: choosing correct key for packet processing
* further propagation of key rotation information
* attaching key rotation information to reply surbs
* added basic key rotation information to mixnet contract
* wip: introducing cached queries for key rotation info from nym api
* unified nym-api contract cache refreshing
* finish packet decoding
* multi api client + retrieving rotation id
* rotating sphinx key files
* logic for migrating config file
* wip: putting new sphinx keys to self described endpoints
* processing loop of KeyRotationController
* fixed sphinx key loading
* rotating bloomfilters
* wired up KeyRotationController
* flushing bloomfilters to disk and loading
* most of nym-node changes
* post rebase fixes
* fixes due to backwards compatible hostkeys
* split http state.rs file
* dont use deprecated fields
* fixed backwards compatible deserialisation of host information
* split up node describe cache
* added a dedicated CacheRefresher listener to perform full refresh outside the set interval
* controlling announced sphinx keys within nym-api
* retrieving rotation id when pulling topology
* split nym-nodes http handlers
* v2 nym-api endpoints to retrieve nodes with additional metadata information
* bug fixes...
* additional bugfixes and guards against stuck epoch
* testnet manager: set first nym-api as the rewarder
* fixed host information deserialisation
* fixed panic during first key rotation
* post rebase fixes
* clippy
* more guards against stuck epochs
* added helper method to reset node's sphinx key
* instantiate mixnet contract with custom key rotation validity
* additional bugfixes and debugging nym-api deadlock
* passing shutdown to nym apis client
* remove dead test
* post rebasing fixes
* missing MixnetQueryClient variants
* remove usage of deprecated methods in sdk example
* fix: incorrect method signature
* post rebasing fixes
* attempt to retrieve key rotation id before doing any config migration work
* ignore tests relying on networking behaviour
* allow networking failures in certain tests
* new NymNodeTopologyProvider to also keep track of ips of all nodes
* added nym-api endpoint for nodes existence by ip
* change behaviour of updating allowed nodes alongside the topology
* clippy
* license fix
* fix default filtering limit
* added explicit SP suffix to started tasks
* added 'GatewayTopologyProvider' that always injects itself into the network
* use the new topology provider to bypass described bootstrapping problem
WIP; rebasing
Another branch squash
Squashing the v3 branch
changing min pledge amounts
logic for adding new nymnode into the contract
converting mixnode/gateway bonding into nym-node bonding
logic for migrating gateways into nymnodes
ibid for mixnodes
further nym-node work + fixed most existing unit tests
forbid nymnode migration with pending cost params changes
preassign nodeid for gateways
changing role assignment and epoch progression
changing role assignment and epoch progression
optional custom http port
logic for unbonding a nym-node
updating Delegation struct
logic for increasing pledge of either mixnode or nymnode
logic for decreasing pledge of either mixnode or a nym node
logic for changing cost params of either mixnode or a nym node
wip
initialise nymnodes storage
fixing transaction tests
fixed naive family tests
reward-compatibility related works
resolving delegation events
introduced rewarded set metadata
another iteration of restoring old tests
updated rewarding part of nym-api
parking the branch
unparking the branch
wip
purged families
added 'ExitGateway' role
passing explicit work factor for rewarding function
remove legacy layers storage
wip: node description queries
added announced ports to self-described api
step1 in gruelling journey of adding node_id to gateways
ensure epoch work never goes above 1.0
changed active set to contain role distribution
[theoretically] sending rewarding messages for the new rewarded set
[theoretically] assigning new rewarded set
reimplementing more nym-api features
remove legacy types
re-implement legacy network monitor
restoring further routes + minor refactor of NodeStatusCache
skimmed routes now return legacy nodes alongside nym-nodes
seemingly restored all functionalities in nym-api
removing more legacy things from the contract
initial contract cleanup
added nym-api endpoints to return generic annotations regardless of type
updated simulator to use new rewarding parameters
more contract cleanup
made existing mixnet contract tests compile
extra validation of nym-node bonding parameters
fixed additional compilation issues
fixed nym-api v3 database migration failure
added additional nym-node contract queries
updated the schema
made additional delegation/rewards queries compatible with both legacy mixnodes and nym-nodes
fixing existing unit tests in mixnet contract
wip
resolved first batch of 500 compiler errors
re-deprecating routes
making wallet's rust backend compile
fixed non-determinism in contract + nym-api build
fixes to the build
populating cotracts-cache with nym-nodes data
more missing nymnodes queries
temp mixnet contract methods + restored result submission in nym-api
allow deprecated routes
submitting correct results for mixnode results
removed deprecated re-export of AxumAppState and removed smurf naming
moved axum modules into support::http
cleaning up nym-api warnings
determine entry gateways before exits
exposed transaction to update nym-node config
missing memo for updating node config
new routes
added routes to swagger and fixed relative paths
fixed some macro derivations
added nym-node commands to nym-cli
* Regenerate deny.toml
* Backport old settings to deny.toml
* Explicitly allow GPL-3 only on our own specific crates
* Update deny.toml for latest changes
* Fix cargo-deny warnings for duplicate crates
* Update cargo-deny-action to v2
* Squashing all the changes
initial router
started expanding the API
initial empty openapi/swagger
populated build-info endpoint
wip: populating rest of swagger
missing swagger data + using closure capture for immutable state
running the api as a proper task in gateway 'run'
fixing some version/feature clashes
refactored routes structures
initial host information endpoint
expanded on gateway-related endpoints
signing host information
moved all models to separate crate
unified http api client
routes unification + node api client
new generic cache and refresher
nym-api caching node self described information
removed old cache type
temporarily wired up NymContractCache to NodeDescriptionProvider
caching self reported host info
clients using self-described gateway information
fixed request timeouts for wasm
fixed wasm builds
post rebase fixes
cargo fmt
brought in wg routes into nym-node router
added ErrorResponse for wireguard routes
basic swagger support for wg endpoints
turns out swagger can be happy with strongly typed requests
output type support for wg routes
using concrete error type for nym node request error
fixed the registration test
landing page configurability
increased configurability
fixed build and lints of other crates
added default user-agent to http-api-client
reduced severity of gateway details lookup failure
changed default http port from 80 to 8080
nym-api using new default port for queries
added health endpoint
nym-api trying multiple ports for the client
using camelcase for node status
corrected health endpoint description
restored and revamped 'force_tls' flag to filter all gateways that support the wss protocol
fixed 'pub_key' path param in open api schema
derived Debug on 'NymNodeDescription'
ensuring valid public ips
added init and run flags to set hostname and public ips
fixed listening address being pushed to public ip
fixed the positional local flag
logging remote ip address of the request
updated helper function to query for described gateways
enabled tls in gateway client
removed hack-opts from mix fetch
additional changes after rebasing against origin/develop
* clippy
* wasm-related target locking
* more clippy, but this time in tests
* always using port 443 for tls gateway connection
* bumping up rc version
* hack: additional hardcoded gateways
it disables topology refresh
* Add NPM packages to tool for bumping versions
* Bump RC versions of SDK packages
* Use manual gateway details
* Add progress to `publish.sh` script
* Add docs for WSS workaround
---------
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
* Squashing commits
added forceTls argument to 'MixFetchOptsSimple'
made forceTls flag optional in mix fetch opts
whacking those moles
* GitHub Actions: install node
---------
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
* changed NymConfigTemplate trait to call 'template' by reference
* network requester lib
* introduced generic parameter to 'MixTrafficController' to allow non-remote gateways
* allowing for custom gateway sender
* types cleanup
* minor gateway cmds refactor + initial NR work
* wip
* running a NR inside gateway
note: this NR isnt tied to the gateway yet
* rebase fixes
* propagating same shutdown handle
* wip
* starting NR with appropriate local transceiver
* fixed premature shutdown
* wiring up PacketRouter
* both ends wired together
* actually working
so much cleanup to do now
* started removing dead code
* wip
* temp: hardcode gateway
* further cleanup
* fixed build of other binaries
* setup-network-requester subcmd
* overriding NR config in gateway init/run
* wip making it wasm-compatible [again]
* refactored 'GatewaySetup'
* clippy and friends
* removed debug code
* rust 1.72 lints
* ensuring local gateway is available + some comments
* correctly putting network requester data in the same underlying details struct
* improved gateway errors
* changed 'network_requester_config' deserialization
* missing clap annotation for 'enabled' flag in 'setup-network-requester' command
* saving config file after 'setup-network-requester'
* removed dead code
* review comments
* make embedded NR wait for gateway to come online (for at most 70min)
* fixed shutdown on successful gateway wait
* updated NR config override
* wip
* semi-hacky way of persisting wasm-client gateway config
a better way shall be introduced after config refactoring
* cargo fmt
* wasm client clippy
* removed artifacts from other branches
* added cargo config file to explicitly specify build target
* wip
* Config option to disable topology refreshing
* extracted common parsing code
* helper trait for working on wasm topology
* wasm topology parsing
* restored (slightly modified) old js-example
* wip
* Moved message preparation into a trait
* wip
* long-winded way of sending test packet
* standalone NymNodeTester
* finishing the test upon receiving all packets even if timeout wasnt reached
* initial round of cleanup
* sending multiple test packets in normal NymClient
* javascript-side cleanup
* starting mixnode test on btn click
* Improved NymNodeTester constructors
* improved error handling and constructors
* tester utils error handling
* further cleanup + using BTreeMap for NymTopology mixnodes
* handling missed errors
* splitting up 'test_node'
* split up and cleaned up generation of test result
* clippy + fixed example
* post rebase fixes
* another broken test
* prevent running multiple parallel tests
* cargo fmt
* Added nym- prefix to node tester utils
Jędrzej Stuczyński