dont build netstack in CI
additional rust 2024 fixes
fixes
removed temp.rs
first round of cleanup
removed duplicated NS types
moved gateway probe to the monorepo
* wip: changes to surb logic + stronger db typing
* surb invalidation logic
* chore: remove unused deps
* resolving todos
* a lot of additional bugfixes
* 1.88 clippy
* wasm fixes
* wasm clippy
* wallet clippy
* wait for epoch end when setting up new network
* split ReplyController into Sender and Receiver for easier reasoning
* additional reply surbs improvements
includes, but is not limited to: unconditionally reseting sender tag on restart, limiting number of surb re-requests, resetting stale surbs on load
* fixed calculation of number of removed surbs
* add additional calculated field to key rotation info
* DBG: 'request_reply_surbs_for_queue_clearing' temp logs
* fixes for silly mistakes
* conditionally reduce log severity
* wip
* wip: wrap node's sphinx key with a manager
* wip: choosing correct key for packet processing
* further propagation of key rotation information
* attaching key rotation information to reply surbs
* added basic key rotation information to mixnet contract
* wip: introducing cached queries for key rotation info from nym api
* unified nym-api contract cache refreshing
* finish packet decoding
* multi api client + retrieving rotation id
* rotating sphinx key files
* logic for migrating config file
* wip: putting new sphinx keys to self described endpoints
* processing loop of KeyRotationController
* fixed sphinx key loading
* rotating bloomfilters
* wired up KeyRotationController
* flushing bloomfilters to disk and loading
* most of nym-node changes
* post rebase fixes
* fixes due to backwards compatible hostkeys
* split http state.rs file
* dont use deprecated fields
* fixed backwards compatible deserialisation of host information
* split up node describe cache
* added a dedicated CacheRefresher listener to perform full refresh outside the set interval
* controlling announced sphinx keys within nym-api
* retrieving rotation id when pulling topology
* split nym-nodes http handlers
* v2 nym-api endpoints to retrieve nodes with additional metadata information
* bug fixes...
* additional bugfixes and guards against stuck epoch
* testnet manager: set first nym-api as the rewarder
* fixed host information deserialisation
* fixed panic during first key rotation
* post rebase fixes
* clippy
* more guards against stuck epochs
* added helper method to reset node's sphinx key
* instantiate mixnet contract with custom key rotation validity
* additional bugfixes and debugging nym-api deadlock
* passing shutdown to nym apis client
* remove dead test
* post rebasing fixes
* missing MixnetQueryClient variants
* remove usage of deprecated methods in sdk example
* fix: incorrect method signature
* post rebasing fixes
* attempt to retrieve key rotation id before doing any config migration work
* ignore tests relying on networking behaviour
* allow networking failures in certain tests
* removed support for legacy packet types from NymCodec
I think nodes had plenty of time to upgrade given versioned variant was introduced in 2022
* temp: use local sphinx packet for development
* introduce new messages that use more efficient reply surbs encoding
* checks for incorrect encoding
* generate correct message depending on config value
* fixed current packet version
* made packet type selection configurable
* updated sphinx packet crate to the published version
* fixed wasm build
* fixes in outfox due to sphinx api changes
* additional tests
* clippy
* fixed log/tracing import
* remove old packettype + fix: apply routing filter BEFORE delaying
* updated sphinx crate for allow usage of reply tags
* full pipeline for placeholder checking of packet replay
* replay protection with batched insertion
* running background task for clearing/flushing the BF
* allow disabling the replay detection + cleanup
* allow unwrap in bench code
* removed mnemonic from gateway config struct
scaffolding for common mixnet listener
running verloc unconditionally in a nym-node
remove filtering by mixnode
extracted verloc to separate crate
integrated nym-node-http-server more tightly with the binary
most logic for handling forward packets
running all mixnode-related tasks natively inside nymnode
removed gateway storage trait in favour of the only concrete implementation
most logic for handling final hop packets
using nym-node owned socket listener for gateways
utility for sending plain message through mixnet + gateway fix
using common packet forwarding in both modes
nifying nym-node metrics
reproduce behaviour of the console logger
cleaned up cli args
redesigned gateway tasks startup procedure
removing dead code
scaffolding for old config v6
config migration
implemented MixnetMetricsCleaner
* clippy
* require entry/exit for wireguard
* removed dead code in migration code
* updated config template
* use custom user agent for verloc queries
* fixed premature shutdown of gateway tasks
* hidden nym-api flag to allow illegal node ips
* experiment: final hop handing with wireguard
* added additional startup logs
* typo
* fixed legacy stats endpoint data
* additional logs
* apply review comments
* fixed local testnet manager
* oak-2
* oak-8
* oak-13
* oak-15
* oak-18
* Minor clippy nit
* 2023-01-13-OAK-6
* 2023-01-13-OAK-3
* 2023-01-13-OAK-13
Implemented via direct dependency on cw2 and calling the appropriate code on migration
* Removed few instances of password being unecessarily copied
* 2023-01-13-OAK-10
* 2023-01-13-OAK-12
* 2021-09-13-JP-S-NYM-02
* 2021-09-13-JP-S-NYM-03
* Removed further instances of needlessly copying the mnemonic
* 2021-09-13-JP-O-PROT-03
* 2021-09-13-JP-S-NYM-01*
*: we still have one vulnerability on 'time' pulled from chrono via sqlx. However, apparently its usage is fine... Having said that, I'd still recommend removing all dependencies on chrono, but this will require some database migrations...
* 2023-01-13-OAK-11 (#3009)
* wip
* Introducing the concept of starting epoch transition in `nym-api`
* split epoch operations into multiple files
* epoch operation failure recovery
* sending rewarding transactions in correct order
* tests and fixes due to epoch state progression
* lint
* missed rebasing import changes
* Setting cw2 contract version during first migration run
* calling 'reconcile_epoch_events' at least once
* Made message to BeginEpochTransition more consistent with other variants
* Merge layer assignment updates
---------
Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
* cleaned up MixProcessingError
* Added Error impl to (hopefully) all error enums in the codebase
* Replaced all occurences of error("{0}") with error(transparent)
* Changelog entry
* Rename to ExtendedPacketSize32
* Add two more extended packet sizes
* Update config handling for new packet sizes
* Update wasm-client
* Changelog: update
* wasm-client: fix ref
* Switch use enum instead of string for config
* Update hmac and blake3
* Remain paranoid for `0.*` crates
* Most paranoid versions :)
* Updated aes and using published version of blake3
Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
* Basic storage stub
* New models for new node status api
* Route handling
* Mounting new routes
* Missing selective commit
* Moved network monitor related files to separate package
* Starting to see some sqlx action
* Schema updates
* Log statement upon finished migration
* Removed old diesel related imports
* Converted mixnode cache initialisation into a fairing
* Moved cache related functionalities to separate package
Also defined staging there
* Created run method for validator cache + removed unwrap
* Removed old node-status-api types and left bunch of todo placeholders in their place
* Fixed managing validatorcache
* Status reports are starting to get constructed
* Submitting some dummy results to the database
* Removing duplicate code for generating reports
* Removed statuses older than 48h
* Initial attempt at trying to obtain reports for all active nodes
* Removed duplicates from the full report
* Grabbing uptime history
* Updating historical uptimes of active nodes
* Updated sqlx-data.json
* Removed all placeholder foomp owner values
* Changed Layer serde behaviour for easier usage
* Extended validator api config
* Initial (seems working !) integration with network monitor
* Added database path configuration to config
* Using ValidatorCache in NetworkMonitor
* Flag indicating whether validator cache has been initialised
* Introduced a locla-only route for reward script to perform daily chores
* Flag to save config to a file
* Moved spawning of receiving future to run method rather than new
* Removed arguments that dont make sense to be configured via CLI
* Removed dead code from config file
* More dead code removal
* Added validator API to CI
* Corrected manifest-path arguments
* Constructing network monitor by passing config
* Combined validator API CI with the main CI file
* Using query_as for NodeStatus
* Checking if historical uptimes were already calculated on particular day
* Making id field NOT NULL
* More query_as! action
* Updated sqlx-data.json
* Removed unused chrono feature
* Renamed the migration file
* Changed default validator endpoint to point to local validator
* Removing unnecessary clone
* More appropriate naming
* Removed dead code
* Lock file updates
* Updated network monitor address in contract code
* Don't stage node status api if network monitor is disabled
* cargo fmt
* Updated all license notices to SPDX
* Running CI also on windows and macOS
* Rust 2021 formatting
* clippy::upper_case_acronyms where appropriate
* Allowing unknown clippy lints
* Further clippy updates
* Building wasm client during CI
* wasm actions update
* added working directory to cargo jobs
* Temporarily disabled wasm test and clippy
* Changed AvailableReader from Future to Stream
* comment formatting
* WIP
* Maximum number of bytes read per poll
* More detailed socks5 printing
* Split up socks5 service provider runner
* Fixed closing proxy too soon on one side
* Additional log information
* Printing connection id in log target
* Adjusted some constants
* Target field in log
* Removed dead code
* Decreased logging level
* Explicitly seperated 'Send' and 'Connect' request + data buffering
* Temporarily commented out test that fails due to not understanding limitations of tokio test io builder
* Fixed socks5 tests
* Ability to set client in vpn mode
* Connection handler for mixnode
* Initial vpn mode for mixes
* Updated SphinxCodec to contain more metadata
* Renaming
* Removed handle from mixnet client and introduced forwarder
* Mixnode using new forwarder
* Mixnode common module containing shared packet processing
* ibid. incorporated inside mixnode
* New processing for gateway
* Type cleanup
* Wasm fix
* Fixed client config
* Fixed mixnode runtime issues
* Formatting
* Client re-using secret on 'normal' packets
* Using the same key for acks
* WIP
* vpn key manager cleanup
* wasm fix
* VPN_KEY_REUSE_LIMIT moved to config
* Moved AckDelayQueue to separate common crate
* Key cache invalidator
* Updated dashmap used in gateway
* Old typo
* Additional comment
* Cargo fmt
* Fixed tests
* Sphinx update
* cache ttl as config option
* Cargo fmt
* Merge branch 'develop' into feature/socks5-no-core
common/client-libs/directory-client/models/src/presence/topology.rs
* More diagnostics
* WIP commit, not properly multithreaded
* WIP, now sort of multithreaded but errors all over the place
* Replaced expect with a return
* Adding abort on panic
* Removing a few noisy prints
* Another ack-related expect replaced
* Moved profile.dev annotation to workspace cargo.toml
* Semi-stabilised socks
* Assigning connection shared key post registration (#309)
* Sending return address with connect request
* WIP commit
* Reading all available data on reader + removed duplicate code
Co-authored-by: jstuczyn <jedrzej.stuczynski@gmail.com>
* Split text and binary client apis
* Very initial attempt at new serialization
* Defined ser+de for Recipient and ReplySURB
* Response errors
* builds with changes
* Working WS API + moved to separate crate
* updated python examples
* Fixed parsing bug
* Updated go examples
* Updated rust examples
* formatting
* Removed unused imports
* Removed accidentally left panic call
* Changed identity keypair to use ed25519
* Encryption key is now x25519 based + compatibiltiy with sphinx
* Pathing and import fixes
* Moved all asymmetric keys to sub-module in crypto
* Extracted aes to separate module
* kdf module in crypto
* Ability to perform diffie hellman on encryption keys
* ecdsa on identity keys
* Extremely rough and incomplete registration handshake
* Authentication primitives
* Creating new random authenticationIV
* Wrapper type for the derived shared key
* Removed AuthToken in favour of using SharedKey for authentication
* Gateway identity keys
* Registration handshake without error mapping
* Gateway address in client config
* Added extra key for gateway presence
* Updated pemstore to work on borrows instead
* Gateway client trying to perform the handshake
* Gateway changes to allow for handshake and shared key
* Debug trait on sharedkey
* native client using updated gateway client
* Slightly updated gateway API
* Minor cleanup
* Fixed pemstore to correctly save multiple keypairs
* Gateway actually deriving shared key during handshake
* Gateway sending correct mid-handshake message
* Missing quotation mark in client config template
* Fixed template for correct shared key serialization
* Fixed gateway authentication
* Fixed tests
* Using correct gateway key when converting to sphinx node
* "get_all_clients" takes them from gateways as opposed to providers now
* cargo fmt
* Renamed pemstore methods
* Unused import
* Encryption of forward requests between client and gateway
* Updated sphinx dependency to use public revision
* Sending 'error' on handshake processing error
* Removed some dead code
* Dead code I forgot to remove before
* Extracted AckAes128Key into a struct
* Slight pemstore revamp allowing for symmetric key store
* ibid.
* PemStorableKey for SharedKey
* Introduced single location responsible for key management for client
* WIP
* Sphinx version update
* Stop using NodeAddressBytes for two distinct and confusing purposes
* Abstracting away SocketAddr from sphinx forwarding
* Passing the bool for reply surbs
* Attack plan for replies + encryption
* Comment + removed variable binding
* ReplySURB usage
* Topology import in nymsphinx
* Sphinx version update
* Changed 'Recipient' to contain client's encryption key
* Message preparation taking shape!
* reply surb also containing the encryption key
* Very initial message receiver
* Sphinx version update
* A possibly working way of receiving surbs
* Fixed incorrect field name in client config template
* camel casing all request arguments
* Renamed and moved `MessageMode` to more appropriate file
* Restored reconstruction tests
* Removed dead code from chunking
* Made rust examples compilable
* reply SURB key storage
* Replies as an InputMessage
* Forgotten commented code
* No retransmission processing for cover or replies
* Received reply processing
* Renamed client pathfinder to something more appropriate
* Made HasherOutputSize public
* Added key store path to config
* Reply surb attaching key digest when used
* Changes due to previous renaming
* Removed comment
* Fixed insert_encryption_key
* Assigning initial value of key store path
* Computing key digest with correct algorithm
* Initial and presumably temporary request serialization
* hacky way of introducing 'FragmentIdentifier' for replies
* Moved responsibility of reply encryption, padding, etc, to message preparer
* Optional recipient in try_get_valid_topology_ref
* Handling new reply surbs with acks and padding
* Updated go and python examples to include replies in text and binary cases
* Updated rust examples + binaryserverresponse
* Helpers in rust examples
* And updated JS example
* Moved shared key generation function to crypto crate
* Cover traffic encryption!
* hmac computation in crypto
* Updated aes imports due to new dependencies
* hkdf made more generic
* crypto cleanup + algorithms in params
* Clippy cleanup pass
* Generating encryption+mac shared keys between client and gateway
* MACs attached to forward requests to gateway
* Gateway messages encrypted and mac'd
* Lowered logging level
* compiler warning cleanup
* Some minor cleanup
* Generic stream cipher
* Generic shared key derivation + algorithm definitions
* Project-wide AES clean-up
* Comment fix
* Removed commented imports
* Updated comments
* Fixed topology test fixture
* VersionFilterable for HashMap
* Removed NymTopology trait in favour of concrete type
* Removed providers from NymTopology
* Made gateway conversion use reference, similarly to mixes
* Using more concrete types in topology rather than b58 strings
* Allowing gateways to have DNS-resolvable mix listener address
* Error propagation for gateway key conversion
* Reducing ACK packet size to exactly what we need
* Made fragmentidentifier into an array
* Padding all sent messages so they'd be split into constant length packets
* Ability to send sphinx packets of different sizes + more efficient decoding
* Closing connection on connection corruption
* Missing semicolons
* Missing license notices
* Default for packetsize
* Split nymsphinx
* Replaced Mutex with RwLock for TopologyAccessor + impl Deref
* Sphinx update + import cleanup
* Moved packet_sizes file
* Updated NymTopology API
* sphinx version bump
* Missing license notice
* nymsphinx-params crate
* Changes due to ibid.
* Chunking rework to allow variable size fragments
* Initial ack crate
* Version bump to new dev build
* Cargo lock changes
* random_route_to_gateway by node address
* exposing getting read permit
* Very initial draft on ack control
* Correctly dereferencing out of topology read permit
* All pending changes + compilation todo!s
* Restricted scope of deref on TopologyAccessorInner
* Type path alias for generate_key
* Derived traits for MessageChunker
* Ack control starting to take shape!
* Awaiting callbacks
* Most of work done on acks. Now to wire it all together
* Import cleanup
* rng generalization
* Connected real traffic together; only acks from gateway left
* Removed redundant things from nymsphinx::utils
* nymsphinx-cover crate
* Ack-related fields in client config
* Decreased packet store log level
* Restored forward sphinx request
* Slight adjustements to surb acks
* Changed TopologyReadPermit from type alias into a struct
* Changes due to ibid.
* Sphinx version upgrade
* Gateway being able to understand and handle acks
* Special Cover FragmentIdentifier + removal of dead code
* Initial packet router for gateway client
* Kill client if it fails to send to gateway too many times
* Cover messages with acks
* Moved out gateway client errors
* Ignoring cover traffic acks
* Changes in ack control
* Another sphinx version upgrade
* websocket handler delegating message chunking
* Using config defined ack wait additions
* Other minor changes I should have been more dilligent with splitting
* Import path fix
* sphinx_receiver => mixnet_receiver
* Missing renamed variable instance
* Updated aes-ctr to 0.4.0
* Removed concept of 'unfragmented' single fragment
* Replay fragments detection
* Long method split
* typo
* Cleaner client init
* Fixed race condition
* Fixed similar issue for retransmission
* Cargo fmt
* Minor clenaup
Jędrzej Stuczyński