Commit Graph

51 Commits

Author SHA1 Message Date
Jędrzej Stuczyński 0ee387d983 Feature/cancellation migration (#6014)
* squashing work on using cancellation in nym crates

making nym-task wasm compilable

removed sending of status messages

replaced TaskManager with ShutdownManager in the validator rewarder

additional helpers for ShutdownManager

simplified ShutdownToken by removing the name field

TaskClient => ShutdownToken within all client tasks

wip: remove TaskHandle

* track all long-living client tasks

* add task tracking for most top level tasks within nym-node

* improved default builder

* split up cancellation module

* module documentation and unit tests

* nym node fixes and naming consistency

* wasm fixes

* assert_eq => assert

* wasm fixes and made 'run_until_shutdown' take reference instead of ownership

* linux-specific fixes to IpPacketRouter

* post rebasing fixes for signing monitor

* add ShutdownManager constructor to build it from an external token

* applying PR review suggestions
2025-09-10 13:56:39 +01:00
Jędrzej Stuczyński 2ae38b9e49 chore: 1.88 clippy (#5877)
* 1.88 clippy

* wasm clippy

* wallet clippy
2025-07-01 10:28:57 +01:00
Jon Häggblad ede4b23e8a Fix clippy::too-long-first-doc-paragraph (#4897) 2024-09-18 10:25:49 +02:00
Drazen Urch 1ac262ec90 New Network Monitor (#4610)
* Initial commit

* Cherry pick from develop

* Keep track of fragments

* A bunch of data formats, graphs

* Use mix_id for display

* Proper API routes

* Add openapi + swagger ui

* Update locustfile

* Add node stats endpoint

* Add Swagger and locust to readme

* All node stats endpoint

* Update dependencies to use workspace

* Bunch of pedantic fixes

* More version updates, fmt

* More lints

* Add new_from_env for NymTopology

* Nym API endpoint to submit monitoring results (#4616)

* Nym API endpoint to submit monitoring results

* Add gateway monitoring results

* Cleanup, ergonomics

* Weaponize

* Finalize results submissions

* Monitor message signing and verification

* Update README

* Axum graceful shutdown

* More grtacefulness

* Restructure result submission

* Less fragile routes

* Remove gateway unique index on node_id
2024-08-22 11:29:36 +02:00
Jon Häggblad cb4b0403b5 Remove deprecated mark_as_success and use new disarm (#4751) 2024-08-13 15:09:48 +02:00
Jon Häggblad 369330f517 Fix clippy for latest stable toolchain (#3935)
* clippy::redundant-guards

* clippy::incorrect_partial_ord_impl_on_ord_type

* clippy::redundant-guards

* clippy::unwrap-or-default

* rustfmt

* noop_method_call

* nym-wallet Cargo.lock

* Unpin rust toolchain for nym-wallet

* cargo clippy --fix --workspace

* clippy::redundant_locals

* Reorder Makefile targets for more logical ordering
2023-10-06 11:04:26 +02:00
Jędrzej Stuczyński 114d92f93f Feature/gateway inbuilt nr (#3877)
* changed NymConfigTemplate trait to call 'template' by reference

* network requester lib

* introduced generic parameter to 'MixTrafficController' to allow non-remote gateways

* allowing for custom gateway sender

* types cleanup

* minor gateway cmds refactor + initial NR work

* wip

* running a NR inside gateway

note: this NR isnt tied to the gateway yet

* rebase fixes

* propagating same shutdown handle

* wip

* starting NR with appropriate local transceiver

* fixed premature shutdown

* wiring up PacketRouter

* both ends wired together

* actually working

so much cleanup to do now

* started removing dead code

* wip

* temp: hardcode gateway

* further cleanup

* fixed build of other binaries

* setup-network-requester subcmd

* overriding NR config in gateway init/run

* wip making it wasm-compatible [again]

* refactored 'GatewaySetup'

* clippy and friends

* removed debug code

* rust 1.72 lints

* ensuring local gateway is available + some comments

* correctly putting network requester data in the same underlying details struct

* improved gateway errors

* changed 'network_requester_config' deserialization

* missing clap annotation for 'enabled' flag in 'setup-network-requester' command

* saving config file after 'setup-network-requester'

* removed dead code

* review comments

* make embedded NR wait for gateway to come online (for at most 70min)

* fixed shutdown on successful gateway wait

* updated NR config override
2023-09-20 15:47:05 +01:00
Jędrzej Stuczyński 4890c528bc feat: mixFetch - the final countdown (#3737)
* mixFetch

* clippy

* removed redundant Arc over 'WasmStorage' in the 'ClientStorage'

---------

Co-authored-by: Fouad <fmtabbara@hotmail.co.uk>
2023-09-07 12:45:03 +01:00
Jędrzej Stuczyński 687b437ea0 cargo fmt 2023-06-12 10:20:13 +01:00
Jędrzej Stuczyński a8bf690c17 tests 2023-06-12 10:20:13 +01:00
Jędrzej Stuczyński 8b046d4139 split up functions a bit more 2023-06-12 10:20:13 +01:00
Jędrzej Stuczyński f4cd372808 simplified keeping track of message sequence 2023-06-12 10:20:13 +01:00
Jędrzej Stuczyński 45f3f3ec01 cleanup 2023-06-12 10:20:13 +01:00
Jędrzej Stuczyński 5f9e54c83c buggy, but backwards compatible, explicit sequencing 2023-06-12 10:20:13 +01:00
Jędrzej Stuczyński f8c2f90502 initial work on putting data sequence explicitly inside socks5 request 2023-06-12 10:20:13 +01:00
farbanas 7c55483585 Merge branch 'master' into develop 2023-06-07 10:43:11 +02:00
Jon Häggblad 07740cbf08 Reduce SHUTDOWN_TIMEOUT to 3 sec
Can't think of a scenario where we don't want to close one at the same
time as the other, but let's be conservative and keep it a very low
number for now.
2023-06-05 10:42:38 +02:00
Jon Häggblad 5075894ff5 socks5: abort sending data if the connection is closed (#3365)
* socks5 inbound: stop reading when closing connection

* Wait for lane at select top-level

* Allow closing connection while waiting for lanes to clear

* Some tidy in inbound.rs

* Put chained future back inline in the select

* Remove commented out line

* Disable the read data branch on is_finished
2023-05-23 15:01:13 +02:00
Jon Häggblad cc83ecf7e4 socks5: send empty keepalive msg to avoid triggering MIX_TTL during long downloads (#3364)
* socks5: send empty keepalive msg to avoid triggering MIX_TTL during long downloads

* rustfmt

* reset timer after each normal send
2023-04-26 16:44:27 +02:00
Tommy Verrall f822d3db7b cargo fmt 2023-04-24 15:29:51 +01:00
Jędrzej Stuczyński fd4930b198 removed old leftover log statement 2023-04-24 15:29:51 +01:00
Jędrzej Stuczyński 5d7be89edb replaced inner implementation with tokio's 'ReaderStream' 2023-04-24 15:29:51 +01:00
Jędrzej Stuczyński 47f5b4ceac limit the maximum buffer size of AvailableReader by PacketSize of our mix packets 2023-04-24 15:29:51 +01:00
Jędrzej Stuczyński 790220039b added read deadline to AvailableReader 2023-04-24 15:29:51 +01:00
Jon Häggblad 24b9b17e64 Add a few more nym- crate prefixes (#3158)
* Add nym- to socks5-prefixes crate

* Update imports

* rustfmt

* Add nym-socks5 prefix to proxy-helpers crate

* rustfmt

* Add nym prefix to ordered-buffer crate

* rustfmt

* Add nym prefix to service-providers-common crate

* rustfmt

* Add nym prefix to dkg crate

* Add nym prefix to credentials crate

* rustfmt

* fix build fail in tests
2023-03-08 11:56:29 +01:00
Jon Häggblad 586d5f658b Move client-connections crate to nym-task (#3068)
* Move client-connections to nym-task

* rustfmt
2023-02-21 13:54:27 +01:00
Jon Häggblad 46e2c74a98 Rename crate to nym-task (#3064)
* Rename to nym-task

* Update imports to use nym_task

* Update to nym_task in explicit module prefix

* Touchups and rustfmt

* rustfmt

* crate metadata

* fix nym-connect-android
2023-02-20 21:55:29 +01:00
Jon Häggblad bbb3287029 Add client functionality into nym-network-requester (#2972)
* network-requester: replace websocket with mixnet client

* network-requester: mini tidy

* network-requester: add info line about own address

* network-requester: update to nym-crypto rename
2023-02-20 13:00:34 +01:00
Jędrzej Stuczyński 3e23bdf3c0 Feature/service provider interface (#2934)
* added additional workspace-wide dependencies

* Added conditional serialization on 'BinaryBuildInformationOwned'

* initial framework for service provider messages

* updated request/response tags to account for existing variants of Socks5Message

* handling legacy deserialization

* another serialization revamp to account for legacy version

* legacy client working with versioned network requester

* socks5 client deserializing responses into updated structures

* using new structures for sending in socks5 client

* SendRequest wrapper for Request::Send variant

* created named fields for all variants of 'ControllerCommand'

* Versioning socks5 requests + moving to proper Socks5Request struct

* Updated backwards compatible Socks5Response

* unused imports

* poc ServiceProvider trait

* wip

* implemented 'ServiceProvider' trait on the Network Requester

* Socks5RequestError

* added properly serialized ErrorResponse to ControlResponse

* fixed version serialization + feature selection

* handling of version control requests

* improved SocksProxyError by providing more concrete variants and removing generic case

* got rid of ServiceProviderClient trait and wrote simple example showing control requests

* tests for serialisation backwards compatibility

* post-merge fixes due to method renaming

* enum boxing to make clippy happier

* making sure to not drop buffer channel when starting `DirectClient`

* Using nym-sdk in the example

* Replaced printing version to stdout with proper log call
2023-02-08 08:59:48 +01:00
Jędrzej Stuczyński 50c7d717c0 chore: fixed clippy warnings from the nightly toolchain (#2838) 2023-01-13 13:16:47 +00:00
Jon Häggblad 775ce0f95d rust-sdk: fix some issues identified during draft review 2023-01-05 16:18:39 +01:00
Jon Häggblad eb07ec8580 client: sort out shutdown procedure and harmonize with socks5-client (#2695)
* common/task: rename ShutdownNotifier to TaskManager

* nym-client: return boxed error

* nym-client: enable graceful shutdown

* nym-client: task wait on shutdown to instead exit on closed channel

* Fix build

* Fix unused

* changelog: update
2022-12-14 17:13:00 +01:00
Jędrzej Stuczyński 97b01db23e Chore/more error macros (#2686)
* cleaned up MixProcessingError

* Added Error impl to (hopefully) all error enums in the codebase

* Replaced all occurences of error("{0}") with error(transparent)

* Changelog entry
2022-12-13 17:42:11 +00:00
Jon Häggblad c720481a45 socks5: rework waiting in inbound.rs (#1880)
* socks5: rework waiting in inbound.rs

* changelog: add note

* out queue control: reduce old set size to 4
2022-12-13 11:39:02 +01:00
Jędrzej Stuczyński 8de9f36b69 Experiment/client refactoring (#1814)
* experimenting with extracting more common client code

* drying up the wasm client

* allowing some dead code for the time being

* fixed formatting in nym-connect

* made socks5 client inside nym-connect immutable

* made clippy a bit happier

* hidden away target locking for recv timeout
2022-11-29 15:48:12 +00:00
Jon Häggblad 65a69b2cba socks5: if any task panics, signal all other tasks to shutdown (#1805)
* socks5: signal shutdown on error

* Mark as success

* Tidy

* Reduce wait to 5 sec

* Replace unwrap with expect

* Two more unwraps

* Update changelog
2022-11-28 10:49:00 +01:00
Jędrzej Stuczyński 0be6fe5079 Feature/use expect instead of panicking (#1797)
* Implementation of 'Debug' on 'RealMessage'

* expect with failed channel name instead of throwing empty panics

* Introduced Debug trait constraint in ProxyRunner

* Derive Debug for socks5_requests::Message
2022-11-25 10:54:12 +01:00
Jon Häggblad c6d624a3b3 Network-requester: throttle inbound connections (#1789)
* Return and handle ClientRequest::LaneQueueLenghts

* Pass lane queue lengths to inbound future

* Remove unused self reference

* Request lane queue lengths periodically for all open connections

* Add timeouts

* Rename to ConnectionCommandSender and Receiver

* Rename to client_connection_tx/rx

* Fix wasm build

* Replace bool with enum
2022-11-25 10:54:12 +01:00
Jon Häggblad e873845178 Fix some client send unwraps encountered during use (#1787) 2022-11-23 10:09:22 +01:00
Jon Häggblad fa95d15eac socks5-client: throttle connection inbound from application until data is sent (#1783)
* socks5: throttle send

* client-connections: add additional methods

* WIP

* Update

* Input message sender bounded

* WIP

* Remove the delay that is no longer needed

* rustfmt

* clippy

* Fix wasm build

* clippy

* Try to use MixProxySender/Reader type alias

* Extract out wait function

* Wait on every msg

* changelog: add note

* rustfmt
2022-11-21 23:52:30 +01:00
Jon Häggblad d912844543 Client: multiplex connection data streams (#1720)
* WIP: QA network details

* Initial implementation to multiplex socks5-client sends

* Introduce TransmissionLane enum

* WIP

* WIP: client requests connection id

* WIP

* mulitplex somewhat done

* Remove closed lanes

* WIP: connection handling over ws

* Remove unused published active connections shared data

* Start on status timer

* Max number of connections, and prune

* Some tidy

* Remove commented out code and tweak log

* Tidy

* Tweak log output

* Rename to TransmissionBuffer

* Use number of msg sent instead of time to rank age of lanes

* Create client-connections crate

* Remove waker call that probably are not needed

* Extract out some types from real traffic stream module

* Revert to develop qa.env

* Tweak comments, tidy for getting ready to merge

* Update changelog

* wasm client compile fixes

* rustfmt
2022-11-11 11:04:49 +01:00
Jon Häggblad cf65bc1295 socks5: wait to close buffer (#1702)
* socks5: wait to close buffer

This is the fix proposed by @simonwicky in
https://github.com/nymtech/nym/issues/1701

* socks5: fix typo in patch

* socks5: fix tests

* socks5: add type for returned data and index

* socks5: make closed_at_index an Option

* changelog: add note
2022-10-27 12:42:01 +02:00
Jon Häggblad baed6c89fc socks5: assert that tasks are not exiting when not shutdown 2022-09-09 12:10:32 +02:00
Jon Häggblad 10be112279 socks5-client: graceful shutdown 2022-09-09 12:09:59 +02:00
Jon Häggblad 9d03bec14b clippy: fix lints from beta toolchain 2022-03-11 10:17:39 +01:00
Jędrzej Stuczyński e5afd54ce0 Feature/node status api (#680)
* Basic storage stub

* New models for new node status api

* Route handling

* Mounting new routes

* Missing selective commit

* Moved network monitor related files to separate package

* Starting to see some sqlx action

* Schema updates

* Log statement upon finished migration

* Removed old diesel related imports

* Converted mixnode cache initialisation into a fairing

* Moved cache related functionalities to separate package

Also defined staging there

* Created run method for validator cache + removed unwrap

* Removed old node-status-api types and left bunch of todo placeholders in their place

* Fixed managing validatorcache

* Status reports are starting to get constructed

* Submitting some dummy results to the database

* Removing duplicate code for generating reports

* Removed statuses older than 48h

* Initial attempt at trying to obtain reports for all active nodes

* Removed duplicates from the full report

* Grabbing uptime history

* Updating historical uptimes of active nodes

* Updated sqlx-data.json

* Removed all placeholder foomp owner values

* Changed Layer serde behaviour for easier usage

* Extended validator api config

* Initial (seems working !) integration with network monitor

* Added database path configuration to config

* Using ValidatorCache in NetworkMonitor

* Flag indicating whether validator cache has been initialised

* Introduced a locla-only route for reward script to perform daily chores

* Flag to save config to a file

* Moved spawning of receiving future to run method rather than new

* Removed arguments that dont make sense to be configured via CLI

* Removed dead code from config file

* More dead code removal

* Added validator API to CI

* Corrected manifest-path arguments

* Constructing network monitor by passing config

* Combined validator API CI with the main CI file

* Using query_as for NodeStatus

* Checking if historical uptimes were already calculated on particular day

* Making id field NOT NULL

* More query_as! action

* Updated sqlx-data.json

* Removed unused chrono feature

* Renamed the migration file

* Changed default validator endpoint to point to local validator

* Removing unnecessary clone

* More appropriate naming

* Removed dead code

* Lock file updates

* Updated network monitor address in contract code

* Don't stage node status api if network monitor is disabled

* cargo fmt

* Updated all license notices to SPDX
2021-07-19 14:02:47 +01:00
Jędrzej Stuczyński 596bc76cc6 Chore/dependency updates (#549)
* Updated all non-breaking dependencies

* Updated common/crypto dependencies

* Updated all tokio [and associated] dependencies to most recent version

* Bumped version of rand_distr

* Fixed api changes in tests

* Made clippy happier about the acronym

* Fixed the type while trying to make clippy even happier

* nightly cargo fmt
2021-03-29 15:32:34 +01:00
Jędrzej Stuczyński 6f8ae53f0c Feature/socks5 adjustments (#510)
* Increased default connection buffer size to help with packet bursts

* Informing remote about failure to establish connection

* Properly closing outbound proxy on failure to read from socket

* Split proxy runner into separate inbound and outbound files

* Flag indicating whether socks client actually started proxying

* Clearing stale proxies if other side is closed

* Decreased log level on not a warning

* Closing proxy if no messages were received in a while

* Dont print harmless false-positive errors

* Formatting + future notes

* Fixed packet delay in higher volume traffic

* Don't attempt writing empty messages to connection
2021-02-01 14:45:50 +00:00
Jędrzej Stuczyński 2d3b4f4b91 Feature/GitHub actions and clippy cleanup (#493)
* Added github actions templates

* removed travis .yml file

* initial clippy cleanup pass

* fixed the rest of clippy warnings

* Made github badges more fancy and consistent with the ones in sphinx

* Updated local rustc version and removed compilation warningns

* ... and fresh clippy warnings

* formatting

* beta clippy specific warnings fixed

* Fixed all nightly clippy warnings

* Fixed trying to unwrap a ()

* Actually running all tests

* Correctly passing the --all flag

* Hopefullly third time's a charm in fixing argument passing
2021-01-18 11:50:29 +00:00
Jędrzej Stuczyński be49c3843a Feature/socks improvements (#423)
* Changed AvailableReader from Future to Stream

* comment formatting

* WIP

* Maximum number of bytes read per poll

* More detailed socks5 printing

* Split up socks5 service provider runner

* Fixed closing proxy too soon on one side

* Additional log information

* Printing connection id in log target

* Adjusted some constants

* Target field in log

* Removed dead code

* Decreased logging level

* Explicitly seperated 'Send' and 'Connect' request + data buffering

* Temporarily commented out test that fails due to not understanding limitations of tokio test io builder

* Fixed socks5 tests
2020-11-10 12:47:26 +00:00