Commit Graph

25 Commits

Author SHA1 Message Date
Bogdan-Ștefan Neacşu 0c66cc7393 Feature/service stats (#1267)
* Send message from service provider to stats service

* Put some actual data in stats

* Put stats sender on its own thread and send response data too

* Use SQLite for storing stats

* Add the data interval and timestamp

* Fix clippy

* Set description at boot

* Guard stats service functionality under a feature for now

* Make stats service address data into consts

* Add README to network requester

* Retrieve sql data in interval

* Expose sql data via rocket rest api

* Add entry to changelog
2022-05-16 12:06:15 +03:00
Jędrzej Stuczyński ea62d01e65 Feature/updated network monitor (#604)
* Connecting to gateway with a timeout

* Added address value to InvalidAddress error

* Initial updated network monitor (does not submit results yet)

* Created client for node status api

* Changed default address to the local one

* Removed old validator client

* Renamed validator client rest to validator client

* Print conversion warnings using Display rather than Debug formattingn

* WIP for adding owner field in monitor

* Preserving changes before branch switch

* Keeping track of node owners during monitoring + using those for node status api

* Removed temporary log statement

* Dont notify node status api if good nodes are broken

* Changed default monitor interval to 5min

* Post merge issues

* Allowing dead version field in seocksrequest

* Updated monitor run interval to 15min

* Reporting gateways and mixnodes separately with the new api

* Unused imports

* Decreased testing interval to temporarily bypass the silent timeout issue

* Formattingn
2021-05-18 17:33:31 +01:00
Jędrzej Stuczyński 596bc76cc6 Chore/dependency updates (#549)
* Updated all non-breaking dependencies

* Updated common/crypto dependencies

* Updated all tokio [and associated] dependencies to most recent version

* Bumped version of rand_distr

* Fixed api changes in tests

* Made clippy happier about the acronym

* Fixed the type while trying to make clippy even happier

* nightly cargo fmt
2021-03-29 15:32:34 +01:00
Dave Hrycyszyn 4ca40fd3bc Feature/initial mixnet contract (#515)
* Starting on cosmwasm smart contracts

* Mixnet contract now builds

* Removing license and notice files, the monorepo already has these.

* Removing generated README content

* Simplified development instructions a bit.

* Converted some network monitor files to use SPDX license headers

* Renamed packaget to mixnet-contracts

* Depending on the Nym topology crate

* Renaming contract package in usage

* Renamed "announce" to "register_node" in the defined messages

* Fixed package name for mixnet contracts in defined release annotations

* Added the mixnet contracts to the Cargo lock file

* Renamed some fields in our contract topology

* Using the stringy mixnode from the validator client.

* Removing mix nodes count from state, we can infer that

* Saving generated code in comment as it's a useful example for now

* Renamed "count" to "get_topology"

* Adding the beginnings of a validator client (in Typescript)

* Starting to integrate example code. WIP.

* Ignoring generated accounts

* Making a few less mixnodes :)

* Adding shebang to start script

* Cranking up the Nym-related gas limits, as otherwise contract upload fails.

* Simplest mixnode example is now working

* Removing the external client code, it messed up wasm compilation. Will copy/paste for now.

* Contract now wants to add a MixNode rather than an IP string

* Adding mixnodes via contract now works (!)

* Simplified mixnode registration example

* Further mixnode-adding simplification.

* Adding author name

* Fixed description

* Sent funds are now required to bond a mixnode

* Ensuring that we send correct coin denomination

* Unbonding now works (!). Quite primitivist.

* Checking that unbonding works from the client.

* Setting up a thief account to play with

* Checking to see whether thief can unbond a node (it fails, happily)

* Adding a more specific error for when an account attempts to unbond but owns no bonds

* Figured out how to test contract balances

* Set the console messages to explain things a little more nicely

* Tests for insufficient funds result

* Using more async in driver example

* Added a bit more explanation of the actions taken by the driver example

* Locking down wasm instantiate a bit more

* Docs clarifications on how to run example

* README clarification

* Corrected the commit hash in the wasmd build command, it was still set for 0.14.0

* Moved models from types into state

* Starting work on range queries

* whitespace

* Going back to slow but reliabel node uploads and disabling new contract upload

* Cranking gas fees temporarily

* Mixnode key retrieval working and tested

* Range retrievals now working well

* Removing unused clone

* Compressing tests a bit

* Testing node retrieval on large numbers of nodes.

Not sure whether MockStorage has the same space limitations as
production storage does.

* Getting rid of spelling warning

* Removing unused responses

* Minor cleanup

* Starting to map my way out of the tuples

* Slightly more meaningful variable names

* Returning a StdResult from nodes query

* Fighting through the unwraps :)

* Unfucking a bit more

* Starting to use ranged nodes in contract

* Testing node retrieval from range store

* Ditching generated tests

* Adding works, still need to test removing mixnodes

* Attempting to remove a mixnode returns an error when no nodes exist

* Un-registering when no accounts exist (edit)

* un-registering someone else's mixnode fails

* Ensuring proper ownership

* Testing for only 1 mixnode getting deleted

* Testing single-node retrieval

* Removing mixnode working

* Removed unused imports and unused variable warnings

* Made handler functions private

* Tested for error response on mixnode removal

* Ensured proper post-state on mixnode removal

* Using Vec<Coin> for currency equality comparisons

* Removed todo, this amount is only for logging purposes anyway

* Refactoring tests a bit

* Adding a few storytelling comments

* Putting helper methods into alphabetical order

* Drying up mixnode adding in tests

* Using the new add_mixnode helper

* Checking full object equality in test

* Removing the GetNodes handler

* Taking a more "storytelling" approach to the contract tests

* We need a few more methods to run our example driver

* We now need to make a new address for each node we want to have,

as each sending account can only have one node

* HumanAddr not needed

* Making call sequence a little more readable

* Added the results of today's experiments with the REST API to the validator client readme

* Corrected console.log message

* Adding a note about how to run tests

* More contract exercising fun

* Updating mocha

* Whitespace

* Adding a note about running tests

* Adding typed rest client

* Starting to mess with typescript paging client

* Removing the rest client, we'll use the cosmjs one for this

* Noting a few more contract requirements

* Starting client restructuring

* Importing cosmjs stargate client

* Starting to work on the chain cache

* Cleanup

* Removing type annotations which hilariously worked, confusing the compiler

* Might as well do each cache individually

* Renaming chaincache so that it handles only mixnodes

* Renaming chaincache

* Setting dynamic per-page value to ease testing

* Using perPage in tests

* Moving tests back into their own special home so they don't bloat our package

* Ignoring generated docs

* Adding TypeDoc documentation generator

* Removing unused NetClient import

* Added docs generation

* Noting existence of docs generation

* Starting to test paged responses

* Working paging tests

* Clarified test names a bit

* Removed console.logs

* Added a test for two full pages.

* Formatting

* Starting to query for mix nodes

* Removed the topology in preparation for paging

* Removing unused struct

* Getting ready for series-based paging

* We're now setting page size limits on list retrievals

* Pagination starting to work, needs more testing

* Moved test support stuff into its own home

* Removing duplicate testy code

* Testing all paging stuff in the contract

* Removed useless method duplicate

* Moving queries into their own file

* Removing redundant tests

* Testing default paging limit

* Testing max paging limit

* We don't need to c/p pagination stuff from the cw-plus contracts, removing

* Testing pagination

* Making next key calculation explict via a function

* Removing temporary variable

* Commenting final state

* Incorporating the PagedResponse

* On the road to a working TypeScript client

* Adding some logging utilities

* Paged retrieval working but needs improvement - it's very brittle

* Getting the loop right

* Removing unused logger

* Setting up a request count

* Documenting the ins and outs of the client network interface

* Removing requestCount as we're not using it yet

* Success! Making paginated requests for mixnodes!

* Differentiating between MixNode and MixNodeBond

* Checking that Fred can upload a mixnode

* Fixing export

* Adding the ability for client to get balances

* Docs fix

* Converting interfaces to types

* Changing `mixNodes()` to `getMixNodes()` on client

* We might as well return the nodes we've just retrieved when we refresh

* Starting work on unbonding

* Fixed a caching bug which was causing multiple result sets to be cached

* Using the sender address as the key for removal

* Importing some result stuff so we can find out what happened on execution

* Minor messing around to prove that the sequence fully works

* Displaying a nicer message on mixnode unbond

* Renamed announce to bond in validator client

* Fixed unstable clippy warnings

* Removing commented fields

* Comment spacing

* Changed announce to bond in example code

* Making the test accounts directory configurable

* Rebuilt

* Loading keys from the local ./accounts directory

* Ignoring contract lockfile

* Saving out a contract lockfile so things continue working after contract upload

* Splitting the driver example into smaller self-contained examples

* Deleting the example that Andrew hates so much

* Making dependabot happy

* Stricter equals

* Removing unused import
2021-03-09 11:32:51 +00:00
Jędrzej Stuczyński 6f8ae53f0c Feature/socks5 adjustments (#510)
* Increased default connection buffer size to help with packet bursts

* Informing remote about failure to establish connection

* Properly closing outbound proxy on failure to read from socket

* Split proxy runner into separate inbound and outbound files

* Flag indicating whether socks client actually started proxying

* Clearing stale proxies if other side is closed

* Decreased log level on not a warning

* Closing proxy if no messages were received in a while

* Dont print harmless false-positive errors

* Formatting + future notes

* Fixed packet delay in higher volume traffic

* Don't attempt writing empty messages to connection
2021-02-01 14:45:50 +00:00
jstuczyn d7ec81ad0a Shutting up nightly clippy to fix all pending PRs 2021-01-28 11:59:05 +00:00
Jędrzej Stuczyński 2d3b4f4b91 Feature/GitHub actions and clippy cleanup (#493)
* Added github actions templates

* removed travis .yml file

* initial clippy cleanup pass

* fixed the rest of clippy warnings

* Made github badges more fancy and consistent with the ones in sphinx

* Updated local rustc version and removed compilation warningns

* ... and fresh clippy warnings

* formatting

* beta clippy specific warnings fixed

* Fixed all nightly clippy warnings

* Fixed trying to unwrap a ()

* Actually running all tests

* Correctly passing the --all flag

* Hopefullly third time's a charm in fixing argument passing
2021-01-18 11:50:29 +00:00
Dimitris Apostolou 131574cd3c Fix typos (#492) 2021-01-04 09:31:50 +00:00
Jędrzej Stuczyński 69eefaf91f Fixed bunch of clippy warnings (#427) 2020-11-10 15:28:55 +00:00
Jędrzej Stuczyński be49c3843a Feature/socks improvements (#423)
* Changed AvailableReader from Future to Stream

* comment formatting

* WIP

* Maximum number of bytes read per poll

* More detailed socks5 printing

* Split up socks5 service provider runner

* Fixed closing proxy too soon on one side

* Additional log information

* Printing connection id in log target

* Adjusted some constants

* Target field in log

* Removed dead code

* Decreased logging level

* Explicitly seperated 'Send' and 'Connect' request + data buffering

* Temporarily commented out test that fails due to not understanding limitations of tokio test io builder

* Fixed socks5 tests
2020-11-10 12:47:26 +00:00
Jędrzej Stuczyński c50e9a9ed7 Feature/wasm update (#341)
* Split text and binary client apis

* Very initial attempt at new serialization

* Defined ser+de for Recipient and ReplySURB

* Response errors

* builds with changes

* Working WS API + moved to separate crate

* updated python examples

* Fixed parsing bug

* Updated go examples

* Updated rust examples

* formatting

* Removed unused imports

* dependency updates

* Further dependency changes

* nymsphinx exposingn framing only if not in wasm32

* Cargo lock changes before develop merge

* Pending work

* Actually sending and receiving websocket from rust!

* more WIP

* Initial wasm client + establishing shared key with gateway!

* Splitting and sending a message!

* WIP

* WIP

* Initial wasm-ification of the gateway client

* Passing reconstruction result to js callback!

* Initial WASM cleaning pass

* Dependency pruning

* Moved processing loop to received_processor + at least ack unwrappingn

* Post merge fix

* Kinda updated react example

* Old print statement removed

* Removed yarn.lock

* Fixed building issue for other clients

* Fixed travis test command
2020-09-16 15:18:04 +01:00
Jędrzej Stuczyński 533de18175 Socks client no longer logging dns resolved addresses (#329) 2020-09-09 09:47:38 +01:00
Jędrzej Stuczyński ae40b0d857 snake_cased replySURBs (#322) 2020-09-08 13:11:22 +01:00
Jędrzej Stuczyński 0e7f29c48d Feature/socks5 sequencing (#318)
* Adding ordered buffer, moving socks5 request location into common

* Moved utils to available-reader

* Starting to use OrderedMessage in socks5 requests

* Renamed available reader module

* Initially included sequence numbers

* OrderedBuffer fix

* Buffer bug fix + extra test

* Bug fixes

* Removed test from an old experiment

Co-authored-by: Dave <futurechimp@users.noreply.github.com>
2020-09-01 16:53:48 +01:00
Jędrzej Stuczyński 993d3d10a1 Feature/socks client config (#316)
* Made base client_core config

* Updated socks5 config with mandatory provider field
2020-08-27 16:28:53 +01:00
Jędrzej Stuczyński 9efc195046 Explicit proxy runner + closing local connection if remote is over (#314)
* Updated built dependency

So that it wouldn't fail on nwindows

* Concurrent RW client-side

* Draft of a ProxyRunner

* Super experimental proxy runner for provider

Very much WIP

* Extracting shared code + cleanup

* Logging cleanup

* Removed redundant close request
2020-08-27 13:59:37 +01:00
Jędrzej Stuczyński 7f0ccea39c Removed unused dependencies (#315) 2020-08-27 12:10:12 +01:00
Dave Hrycyszyn 7b2036bd33 Feature/perf messing (#311)
* Merge branch 'develop' into feature/socks5-no-core

common/client-libs/directory-client/models/src/presence/topology.rs

* More diagnostics

* WIP commit, not properly multithreaded

* WIP, now sort of multithreaded but errors all over the place

* Replaced expect with a return

* Adding abort on panic

* Removing a few noisy prints

* Another ack-related expect replaced

* Moved profile.dev annotation to workspace cargo.toml

* Semi-stabilised socks

* Assigning connection shared key post registration (#309)

* Sending return address with connect request

* WIP commit

* Reading all available data on reader + removed duplicate code

Co-authored-by: jstuczyn <jedrzej.stuczynski@gmail.com>
2020-08-20 13:43:26 +01:00
Jędrzej Stuczyński 7d2d512405 Feature/client binary api update (#306)
* Split text and binary client apis

* Very initial attempt at new serialization

* Defined ser+de for Recipient and ReplySURB

* Response errors

* builds with changes

* Working WS API + moved to separate crate

* updated python examples

* Fixed parsing bug

* Updated go examples

* Updated rust examples

* formatting

* Removed unused imports

* Removed accidentally left panic call
2020-08-17 10:44:40 +01:00
Dave Hrycyszyn eec6034684 Revert "Starting to unify the slightly different interfaces of NymClient"
This reverts commit 4d444ceb07.
2020-08-15 12:06:44 +01:00
Dave Hrycyszyn 4d444ceb07 Starting to unify the slightly different interfaces of NymClient 2020-08-13 19:10:34 +01:00
Dave Hrycyszyn 297e212063 Minor docs on socks5 authentication 2020-08-13 12:14:41 +01:00
Dave Hrycyszyn ba81dde4a9 Tightening up accessor visibility on Authenticator. 2020-08-13 12:12:13 +01:00
Dave Hrycyszyn bdfe91676c Feature/client core (#303)
* Adding a client-core crate.

* Moved config into client_core

* Socks5 client now depends on client-core/config

* Native client now mostly using client-core internals

* Socks client now uses client-core internals.
2020-08-13 11:21:59 +01:00
Dave Hrycyszyn 1546088904 Feature/socks5 (#302)
* Adding the beginnings of a socks5 crate

* Removing unused import

* Adding built.rs

* Figured out test failure, stuck a note in detailing under what conditions it fails.

* Added lib

* wip on the way to compile

* First compile with much of the client in place

* Comment reflow to 80 lines

* Changed Socks5 client help message

* Latest changes to from develop applied to socks5 client

* Minor cleanup on unused code

* Adding snafu dependency

* Adding socks library

* Getting socks into the module structure

* Tokio conversion for socks code nearly completed.

* Starting traffic controllers again

* Bitcoin SP starting to breathe with Socks5 proxy. Responses not yet being sent.

* Adding in some hugely verbose print action so we can see things happening

* WIP refactor of socks code.

* Renamed structs to be more rubyish

* Refactored the run command a bit.

* handle_client doesn't need to be public

* Starting to split the handle method up into smaller, refactorable chunks

* Renamed a test

* Finished initial refactor

* Minor cleanup

* Made a few notes for my future self

* Being a bit more explicit in authtentication test

* Ensuring that user/password authentication attempts fail if that auth mode is off

* Documentation

* Refactord types into a types module

* Sending the request ID across and reading the response when it comes back.

* Added the request_id to the response header

* Adding exception handling to websocket send

* Semi-working...

* Removing non-functional examples.

* Minor output clarification

* Adding a Socks5 service provider

* Websocket connection is now being made.

* Added some simple and ungraceful websocket connection error handling

* Renamed socks5_proxy back to proxy

* ibid

* Nicer websocket start method

* Receiving messages via websocket

* Socks requests work in the simple case, SSL requests don't (yet).

* Minor cleanup, renaming variables and moving private functions around

* Comments on try_read_request

* Moved some code around

* Removed commented code and printlns

* Comments sp request

* Commented response data read

* Changing Request to Connection

* ibid

* Added a controller and split connection / request parsing

* Built out error handling on requests a bit

* Initial router action

* Request deserialization tests back in action

* Request constructors

* Constructor for controller

* Renamed message_router to controller

* Starting to build out the responses

* Returning proxied connection data

* Moving towards new Socks5 request crate

* Sending Socks5 multi-part requests through mixnet

* Removed the detritus of exploratory coding.

* Breaking the socks client read loop when empty bytes are read

* Documenting the message format for serialized socks requests

* Returning a response from the socks proxy

* Removing unused import

* Removing more detritus

* Restarting loop if no response is received

* The off-by-one change that fixed it all

* Removing unused response.rs module

* Removed unused import

* Comment cleanup

* More detritus

* Cleaning...

* Docs for socks5-requests

* Using the simple-socks5-requests crate Response in the socks5 client

* Removing unused error types

* Split request/response into their own files and wrote more tests

* Removing temporary README notes

* Renamed all instances of request_id to connection_id

* Docs on Connection struct

* Caving in to connecting inside the constructor for the moment

* Fixing up comments on socks5 service provider start

* Simplified errors in the Socks5 requests crate

* Flattened service provider module hierarchy a bit.

* Removed println

* Comment to explain return on timeout

* Logging controller connect errors

* Renamed websocket reads and writes to make them a bit more understandable

* Renamed TodoError to ConnectionError

* Logging errors instead of panicking on connection read/write failures

* Fixed error handling in controller

* Removing dead comments

* Cargo fmt applied

* Removing print statements

* Removed more comments, prints, etc

Co-authored-by: jstuczyn <jedrzej.stuczynski@gmail.com>
2020-08-12 16:23:23 +01:00