1421 Commits

Author SHA1 Message Date
Jędrzej Stuczyński 2fa5c3c3b7 Release/v0.9.0 (#453)
* Feature/wasm update (#341)

* Split text and binary client apis

* Very initial attempt at new serialization

* Defined ser+de for Recipient and ReplySURB

* Response errors

* builds with changes

* Working WS API + moved to separate crate

* updated python examples

* Fixed parsing bug

* Updated go examples

* Updated rust examples

* formatting

* Removed unused imports

* dependency updates

* Further dependency changes

* nymsphinx exposingn framing only if not in wasm32

* Cargo lock changes before develop merge

* Pending work

* Actually sending and receiving websocket from rust!

* more WIP

* Initial wasm client + establishing shared key with gateway!

* Splitting and sending a message!

* WIP

* WIP

* Initial wasm-ification of the gateway client

* Passing reconstruction result to js callback!

* Initial WASM cleaning pass

* Dependency pruning

* Moved processing loop to received_processor + at least ack unwrappingn

* Post merge fix

* Kinda updated react example

* Old print statement removed

* Removed yarn.lock

* Fixed building issue for other clients

* Fixed travis test command

* Updated wasm publishing instructions

* Feature/instant sending (#359)

* Ability to set client in vpn mode

* Connection handler for mixnode

* Initial vpn mode for mixes

* Updated SphinxCodec to contain more metadata

* Renaming

* Removed handle from mixnet client and introduced forwarder

* Mixnode using new forwarder

* Mixnode common module containing shared packet processing

* ibid. incorporated inside mixnode

* New processing for gateway

* Type cleanup

* Wasm fix

* Fixed client config

* Fixed mixnode runtime issues

* Formatting

* Client re-using secret on 'normal' packets

* Using the same key for acks

* WIP

* vpn key manager cleanup

* wasm fix

* VPN_KEY_REUSE_LIMIT moved to config

* Moved AckDelayQueue to separate common crate

* Key cache invalidator

* Updated dashmap used in gateway

* Old typo

* Additional comment

* Cargo fmt

* Fixed tests

* Sphinx update

* cache ttl as config option

* Cargo fmt

* Feature/network monitor (#369)

* WIP commit

* Further into errors (WIP)

* WIP commit, more stuff compiling, but now we switch to sneaky Jedrzej trick

* Websocket connections starting to work

* WIP

* Constructing the socket stream in one go

* Nicer connections. Getting topology now works.

* Fixing startup message

* Injecting directory uri

* Injecting good mixndodes

* Deleting old healthcheck

* Starting to wrap gateway client

* Splitting out good topology contstruction

* Starting to breathe finally

* Fixed layer assignment error

* Starting gateway client in a better spot

* Cleanup

* Renamed construct() to new()

* Injected channels, removed websockets

* Added a MixnetListener to deal with returned packet traffic

* Simplification

* Renaming and commenting a few things

* Renamed temp variables

* Fixed variable names

* Made errors a bit more explicit on message reconstruction

* Added the mixmining route / object to the directory server client

* Recipient is always "me", a bit easier to understand what's going on

* Subbing in test nodes works

* Using QA directory

* Noting which directory server we're using at monitor start

* Adding Debug to MixStatus so we can print it more easily

* Prettification of startup messages

* Notifying directory of successful packet receipt

* All nodes now being tested

* Now able to do timer-based test runs and detect the last node tested

* Renamed mixnet_listener to notifier

* Moved message chunking into its own home

* Starting to pull out packet sends

* Basic functionality working!

* Enabled timer runs forever

* License notices

* Depdendency cleanup

* Import cleanup

* Moved to implicit tokio runtime

* Removed some unnecessary data clones

* Fixed monitor doing two runs at startup rather than one

* dalek version updates

* This should have been split but got carried away

* Initial ipv6 support

* Don't test outdated nodes

* Added network monitor to default workspace members

* Actually sending reports

* Batch-sending mix status

* Cargo fmt

* Minor cleanup

* Malformed Cargo.lock

* Post-merge fixes

* Defined constant in main.rs for enabling detailed reporting

* Updated package versions of the system

* Checking version compatibility on startup

Co-authored-by: Dave Hrycyszyn <futurechimp@users.noreply.github.com>

* Added sphinx socks to default workspace members (#358)

* Feature/version in config (#376)

* Added version field to config files

* Removed built dependency

* Missing non-removed call to built

* Corrected version on client-core (#377)

* Feature/remove validator (#380)

* Replacing Rust validator sketch with Cosmos-based code.

* Removing old build instructions

* go fmt on generated code

* Removing build files for github and pi

* Starting transplant surgery

* Directory now running in validator, sans metrics

* Updating go sum

* A partly-working directory startup integrated into the nymcli app.

* Removing the Swagger docs header from nymcli, it can live in server.go

* Working swagger docs.

* Updating test title on Swagger docs

* Directory server now integrated into validator (primitive but working)

* Noted location of mixmining reports database.

* Bumped golang version

* Removing validator cargo files.

* Moving directory API init method into the nymd module

* Ditching a test which is causing tooling problems - presence will go away soon anyway

* Removing the validator and merging develop

* Update README.md

* Update README.md

* Feature/upgrade command (#381)

* Exposed version parsing from version checker

* Ability to upgrade mixnode from 0.8.X to 0.9.0

* Ability to upgrade gateway from 0.8.X to 0.9.0

* Ability to upgrade native client from 0.8.X to 0.9.0

* Ability to upgrade socks5 client from 0.8.X to 0.9.0

* Typo

* Minor renaming

* Preventing upgrade if current is not a release version

* Additional upgrade restrictions

* Corrected version bound

* Feature/mix ed25519 identity (#388)

* Introduced identity keypair to mixnode

* Updated upgrade 0.9.0 mix upgrade instructions

* Printing failed upgrade notice when it failed

* Moved printing upgrade start notice to beginning of function

* Feature/double init prevention (#386)

* Updated NymConfig trait making id always obligatory. Also added extra default method implementations

* Preventing init on mixnodes that were initialised before

* Quotes before mix id

* Preventing init on gateways that were initialised before

* Preventing init on native clients that were initialised before

* Preventing init on socks5 clients that were initialised before

* Feature/duration cleanup (#391)

* Mixnode using Duration for time intervals + human readable config fields for them

* Removed old comment

* Native client using Duration for time intervals + human readable config fields for them

* Socks5 client using Duration for time intervals + human readable config fields for them

* Gateway using Duration for time intervals + human readable config fields for them

* Config compatibility with 0.8.0 debug values

* Fixed previous merge issue

* Moving subsidiary functions down below main so we can see where thing start

* Feature/fix dependabot alerts (#393)

* Fixing a dependabot alert

* Fixing npm audit failures in js example for webassembly client

* Fixed some more npm audit problems

* Explicitly handling base58 key recovery errors (#396)

* Fix compiler warnings for unneeded mut (#398)

* moved new_v4_with_node to test only section (#392)

* moved new_v4_with_node to test only section

* cargo fmt

* Feature/directory server transition (#401)

* Initial changes to validator client API

* Updated models

* GatewayRegistrationInfo constructor

* Change validator topology to convert into NymTopology without failure

* Mixnode registering and unregistering presence

* Directory -> Validator renamings + adjustments

* Updated upgrade command for mixnode

* Extracted metrics part of directory client into separate library

* Removed no longer needed traits

* Integrated new metrics client into mixnode

* Introduced the same set of changes to the gateway

* Getting active topology in client core via validator client

* Updated clients to get correct topology

* Introduced mix mining endpoints to validator client

* Network monitor using validator client

* Removed directory client

* Updated wasm client

* Temporarily disabled the test

* Checking ok status for validator client response

* Updated upgrade command for clients

* Allowing using old presence directory as new validator endpoint for mixnodes and gateways

* Fixed tests in non-default crates

* Feature/validator api update (#402)

* Updated validator API routes

* missing comment

* Feature/wasm client fix (#399)

* Disabling wasm optimization for the moment, it isn't working on my machine

* Updating the React example back to a working state, 0.8.1 npm package is published.

* Merging develop into bugfix

* Upgrading react-scripts to fix a major vulnerability

* Using metrics interval received from server (#403)

* Using metrics interval received from server

* Test fix

* Feature/error on noninit (#404)

* More graceful shutdown in case of uninitialised client/node

* Performing version check on binary `run`

* Feature/no run config flag (#405)

* Removed `--config` parameter in `run` commands

* Removed old tests that were doing disk IO

* Renamed directory arguments to validator (#408)

* Feature/max retry (#409)

* Maximum number of allowed reconnection attempts

* Requiring initial internode connection to be successful

* Decreased logging level for failing to establish initial connection

* Feature/controlled reinit (#410)

* Allowing init with existing id but keeps existing keys

* Removed unused imports

* Removed debug print statement (#411)

* Updated upgrade commands to set new default validator (#413)

* Feature/network monitor file topology (#412)

* Network monitor loading 'good' topology from files instead

* Update .gitignore

* Passing address of validator as an argument

* Made detailed report const flag into an argument

* Flag to start network requester in open proxy mode (#418)

* Renamed 'sphinx-socks' to 'network-requester' (#417)

* Added option to set incentives address during mix and gateway init (#419)

* Updated validator topology (#420)

* Bugfix/upgrade fix (#421)

* Added default for incentives address when deserializing

* Ability to optionally pass incentives address during upgrade

* Feature/socks improvements (#423)

* Changed AvailableReader from Future to Stream

* comment formatting

* WIP

* Maximum number of bytes read per poll

* More detailed socks5 printing

* Split up socks5 service provider runner

* Fixed closing proxy too soon on one side

* Additional log information

* Printing connection id in log target

* Adjusted some constants

* Target field in log

* Removed dead code

* Decreased logging level

* Explicitly seperated 'Send' and 'Connect' request + data buffering

* Temporarily commented out test that fails due to not understanding limitations of tokio test io builder

* Fixed socks5 tests

* Starting cover traffic stream under correct condition (#422)

* Fixed bunch of clippy warnings (#427)

* Updated default sending rates (#430)

* Feature/explorer (#431)

* Initial commit of the new dashboard code.

* Periodically grabbing topology json

* Pulling file saving out into its own module

* Ignoring downloaded topology file

* Moved everything public into a public folder

* Refreshing the mixmining report

* Mounting static files from /public

* Including mixminiming report grabber

* Leaving the route in place to pick up later. It's not used right now.

* Removing json download from git

* Ignoring topology download

* Moving recurrent jobs in to a jobs module

* Adding websocket dependencies

* Starting to get client/server websocket functionality running.

* Fixing unused imports

* Separating client and server functionality a bit more cleanly

* WIP to sketch out the ws client and server a bit more

* Initial metrics broadcaster

* Import fixup

* Spawning rocket in tokio task

* Removed outdated comment

* removed the js file

Co-authored-by: Dave <futurechimp@users.noreply.github.com>

* Bugfix/metrics fixes (#434)

* Updated dashboard html file

* Changed explorer to listen on the same port as old dashboard

* Using correct field names in js file

* Sending correct key in metrics

* Added nym prefix to binary names (#436)

* Passing validator base url as an argument (#439)

* Spawning rocket as a blocking task (#440)

* Update main.js (#441)

* Update main.js

* Changed websocket port to 1648

* Changing default validator location to make it clear we're testnet (#437)

* Changing default validator location to make it clear we're on a testnet here.

* Updated validator url

* Updated metrics url

Co-authored-by: jstuczyn <jedrzej.stuczynski@gmail.com>

* Adding keybase to allowed.list.sample (#442)

* Updated nym-run gateway id (#445)

* Adjusted 'fastmode' settings (#444)

* Added validators to dashboard + validator and block count (#443)

* Slightly friendlier upgrade argument description (#446)

* Slightly friendlier upgrade argument description

* Added right format examples

* Explorer public folder being relative to the binary (#447)

* Explorer public folder being relative to the binary

* Qualifying the public mounting path

Co-authored-by: Dave <futurechimp@users.noreply.github.com>

* Extra argument to specify metrics websocket + long attribute (#448)

* Extra argument to specify metrics websocket + long attribute

* Default testnet values

* Fixed the URL of the websocket (#449)

* Removed hardcoded 'good gateways' in favour of pseusorandom choice from the active set (#450)

* Adding graceful de-registration to service

* Removed incentives form url (#451)

* Updated version numbers

* Updated generate changelog script

* Updated changelog

* wasm examples fixes

Co-authored-by: Dave Hrycyszyn <futurechimp@users.noreply.github.com>
Co-authored-by: Ethan Frey <ethanfrey@users.noreply.github.com>
v0.9.0
2020-11-13 12:44:58 +00:00
Jędrzej Stuczyński d05b313a99 Hotfix/0.8.1 (#357)
* Feature/wasm update (#341)

* Split text and binary client apis

* Very initial attempt at new serialization

* Defined ser+de for Recipient and ReplySURB

* Response errors

* builds with changes

* Working WS API + moved to separate crate

* updated python examples

* Fixed parsing bug

* Updated go examples

* Updated rust examples

* formatting

* Removed unused imports

* dependency updates

* Further dependency changes

* nymsphinx exposingn framing only if not in wasm32

* Cargo lock changes before develop merge

* Pending work

* Actually sending and receiving websocket from rust!

* more WIP

* Initial wasm client + establishing shared key with gateway!

* Splitting and sending a message!

* WIP

* WIP

* Initial wasm-ification of the gateway client

* Passing reconstruction result to js callback!

* Initial WASM cleaning pass

* Dependency pruning

* Moved processing loop to received_processor + at least ack unwrappingn

* Post merge fix

* Kinda updated react example

* Old print statement removed

* Removed yarn.lock

* Fixed building issue for other clients

* Fixed travis test command

* Updated wasm publishing instructions

* Increased default presence and metrics sending interval (#354)

* Defaulting to mix layer with fewest node if no argument provided (#353)

* Defaulting to mix layer with fewest node if no argument provided

* cargo fmt

* Selecting the gateway randomly from 2 known good ones. (#355)

* Selecting the gateway randomly from 2 known good ones. There's a problem with the config file in this commit though.

* Saving value of chosen gateway

* Removed duplicate printing of gateway being used

Co-authored-by: jstuczyn <jedrzej.stuczynski@gmail.com>

* More detailed error response on authentication/registration failure (#356)

* Updated changelog

* Updated version numbers

Co-authored-by: Dave Hrycyszyn <futurechimp@users.noreply.github.com>
v0.8.1
2020-09-28 17:19:07 +01:00
Jedrzej Stuczynski f58d19ab9a Merge branch 'release/v0.8.0' v0.8.0 2020-09-10 14:17:13 +01:00
Jedrzej Stuczynski 68dac6266e Updated version numbers 2020-09-10 14:17:01 +01:00
Jędrzej Stuczyński 04948d3d62 v0.8.0 Changelog update (#328)
* Updated changelog

* Changed link to future tag

* Changelog update
2020-09-10 14:14:13 +01:00
Jędrzej Stuczyński 6aada422ec placeholder contact form url (#334)
* placeholder contact form url

* Gateway URL update

* Mixnode url update
2020-09-10 14:08:48 +01:00
Jędrzej Stuczyński b487126f5f Recommended testnet gateway (#335) 2020-09-10 12:52:04 +01:00
Dave 9d2838425a Reducing default delays 2020-09-10 12:23:46 +01:00
Dave Hrycyszyn 915dba2c32 Knocking down delay on message sending default (#332)
* Knocking down delay on message sending default

* accuratizing comments
2020-09-10 11:03:16 +01:00
Jędrzej Stuczyński 66eb1913ba Made gateway mandatory during init (#331) 2020-09-10 10:57:35 +01:00
Dave Hrycyszyn 82b0d7cbcf Renaming client binary (#330) 2020-09-10 10:04:47 +01:00
Jędrzej Stuczyński 533de18175 Socks client no longer logging dns resolved addresses (#329) 2020-09-09 09:47:38 +01:00
Dave Hrycyszyn 4e88960d60 Removing onion addresses.
Tor over a mixnet would be fun though.
2020-09-08 18:43:49 +01:00
Dave Hrycyszyn 256c04bbfa Feature/sphinx socks (#326)
* Renamed simple-socks5 to sphinx-socks

* Changing default storefile path to rename

* Adding a sample list of allowed network requests

* Fixing test
2020-09-08 16:20:27 +01:00
Jędrzej Stuczyński d3ce909916 Feature/websocket js example dependency update (#323)
* Removed yarn.lock file

* Updated dependencies
2020-09-08 15:19:47 +01:00
Jędrzej Stuczyński 29074723c3 Feature/temp wasm example removal (#324)
* Removed yarn.lock file

* wasm js-example dependency updates

* Disabled wasm js-example

* Disabled wasm react example
2020-09-08 15:18:44 +01:00
Jędrzej Stuczyński 0f01893ac2 Feature/print client address on startup (#325)
* Decreased logging level

* Printing clients full address on startup
2020-09-08 15:18:09 +01:00
Jędrzej Stuczyński ae40b0d857 snake_cased replySURBs (#322) 2020-09-08 13:11:22 +01:00
Jędrzej Stuczyński 80d01f1f75 Feature/outbound request lists (#321)
* Moving main upwards

* WIP commit

* Checking outbound requests starting to happen

* Renamed Persistence to HostsStore

* Ensured that unknowns are only added once

* Ownership improvements

* Adding rand crate so we can generate test output files

* Adding dirs crate so we can save to disk

* ibid

* ibid

* Unknown hosts are written to file

* Tests ensuring that unknown requests are written only once

* Fixed up paths for easier testing

* Loading host lists from disk

* Saving domain roots in unknown file

* Only fetching the publicsuffix domains list once

* Documenting use of publicsuffix list

* Beefed up error handling

* Knocking down log level on high water mark

* Minor cleanup on domain parsing

* Noted failing filter check
2020-09-08 12:06:32 +01:00
Dave 83e78e7df1 Setting log message to trace 2020-09-08 12:02:04 +01:00
Dave 406086aec7 Removing log message 2020-09-08 12:01:43 +01:00
Dave 80a86e3bec Noted failing filter check 2020-09-05 14:14:29 +01:00
Dave eb1737264e Minor cleanup on domain parsing 2020-09-05 14:14:03 +01:00
Dave 4b16d14121 Knocking down log level on high water mark 2020-09-05 14:13:35 +01:00
Dave 4c055a8e35 Beefed up error handling 2020-09-05 13:00:18 +01:00
Dave 9ce8c42455 Documenting use of publicsuffix list 2020-09-05 12:35:34 +01:00
Dave d3cb0d25c8 Only fetching the publicsuffix domains list once 2020-09-05 11:59:58 +01:00
Dave 4d4ee26a9b Saving domain roots in unknown file 2020-09-04 20:27:19 +01:00
Dave b7a8aefbde Loading host lists from disk 2020-09-04 15:08:10 +01:00
Dave ac3829c65f Fixed up paths for easier testing 2020-09-04 14:02:46 +01:00
Dave 11000a99c1 Tests ensuring that unknown requests are written only once 2020-09-04 13:04:20 +01:00
Dave 5340716cce Unknown hosts are written to file 2020-09-04 10:53:02 +01:00
Dave 334e8f2096 ibid 2020-09-04 10:48:17 +01:00
Dave 244a8bf957 ibid 2020-09-04 10:48:11 +01:00
Dave c4e9577f78 Adding dirs crate so we can save to disk 2020-09-04 10:47:57 +01:00
Dave 80b16069e8 Adding rand crate so we can generate test output files 2020-09-04 10:47:43 +01:00
Dave 4cb9720758 Ownership improvements 2020-09-03 15:10:31 +01:00
Dave 6d1a499cbe Ensured that unknowns are only added once 2020-09-03 15:10:21 +01:00
Dave 918e134c37 Renamed Persistence to HostsStore 2020-09-03 15:04:06 +01:00
Dave dc5576e1d7 Checking outbound requests starting to happen 2020-09-03 15:02:17 +01:00
Dave 822f6350cb WIP commit 2020-09-03 11:47:28 +01:00
Dave d1b7ed8603 Moving main upwards 2020-09-02 15:25:00 +01:00
Jędrzej Stuczyński 0e7f29c48d Feature/socks5 sequencing (#318)
* Adding ordered buffer, moving socks5 request location into common

* Moved utils to available-reader

* Starting to use OrderedMessage in socks5 requests

* Renamed available reader module

* Initially included sequence numbers

* OrderedBuffer fix

* Buffer bug fix + extra test

* Bug fixes

* Removed test from an old experiment

Co-authored-by: Dave <futurechimp@users.noreply.github.com>
2020-09-01 16:53:48 +01:00
Dave Hrycyszyn be6486665b Feature/temp fix for ipv6 (#317)
* A temporary fix for IPv6 connections, which crash the SP

* Showing fewer timeouts in terminal while running
2020-08-28 10:03:48 +01:00
Jędrzej Stuczyński 993d3d10a1 Feature/socks client config (#316)
* Made base client_core config

* Updated socks5 config with mandatory provider field
2020-08-27 16:28:53 +01:00
Jędrzej Stuczyński 9efc195046 Explicit proxy runner + closing local connection if remote is over (#314)
* Updated built dependency

So that it wouldn't fail on nwindows

* Concurrent RW client-side

* Draft of a ProxyRunner

* Super experimental proxy runner for provider

Very much WIP

* Extracting shared code + cleanup

* Logging cleanup

* Removed redundant close request
2020-08-27 13:59:37 +01:00
Jędrzej Stuczyński 7f0ccea39c Removed unused dependencies (#315) 2020-08-27 12:10:12 +01:00
Jędrzej Stuczyński a0484ce082 Feature/ack messing (#313)
* Added Display for FragmentIdentifier

* AckDelayQueue as wrapper for DelayQueue

* First pass on ack controller

* Controller very initially incorporated into the client

* Cleanup

* Using socks5 build.rs

* Extra (temporary) log statements

* Decreased logging levels
2020-08-25 10:45:31 +01:00
Dave Hrycyszyn 7b2036bd33 Feature/perf messing (#311)
* Merge branch 'develop' into feature/socks5-no-core

common/client-libs/directory-client/models/src/presence/topology.rs

* More diagnostics

* WIP commit, not properly multithreaded

* WIP, now sort of multithreaded but errors all over the place

* Replaced expect with a return

* Adding abort on panic

* Removing a few noisy prints

* Another ack-related expect replaced

* Moved profile.dev annotation to workspace cargo.toml

* Semi-stabilised socks

* Assigning connection shared key post registration (#309)

* Sending return address with connect request

* WIP commit

* Reading all available data on reader + removed duplicate code

Co-authored-by: jstuczyn <jedrzej.stuczynski@gmail.com>
2020-08-20 13:43:26 +01:00
Dave Hrycyszyn da25b23898 Upgrades tungstenite libraries to new versions with 64MB message sizes. (#310) 2020-08-20 10:43:38 +01:00