Merge branch 'release/0.6.0'

* Added timeout values to configs

* Fixed possible crash when using delay larger than 2 years

* Connection timeout with hardcoded value

* Using provided timeout value

* tcp client requiring timeout value at construction

* Client using updated tcp client

* Mixnode using updated tcp client

* Healthchecker having separate timeout + new client config field

* Updated validator with connection timeout field

* Added connection_timeout to validator config template

.env.sample-dev

@@ -0,0 +1,2 @@
+RUST_LOG=info
+RUST_BACKTRACE=1

.gitignore

@@ -1,3 +1,9 @@
-/target
+/targetString
 **/*.rs.bk
-/.idea/
+/*/target
+/test_inbox
+.idea
+target
+.env
+/.vscode/settings.json
+sample-configs/validator-config.toml

.travis.yml

@@ -0,0 +1,15 @@
+language: rust
+rust:
+  - stable
+  - beta
+  - nightly
+jobs:
+  allow_failures:
+    - rust: nightly
+  fast_finish: true
+before_script:
+  - rustup component add rustfmt
+script:
+  - cargo build
+  - cargo test -- --test-threads=1
+  - cargo fmt -- --check

CHANGELOG.md

@@ -1,12 +1,268 @@
-# nym-client Changelog
+# Changelog
 
-* removed the `--local` flag
-* introduced `--directory` argument to support arbitrary directory servers. Leaving it out will point the node at the "https://directory.nymtech.net" alpha testnet server
-* IPv6 support
-* client version number is now shown at node start
-* directory server location is now shown at node start
-* decrease default delays
+## [v0.6.0](https://github.com/nymtech/nym/tree/HEAD)
 
-## 0.1.0 - Initial Release
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.5.0...HEAD)
 
-* The bare minimum set of features required by a Nym Client
+This  release fixes bugs in v0.5.0. All testnet node operators are advised to upgrade from v0.5.0.
+
+* fixed premature EOFs on socket connections by using the new multi-TCP client
+* fixed a bug causing client and mixnode connection hangs for misconfigured nodes
+* by default 'Debug' section of saved configs is now empty and default values are used unless explicitly overridden
+* introduced packet chunking allowing clients to send messages of arbitrary length. Note that packet retransmission is not implemented yet, so for longer messages, you might not get anything
+* mixnodes now periodically log stats regarding number of packets mixed
+* fixed possible client hang ups when sending high rates of traffic 
+* preventing mixes from starting with same announce-host as an existing node
+* fixed overflow multiplication if connection backoff was set to a high value
+
+
+**Closed issues:**
+
+- Periodic activity summary [\#172](https://github.com/nymtech/nym/issues/172)
+- Move contents of 'common/addressing' into 'common/nymsphinx' [\#161](https://github.com/nymtech/nym/issues/161)
+- Make builds simpler for node operators [\#114](https://github.com/nymtech/nym/issues/114)
+- Chunking in `nym-client` \(receive\) [\#83](https://github.com/nymtech/nym/issues/83)
+- Chunking in `nym-client` \(send\) [\#82](https://github.com/nymtech/nym/issues/82)
+
+**Merged pull requests:**
+
+- Feature/tcp client connection timeout [\#176](https://github.com/nymtech/nym/pull/176) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/mixing stats logging [\#175](https://github.com/nymtech/nym/pull/175) ([jstuczyn](https://github.com/jstuczyn))
+- Preventing multiplication overflow for reconnection backoff [\#174](https://github.com/nymtech/nym/pull/174) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/non mandatory debug config [\#173](https://github.com/nymtech/nym/pull/173) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/addressing move [\#169](https://github.com/nymtech/nym/pull/169) ([jstuczyn](https://github.com/jstuczyn))
+- Checking if any other node is already announcing the same host [\#168](https://github.com/nymtech/nym/pull/168) ([jstuczyn](https://github.com/jstuczyn))
+- Bugfix/closing tcp client connections on drop [\#167](https://github.com/nymtech/nym/pull/167) ([jstuczyn](https://github.com/jstuczyn))
+- Yielding tokio task upon creating loop/real traffic message [\#166](https://github.com/nymtech/nym/pull/166) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/minor healthchecker improvements [\#165](https://github.com/nymtech/nym/pull/165) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/packet chunking [\#158](https://github.com/nymtech/nym/pull/158) ([jstuczyn](https://github.com/jstuczyn))
+
+## [v0.5.0](https://github.com/nymtech/nym/tree/v0.5.0) (2020-03-23)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.5.0-rc.1...v0.5.0)
+
+1. Introduced proper configuration options for mixnodes, clients and providers. Everything is initialised with the `init` command that creates a saved config.toml file. To run the binary you now use `nym-<binary-name> run`, for example `nym-mixnode run`. Each flag can be overwritten at any stage with the following priority: run flags, data in config.toml and finally init flags.
+2. Made mixnet TCP connections persistent. When sending a Sphinx packet, it should no longer go through the lengthy process of establishing a TCP connection only to immediately tear it down after sending a single packet. This significantly boosts throughput. 
+3. A lot of work on code clean up and refactoring including some performance fixes.
+4. Client now determines its default nym-sfw-provider at startup and should always try to connect to the same one. Note: we still can't reliably run more than a single provider on the network.
+5. Logging messages now have timestamps and when running at more aggressive log mode (like debug or even trace) we should no longer be overwhelmed with messages from external crates.
+6. Initial compatibility with Windows. Please let us know if you have problems.
+7. More work on validator, including initial Tendermint integration in Rust, and the start of the mixmining system.
+
+**Closed issues:**
+
+- Introduce timestamps to log messages [\#124](https://github.com/nymtech/nym/issues/124)
+
+**Merged pull requests:**
+
+- removing spooky startup warning message [\#155](https://github.com/nymtech/nym/pull/155) ([futurechimp](https://github.com/futurechimp))
+- Some more startup fixes [\#154](https://github.com/nymtech/nym/pull/154) ([futurechimp](https://github.com/futurechimp))
+- Entering runtime context when creating mix traffic controller [\#153](https://github.com/nymtech/nym/pull/153) ([jstuczyn](https://github.com/jstuczyn))
+- Friendlification of startup messages [\#151](https://github.com/nymtech/nym/pull/151) ([futurechimp](https://github.com/futurechimp))
+- Entering runtime context when creating packet forwarder [\#150](https://github.com/nymtech/nym/pull/150) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/add topology to validator [\#149](https://github.com/nymtech/nym/pull/149) ([futurechimp](https://github.com/futurechimp))
+- Making code work on windows machines [\#148](https://github.com/nymtech/nym/pull/148) ([jstuczyn](https://github.com/jstuczyn))
+- validator: adding HTTP interface [\#146](https://github.com/nymtech/nym/pull/146) ([futurechimp](https://github.com/futurechimp))
+- Extracting the log setup [\#145](https://github.com/nymtech/nym/pull/145) ([futurechimp](https://github.com/futurechimp))
+- Feature/optional location in configs [\#144](https://github.com/nymtech/nym/pull/144) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/concurrent connection managers [\#142](https://github.com/nymtech/nym/pull/142) ([jstuczyn](https://github.com/jstuczyn))
+- Defaulting for global 'Info' logging level if not set in .env [\#140](https://github.com/nymtech/nym/pull/140) ([jstuczyn](https://github.com/jstuczyn))
+- Provider not storing loop cover messages [\#139](https://github.com/nymtech/nym/pull/139) ([jstuczyn](https://github.com/jstuczyn))
+- Using log builder to include timestamps + filters [\#138](https://github.com/nymtech/nym/pull/138) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/client ws refactoring [\#134](https://github.com/nymtech/nym/pull/134) ([jstuczyn](https://github.com/jstuczyn))
+- Bugfix/metrics presence delay fix [\#133](https://github.com/nymtech/nym/pull/133) ([jstuczyn](https://github.com/jstuczyn))
+- Removed outdated and redundant sample-configs [\#131](https://github.com/nymtech/nym/pull/131) ([jstuczyn](https://github.com/jstuczyn))
+- If not overridden, 'announce-host' should default to 'host' [\#130](https://github.com/nymtech/nym/pull/130) ([jstuczyn](https://github.com/jstuczyn))
+- Nice to know who we're talking to at startup... [\#129](https://github.com/nymtech/nym/pull/129) ([futurechimp](https://github.com/futurechimp))
+
+## [v0.5.0-rc.1](https://github.com/nymtech/nym/tree/v0.5.0-rc.1) (2020-03-06)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.1...v0.5.0-rc.1)
+
+**Closed issues:**
+
+- COMPILE: Could not compile project using Cargo [\#118](https://github.com/nymtech/nym/issues/118)
+- Wherever unbounded mpsc channel is used, prefer unbounded\_send\(\) over send\(\).await [\#90](https://github.com/nymtech/nym/issues/90)
+- Add a `Send` method in nym-client [\#81](https://github.com/nymtech/nym/issues/81)
+- Start on Tendermint integration [\#79](https://github.com/nymtech/nym/issues/79)
+- Ditch DummyKeyPair [\#75](https://github.com/nymtech/nym/issues/75)
+- Replace args with proper config files [\#69](https://github.com/nymtech/nym/issues/69)
+- Fix incorrectly used Arcs [\#47](https://github.com/nymtech/nym/issues/47)
+- nym-mixnode mandatory host option [\#26](https://github.com/nymtech/nym/issues/26)
+- Create config struct for mixnode \(possibly also for client\) [\#21](https://github.com/nymtech/nym/issues/21)
+- Reuse TCP socket connection between client and mixnodes [\#20](https://github.com/nymtech/nym/issues/20)
+- Once implementation is available, wherever appropriate, replace `futures::lock::Mutex` with `futures::lock::RwLock` [\#9](https://github.com/nymtech/nym/issues/9)
+- Check if RwLock on MixProcessingData is still needed [\#8](https://github.com/nymtech/nym/issues/8)
+- Reuse TCP socket connection between mixnodes and providers [\#3](https://github.com/nymtech/nym/issues/3)
+- Persistent socket connection with other mixes [\#2](https://github.com/nymtech/nym/issues/2)
+
+**Merged pull requests:**
+
+- Feature/client refactoring [\#128](https://github.com/nymtech/nym/pull/128) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/provider refactoring [\#125](https://github.com/nymtech/nym/pull/125) ([jstuczyn](https://github.com/jstuczyn))
+- all: fixing mis-spelling [\#123](https://github.com/nymtech/nym/pull/123) ([futurechimp](https://github.com/futurechimp))
+- Feature/further clippy fixes [\#121](https://github.com/nymtech/nym/pull/121) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/tokio tungstenite dependency fix [\#120](https://github.com/nymtech/nym/pull/120) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/config files cleanup [\#119](https://github.com/nymtech/nym/pull/119) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/config files [\#117](https://github.com/nymtech/nym/pull/117) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/un genericize keys [\#111](https://github.com/nymtech/nym/pull/111) ([futurechimp](https://github.com/futurechimp))
+- Feature/abci [\#110](https://github.com/nymtech/nym/pull/110) ([futurechimp](https://github.com/futurechimp))
+- Simplified the use of generics on identity keypair by using output types [\#109](https://github.com/nymtech/nym/pull/109) ([jstuczyn](https://github.com/jstuczyn))
+
+## [v0.4.1](https://github.com/nymtech/nym/tree/v0.4.1) (2020-01-29)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0...v0.4.1)
+
+**Closed issues:**
+
+- Change healthcheck to run on provided topology rather than pull one itself [\#95](https://github.com/nymtech/nym/issues/95)
+
+**Merged pull requests:**
+
+- Bugfix/healthcheck on provided topology [\#108](https://github.com/nymtech/nym/pull/108) ([jstuczyn](https://github.com/jstuczyn))
+
+## [v0.4.0](https://github.com/nymtech/nym/tree/v0.4.0) (2020-01-28)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/0.4.0-rc.2...v0.4.0)
+
+Nym 0.4.0 Platform
+
+In this release, we're taking a lot more care with version numbers, so that we can ensure upgrade compatibility for mixnodes, providers, clients, and validators more easily. 
+
+This release also integrates a health-checker and network topology refresh into the Nym client, so that the client can intelligently choose paths which route around any non-functional or incompatible nodes. 
+
+## [0.4.0-rc.2](https://github.com/nymtech/nym/tree/0.4.0-rc.2) (2020-01-28)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0-rc.2...0.4.0-rc.2)
+
+## [v0.4.0-rc.2](https://github.com/nymtech/nym/tree/v0.4.0-rc.2) (2020-01-28)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0-rc.1...v0.4.0-rc.2)
+
+**Merged pull requests:**
+
+- Hotfix/semver compatibility [\#106](https://github.com/nymtech/nym/pull/106) ([jstuczyn](https://github.com/jstuczyn))
+
+## [v0.4.0-rc.1](https://github.com/nymtech/nym/tree/v0.4.0-rc.1) (2020-01-28)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.3...v0.4.0-rc.1)
+
+**Closed issues:**
+
+- Check Sphinx packet length in client [\#98](https://github.com/nymtech/nym/issues/98)
+- workflow test [\#97](https://github.com/nymtech/nym/issues/97)
+- Client SemVer [\#85](https://github.com/nymtech/nym/issues/85)
+- Move PemStore [\#74](https://github.com/nymtech/nym/issues/74)
+- Periodic client refresh [\#70](https://github.com/nymtech/nym/issues/70)
+- Logging [\#68](https://github.com/nymtech/nym/issues/68)
+- Nym-client refactor [\#67](https://github.com/nymtech/nym/issues/67)
+- Stop panicking! [\#66](https://github.com/nymtech/nym/issues/66)
+- Fix Mixnode Panic on Sphinx packet replay [\#65](https://github.com/nymtech/nym/issues/65)
+- Convert older code to start using logging [\#35](https://github.com/nymtech/nym/issues/35)
+- Convert to non-url-safe base64 \(everywhere\) [\#28](https://github.com/nymtech/nym/issues/28)
+- If a thread blows at startup, panic the entire application [\#15](https://github.com/nymtech/nym/issues/15)
+- Split provider/mod.rs [\#10](https://github.com/nymtech/nym/issues/10)
+
+**Merged pull requests:**
+
+- Feature/health checker with existing keys [\#105](https://github.com/nymtech/nym/pull/105) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/remove topology equality check [\#104](https://github.com/nymtech/nym/pull/104) ([futurechimp](https://github.com/futurechimp))
+- Feature/base58 [\#102](https://github.com/nymtech/nym/pull/102) ([futurechimp](https://github.com/futurechimp))
+- Feature/panic improvements [\#101](https://github.com/nymtech/nym/pull/101) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/fix sphinx unwraps [\#100](https://github.com/nymtech/nym/pull/100) ([futurechimp](https://github.com/futurechimp))
+- Feature/check packet length [\#99](https://github.com/nymtech/nym/pull/99) ([futurechimp](https://github.com/futurechimp))
+- Feature/version filtering improvements [\#96](https://github.com/nymtech/nym/pull/96) ([futurechimp](https://github.com/futurechimp))
+- Feature/refreshing topology [\#94](https://github.com/nymtech/nym/pull/94) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/consistent logging [\#93](https://github.com/nymtech/nym/pull/93) ([futurechimp](https://github.com/futurechimp))
+- Feature/semver client [\#92](https://github.com/nymtech/nym/pull/92) ([futurechimp](https://github.com/futurechimp))
+- Feature/client refactor [\#91](https://github.com/nymtech/nym/pull/91) ([jstuczyn](https://github.com/jstuczyn))
+- Release builds should no longer silently fail - everything will be im… [\#89](https://github.com/nymtech/nym/pull/89) ([jstuczyn](https://github.com/jstuczyn))
+
+## [v0.3.3](https://github.com/nymtech/nym/tree/v0.3.3) (2020-01-20)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.2...v0.3.3)
+
+**Fixed bugs:**
+
+- Nym client crashing and disconnecting local websocket with complaint about binary data [\#56](https://github.com/nymtech/nym/issues/56)
+
+**Closed issues:**
+
+- Websocket text fix [\#64](https://github.com/nymtech/nym/issues/64)
+- Restore nym-client lib in crate [\#63](https://github.com/nymtech/nym/issues/63)
+- Make websocket not crash on ping or pong messages [\#62](https://github.com/nymtech/nym/issues/62)
+- Messages returned by fetch are base64 encoded [\#55](https://github.com/nymtech/nym/issues/55)
+- Check layer 1 connectivity at client start [\#38](https://github.com/nymtech/nym/issues/38)
+- Check required sfw-provider args [\#27](https://github.com/nymtech/nym/issues/27)
+- Make Electron app work with new Rust mixnet client [\#16](https://github.com/nymtech/nym/issues/16)
+- Take version numbers into account when picking routes [\#14](https://github.com/nymtech/nym/issues/14)
+
+**Merged pull requests:**
+
+- Feature/websocket improvements [\#88](https://github.com/nymtech/nym/pull/88) ([jstuczyn](https://github.com/jstuczyn))
+- Using println rather than log for startup banner, it's not an error [\#87](https://github.com/nymtech/nym/pull/87) ([futurechimp](https://github.com/futurechimp))
+- Feature/nym client lib [\#61](https://github.com/nymtech/nym/pull/61) ([jstuczyn](https://github.com/jstuczyn))
+
+## [v0.3.2](https://github.com/nymtech/nym/tree/v0.3.2) (2020-01-17)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.1...v0.3.2)
+
+**Merged pull requests:**
+
+- Feature/separate presence address [\#59](https://github.com/nymtech/nym/pull/59) ([jstuczyn](https://github.com/jstuczyn))
+
+## [v0.3.1](https://github.com/nymtech/nym/tree/v0.3.1) (2020-01-16)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.0...v0.3.1)
+
+**Merged pull requests:**
+
+- Bugfix/presence client crash [\#58](https://github.com/nymtech/nym/pull/58) ([jstuczyn](https://github.com/jstuczyn))
+
+## [v0.3.0](https://github.com/nymtech/nym/tree/v0.3.0) (2020-01-14)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/v0.2.0...v0.3.0)
+
+**Closed issues:**
+
+- Error referring to mismatched types caused by topology [\#46](https://github.com/nymtech/nym/issues/46)
+- Provider needs to announce two of its addresses \(+ issue of hardcoded port\) [\#32](https://github.com/nymtech/nym/issues/32)
+- Port run\_network.sh from old Go mixnet [\#30](https://github.com/nymtech/nym/issues/30)
+- Health Checker inside Validator [\#29](https://github.com/nymtech/nym/issues/29)
+- Combine Rust repositories into a single master repo containing multiple crates [\#24](https://github.com/nymtech/nym/issues/24)
+- Fix the version numbers on provider and mixnode [\#23](https://github.com/nymtech/nym/issues/23)
+- Travis releases for Nym [\#22](https://github.com/nymtech/nym/issues/22)
+- Add version number to presence [\#4](https://github.com/nymtech/nym/issues/4)
+- Add version number to presence [\#1](https://github.com/nymtech/nym/issues/1)
+
+**Merged pull requests:**
+
+- Feature/client topology filtering [\#54](https://github.com/nymtech/nym/pull/54) ([jstuczyn](https://github.com/jstuczyn))
+- print public key for nym client tools [\#53](https://github.com/nymtech/nym/pull/53) ([mileschet](https://github.com/mileschet))
+- Showing binding warning on binding to localhost, 0.0.0.0 or 127.0.0.1 [\#52](https://github.com/nymtech/nym/pull/52) ([jstuczyn](https://github.com/jstuczyn))
+- validator: moving sample config files into sample configs directory [\#51](https://github.com/nymtech/nym/pull/51) ([futurechimp](https://github.com/futurechimp))
+- Feature/validator health checker [\#50](https://github.com/nymtech/nym/pull/50) ([jstuczyn](https://github.com/jstuczyn))
+- Improving websocket connection errors [\#49](https://github.com/nymtech/nym/pull/49) ([futurechimp](https://github.com/futurechimp))
+- Feature/crypto crate [\#48](https://github.com/nymtech/nym/pull/48) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/chill log messages [\#43](https://github.com/nymtech/nym/pull/43) ([futurechimp](https://github.com/futurechimp))
+- persistence: improving PEM file reading and parsing failure messages [\#42](https://github.com/nymtech/nym/pull/42) ([futurechimp](https://github.com/futurechimp))
+- Providing a nicer error than "failed on unwrap\(\)" when topology retri… [\#41](https://github.com/nymtech/nym/pull/41) ([futurechimp](https://github.com/futurechimp))
+- Prettying up sfw-provider start sequence a bit. [\#40](https://github.com/nymtech/nym/pull/40) ([futurechimp](https://github.com/futurechimp))
+- Removing the run command from code and documentation [\#39](https://github.com/nymtech/nym/pull/39) ([futurechimp](https://github.com/futurechimp))
+- Feature/script/localnet [\#37](https://github.com/nymtech/nym/pull/37) ([futurechimp](https://github.com/futurechimp))
+- Feature/arguments fix [\#36](https://github.com/nymtech/nym/pull/36) ([jstuczyn](https://github.com/jstuczyn))
+- Feature/crates cleanup [\#34](https://github.com/nymtech/nym/pull/34) ([jstuczyn](https://github.com/jstuczyn))
+- Features/version number to presence [\#25](https://github.com/nymtech/nym/pull/25) ([aniampio](https://github.com/aniampio))
+- Feature/minor client changes [\#7](https://github.com/nymtech/nym/pull/7) ([jstuczyn](https://github.com/jstuczyn))
+
+## [v0.2.0](https://github.com/nymtech/nym/tree/v0.2.0) (2020-01-07)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/0.2.0...v0.2.0)
+
+## [0.2.0](https://github.com/nymtech/nym/tree/0.2.0) (2020-01-06)
+
+[Full Changelog](https://github.com/nymtech/nym/compare/0.1.0...0.2.0)
+
+
+
+\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*

Cargo.toml

@@ -1,39 +1,23 @@
-[package]
-build = "build.rs"
-name = "nym-client"
-version = "0.2.0"
-authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
-edition = "2018"
+[profile.release]
+panic = "abort"
 
-[lib]
-name = "nym_client"
-path = "src/lib.rs"
+[workspace]
 
-# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
-
-[dependencies]
-base64 = "0.11.0"
-clap = "2.33.0"
-curve25519-dalek = "1.2.3"
-dirs = "2.0.2"
-futures = "0.3.1"
-hex = "0.4.0"
-pem = "0.7.0"
-rand = "0.7.2"
-rand_distr = "0.2.2"
-reqwest = "0.9.22"
-serde = { version = "1.0.104", features = ["derive"] }
-serde_json = "1.0.44"
-sphinx = { path = "../sphinx" }
-sfw-provider-requests = { path = "../nym-sfw-provider/sfw-provider-requests" }
-tokio = { version = "0.2", features = ["full"] }
-tungstenite = "0.9.2"
-
-# putting this explicitly below everything and most likely, the next time we look into it, it will already have a proper release
-tokio-tungstenite = { git = "https://github.com/dbcfd/tokio-tungstenite", rev="6dc2018cbfe8fe7ddd75ff977343086503135b38" }
-
-[build-dependencies]
-built = "0.3.2"
-
-[dev-dependencies]
-mockito = "0.22.0"
+members = [
+    "common/clients/directory-client",
+    "common/clients/mix-client",
+    "common/clients/multi-tcp-client",
+    "common/clients/provider-client",
+    "common/clients/validator-client",
+    "common/config",
+    "common/crypto",
+    "common/healthcheck",
+    "common/nymsphinx",
+    "common/pemstore",
+    "common/topology",
+    "mixnode",
+    "nym-client",
+    "sfw-provider",
+    "sfw-provider/sfw-provider-requests",
+    "validator",
+]

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -1,10 +1,24 @@
-# Nym Client
+## The Nym Privacy Platform
 
-The Nym Client communicates with the remote, decentralised nodes which make up the Nym system as a whole. 
+This repository contains the full Nym platform.
 
-It needs to handle communication with 3 types of remote nodes. Here's the development status of each:
+The platform is composed of multiple Rust crates. Top-level executable binary crates include:
 
-- [ ] Directory nodes
-- [ ] Mix nodes
-- [ ] Validator nodes
+* nym-mixnode - shuffles [Sphinx](https://github.com/nymtech/sphinx) packets together to provide privacy against network-level attackers.
+* nym-client - an executable which you can build into your own applications. Use it for interacting with Nym nodes.
+* nym-sfw-provider - a store-and-forward service provider. The provider acts sort of like a mailbox for mixnet messages.
+* nym-validator - currently just starting development. Handles consensus ordering of transactions, mixmining, and coconut credential generation and validation. 
 
+[![Build Status](https://travis-ci.com/nymtech/nym.svg?branch=develop)](https://travis-ci.com/nymtech/nym)
+
+### Building
+
+Platform build instructions are available on [our docs site](https://nymtech.net/docs/mixnet/installation/).
+
+### Developing
+
+There's a `.env.sample-dev` file provided which you can rename to `.env` if you want convenient logging, backtrace, or other environment variables pre-set. The `.env` file is ignored so you don't need to worry about checking it in.
+
+### Developer chat
+
+You can chat to us in [Keybase](https://keybase.io). Download their chat app, then click **Teams -> Join a team**. Type **nymtech.friends** into the team name and hit **continue**. For general chat, hang out in the **#general** channel. Our development takes places in the **#dev** channel. Node operators should be in the **#node-operators** channel.

common/clients/directory-client/Cargo.toml

@@ -0,0 +1,19 @@
+[package]
+name = "directory-client"
+version = "0.1.0"
+authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+log = "0.4"
+pretty_env_logger = "0.3"
+reqwest = "0.9.22"
+serde = { version = "1.0.104", features = ["derive"] }
+
+## internal
+topology = {path = "../../topology"}
+
+[dev-dependencies]
+mockito = "0.23.0"

common/clients/directory-client/src/lib.rs

@@ -1,22 +1,16 @@
-use crate::clients::directory::requests::health_check_get::{
-    HealthCheckRequester, Request as HealthCheckRequest,
-};
-use crate::clients::directory::requests::metrics_mixes_get::{
-    MetricsMixRequester, Request as MetricsMixRequest,
-};
-use crate::clients::directory::requests::metrics_mixes_post::{
-    MetricsMixPoster, Request as MetricsMixPost,
-};
-use crate::clients::directory::requests::presence_coconodes_post::{
+use crate::requests::health_check_get::{HealthCheckRequester, Request as HealthCheckRequest};
+use crate::requests::metrics_mixes_get::{MetricsMixRequester, Request as MetricsMixRequest};
+use crate::requests::metrics_mixes_post::{MetricsMixPoster, Request as MetricsMixPost};
+use crate::requests::presence_coconodes_post::{
     PresenceCocoNodesPoster, Request as PresenceCocoNodesPost,
 };
-use crate::clients::directory::requests::presence_mixnodes_post::{
+use crate::requests::presence_mixnodes_post::{
     PresenceMixNodesPoster, Request as PresenceMixNodesPost,
 };
-use crate::clients::directory::requests::presence_providers_post::{
+use crate::requests::presence_providers_post::{
     PresenceMixProviderPoster, Request as PresenceProvidersPost,
 };
-use crate::clients::directory::requests::presence_topology_get::{
+use crate::requests::presence_topology_get::{
     PresenceTopologyGetRequester, Request as PresenceTopologyRequest,
 };
 
@@ -28,6 +22,12 @@ pub struct Config {
     pub base_url: String,
 }
 
+impl Config {
+    pub fn new(base_url: String) -> Self {
+        Config { base_url }
+    }
+}
+
 pub trait DirectoryClient {
     fn new(config: Config) -> Self;
 }
@@ -54,7 +54,7 @@ impl DirectoryClient for Client {
         let presence_mix_nodes_post: PresenceMixNodesPost =
             PresenceMixNodesPost::new(config.base_url.clone());
         let presence_providers_post: PresenceProvidersPost =
-            PresenceProvidersPost::new(config.base_url.clone());
+            PresenceProvidersPost::new(config.base_url);
         Client {
             health_check,
             metrics_mixes,

common/clients/directory-client/src/presence/coconodes.rs

@@ -0,0 +1,36 @@
+use serde::{Deserialize, Serialize};
+use topology::coco;
+
+#[derive(Clone, Debug, Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct CocoPresence {
+    pub location: String,
+    pub host: String,
+    pub pub_key: String,
+    pub last_seen: u64,
+    pub version: String,
+}
+
+impl Into<topology::coco::Node> for CocoPresence {
+    fn into(self) -> topology::coco::Node {
+        topology::coco::Node {
+            location: self.location,
+            host: self.host,
+            pub_key: self.pub_key,
+            last_seen: self.last_seen,
+            version: self.version,
+        }
+    }
+}
+
+impl From<topology::coco::Node> for CocoPresence {
+    fn from(cn: coco::Node) -> Self {
+        CocoPresence {
+            location: cn.location,
+            host: cn.host,
+            pub_key: cn.pub_key,
+            last_seen: cn.last_seen,
+            version: cn.version,
+        }
+    }
+}

common/clients/directory-client/src/presence/mixnodes.rs

@@ -0,0 +1,52 @@
+use serde::{Deserialize, Serialize};
+use std::convert::TryInto;
+use std::io;
+use std::net::ToSocketAddrs;
+use topology::mix;
+
+#[derive(Clone, Debug, Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct MixNodePresence {
+    pub location: String,
+    pub host: String,
+    pub pub_key: String,
+    pub layer: u64,
+    pub last_seen: u64,
+    pub version: String,
+}
+
+impl TryInto<topology::mix::Node> for MixNodePresence {
+    type Error = io::Error;
+
+    fn try_into(self) -> Result<topology::mix::Node, Self::Error> {
+        let resolved_hostname = self.host.to_socket_addrs()?.next();
+        if resolved_hostname.is_none() {
+            return Err(io::Error::new(
+                io::ErrorKind::Other,
+                "no valid socket address",
+            ));
+        }
+
+        Ok(topology::mix::Node {
+            location: self.location,
+            host: resolved_hostname.unwrap(),
+            pub_key: self.pub_key,
+            layer: self.layer,
+            last_seen: self.last_seen,
+            version: self.version,
+        })
+    }
+}
+
+impl From<topology::mix::Node> for MixNodePresence {
+    fn from(mn: mix::Node) -> Self {
+        MixNodePresence {
+            location: mn.location,
+            host: mn.host.to_string(),
+            pub_key: mn.pub_key,
+            layer: mn.layer,
+            last_seen: mn.last_seen,
+            version: mn.version,
+        }
+    }
+}

common/clients/directory-client/src/presence/mod.rs

@@ -0,0 +1,110 @@
+use crate::requests::presence_topology_get::PresenceTopologyGetRequester;
+use crate::{Client, Config, DirectoryClient};
+use log::*;
+use serde::{Deserialize, Serialize};
+use std::convert::TryInto;
+use topology::coco;
+use topology::mix;
+use topology::provider;
+use topology::NymTopology;
+
+pub mod coconodes;
+pub mod mixnodes;
+pub mod providers;
+
+// Topology shows us the current state of the overall Nym network
+#[derive(Clone, Debug, Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct Topology {
+    pub coco_nodes: Vec<coconodes::CocoPresence>,
+    pub mix_nodes: Vec<mixnodes::MixNodePresence>,
+    pub mix_provider_nodes: Vec<providers::MixProviderPresence>,
+}
+
+impl NymTopology for Topology {
+    fn new(directory_server: String) -> Self {
+        debug!("Using directory server: {:?}", directory_server);
+        let directory_config = Config {
+            base_url: directory_server,
+        };
+        let directory = Client::new(directory_config);
+
+        directory
+            .presence_topology
+            .get()
+            .expect("Failed to retrieve network topology.")
+    }
+
+    fn new_from_nodes(
+        mix_nodes: Vec<mix::Node>,
+        mix_provider_nodes: Vec<provider::Node>,
+        coco_nodes: Vec<coco::Node>,
+    ) -> Self {
+        Topology {
+            coco_nodes: coco_nodes.into_iter().map(|node| node.into()).collect(),
+            mix_nodes: mix_nodes.into_iter().map(|node| node.into()).collect(),
+            mix_provider_nodes: mix_provider_nodes
+                .into_iter()
+                .map(|node| node.into())
+                .collect(),
+        }
+    }
+
+    fn mix_nodes(&self) -> Vec<mix::Node> {
+        self.mix_nodes
+            .iter()
+            .filter_map(|x| x.clone().try_into().ok())
+            .collect()
+    }
+
+    fn providers(&self) -> Vec<provider::Node> {
+        self.mix_provider_nodes
+            .iter()
+            .map(|x| x.clone().into())
+            .collect()
+    }
+
+    fn coco_nodes(&self) -> Vec<topology::coco::Node> {
+        self.coco_nodes.iter().map(|x| x.clone().into()).collect()
+    }
+}
+
+#[cfg(test)]
+mod converting_mixnode_presence_into_topology_mixnode {
+    use super::*;
+
+    #[test]
+    fn it_returns_error_on_unresolvable_hostname() {
+        let unresolvable_hostname = "foomp.foomp.foomp:1234";
+
+        let mix_presence = mixnodes::MixNodePresence {
+            location: "".to_string(),
+            host: unresolvable_hostname.to_string(),
+            pub_key: "".to_string(),
+            layer: 0,
+            last_seen: 0,
+            version: "".to_string(),
+        };
+
+        let _result: Result<mix::Node, std::io::Error> = mix_presence.try_into();
+        // assert!(result.is_err()) // This fails only for me. Why?
+        // ¯\_(ツ)_/¯ - works on my machine (and travis)
+    }
+
+    #[test]
+    fn it_returns_resolved_ip_on_resolvable_hostname() {
+        let resolvable_hostname = "nymtech.net:1234";
+
+        let mix_presence = mixnodes::MixNodePresence {
+            location: "".to_string(),
+            host: resolvable_hostname.to_string(),
+            pub_key: "".to_string(),
+            layer: 0,
+            last_seen: 0,
+            version: "".to_string(),
+        };
+
+        let result: Result<topology::mix::Node, std::io::Error> = mix_presence.try_into();
+        assert!(result.is_ok())
+    }
+}

common/clients/directory-client/src/presence/providers.rs

@@ -0,0 +1,72 @@
+use serde::{Deserialize, Serialize};
+use topology::provider;
+
+#[derive(Clone, Debug, Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct MixProviderPresence {
+    pub location: String,
+    pub client_listener: String,
+    pub mixnet_listener: String,
+    pub pub_key: String,
+    pub registered_clients: Vec<MixProviderClient>,
+    pub last_seen: u64,
+    pub version: String,
+}
+
+impl Into<topology::provider::Node> for MixProviderPresence {
+    fn into(self) -> topology::provider::Node {
+        topology::provider::Node {
+            location: self.location,
+            client_listener: self.client_listener.parse().unwrap(),
+            mixnet_listener: self.mixnet_listener.parse().unwrap(),
+            pub_key: self.pub_key,
+            registered_clients: self
+                .registered_clients
+                .into_iter()
+                .map(|c| c.into())
+                .collect(),
+            last_seen: self.last_seen,
+            version: self.version,
+        }
+    }
+}
+
+impl From<topology::provider::Node> for MixProviderPresence {
+    fn from(mpn: provider::Node) -> Self {
+        MixProviderPresence {
+            location: mpn.location,
+            client_listener: mpn.client_listener.to_string(),
+            mixnet_listener: mpn.mixnet_listener.to_string(),
+            pub_key: mpn.pub_key,
+            registered_clients: mpn
+                .registered_clients
+                .into_iter()
+                .map(|c| c.into())
+                .collect(),
+            last_seen: mpn.last_seen,
+            version: mpn.version,
+        }
+    }
+}
+
+#[derive(Clone, Debug, Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct MixProviderClient {
+    pub pub_key: String,
+}
+
+impl Into<topology::provider::Client> for MixProviderClient {
+    fn into(self) -> topology::provider::Client {
+        topology::provider::Client {
+            pub_key: self.pub_key,
+        }
+    }
+}
+
+impl From<topology::provider::Client> for MixProviderClient {
+    fn from(mpc: topology::provider::Client) -> Self {
+        MixProviderClient {
+            pub_key: mpc.pub_key,
+        }
+    }
+}

common/clients/directory-client/src/requests/metrics_mixes_get.rs

@@ -1,4 +1,4 @@
-use crate::clients::directory::metrics::PersistedMixMetric;
+use crate::metrics::PersistedMixMetric;
 
 pub struct Request {
     base_url: String,

common/clients/directory-client/src/requests/metrics_mixes_post.rs

@@ -1,4 +1,4 @@
-use crate::clients::directory::metrics::MixMetric;
+use crate::metrics::MixMetric;
 use reqwest::Response;
 
 pub struct Request {
@@ -69,7 +69,7 @@ mod metrics_get_request {
 
     #[cfg(test)]
     mod fixtures {
-        use crate::clients::directory::metrics::MixMetric;
+        use crate::metrics::MixMetric;
 
         pub fn new_metric() -> MixMetric {
             MixMetric {

common/clients/directory-client/src/requests/presence_coconodes_post.rs

@@ -1,4 +1,4 @@
-use crate::clients::directory::presence::CocoPresence;
+use crate::presence::coconodes::CocoPresence;
 use reqwest::Response;
 
 pub struct Request {
@@ -73,13 +73,15 @@ mod metrics_get_request {
 
     #[cfg(test)]
     mod fixtures {
-        use crate::clients::directory::presence::CocoPresence;
+        use crate::presence::coconodes::CocoPresence;
 
         pub fn new_presence() -> CocoPresence {
             CocoPresence {
+                location: "foomp".to_string(),
                 host: "foo.com".to_string(),
                 pub_key: "abc".to_string(),
                 last_seen: 666,
+                version: "0.2.0".to_string(),
             }
         }
     }

common/clients/directory-client/src/requests/presence_mixnodes_post.rs

@@ -1,4 +1,4 @@
-use crate::clients::directory::presence::MixNodePresence;
+use crate::presence::mixnodes::MixNodePresence;
 use reqwest::Response;
 
 pub struct Request {
@@ -73,14 +73,16 @@ mod metrics_get_request {
 
     #[cfg(test)]
     mod fixtures {
-        use crate::clients::directory::presence::MixNodePresence;
+        use crate::presence::mixnodes::MixNodePresence;
 
         pub fn new_presence() -> MixNodePresence {
             MixNodePresence {
+                location: "foomp".to_string(),
                 host: "foo.com".to_string(),
                 pub_key: "abc".to_string(),
                 layer: 1,
                 last_seen: 0,
+                version: "0.1.0".to_string(),
             }
         }
     }

common/clients/directory-client/src/requests/presence_providers_post.rs

@@ -1,4 +1,4 @@
-use crate::clients::directory::presence::MixProviderPresence;
+use crate::presence::providers::MixProviderPresence;
 use reqwest::Response;
 
 pub struct Request {
@@ -72,13 +72,17 @@ mod metrics_get_request {
     }
     #[cfg(test)]
     mod fixtures {
-        use crate::clients::directory::presence::MixProviderPresence;
+        use crate::presence::providers::MixProviderPresence;
 
         pub fn new_presence() -> MixProviderPresence {
             MixProviderPresence {
-                host: "foo.com".to_string(),
+                location: "foomp".to_string(),
+                client_listener: "foo.com".to_string(),
+                mixnet_listener: "foo.com".to_string(),
                 pub_key: "abc".to_string(),
                 registered_clients: vec![],
+                last_seen: 0,
+                version: "0.1.0".to_string(),
             }
         }
     }

common/clients/directory-client/src/requests/presence_topology_get.rs

@@ -1,4 +1,4 @@
-use crate::clients::directory::presence::Topology;
+use crate::presence::Topology;
 
 pub struct Request {
     base_url: String,
@@ -72,71 +72,93 @@ mod topology_requests {
             r#"{
                    "cocoNodes": [
                      {
+                       "location": "unknown",
                        "host": "3.8.244.109:4000",
                        "pubKey": "AAAAAAAAAAEKwAECSqKy8I8KkSYIBSctxRBRxuR61PpAOwK0UQtkeuPRdwusAyaoBbvv1IBWyMEhvbgT4CtgUnGfYH2s06CIJ09lWWvQ0Jkgthq12mG73H9QSTNM8RITlF1X5ax9BV0EK34M5dUncn1uEYzJzcbaLjUarf2bqoy906dtQpppUWDRLJI6ycw7rKKJ4ZNUhgi4KAEGBsSgLqc0zDKs0rArwouZyz4ofoWnY68mdJKrVy6Zqz83DSdc7B2hqqkHX_Bfeb4SwAEFuhRpy4HfcuxwcRI9sIWMo_LVmbk19g1gfMRlBrmZqoEQL6rDApVLZ9eMp-5IQK8WLlZpWf4Zjy7kZolARAyp_rHUQkH4PrDjgoPrKbm6qK_iejYpL7qx28Q3VeInMpwMIMaSbbW9y36sEVtGc2I0Iu5vS0sp8ESiVlQ5NaBz72deZ8oKJJ4IEPPHP99-b0UQX80fVIrNM88mMzKy0bHri9NFlmIG-e0G1cqmw_ry3XWGQkcr1M5RuNa6oX50w5QawAEVxd5FP5bE8bS4x54Csof11sQWUTwMp6Q7_3H7ZCTSlKSqujlOhmfqSHfGPO2sDIYPHDhDzjakZpKAZWWhn_hiR6DfPpomQ01ZYUhVKKSMxz7_VPjsQplP0bZXA2gfnkADUN8UQ0N9g_usIw73r4aZsOviMsRM8oByvsjVfUWc4_HTLSdnQyImFkHz9CiCmrIYL2dYQRePRatWggvBAyeRzntxI4jDqLKiBdi54ZlAKgV6MCRaJ7Bu7BtmLXrtK4sawAED3QYxuvOSZrbZdUr4yG-U9yVvJ9Klkf-5Mo4EYp3qTL2KBB6_LrZepjAQqp486YkZ03mTIezcsZ48EboXVTWKBZ3QnTI5tX-j4gGxQb7klOJc97qJkDxsvpz4F0ChgCUIZhpIItWHia7_R3Gi-b5siLIdQdUho9isn3kiDGm6t0NED2Bgy3ZxxQwzqsBZm4kPr2_fPX4YyvIoP9895YcGjZyE5iiRC_TE41RJmB1GZYdxegTMq3lNDllKgiqaiPgawAEJASDkmZHTwlg9YOev5OWpQD-FnhPkqVNo_QcDyRu9eoGcWSGFp2sYqjG2SpmiXq0VNnAO7AcKxRzDFu7TjfhlU3Kt0uTKIcrWVU1zFNbJNMjYEq90pp50nowwx8INz20IXET2ZNX6kIXYFCsEvPLZFlG2OoL6xg3uQS1qMl3lIS_VxdO_JfVe0rT65WsJ_P4Nkc1jYiuNPHY6d_iFO0BVYqX0sOCX73GC_TT13BR0jnPwDAVw0rGtYHsXBb8TKOsawAEZIClauuT1V3qOZnb7uRZhFXO-PKTxgc1LCzJt2ChOrMZaBpjlkf3IPpJ2UF4JH4kGaDeBf2k_S-FLAs3drK21efbi5P6_a4QTxAiiRimXGoQIyvOg462s6kP_ZRFufo8YYQHS4olaOeqU4564dNskg_uBPsFMz_2GNOhmn_15cJqP1jfkyD49Z16GTS5YLHgVl9bJKqyvLuypsToLbt1BJzipEP0L2OohuRm-_MvqvwwWKyjNQsubgee1K728d9AawAEBkGggcNVCtXyhoSqi3_w0tVxtkAYeud8sBeAtZHGs06me_QL8co0MFLlO-zdkUb4ZBq08rFEbgLOma8_3whleM8NIPaHNISp1q3IsIhB5zdXcZoGsqLixODBFHtID3YEHAlr4f9T_yh11yJ95xGCl_6Y37hpwLQVGyrfSfccM24mVFqnV3TT5Wdq3ile-jesUx1Q2G1yK_xVqc6itmk-kDuBjyZgzYi1-jsIXAjnhM9G7t8J_Bv5yGGZhLK2dCzM=",
                        "type": "validator",
-                       "lastSeen": 1575915097085539300
+                       "lastSeen": 1575915097085539300,
+                       "version": "0.1.0"
                      },
                      {
+                       "location": "unknown",
                        "host": "3.9.129.61:4000",
                        "pubKey": "AAAAAAAAAAMKwAECSqKy8I8KkSYIBSctxRBRxuR61PpAOwK0UQtkeuPRdwusAyaoBbvv1IBWyMEhvbgT4CtgUnGfYH2s06CIJ09lWWvQ0Jkgthq12mG73H9QSTNM8RITlF1X5ax9BV0EK34M5dUncn1uEYzJzcbaLjUarf2bqoy906dtQpppUWDRLJI6ycw7rKKJ4ZNUhgi4KAEGBsSgLqc0zDKs0rArwouZyz4ofoWnY68mdJKrVy6Zqz83DSdc7B2hqqkHX_Bfeb4SwAEEv6RMevAQmLGkeK0uJKnMPPAtm8GgXjWSQijYdnxlPh5SJSNeJUbPZKWFFWdk8yIFXKa8jnzETtdGFKgUUt5AVUDpTBmEdwaHCzlFhXrttshy0V5OhPUlV8cGABmxbagMYm0bFPg0r-snSkrB9YG6wqJYQVeIMOCGYCPbHmDA8R_0-h8VkRKWs1d9KvQOK4kShqgZtYN71KJW8uDE4q2jsGDVvxFt1AgmU9b93xsXF17KrpZy5WxlLZ73HtnTD_oawAED4vd_rK-Kx_n8x_OdDiiEOPUlYDlDCQUqenU9XHKH3B6ijfkJ368wd3LDDVStjDwNORrAyUSw_VlSNUpd1XLC8d17gTaIq5ZI2fWuwwZaoN1JCsYU8fQ6USgtIehQX7IPP8EkFuNmuCBCmpr4schtYniGe9J8Q4dsV-TYPr2uLJkdx1r7luzF--I22k7NfQQM14QDci_0kgrgmZ54CJGkjXyOhCppBXg3fqLC6aFvT3ZocfiiXBJt0huGgPMDtYsawAECLh8KUdNsDolERwJ8v04bS5jI_KKf7uUnCHWuCELwbJSUI3OK1ufS1qSpauvSzVQSbrhEzrEfwQn4VtxQxJlX4UdDU-R-hafiZvVC6DLLAbuORBAC3FScn9W58CnezH4DvCp_w7nftDfdxeuungbZT9XaxS3iNC6PnFsWF6WM3DxMwrzOrFe6wEEoTSPe1mcUDrtwM5UksIvJr6MBRAXrdl0IdBTQr7cLwKe_KYi4siwdjfJEJtOh7oxQBxBg2UkawAEJAPZK2Gg2MQwpxdDT24lNQHF7FVfkO_LuhJwn0RbwNDSVeA4P6-tWL5TkCpqr8xYHfwQ6Z3ILfpGCZr8PspwIoRzqZHQ16f8Pq9xnr0hLEI9BOQU0FS2EtuyPgju5iwsAJAfehUzu6kNLphuLGsXoIZdXDG5mbylwh9JzAVXTwgaR0hNqyXVJxgbt7jcYaSEBFcMGV-hjXyVVNzBleE-G9o_noI_KWU4Ce7K-qOMcewMKfy_VEw-gVaD6dHz6AMoawAEE9XuOLwRttvKybAssZ9gsK-_YRUwuFOeRDIr3NX___9bx6pCc18adCIlH_8EJWFwXZ05ZpNNE88mYx7ZQ3aqaArZJRoWeZeKhqH_s05V10xbzkYX71G5cqz--8vr9ZlQRb2BeETF_Tdq_PLk7qbT8WTGIoq7ZwyDRQTgzvkCgyzj_hBLh2o7sSVNgUo38SFUTMn7YtvVFYlSrTDE3WKE-T-nh5SWdDBxgDTc3Bw8JpzNH-WkoJ4Lim7sB4Op1gEUawAEW4-kenlffwsNr_3b3aV0YuusLpxB03sxPzQ5B0CWNiVtbja1Z4tWhKGUUrdq_eUgMV0y5Of-BqNi5FspAQnhJBFSSxtOzRGV1h3qyUTksfZyed9z8zPI-ZPP9XXm7hYgJgDz_kxte-NfS9UG9q5AZetHUN4kGxXutjjzfUQZ9yTvhBKgKgTI2Dp_R_jZrWQ8F1BoWzIJzjddT1K2MvCQEkARYw08isbOeFmCwgVUcjxYZO45WyOmLQA7QJRL9WvA=",
                        "type": "validator",
-                       "lastSeen": 1575915097388409000
+                       "lastSeen": 1575915097388409000,
+                       "version": "0.1.0"
                      },
                      {
+                       "location": "unknown",
                        "host": "3.9.222.1:4000",
                        "pubKey": "AAAAAAAAAAQKwAECSqKy8I8KkSYIBSctxRBRxuR61PpAOwK0UQtkeuPRdwusAyaoBbvv1IBWyMEhvbgT4CtgUnGfYH2s06CIJ09lWWvQ0Jkgthq12mG73H9QSTNM8RITlF1X5ax9BV0EK34M5dUncn1uEYzJzcbaLjUarf2bqoy906dtQpppUWDRLJI6ycw7rKKJ4ZNUhgi4KAEGBsSgLqc0zDKs0rArwouZyz4ofoWnY68mdJKrVy6Zqz83DSdc7B2hqqkHX_Bfeb4SwAECh9xcxpjOp1r7kiNIgrI9GgAlvXwgHkTchOxUiyOzTq6FDWdGN64KiC3NDeyGTg8FmzvGzS3jREeJqOdr4G9ZGtWkauAITgLFiH62t-YntRslhr8_1shxlmzKiNKJN_QFflEq79pZIlWtp3N8LIHMvXRtl-zt2DMze4s02XDmEkviyVE4CkQUDtCc-2MfPT4JcmEFqtFIxjrXn18SbYg3c6XUQHsGIkuDrKuCTRlpC8kvmM0uVoIeWdmwDlZk4jUawAEJhRwK5ozjqIWRP1bFzBPS9VhaJnfKU9PeFYtN5beiAHrYr2ylIB3yDfmAQUdKDowDUm5nfJATejEjEnrTGxh70QtfoNV391rSns3F71tBwY62KLaNr8qnVfeSFHV3FcQTMHHF_8mDb5_11Rj6aiMvW0y6eetHo7CDPMdEyDPmok_U2ZM5BzOUnwjT21HtnvcKxKKwHJ_QGfnAHPyDIhNOMgxJCrVazOidLCHeYGpyCLw1ipeTyKOQX0_ByB8dH6AawAEGV1GuF5SSlT67B1ityPJK2ZwXjeeKB4gGdCG3qRtWxLTZfGhVm7YAYm2f5tw_wrsJAZ9FubVhateGg0ZN67NxZtsvOOejXz6743f7ijnQopPgd_8pH-iVf6BEcSO8ZdcHxNRUTayzjVLs99bwMo2zaPevW4X4G_bN4mh---aPkdGYHwaiklzUhqJ-eqycrYAFyjyEXaPBXLQm1rpczqluNvnKbd8Q9LZWukgm7_uWv_HxufIvdWgoq8bAt78UU3oawAEP9VDehhqrQG5-WHMB66XVxo1TgMM8aVV0SwAq3lCRkpiFBz_9kw8T1F9Hx2AiNrEGT1QLbdMkpms1cG_5gBBahQofdt_NmUs1jfTFXY9iyMy1Q7A6ZYaLP8Z6q-orc1cKqySY-BJZQ_CpGFfXS0OVniFDQ6v78ytPK7K-yRgT1PxFgm3rZqrG0Tjbrpsg2PUL5S5fuXfMhUosP0uoLj0D1guWAR9Y7kfFBIXaTSFMoa8fghVBUTRNhK9f72a8SxQawAEOiv71taLjKqaaWQ_QjcDhWbvjG1EnsCyI0toNjGkcF19x4Vk-5NC96_4ioUGz404IC0XN03roRnibRT_78D9vZFVCWCqve9EjdF5TcApx03zIP4JT2g2q0MKIGgGrwt4Pz6LO6yOfMm7B8Yraps8IV-nP1w7K1m9XKP_FvH8egl5GHJe-_omlC2YyL_b28jMLENbxDFD-3KPjZFBhSLrRukX2PlayYTwEiTtokA2R9_11vQvJgP8KFEjGHg6zsAMawAEBn2H_hz2knb8ltnpEA5YSKVcV3nUtojkCNi_WUz7xUKd7efw1oI_lbnKrS7HkyC0JkQUZ1pCWUlSXNmgjMEhsn823a1LFzpV7rOv4vayYvvFX61hB9R78VjpyxJiYpDwRZLiUY3AK4WY8NqFDbjXR7rT4CkFHEf-VhSQQ8ZNvlpod1nmeVQVizHH9e7Tq7wsWz-LWEk3Hx6LmcrgDsL79LZYG9JXU5IdvG8RvLNx9cSwEI8yxcchpISAaot7UoYQ=",
                        "type": "validator",
-                       "lastSeen": 1575915094734973000
+                       "lastSeen": 1575915094734973000,
+                       "version": "0.1.0"
                      },
                      {
+                       "location": "unknown",
                        "host": "3.9.102.214:4000",
                        "pubKey": "AAAAAAAAAAIKwAECSqKy8I8KkSYIBSctxRBRxuR61PpAOwK0UQtkeuPRdwusAyaoBbvv1IBWyMEhvbgT4CtgUnGfYH2s06CIJ09lWWvQ0Jkgthq12mG73H9QSTNM8RITlF1X5ax9BV0EK34M5dUncn1uEYzJzcbaLjUarf2bqoy906dtQpppUWDRLJI6ycw7rKKJ4ZNUhgi4KAEGBsSgLqc0zDKs0rArwouZyz4ofoWnY68mdJKrVy6Zqz83DSdc7B2hqqkHX_Bfeb4SwAEOVCUN3EwiVroS5-TOq2o7hYSxphK9X0G23N-IBZ0Tr1Rl8XEiJ-OEy0rqnAKwmhAZJWnx3u8oXqbZtOWIZmzQSpcoxhgwfhdmTZJCqT2RVzZyeFItX4sVeilEP3z2xdsJs8-a1kg6UZnx1s1BNLBo7eZrreZygWojPCIDBn03fSAflXoVc5PpY2CGy5MA_IgWgSYBHDdoZEtigp_amjqK7Us44Db20XpLxMXfbahiqa7WKNnMgi6Ca2H67VtaaD8awAEF3zbE1nZRAa7a8vbU25c80YBYJBaW8P6FwXQI-K0Xk5MakwYeMMnIrm6w6IS_0XAO5YlD453GLqnxY8H1BEnRpfOnT7PE4el9mJ8MuYQMo6R2up0lGCmYM0YA9FORjroM3ng69SEPfJPCReG7LfJkERl_m2U403ertDRBYrlqCDagDfyI500srBcMrjSvV3oNouyyx3yZUrjLQfbHhDteQFsYdmakJs8Y-Q9-5MXCcrz6Qa4xwv522Euv0CCxkHcawAEYjfsU_zDhUZA1ey1aquWXlFOnx-iEALqxW1slDYHwQ1M2SILc-v_E6i1doa5e_bAZHVezBHFAlaNAVedNyHFFJxYAqAK3hbzbvl2glw3Q6h_rTXElymloqtaqVFIJ-oUWWOHsZBmu8EDA-HzvGCiBa_GbRaVfh2lE4ObeMXoJrEm_5dbxxeEic2l3IYeIz40N9ooQQOkQcOZdY4AXWYCavIAwWEJBjLtptJgCLu9a_zM1S5GsiyJHpdDs46WbP0EawAEWZ-95Sf0YAHujxRNLdXgpqe0ZF8loVwzZfvyMvqaxF1Ug274BqHuY_c5NdPAzuqoTwjfEn8NKEoaNqlumM75FUYbaTd7mXvk4WVYWjVnkO40dfQjRB7DYhvj0LBlbndAJ4wJIA2ilPYgjZsXVbNNh3e2j3u9eABd0VaFMbSb8Sz5_31r8HzoWmPJs3HiyuyANGFUA6CvAnMN6K3b-D8BhFZU_nPUTgu80o8_n6LQt-XWbaC_mTHzsnOjzBiPJxlYawAEW3bmOEtStH2T8q7vMkhchImp2-hg9MFYGBmEe9sSByTn3NUf8eksqXOC1dUjHkXoZm298FgUYLkNdnlxWpf993j5mEDoFxjcTB7scBD7k6nu6Nrs_wK0-seS8gsHrx9UK7GwAsi10q82Cm4PFyAtrWjmy_d9WLHuZt6VIOKunTs8cf0FwNUiMcvZsruqIFJcP7iWxdiFdUkh65P_iCz1ZEjJcj2GEZoq4v3a3by1aizGPaaiKc1jd_T-XJg_YpncawAEWnstu5b9WiZv0x8xfsiMk6YRlU0Cnj5svxLLXz_8drvwAa--GBY5yH0ke2EM6udMEi2EPeFcGTe6Sjs0YEhSbY7Uad_8suD2J4tIWJSWBbiyvh7rSqzv57m7BlsVcHfQJn_wNH-UlC9xkx8vg-LwfN8_FlxvHNPTc7XZG3lKYbwpUWlZxAziOYT1VQ-2K2bQQBBMdix-ht_SjccL1Dc2dP5kDazQ8yZV_8xnyeheazEedWe63uutfkHlZRg9YwP8=",
                        "type": "validator",
-                       "lastSeen": 1575915094967382800
+                       "lastSeen": 1575915094967382800,
+                       "version": "0.1.0"
                      }
                    ],
                    "mixNodes": [
                      {
+                       "location": "unknown",
                        "host": "35.176.155.107:1789",
                        "pubKey": "zSob16499jT7C3S3ky4GihNOjlU6aLfSRkf1xAxOwV0=",
                        "layer": 3,
-                       "lastSeen": 1575915096805374500
+                       "lastSeen": 1575915096805374500,
+                       "version": "0.1.0"
                      },
                      {
+                       "location": "unknown",
                        "host": "18.130.86.190:1789",
                        "pubKey": "vCdpFc0NvW0NSqsuTxtjFtiSY35aXesgT3JNA8sSIXk=",
                        "layer": 1,
-                       "lastSeen": 1575915097370376000
+                       "lastSeen": 1575915097370376000,
+                       "version": "0.1.0"
                      },
                      {
+                       "location": "unknown",
                        "host": "3.10.22.152:1789",
                        "pubKey": "OwOqwWjh_IlnaWS2PxO6odnhNahOYpRCkju50beQCTA=",
                        "layer": 1,
-                       "lastSeen": 1575915097639423500
+                       "lastSeen": 1575915097639423500,
+                       "version": "0.1.0"
                      },
                      {
+                       "location": "unknown",
                        "host": "35.178.213.77:1789",
                        "pubKey": "nkkrUjgL8UJk05QydvWvFSvtRB6nmeV8RMvH5540J3s=",
                        "layer": 2,
-                       "lastSeen": 1575915097895166500
+                       "lastSeen": 1575915097895166500,
+                       "version": "0.1.0"
                      },
                      {
+                       "location": "unknown",
                        "host": "52.56.99.196:1789",
                        "pubKey": "whHuBuEc6zyOZOquKbuATaH4Crml61V_3Y-MztpWhF4=",
                        "layer": 2,
-                       "lastSeen": 1575915096255174700
+                       "lastSeen": 1575915096255174700,
+                       "version": "0.1.0"
                      },
                      {
+                       "location": "unknown",
                        "host": "3.9.12.238:1789",
                        "pubKey": "vk5Sr-Xyi0cTbugACv8U42ZJ6hs6cGDox0rpmXY94Fc=",
                        "layer": 3,
-                       "lastSeen": 1575915096497827600
+                       "lastSeen": 1575915096497827600,
+                       "version": "0.1.0"
                      }
                    ],
                    "mixProviderNodes": [
                      {
-                       "host": "3.8.176.11:1789",
+                       "location": "unknown",
+                       "clientListener": "3.8.176.11:8888",
+                       "mixnetListener": "3.8.176.11:9999",
                        "pubKey": "54U6krAr-j9nQXFlsHk3io04_p0tctuqH71t7w_usgI=",
                        "registeredClients": [
                          {
@@ -248,10 +270,13 @@ mod topology_requests {
                            "pubKey": "COGdpfhmzNGR6YX820GqJIkjOihL8mr6-h-d3JlTDFA="
                          }
                        ],
-                       "lastSeen": 1575915097358694100
+                       "lastSeen": 1575915097358694100,
+                       "version": "0.1.0"
                      },
                      {
-                       "host": "35.178.212.193:1789",
+                       "location": "unknown",
+                       "clientListener": "3.8.176.12:8888",
+                       "mixnetListener": "3.8.176.12:9999",
                        "pubKey": "sA-sxi038pEbGy4lgZWG-RdHHDkA6kZzu44G0LUxFSc=",
                        "registeredClients": [
                          {
@@ -297,7 +322,8 @@ mod topology_requests {
                            "pubKey": "w1bfLpnd3rWu5JczB0nQfnE2S6nUCbx2AA7HDE48DQo="
                          }
                        ],
-                       "lastSeen": 1575915097869025000
+                       "lastSeen": 1575915097869025000,
+                       "version": "0.1.0"
                      }
                    ]
                  }"#.to_string()

common/clients/mix-client/Cargo.toml

@@ -0,0 +1,23 @@
+[package]
+name = "mix-client"
+version = "0.1.0"
+authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+log = "0.4.8"
+pretty_env_logger = "0.3"
+rand = "0.7.2"
+rand_distr = "0.2.2"
+tokio = { version = "0.2", features = ["full"] }
+
+## internal
+nymsphinx = {path = "../../nymsphinx"}
+topology = {path = "../../topology"}
+
+## will be moved to proper dependencies once released
+sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" }
+# sphinx = { path = "../../../../sphinx"}
+

common/clients/mix-client/src/lib.rs

@@ -1,11 +1,15 @@
-use sphinx::route::NodeAddressBytes;
+use log::*;
 use sphinx::SphinxPacket;
 use std::net::SocketAddr;
 use tokio::prelude::*;
 
+pub mod packet;
+pub mod poisson;
+
 pub struct MixClient {}
 
 impl MixClient {
+    #[allow(clippy::new_without_default)]
     pub fn new() -> MixClient {
         MixClient {}
     }
@@ -17,8 +21,7 @@ impl MixClient {
         mix_addr: SocketAddr,
     ) -> Result<(), Box<dyn std::error::Error>> {
         let bytes = packet.to_bytes();
-
-        println!("socket addr: {:?}", mix_addr);
+        debug!("Sending to the following address: {:?}", mix_addr);
 
         let mut stream = tokio::net::TcpStream::connect(mix_addr).await?;
         stream.write_all(&bytes[..]).await?;

common/clients/mix-client/src/packet.rs

@@ -0,0 +1,119 @@
+use nymsphinx::addressing::nodes::{NymNodeRoutingAddress, NymNodeRoutingAddressError};
+use sphinx::route::{Destination, DestinationAddressBytes, SURBIdentifier};
+use sphinx::SphinxPacket;
+use std::convert::TryFrom;
+use std::net::SocketAddr;
+use std::time;
+use topology::{NymTopology, NymTopologyError};
+
+pub const LOOP_COVER_MESSAGE_PAYLOAD: &[u8] = b"The cake is a lie!";
+
+#[derive(Debug)]
+pub enum SphinxPacketEncapsulationError {
+    NoValidProvidersError,
+    InvalidTopologyError,
+    SphinxEncapsulationError(sphinx::header::SphinxUnwrapError),
+    InvalidFirstMixAddress,
+}
+
+impl From<topology::NymTopologyError> for SphinxPacketEncapsulationError {
+    fn from(_: NymTopologyError) -> Self {
+        use SphinxPacketEncapsulationError::*;
+        InvalidTopologyError
+    }
+}
+
+// it is correct error we're converting from, it just has an unfortunate name
+// related issue: https://github.com/nymtech/sphinx/issues/40
+impl From<sphinx::header::SphinxUnwrapError> for SphinxPacketEncapsulationError {
+    fn from(err: sphinx::header::SphinxUnwrapError) -> Self {
+        use SphinxPacketEncapsulationError::*;
+        SphinxEncapsulationError(err)
+    }
+}
+
+impl From<NymNodeRoutingAddressError> for SphinxPacketEncapsulationError {
+    fn from(_: NymNodeRoutingAddressError) -> Self {
+        use SphinxPacketEncapsulationError::*;
+        InvalidFirstMixAddress
+    }
+}
+
+#[deprecated(note = "please use loop_cover_message_route instead")]
+pub fn loop_cover_message<T: NymTopology>(
+    our_address: DestinationAddressBytes,
+    surb_id: SURBIdentifier,
+    topology: &T,
+    average_delay: time::Duration,
+) -> Result<(SocketAddr, SphinxPacket), SphinxPacketEncapsulationError> {
+    let destination = Destination::new(our_address, surb_id);
+
+    #[allow(deprecated)]
+    encapsulate_message(
+        destination,
+        LOOP_COVER_MESSAGE_PAYLOAD.to_vec(),
+        topology,
+        average_delay,
+    )
+}
+
+pub fn loop_cover_message_route(
+    our_address: DestinationAddressBytes,
+    surb_id: SURBIdentifier,
+    route: Vec<sphinx::route::Node>,
+    average_delay: time::Duration,
+) -> Result<(SocketAddr, SphinxPacket), SphinxPacketEncapsulationError> {
+    let destination = Destination::new(our_address, surb_id);
+
+    encapsulate_message_route(
+        destination,
+        LOOP_COVER_MESSAGE_PAYLOAD.to_vec(),
+        route,
+        average_delay,
+    )
+}
+
+#[deprecated(note = "please use encapsulate_message_route instead")]
+pub fn encapsulate_message<T: NymTopology>(
+    recipient: Destination,
+    message: Vec<u8>,
+    topology: &T,
+    average_delay: time::Duration,
+) -> Result<(SocketAddr, SphinxPacket), SphinxPacketEncapsulationError> {
+    let mut providers = topology.providers();
+    if providers.is_empty() {
+        return Err(SphinxPacketEncapsulationError::NoValidProvidersError);
+    }
+    // unwrap is fine here as we asserted there is at least single provider
+    let provider = providers.pop().unwrap().into();
+
+    let route = topology.route_to(provider)?;
+
+    let delays = sphinx::header::delays::generate_from_average_duration(route.len(), average_delay);
+
+    // build the packet
+    let packet = sphinx::SphinxPacket::new(message, &route[..], &recipient, &delays)?;
+
+    // we know the mix route must be valid otherwise we would have already returned an error
+    let first_node_address =
+        NymNodeRoutingAddress::try_from(route.first().unwrap().address.clone())?;
+
+    Ok((first_node_address.into(), packet))
+}
+
+pub fn encapsulate_message_route(
+    recipient: Destination,
+    message: Vec<u8>,
+    route: Vec<sphinx::route::Node>,
+    average_delay: time::Duration,
+) -> Result<(SocketAddr, SphinxPacket), SphinxPacketEncapsulationError> {
+    let delays = sphinx::header::delays::generate_from_average_duration(route.len(), average_delay);
+
+    // build the packet
+    let packet = sphinx::SphinxPacket::new(message, &route[..], &recipient, &delays)?;
+
+    let first_node_address =
+        NymNodeRoutingAddress::try_from(route.first().unwrap().address.clone())?;
+
+    Ok((first_node_address.into(), packet))
+}

common/clients/mix-client/src/poisson.rs

@@ -0,0 +1,10 @@
+use rand_distr::{Distribution, Exp};
+use std::time;
+
+pub fn sample(average_duration: time::Duration) -> time::Duration {
+    // this is our internal code used by our traffic streams
+    // the error is only thrown if average delay is less than 0, which will never happen
+    // so call to unwrap is perfectly safe here
+    let exp = Exp::new(1.0 / average_duration.as_nanos() as f64).unwrap();
+    time::Duration::from_nanos(exp.sample(&mut rand::thread_rng()).round() as u64)
+}

common/clients/multi-tcp-client/Cargo.toml

@@ -0,0 +1,12 @@
+[package]
+name = "multi-tcp-client"
+version = "0.1.0"
+authors = ["Jedrzej Stuczynski <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+futures = "0.3"
+log = "0.4.8"
+tokio = { version = "0.2", features = ["full"] }

common/clients/multi-tcp-client/src/connection_manager/mod.rs

@@ -0,0 +1,148 @@
+use crate::connection_manager::reconnector::ConnectionReconnector;
+use crate::connection_manager::writer::ConnectionWriter;
+use futures::channel::{mpsc, oneshot};
+use futures::future::{abortable, AbortHandle};
+use futures::task::Poll;
+use futures::{AsyncWriteExt, StreamExt};
+use log::*;
+use std::io;
+use std::net::SocketAddr;
+use std::time::Duration;
+use tokio::runtime::Handle;
+
+mod reconnector;
+mod writer;
+
+pub(crate) type ResponseSender = Option<oneshot::Sender<io::Result<()>>>;
+
+pub(crate) type ConnectionManagerSender = mpsc::UnboundedSender<(Vec<u8>, ResponseSender)>;
+type ConnectionManagerReceiver = mpsc::UnboundedReceiver<(Vec<u8>, ResponseSender)>;
+
+enum ConnectionState<'a> {
+    Writing(ConnectionWriter),
+    Reconnecting(ConnectionReconnector<'a>),
+}
+
+pub(crate) struct ConnectionManager<'a> {
+    conn_tx: ConnectionManagerSender,
+    conn_rx: ConnectionManagerReceiver,
+
+    address: SocketAddr,
+
+    maximum_reconnection_backoff: Duration,
+    reconnection_backoff: Duration,
+
+    state: ConnectionState<'a>,
+}
+
+impl<'a> Drop for ConnectionManager<'a> {
+    fn drop(&mut self) {
+        debug!("Connection manager to {:?} is being dropped", self.address)
+    }
+}
+
+impl<'a> ConnectionManager<'static> {
+    pub(crate) async fn new(
+        address: SocketAddr,
+        reconnection_backoff: Duration,
+        maximum_reconnection_backoff: Duration,
+        connection_timeout: Duration,
+    ) -> ConnectionManager<'a> {
+        let (conn_tx, conn_rx) = mpsc::unbounded();
+
+        // the blocking call here is fine as initially we want to wait the timeout interval (at most) anyway:
+        let tcp_stream_res = std::net::TcpStream::connect_timeout(&address, connection_timeout);
+
+        let initial_state = match tcp_stream_res {
+            Ok(stream) => {
+                let tokio_stream = tokio::net::TcpStream::from_std(stream).unwrap();
+                debug!("managed to establish initial connection to {}", address);
+                ConnectionState::Writing(ConnectionWriter::new(tokio_stream))
+            }
+            Err(e) => {
+                warn!("failed to establish initial connection to {} within {:?} ({}). Going into reconnection mode", address, connection_timeout, e);
+                ConnectionState::Reconnecting(ConnectionReconnector::new(
+                    address,
+                    reconnection_backoff,
+                    maximum_reconnection_backoff,
+                ))
+            }
+        };
+
+        ConnectionManager {
+            conn_tx,
+            conn_rx,
+            address,
+            maximum_reconnection_backoff,
+            reconnection_backoff,
+            state: initial_state,
+        }
+    }
+
+    async fn run(mut self) {
+        while let Some(msg) = self.conn_rx.next().await {
+            let (msg_content, res_ch) = msg;
+            let res = self.handle_new_message(msg_content).await;
+            if let Some(res_ch) = res_ch {
+                if let Err(e) = res_ch.send(res) {
+                    error!(
+                        "failed to send response on the channel to the caller! - {:?}",
+                        e
+                    );
+                }
+            }
+        }
+    }
+
+    /// consumes Self and returns channel for communication as well as an `AbortHandle`
+    pub(crate) fn start_abortable(self, handle: &Handle) -> (ConnectionManagerSender, AbortHandle) {
+        let sender_clone = self.conn_tx.clone();
+        let (abort_fut, abort_handle) = abortable(self.run());
+
+        handle.spawn(async move { abort_fut.await });
+
+        (sender_clone, abort_handle)
+    }
+
+    async fn handle_new_message(&mut self, msg: Vec<u8>) -> io::Result<()> {
+        if let ConnectionState::Reconnecting(conn_reconnector) = &mut self.state {
+            // do a single poll rather than await for future to completely resolve
+            let new_connection = match futures::poll!(conn_reconnector) {
+                Poll::Pending => {
+                    return Err(io::Error::new(
+                        io::ErrorKind::BrokenPipe,
+                        "connection is broken - reconnection is in progress",
+                    ))
+                }
+                Poll::Ready(conn) => conn,
+            };
+
+            debug!("Managed to reconnect to {}!", self.address);
+            self.state = ConnectionState::Writing(ConnectionWriter::new(new_connection));
+        }
+
+        // we must be in writing state if we are here, either by being here from beginning or just
+        // transitioning from reconnecting
+        if let ConnectionState::Writing(conn_writer) = &mut self.state {
+            return match conn_writer.write_all(msg.as_ref()).await {
+                // if we failed to write to connection we should reconnect
+                // TODO: is this true? can we fail to write to a connection while it still remains open and valid?
+
+                // TODO: change connection writer to somehow also poll for responses and
+                // change return type of this method from io::Result<> to io::Result<Vec<u8>>
+                Ok(_) => Ok(()),
+                Err(e) => {
+                    trace!("Creating connection reconnector!");
+                    self.state = ConnectionState::Reconnecting(ConnectionReconnector::new(
+                        self.address,
+                        self.reconnection_backoff,
+                        self.maximum_reconnection_backoff,
+                    ));
+                    Err(e)
+                }
+            };
+        };
+
+        unreachable!();
+    }
+}

common/clients/multi-tcp-client/src/connection_manager/reconnector.rs

@@ -0,0 +1,90 @@
+use futures::future::BoxFuture;
+use futures::FutureExt;
+use log::*;
+use std::future::Future;
+use std::io;
+use std::net::SocketAddr;
+use std::pin::Pin;
+use std::task::{Context, Poll};
+use std::time::Duration;
+
+pub(crate) struct ConnectionReconnector<'a> {
+    address: SocketAddr,
+    connection: BoxFuture<'a, io::Result<tokio::net::TcpStream>>,
+
+    current_retry_attempt: u32,
+
+    current_backoff_delay: tokio::time::Delay,
+    maximum_reconnection_backoff: Duration,
+
+    initial_reconnection_backoff: Duration,
+}
+
+impl<'a> ConnectionReconnector<'a> {
+    pub(crate) fn new(
+        address: SocketAddr,
+        initial_reconnection_backoff: Duration,
+        maximum_reconnection_backoff: Duration,
+    ) -> ConnectionReconnector<'a> {
+        ConnectionReconnector {
+            address,
+            connection: tokio::net::TcpStream::connect(address).boxed(),
+            current_backoff_delay: tokio::time::delay_for(Duration::new(0, 0)), // if we can re-establish connection on first try without any backoff that's perfect
+            current_retry_attempt: 0,
+            maximum_reconnection_backoff,
+            initial_reconnection_backoff,
+        }
+    }
+}
+
+impl<'a> Future for ConnectionReconnector<'a> {
+    type Output = tokio::net::TcpStream;
+
+    fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
+        // see if we are still in exponential backoff
+        if Pin::new(&mut self.current_backoff_delay)
+            .poll(cx)
+            .is_pending()
+        {
+            return Poll::Pending;
+        };
+
+        // see if we managed to resolve the connection yet
+        match Pin::new(&mut self.connection).poll(cx) {
+            Poll::Pending => Poll::Pending,
+            Poll::Ready(Err(e)) => {
+                warn!(
+                    "we failed to re-establish connection to {} - {:?} (attempt {})",
+                    self.address, e, self.current_retry_attempt
+                );
+                self.current_retry_attempt += 1;
+
+                // we failed to re-establish connection - continue exponential backoff
+
+                // according to https://github.com/tokio-rs/tokio/issues/1953 there's an undocumented
+                // limit of tokio delay of about 2 years.
+                // let's ensure our delay is always on a sane side of being maximum 1 day.
+                let maximum_sane_delay = Duration::from_secs(24 * 60 * 60);
+
+                let next_delay = std::cmp::min(
+                    maximum_sane_delay,
+                    self.initial_reconnection_backoff
+                        .checked_mul(2_u32.pow(self.current_retry_attempt))
+                        .unwrap_or_else(|| self.maximum_reconnection_backoff),
+                );
+
+                let now = self.current_backoff_delay.deadline();
+                // this can't overflow now because next_delay is limited to one day...
+                // ... well, unless you've been running the system for couple of decades without
+                // any restarts....
+                let next = now + next_delay;
+                self.current_backoff_delay.reset(next);
+
+                self.connection = tokio::net::TcpStream::connect(self.address).boxed();
+
+                Poll::Pending
+            }
+            Poll::Ready(Ok(conn)) => Poll::Ready(conn),
+        }
+    }
+}

common/clients/multi-tcp-client/src/connection_manager/writer.rs

@@ -0,0 +1,46 @@
+use futures::task::{Context, Poll};
+use futures::AsyncWrite;
+use std::io;
+use std::pin::Pin;
+use tokio::prelude::*;
+
+pub(crate) struct ConnectionWriter {
+    connection: tokio::net::TcpStream,
+}
+
+impl ConnectionWriter {
+    pub(crate) fn new(connection: tokio::net::TcpStream) -> Self {
+        ConnectionWriter { connection }
+    }
+}
+
+impl AsyncWrite for ConnectionWriter {
+    fn poll_write(
+        mut self: Pin<&mut Self>,
+        cx: &mut Context<'_>,
+        buf: &[u8],
+    ) -> Poll<io::Result<usize>> {
+        use tokio::io::AsyncWrite;
+
+        let mut read_buf = [0; 1];
+        match Pin::new(&mut self.connection).poll_read(cx, &mut read_buf) {
+            // at least try the obvious check for if connection is definitely down
+            // TODO: can we do anything else?
+            Poll::Ready(Ok(n)) if n == 0 => Poll::Ready(Err(io::Error::new(
+                io::ErrorKind::UnexpectedEof,
+                "trying to write to closed connection",
+            ))),
+            _ => Pin::new(&mut self.connection).poll_write(cx, buf),
+        }
+    }
+
+    fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
+        use tokio::io::AsyncWrite;
+        Pin::new(&mut self.connection).poll_flush(cx)
+    }
+
+    fn poll_close(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
+        use tokio::io::AsyncWrite;
+        Pin::new(&mut self.connection).poll_shutdown(cx)
+    }
+}

common/clients/multi-tcp-client/src/lib.rs

@@ -0,0 +1,259 @@
+use crate::connection_manager::{ConnectionManager, ConnectionManagerSender};
+use futures::channel::oneshot;
+use futures::future::AbortHandle;
+use log::*;
+use std::collections::HashMap;
+use std::io;
+use std::net::SocketAddr;
+use std::time::Duration;
+use tokio::runtime::Handle;
+
+mod connection_manager;
+
+pub struct Config {
+    initial_reconnection_backoff: Duration,
+    maximum_reconnection_backoff: Duration,
+    initial_connection_timeout: Duration,
+}
+
+impl Config {
+    pub fn new(
+        initial_reconnection_backoff: Duration,
+        maximum_reconnection_backoff: Duration,
+        initial_connection_timeout: Duration,
+    ) -> Self {
+        Config {
+            initial_reconnection_backoff,
+            maximum_reconnection_backoff,
+            initial_connection_timeout,
+        }
+    }
+}
+
+pub struct Client {
+    runtime_handle: Handle,
+    connections_managers: HashMap<SocketAddr, (ConnectionManagerSender, AbortHandle)>,
+    maximum_reconnection_backoff: Duration,
+    initial_reconnection_backoff: Duration,
+    initial_connection_timeout: Duration,
+}
+
+impl Client {
+    pub fn new(config: Config) -> Client {
+        Client {
+            // if the function is not called within tokio runtime context, this will panic
+            // but perhaps the code should be better structured to completely avoid this call
+            runtime_handle: Handle::try_current()
+                .expect("The client MUST BE used within tokio runtime context"),
+            connections_managers: HashMap::new(),
+            initial_reconnection_backoff: config.maximum_reconnection_backoff,
+            maximum_reconnection_backoff: config.initial_reconnection_backoff,
+            initial_connection_timeout: config.initial_connection_timeout,
+        }
+    }
+
+    async fn start_new_connection_manager(
+        &mut self,
+        address: SocketAddr,
+    ) -> (ConnectionManagerSender, AbortHandle) {
+        let (sender, abort_handle) = ConnectionManager::new(
+            address,
+            self.initial_reconnection_backoff,
+            self.maximum_reconnection_backoff,
+            self.initial_connection_timeout,
+        )
+        .await
+        .start_abortable(&self.runtime_handle);
+
+        (sender, abort_handle)
+    }
+
+    // if wait_for_response is set to true, we will get information about any possible IO errors
+    // as well as (once implemented) received replies, however, this will also cause way longer
+    // waiting periods
+    pub async fn send(
+        &mut self,
+        address: SocketAddr,
+        message: Vec<u8>,
+        wait_for_response: bool,
+    ) -> io::Result<()> {
+        if !self.connections_managers.contains_key(&address) {
+            debug!(
+                "There is no existing connection to {:?} - it will be established now",
+                address
+            );
+
+            let (new_manager_sender, abort_handle) =
+                self.start_new_connection_manager(address).await;
+            self.connections_managers
+                .insert(address, (new_manager_sender, abort_handle));
+        }
+
+        let manager = self.connections_managers.get_mut(&address).unwrap();
+
+        if wait_for_response {
+            let (res_tx, res_rx) = oneshot::channel();
+            manager.0.unbounded_send((message, Some(res_tx))).unwrap();
+            res_rx.await.unwrap()
+        } else {
+            manager.0.unbounded_send((message, None)).unwrap();
+            Ok(())
+        }
+    }
+}
+
+impl Drop for Client {
+    fn drop(&mut self) {
+        for (_, abort_handle) in self.connections_managers.values() {
+            abort_handle.abort()
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use std::str;
+    use std::time;
+    use tokio::prelude::*;
+
+    const SERVER_MSG_LEN: usize = 16;
+    const CLOSE_MESSAGE: [u8; SERVER_MSG_LEN] = [0; SERVER_MSG_LEN];
+
+    struct DummyServer {
+        received_buf: Vec<Vec<u8>>,
+        listener: tokio::net::TcpListener,
+    }
+
+    impl DummyServer {
+        async fn new(address: SocketAddr) -> Self {
+            DummyServer {
+                received_buf: Vec::new(),
+                listener: tokio::net::TcpListener::bind(address).await.unwrap(),
+            }
+        }
+
+        fn get_received(&self) -> Vec<Vec<u8>> {
+            self.received_buf.clone()
+        }
+
+        // this is only used in tests so slightly higher logging levels are fine
+        async fn listen_until(mut self, close_message: &[u8]) -> Self {
+            let (mut socket, _) = self.listener.accept().await.unwrap();
+            loop {
+                let mut buf = [0u8; SERVER_MSG_LEN];
+                match socket.read(&mut buf).await {
+                    Ok(n) if n == 0 => {
+                        info!("Remote connection closed");
+                        return self;
+                    }
+                    Ok(n) => {
+                        info!("received ({}) - {:?}", n, str::from_utf8(buf[..n].as_ref()));
+
+                        if buf[..n].as_ref() == close_message {
+                            info!("closing...");
+                            socket.shutdown(std::net::Shutdown::Both).unwrap();
+                            return self;
+                        } else {
+                            self.received_buf.push(buf[..n].to_vec());
+                        }
+                    }
+                    Err(e) => {
+                        panic!("failed to read from socket; err = {:?}", e);
+                    }
+                };
+            }
+        }
+    }
+
+    #[test]
+    fn client_reconnects_to_server_after_it_went_down() {
+        let mut rt = tokio::runtime::Runtime::new().unwrap();
+        let addr = "127.0.0.1:6000".parse().unwrap();
+        let reconnection_backoff = Duration::from_secs(1);
+        let timeout = Duration::from_secs(1);
+        let client_config = Config::new(reconnection_backoff, 10 * reconnection_backoff, timeout);
+
+        let messages_to_send = vec![[1u8; SERVER_MSG_LEN].to_vec(), [2; SERVER_MSG_LEN].to_vec()];
+
+        let dummy_server = rt.block_on(DummyServer::new(addr));
+        let finished_dummy_server_future = rt.spawn(dummy_server.listen_until(&CLOSE_MESSAGE));
+
+        let mut c = rt.enter(|| Client::new(client_config));
+
+        for msg in &messages_to_send {
+            rt.block_on(c.send(addr, msg.clone(), true)).unwrap();
+        }
+
+        // kill server
+        rt.block_on(c.send(addr, CLOSE_MESSAGE.to_vec(), true))
+            .unwrap();
+        let received_messages = rt
+            .block_on(finished_dummy_server_future)
+            .unwrap()
+            .get_received();
+
+        assert_eq!(received_messages, messages_to_send);
+
+        // try to send - go into reconnection
+        let post_kill_message = [3u8; SERVER_MSG_LEN].to_vec();
+
+        // we are trying to send to killed server
+        assert!(rt
+            .block_on(c.send(addr, post_kill_message.clone(), true))
+            .is_err());
+
+        let new_dummy_server = rt.block_on(DummyServer::new(addr));
+        let new_server_future = rt.spawn(new_dummy_server.listen_until(&CLOSE_MESSAGE));
+
+        // keep sending after we leave reconnection backoff and reconnect
+        loop {
+            if rt
+                .block_on(c.send(addr, post_kill_message.clone(), true))
+                .is_ok()
+            {
+                break;
+            }
+            rt.block_on(
+                async move { tokio::time::delay_for(time::Duration::from_millis(50)).await },
+            );
+        }
+
+        // kill the server to ensure it actually got the message
+        rt.block_on(c.send(addr, CLOSE_MESSAGE.to_vec(), true))
+            .unwrap();
+        let new_received_messages = rt.block_on(new_server_future).unwrap().get_received();
+        assert_eq!(post_kill_message.to_vec(), new_received_messages[0]);
+    }
+
+    #[test]
+    fn server_receives_all_sent_messages_when_up() {
+        let mut rt = tokio::runtime::Runtime::new().unwrap();
+        let addr = "127.0.0.1:6001".parse().unwrap();
+        let reconnection_backoff = Duration::from_secs(2);
+        let timeout = Duration::from_secs(1);
+        let client_config = Config::new(reconnection_backoff, 10 * reconnection_backoff, timeout);
+
+        let messages_to_send = vec![[1u8; SERVER_MSG_LEN].to_vec(), [2; SERVER_MSG_LEN].to_vec()];
+
+        let dummy_server = rt.block_on(DummyServer::new(addr));
+        let finished_dummy_server_future = rt.spawn(dummy_server.listen_until(&CLOSE_MESSAGE));
+
+        let mut c = rt.enter(|| Client::new(client_config));
+
+        for msg in &messages_to_send {
+            rt.block_on(c.send(addr, msg.clone(), true)).unwrap();
+        }
+
+        rt.block_on(c.send(addr, CLOSE_MESSAGE.to_vec(), true))
+            .unwrap();
+
+        // the server future should have already been resolved
+        let received_messages = rt
+            .block_on(finished_dummy_server_future)
+            .unwrap()
+            .get_received();
+
+        assert_eq!(received_messages, messages_to_send);
+    }
+}

common/clients/provider-client/Cargo.toml

@@ -0,0 +1,19 @@
+[package]
+name = "provider-client"
+version = "0.1.0"
+authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+futures = "0.3.1"
+log = "0.4.8"
+pretty_env_logger = "0.3"
+tokio = { version = "0.2", features = ["full"] }
+
+## internal
+sfw-provider-requests = { path = "../../../sfw-provider/sfw-provider-requests" }
+
+## will be moved to proper dependencies once released
+sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" }

common/clients/provider-client/src/lib.rs

@@ -1,4 +1,5 @@
 use futures::io::Error;
+use log::*;
 use sfw_provider_requests::requests::{ProviderRequest, PullRequest, RegisterRequest};
 use sfw_provider_requests::responses::{
     ProviderResponse, ProviderResponseError, PullResponse, RegisterResponse,
@@ -51,13 +52,8 @@ impl ProviderClient {
         our_address: DestinationAddressBytes,
         auth_token: Option<AuthToken>,
     ) -> Self {
-        // DH temporary: the provider's client port is not in the topology, but we can't change that
-        // right now without messing up the existing Go mixnet. So I'm going to hardcode this
-        // for the moment until the Go mixnet goes away.
-        let provider_socket = SocketAddr::new(provider_network_address.ip(), 9000);
-
         ProviderClient {
-            provider_network_address: provider_socket,
+            provider_network_address,
             our_address,
             auth_token,
         }
@@ -69,34 +65,34 @@ impl ProviderClient {
 
     pub async fn send_request(&self, bytes: Vec<u8>) -> Result<Vec<u8>, ProviderClientError> {
         let mut socket = tokio::net::TcpStream::connect(self.provider_network_address).await?;
-        println!("keep alive: {:?}", socket.keepalive());
-        socket.set_keepalive(Some(Duration::from_secs(2))).unwrap();
+
+        socket.set_keepalive(Some(Duration::from_secs(2)))?;
         socket.write_all(&bytes[..]).await?;
-        if let Err(_e) = socket.shutdown(Shutdown::Write) {
-            // TODO: make it a silent log once we have a proper logging library
-            //            eprintln!("failed to close write part of the socket; err = {:?}", e)
+        if let Err(e) = socket.shutdown(Shutdown::Write) {
+            warn!("failed to close write part of the socket; err = {:?}", e)
         }
 
         let mut response = Vec::new();
         socket.read_to_end(&mut response).await?;
-        if let Err(_e) = socket.shutdown(Shutdown::Read) {
-            // TODO: make it a silent log once we have a proper logging library
-            //            eprintln!("failed to close read part of the socket; err = {:?}", e)
+        if let Err(e) = socket.shutdown(Shutdown::Read) {
+            debug!("failed to close read part of the socket; err = {:?}. It was probably already closed by the provider", e)
         }
 
         Ok(response)
     }
 
     pub async fn retrieve_messages(&self) -> Result<Vec<Vec<u8>>, ProviderClientError> {
-        if self.auth_token.is_none() {
-            return Err(ProviderClientError::EmptyAuthTokenError);
-        }
+        let auth_token = match self.auth_token.as_ref() {
+            Some(token) => token.clone(),
+            None => {
+                return Err(ProviderClientError::EmptyAuthTokenError);
+            }
+        };
 
-        let pull_request = PullRequest::new(self.our_address, self.auth_token.unwrap());
+        let pull_request = PullRequest::new(self.our_address.clone(), auth_token);
         let bytes = pull_request.to_bytes();
 
         let response = self.send_request(bytes).await?;
-        println!("Received the following response: {:?}", response);
 
         let parsed_response = PullResponse::from_bytes(&response)?;
         Ok(parsed_response.messages)
@@ -107,7 +103,7 @@ impl ProviderClient {
             return Err(ProviderClientError::ClientAlreadyRegisteredError);
         }
 
-        let register_request = RegisterRequest::new(self.our_address);
+        let register_request = RegisterRequest::new(self.our_address.clone());
         let bytes = register_request.to_bytes();
 
         let response = self.send_request(bytes).await?;
@@ -115,4 +111,8 @@ impl ProviderClient {
 
         Ok(parsed_response.auth_token)
     }
+
+    pub fn is_registered(&self) -> bool {
+        self.auth_token.is_some()
+    }
 }

common/clients/validator-client/Cargo.toml

@@ -0,0 +1,11 @@
+[package]
+name = "validator-client"
+version = "0.1.0"
+authors = ["Jedrzej Stuczynski <andrew@nymtech.net>, David Hrycyszyn <dave@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+log = "0.4"
+pretty_env_logger = "0.3"

common/clients/validator-client/src/lib.rs

@@ -0,0 +1,7 @@
+#[cfg(test)]
+mod tests {
+    #[test]
+    fn it_works() {
+        assert_eq!(2 + 2, 4);
+    }
+}

common/config/Cargo.toml

@@ -0,0 +1,12 @@
+[package]
+name = "config"
+version = "0.1.0"
+authors = ["Jedrzej Stuczynski <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+handlebars = "3.0.1"
+serde = { version = "1.0.104", features = ["derive"] }
+toml = "0.5.6"

common/config/src/lib.rs

@@ -0,0 +1,64 @@
+use handlebars::Handlebars;
+use serde::de::DeserializeOwned;
+use serde::Serialize;
+use std::path::PathBuf;
+use std::{fs, io};
+
+pub trait NymConfig: Default + Serialize + DeserializeOwned {
+    fn template() -> &'static str;
+
+    fn config_file_name() -> String;
+
+    fn default_root_directory() -> PathBuf;
+
+    // default, most probable, implementations; can be easily overridden where required
+    fn default_config_directory(id: Option<&str>) -> PathBuf {
+        Self::default_root_directory()
+            .join(id.unwrap_or(""))
+            .join("config")
+    }
+
+    fn default_data_directory(id: Option<&str>) -> PathBuf {
+        Self::default_root_directory()
+            .join(id.unwrap_or(""))
+            .join("data")
+    }
+
+    fn root_directory(&self) -> PathBuf;
+    fn config_directory(&self) -> PathBuf;
+    fn data_directory(&self) -> PathBuf;
+
+    fn save_to_file(&self, custom_location: Option<PathBuf>) -> io::Result<()> {
+        let reg = Handlebars::new();
+        // it's whoever is implementing the trait responsibility to make sure you can execute your own template on your data
+        let templated_config = reg.render_template(Self::template(), self).unwrap();
+
+        // make sure the whole directory structure actually exists
+        match custom_location.clone() {
+            Some(loc) => {
+                if let Some(parent_dir) = loc.parent() {
+                    fs::create_dir_all(parent_dir)
+                } else {
+                    Ok(())
+                }
+            }
+            None => fs::create_dir_all(self.config_directory()),
+        }?;
+
+        fs::write(
+            custom_location
+                .unwrap_or_else(|| self.config_directory().join(Self::config_file_name())),
+            templated_config,
+        )
+    }
+
+    fn load_from_file(custom_location: Option<PathBuf>, id: Option<&str>) -> io::Result<Self> {
+        let config_contents = fs::read_to_string(
+            custom_location
+                .unwrap_or_else(|| Self::default_config_directory(id).join("config.toml")),
+        )?;
+
+        toml::from_str(&config_contents)
+            .map_err(|toml_err| io::Error::new(io::ErrorKind::Other, toml_err))
+    }
+}

common/crypto/Cargo.toml

@@ -0,0 +1,18 @@
+[package]
+name = "crypto"
+version = "0.1.0"
+authors = ["Jedrzej Stuczynski <andrew@nymtech.net"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+bs58 = "0.3.0"
+curve25519-dalek = "1.2.3"
+log = "0.4"
+pretty_env_logger = "0.3"
+rand = "0.7.2"
+rand_os = "0.1"
+
+## will be moved to proper dependencies once released
+sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" }

common/crypto/src/encryption/mod.rs

@@ -0,0 +1,138 @@
+use crate::{PemStorableKey, PemStorableKeyPair};
+use curve25519_dalek::montgomery::MontgomeryPoint;
+use curve25519_dalek::scalar::Scalar;
+
+// TODO: ensure this is a proper name for this considering we are not implementing entire DH here
+
+const CURVE_GENERATOR: MontgomeryPoint = curve25519_dalek::constants::X25519_BASEPOINT;
+
+pub struct KeyPair {
+    pub(crate) private_key: PrivateKey,
+    pub(crate) public_key: PublicKey,
+}
+
+impl KeyPair {
+    pub fn new() -> Self {
+        let mut rng = rand_os::OsRng::new().unwrap();
+        let private_key_value = Scalar::random(&mut rng);
+        let public_key_value = CURVE_GENERATOR * private_key_value;
+
+        KeyPair {
+            private_key: PrivateKey(private_key_value),
+            public_key: PublicKey(public_key_value),
+        }
+    }
+
+    pub fn private_key(&self) -> &PrivateKey {
+        &self.private_key
+    }
+
+    pub fn public_key(&self) -> &PublicKey {
+        &self.public_key
+    }
+
+    pub fn from_bytes(priv_bytes: &[u8], pub_bytes: &[u8]) -> Self {
+        KeyPair {
+            private_key: PrivateKey::from_bytes(priv_bytes),
+            public_key: PublicKey::from_bytes(pub_bytes),
+        }
+    }
+}
+
+impl Default for KeyPair {
+    fn default() -> Self {
+        KeyPair::new()
+    }
+}
+
+impl PemStorableKeyPair for KeyPair {
+    type PrivatePemKey = PrivateKey;
+    type PublicPemKey = PublicKey;
+
+    fn private_key(&self) -> &Self::PrivatePemKey {
+        self.private_key()
+    }
+
+    fn public_key(&self) -> &Self::PublicPemKey {
+        self.public_key()
+    }
+
+    fn from_bytes(priv_bytes: &[u8], pub_bytes: &[u8]) -> Self {
+        Self::from_bytes(priv_bytes, pub_bytes)
+    }
+}
+
+#[derive(Debug, Clone, Eq, PartialEq)]
+pub struct PrivateKey(pub Scalar);
+
+impl PrivateKey {
+    pub fn to_bytes(&self) -> Vec<u8> {
+        self.0.to_bytes().to_vec()
+    }
+
+    pub fn from_bytes(b: &[u8]) -> Self {
+        let mut bytes = [0; 32];
+        bytes.copy_from_slice(&b[..]);
+        // due to trait restriction we have no choice but to panic if this fails
+        let key = Scalar::from_canonical_bytes(bytes).unwrap();
+        Self(key)
+    }
+
+    pub fn inner(&self) -> Scalar {
+        self.0
+    }
+}
+
+impl PemStorableKey for PrivateKey {
+    fn pem_type(&self) -> String {
+        String::from("X25519 PRIVATE KEY")
+    }
+
+    fn to_bytes(&self) -> Vec<u8> {
+        self.to_bytes()
+    }
+}
+
+#[derive(Debug, Clone, Eq, PartialEq)]
+pub struct PublicKey(pub MontgomeryPoint);
+
+impl<'a> From<&'a PrivateKey> for PublicKey {
+    fn from(pk: &'a PrivateKey) -> Self {
+        PublicKey(CURVE_GENERATOR * pk.0)
+    }
+}
+
+impl PublicKey {
+    pub fn to_bytes(&self) -> Vec<u8> {
+        self.0.to_bytes().to_vec()
+    }
+
+    pub fn from_bytes(b: &[u8]) -> Self {
+        let mut bytes = [0; 32];
+        bytes.copy_from_slice(&b[..]);
+        let key = MontgomeryPoint(bytes);
+        Self(key)
+    }
+
+    pub fn inner(&self) -> MontgomeryPoint {
+        self.0
+    }
+
+    pub fn to_base58_string(&self) -> String {
+        bs58::encode(&self.to_bytes()).into_string()
+    }
+
+    pub fn from_base58_string(val: String) -> Self {
+        Self::from_bytes(&bs58::decode(&val).into_vec().unwrap())
+    }
+}
+
+impl PemStorableKey for PublicKey {
+    fn pem_type(&self) -> String {
+        String::from("X25519 PUBLIC KEY")
+    }
+
+    fn to_bytes(&self) -> Vec<u8> {
+        self.to_bytes()
+    }
+}

common/crypto/src/identity/mod.rs

@@ -0,0 +1,138 @@
+use crate::{encryption, PemStorableKey, PemStorableKeyPair};
+use bs58;
+use curve25519_dalek::scalar::Scalar;
+use sphinx::route::DestinationAddressBytes;
+
+// for time being define a dummy identity using x25519 encryption keys (as we've done so far)
+// and replace it with proper keys, like ed25519 later on
+#[derive(Clone)]
+pub struct MixIdentityKeyPair {
+    pub private_key: MixIdentityPrivateKey,
+    pub public_key: MixIdentityPublicKey,
+}
+
+impl MixIdentityKeyPair {
+    pub fn new() -> Self {
+        let keypair = encryption::KeyPair::new();
+        MixIdentityKeyPair {
+            private_key: MixIdentityPrivateKey(keypair.private_key),
+            public_key: MixIdentityPublicKey(keypair.public_key),
+        }
+    }
+
+    pub fn private_key(&self) -> &MixIdentityPrivateKey {
+        &self.private_key
+    }
+
+    pub fn public_key(&self) -> &MixIdentityPublicKey {
+        &self.public_key
+    }
+
+    pub fn from_bytes(priv_bytes: &[u8], pub_bytes: &[u8]) -> Self {
+        MixIdentityKeyPair {
+            private_key: MixIdentityPrivateKey::from_bytes(priv_bytes),
+            public_key: MixIdentityPublicKey::from_bytes(pub_bytes),
+        }
+    }
+}
+
+impl Default for MixIdentityKeyPair {
+    fn default() -> Self {
+        MixIdentityKeyPair::new()
+    }
+}
+
+impl PemStorableKeyPair for MixIdentityKeyPair {
+    type PrivatePemKey = MixIdentityPrivateKey;
+    type PublicPemKey = MixIdentityPublicKey;
+
+    fn private_key(&self) -> &Self::PrivatePemKey {
+        self.private_key()
+    }
+
+    fn public_key(&self) -> &Self::PublicPemKey {
+        self.public_key()
+    }
+
+    fn from_bytes(priv_bytes: &[u8], pub_bytes: &[u8]) -> Self {
+        Self::from_bytes(priv_bytes, pub_bytes)
+    }
+}
+
+#[derive(Debug, Clone, Eq, PartialEq)]
+pub struct MixIdentityPublicKey(encryption::PublicKey);
+
+impl MixIdentityPublicKey {
+    pub fn derive_address(&self) -> DestinationAddressBytes {
+        let mut temporary_address = [0u8; 32];
+        let public_key_bytes = self.to_bytes();
+        temporary_address.copy_from_slice(&public_key_bytes[..]);
+
+        DestinationAddressBytes::from_bytes(temporary_address)
+    }
+
+    pub fn to_bytes(&self) -> Vec<u8> {
+        self.0.to_bytes()
+    }
+
+    pub fn from_bytes(b: &[u8]) -> Self {
+        Self(encryption::PublicKey::from_bytes(b))
+    }
+
+    pub fn to_base58_string(&self) -> String {
+        bs58::encode(&self.to_bytes()).into_string()
+    }
+
+    pub fn from_base58_string(val: String) -> Self {
+        Self::from_bytes(&bs58::decode(&val).into_vec().unwrap())
+    }
+}
+
+impl PemStorableKey for MixIdentityPublicKey {
+    fn pem_type(&self) -> String {
+        format!("DUMMY KEY BASED ON {}", self.0.pem_type())
+    }
+
+    fn to_bytes(&self) -> Vec<u8> {
+        self.to_bytes()
+    }
+}
+
+#[derive(Debug, Clone, Eq, PartialEq)]
+pub struct MixIdentityPrivateKey(pub encryption::PrivateKey);
+
+impl<'a> From<&'a MixIdentityPrivateKey> for MixIdentityPublicKey {
+    fn from(pk: &'a MixIdentityPrivateKey) -> Self {
+        let private_ref = &pk.0;
+        let public: encryption::PublicKey = private_ref.into();
+        MixIdentityPublicKey(public)
+    }
+}
+
+impl MixIdentityPrivateKey {
+    pub fn to_bytes(&self) -> Vec<u8> {
+        self.0.to_bytes()
+    }
+
+    pub fn from_bytes(b: &[u8]) -> Self {
+        Self(encryption::PrivateKey::from_bytes(b))
+    }
+}
+
+// TODO: this will be implemented differently by using the proper trait
+impl MixIdentityPrivateKey {
+    pub fn as_scalar(&self) -> Scalar {
+        let encryption_key = &self.0;
+        encryption_key.0
+    }
+}
+
+impl PemStorableKey for MixIdentityPrivateKey {
+    fn pem_type(&self) -> String {
+        format!("DUMMY KEY BASED ON {}", self.0.pem_type())
+    }
+
+    fn to_bytes(&self) -> Vec<u8> {
+        self.to_bytes()
+    }
+}

common/crypto/src/lib.rs

@@ -0,0 +1,22 @@
+pub mod encryption;
+pub mod identity;
+
+// TODO: ideally those trait should be moved to 'pemstore' crate, however, that would cause
+// circular dependency. The best solution would be to remove dependency on 'crypto' from
+// pemstore by using either dynamic dispatch or generics - perhaps this should be done
+// at some point during one of refactors.
+
+pub trait PemStorableKey {
+    fn pem_type(&self) -> String;
+    fn to_bytes(&self) -> Vec<u8>;
+}
+
+pub trait PemStorableKeyPair {
+    type PrivatePemKey: PemStorableKey;
+    type PublicPemKey: PemStorableKey;
+
+    fn private_key(&self) -> &Self::PrivatePemKey;
+    fn public_key(&self) -> &Self::PublicPemKey;
+
+    fn from_bytes(priv_bytes: &[u8], pub_bytes: &[u8]) -> Self;
+}

common/healthcheck/Cargo.toml

@@ -0,0 +1,33 @@
+[package]
+name = "healthcheck"
+version = "0.1.0"
+authors = ["Jedrzej Stuczynski <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+futures = "0.3.1"
+itertools = "0.8.2"
+log = "0.4.8"
+pretty_env_logger = "0.3"
+rand_os = "0.1"
+rand = "0.7.2"
+serde = "1.0.104"
+serde_derive = "1.0.104"
+tokio = { version = "0.2", features = ["full"] }
+
+## internal
+crypto = { path = "../crypto" }
+directory-client = { path = "../clients/directory-client" }
+multi-tcp-client = { path = "../clients/multi-tcp-client" }
+nymsphinx = {path = "../nymsphinx" }
+provider-client = { path = "../clients/provider-client" }
+sfw-provider-requests = { path = "../../sfw-provider/sfw-provider-requests" }
+topology = {path = "../topology" }
+
+## will be moved to proper dependencies once released
+sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" }
+# sphinx = { path = "../../../sphinx"}
+
+[dev-dependencies]

common/healthcheck/src/config.rs

@@ -0,0 +1,15 @@
+use serde_derive::Deserialize;
+
+#[derive(Deserialize, Debug)]
+pub struct HealthCheck {
+    #[serde(rename(deserialize = "directory-server"))]
+    pub directory_server: String,
+
+    pub interval: f64, // in seconds
+
+    #[serde(rename(deserialize = "resolution-timeout"))]
+    pub resolution_timeout: f64, // in seconds
+
+    #[serde(rename(deserialize = "test-packets-per-node"))]
+    pub num_test_packets: usize,
+}

common/healthcheck/src/lib.rs

@@ -0,0 +1,75 @@
+use crate::result::HealthCheckResult;
+use crypto::identity::MixIdentityKeyPair;
+use log::trace;
+use std::fmt::{Error, Formatter};
+use std::time::Duration;
+use topology::{NymTopology, NymTopologyError};
+
+pub mod config;
+mod path_check;
+mod result;
+mod score;
+
+#[derive(Debug)]
+pub enum HealthCheckerError {
+    FailedToObtainTopologyError,
+    InvalidTopologyError,
+}
+
+// required by std::error::Error
+impl std::fmt::Display for HealthCheckerError {
+    fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error> {
+        // just have implementation equivalent to derived debug
+        write!(f, "{:?}", self)
+    }
+}
+
+impl std::error::Error for HealthCheckerError {}
+
+impl From<topology::NymTopologyError> for HealthCheckerError {
+    fn from(_: NymTopologyError) -> Self {
+        use HealthCheckerError::*;
+        InvalidTopologyError
+    }
+}
+
+pub struct HealthChecker {
+    num_test_packets: usize,
+    resolution_timeout: Duration,
+    connection_timeout: Duration,
+    identity_keypair: MixIdentityKeyPair,
+}
+
+impl HealthChecker {
+    pub fn new(
+        resolution_timeout: Duration,
+        connection_timeout: Duration,
+        num_test_packets: usize,
+        identity_keypair: MixIdentityKeyPair,
+    ) -> Self {
+        HealthChecker {
+            resolution_timeout,
+            connection_timeout,
+            num_test_packets,
+            identity_keypair,
+        }
+    }
+
+    pub async fn do_check<T: NymTopology>(
+        &self,
+        current_topology: &T,
+    ) -> Result<HealthCheckResult, HealthCheckerError> {
+        trace!("going to perform a healthcheck!");
+
+        let mut healthcheck_result = HealthCheckResult::calculate(
+            current_topology,
+            self.num_test_packets,
+            self.resolution_timeout,
+            self.connection_timeout,
+            &self.identity_keypair,
+        )
+        .await;
+        healthcheck_result.sort_scores();
+        Ok(healthcheck_result)
+    }
+}

common/healthcheck/src/path_check.rs

@@ -0,0 +1,269 @@
+use crypto::identity::MixIdentityKeyPair;
+use itertools::Itertools;
+use log::{debug, error, info, trace, warn};
+use nymsphinx::addressing::nodes::NymNodeRoutingAddress;
+use provider_client::{ProviderClient, ProviderClientError};
+use sphinx::header::delays::Delay;
+use sphinx::route::{Destination, Node as SphinxNode};
+use std::collections::HashMap;
+use std::convert::TryFrom;
+use std::net::SocketAddr;
+use std::time::Duration;
+use topology::provider;
+
+#[derive(Debug, PartialEq, Clone)]
+pub enum PathStatus {
+    Healthy,
+    Unhealthy,
+    Pending,
+}
+
+pub(crate) struct PathChecker {
+    provider_clients: HashMap<[u8; 32], Option<ProviderClient>>,
+    mixnet_client: multi_tcp_client::Client,
+    paths_status: HashMap<Vec<u8>, PathStatus>,
+    our_destination: Destination,
+    check_id: [u8; 16],
+}
+
+impl PathChecker {
+    pub(crate) async fn new(
+        providers: Vec<provider::Node>,
+        identity_keys: &MixIdentityKeyPair,
+        connection_timeout: Duration,
+        check_id: [u8; 16],
+    ) -> Self {
+        let mut provider_clients = HashMap::new();
+
+        let address = identity_keys.public_key().derive_address();
+
+        for provider in providers {
+            let mut provider_client =
+                ProviderClient::new(provider.client_listener, address.clone(), None);
+            let insertion_result = match provider_client.register().await {
+                Ok(token) => {
+                    debug!("[Healthcheck] registered at provider {}", provider.pub_key);
+                    provider_client.update_token(token);
+                    provider_clients.insert(provider.get_pub_key_bytes(), Some(provider_client))
+                }
+                Err(ProviderClientError::ClientAlreadyRegisteredError) => {
+                    info!("[Healthcheck] We were already registered");
+                    provider_clients.insert(provider.get_pub_key_bytes(), Some(provider_client))
+                }
+                Err(err) => {
+                    warn!(
+                        "failed to register at provider {} - {:?}",
+                        provider.pub_key, err
+                    );
+                    provider_clients.insert(provider.get_pub_key_bytes(), None)
+                }
+            };
+
+            if insertion_result.is_some() {
+                error!("provider {} already existed!", provider.pub_key);
+            }
+        }
+
+        // there's no reconnection allowed - if it fails, then it fails.
+        let mixnet_client_config = multi_tcp_client::Config::new(
+            Duration::from_secs(1_000_000_000),
+            Duration::from_secs(1_000_000_000),
+            connection_timeout,
+        );
+
+        PathChecker {
+            provider_clients,
+            mixnet_client: multi_tcp_client::Client::new(mixnet_client_config),
+            our_destination: Destination::new(address, Default::default()),
+            paths_status: HashMap::new(),
+            check_id,
+        }
+    }
+
+    // iteration is used to distinguish packets sent through the same path (as the healthcheck
+    // may try to send say 10 packets through given path)
+    fn unique_path_key(path: &[SphinxNode], check_id: [u8; 16], iteration: u8) -> Vec<u8> {
+        check_id
+            .iter()
+            .cloned()
+            .chain(std::iter::once(iteration))
+            .chain(
+                path.iter()
+                    .map(|node| node.pub_key.to_bytes().to_vec())
+                    .flatten(),
+            )
+            .collect()
+    }
+
+    pub(crate) fn path_key_to_node_keys(path_key: Vec<u8>) -> Vec<[u8; 32]> {
+        assert_eq!(path_key.len() % 32, 17);
+        path_key
+            .into_iter()
+            .skip(16 + 1) // remove 16 + 1 bytes as it represents check_id and the iteration number which we do not care about now
+            .chunks(32)
+            .into_iter()
+            .map(|key_chunk| {
+                let key_chunk_vec: Vec<_> = key_chunk.collect();
+                let mut key = [0u8; 32];
+                key.copy_from_slice(&key_chunk_vec);
+                key
+            })
+            .collect()
+    }
+
+    fn update_path_statuses(&mut self, messages: Vec<Vec<u8>>) {
+        for msg in messages.into_iter() {
+            // mark path as healthy
+            let previous_status = self.paths_status.insert(msg, PathStatus::Healthy);
+            match previous_status {
+                None => warn!("we received information about unknown path! - perhaps somebody is messing with healthchecker?"),
+                Some(status) => {
+                    if status != PathStatus::Pending {
+                        warn!("we received information about path that WASN'T in PENDING state! (it was in {:?}", status);
+                    }
+                }
+            }
+        }
+    }
+
+    // consume path_checker and return all path statuses
+    pub(crate) fn get_all_statuses(self) -> HashMap<Vec<u8>, PathStatus> {
+        self.paths_status
+    }
+
+    // pull messages from given provider until there are no more 'real' messages
+    async fn resolve_pending_provider_checks(
+        &self,
+        provider_client: &ProviderClient,
+    ) -> Vec<Vec<u8>> {
+        // keep getting messages until we encounter the dummy message
+        let mut provider_messages = Vec::new();
+        loop {
+            match provider_client.retrieve_messages().await {
+                Err(err) => {
+                    error!("failed to fetch provider messages! - {:?}", err);
+                    break;
+                }
+                Ok(messages) => {
+                    let mut should_stop = false;
+                    for msg in messages.into_iter() {
+                        trace!("received provider response: {:?}", msg);
+                        if msg == sfw_provider_requests::DUMMY_MESSAGE_CONTENT {
+                            // finish iterating the loop as the messages might not be ordered
+                            should_stop = true;
+                        } else if msg[..16] != self.check_id {
+                            warn!("received response from previous healthcheck")
+                        } else {
+                            provider_messages.push(msg);
+                        }
+                    }
+                    if should_stop {
+                        break;
+                    }
+                }
+            }
+        }
+        provider_messages
+    }
+
+    pub(crate) async fn resolve_pending_checks(&mut self) {
+        // not sure how to nicely put it into an iterator due to it being async calls
+        let mut provider_messages = Vec::new();
+        for provider_client in self.provider_clients.values() {
+            // if it was none all associated paths were already marked as unhealthy
+            let pc = match provider_client {
+                Some(pc) => pc,
+                None => continue,
+            };
+
+            provider_messages.extend(self.resolve_pending_provider_checks(pc).await);
+        }
+
+        self.update_path_statuses(provider_messages);
+    }
+
+    pub(crate) async fn send_test_packet(&mut self, path: &[SphinxNode], iteration: u8) {
+        if path.is_empty() {
+            warn!("trying to send test packet through an empty path!");
+            return;
+        }
+
+        trace!("Checking path: {:?} ({})", path, iteration);
+        let path_identifier = PathChecker::unique_path_key(path, self.check_id, iteration);
+
+        // check if there is even any point in sending the packet
+
+        // does provider exist?
+        let provider_client = self
+            .provider_clients
+            .get(
+                &path
+                    .last()
+                    .expect("We checked the path to contain at least one entry")
+                    .pub_key
+                    .to_bytes(),
+            )
+            .unwrap();
+
+        if provider_client.is_none() {
+            debug!("we can ignore this path as provider itself is inaccessible");
+            if self
+                .paths_status
+                .insert(path_identifier, PathStatus::Unhealthy)
+                .is_some()
+            {
+                panic!("Overwriting path checks!")
+            }
+            return;
+        }
+
+        let layer_one_mix = path
+            .first()
+            .expect("We checked the path to contain at least one entry");
+
+        // we generated the bytes data so unwrap is fine
+        let first_node_address: SocketAddr =
+            NymNodeRoutingAddress::try_from(layer_one_mix.address.clone())
+                .unwrap()
+                .into();
+
+        let delays: Vec<_> = path.iter().map(|_| Delay::new_from_nanos(0)).collect();
+
+        // all of the data used to create the packet was created by us
+        let packet = sphinx::SphinxPacket::new(
+            path_identifier.clone(),
+            &path[..],
+            &self.our_destination,
+            &delays,
+        )
+        .unwrap();
+
+        debug!("sending test packet to {}", first_node_address);
+
+        match self
+            .mixnet_client
+            .send(first_node_address, packet.to_bytes(), true)
+            .await
+        {
+            Err(err) => {
+                debug!("failed to send packet to {} - {}", first_node_address, err);
+                if self
+                    .paths_status
+                    .insert(path_identifier, PathStatus::Unhealthy)
+                    .is_some()
+                {
+                    panic!("Overwriting path checks!")
+                }
+            }
+            Ok(_) => {
+                if self
+                    .paths_status
+                    .insert(path_identifier, PathStatus::Pending)
+                    .is_some()
+                {
+                    panic!("Overwriting path checks!")
+                }
+            }
+        }
+    }
+}

common/healthcheck/src/result.rs

@@ -0,0 +1,165 @@
+use crate::path_check::{PathChecker, PathStatus};
+use crate::score::NodeScore;
+use crypto::identity::MixIdentityKeyPair;
+use log::{debug, error, warn};
+use rand_os::rand_core::RngCore;
+use sphinx::route::NodeAddressBytes;
+use std::collections::HashMap;
+use std::fmt::{Error, Formatter};
+use std::time::Duration;
+use topology::NymTopology;
+
+#[derive(Debug)]
+pub struct HealthCheckResult(Vec<NodeScore>);
+
+impl std::fmt::Display for HealthCheckResult {
+    fn fmt(&self, f: &mut Formatter) -> Result<(), Error> {
+        write!(f, "NETWORK HEALTH\n==============\n")?;
+        for score in self.0.iter() {
+            writeln!(f, "{}", score)?
+        }
+        Ok(())
+    }
+}
+
+impl HealthCheckResult {
+    pub fn sort_scores(&mut self) {
+        self.0.sort();
+    }
+
+    fn zero_score<T: NymTopology>(topology: &T) -> Self {
+        warn!("The network is unhealthy, could not send any packets - returning zero score!");
+        let mixes = topology.mix_nodes();
+        let providers = topology.providers();
+
+        let health = mixes
+            .into_iter()
+            .map(NodeScore::from_mixnode)
+            .chain(providers.into_iter().map(NodeScore::from_provider))
+            .collect();
+
+        HealthCheckResult(health)
+    }
+
+    // TODO: that is O(n) so maybe not the most efficient considering it will be called n times...
+    fn node_score(&self, node_key: NodeAddressBytes) -> Option<f64> {
+        self.0
+            .iter()
+            .find(|&node_score| node_score.pub_key() == node_key)
+            .map(|node| node.score())
+    }
+
+    pub fn filter_topology_by_score<T: NymTopology>(
+        &self,
+        topology: &T,
+        score_threshold: f64,
+    ) -> T {
+        let filtered_mix_nodes = topology
+            .mix_nodes()
+            .into_iter()
+            .filter(|node| {
+                match self.node_score(NodeAddressBytes::from_base58_string(node.pub_key.clone())) {
+                    None => {
+                        error!("Unknown node in topology - {:?}", node);
+                        false
+                    }
+                    Some(score) => score > score_threshold,
+                }
+            })
+            .collect();
+
+        let filtered_provider_nodes = topology
+            .providers()
+            .into_iter()
+            .filter(|node| {
+                match self.node_score(NodeAddressBytes::from_base58_string(node.pub_key.clone())) {
+                    None => {
+                        error!("Unknown node in topology - {:?}", node);
+                        false
+                    }
+                    Some(score) => score > score_threshold,
+                }
+            })
+            .collect();
+        // coco nodes remain unchanged as no healthcheck is being run on them or time being
+        let filtered_coco_nodes = topology.coco_nodes();
+
+        T::new_from_nodes(
+            filtered_mix_nodes,
+            filtered_provider_nodes,
+            filtered_coco_nodes,
+        )
+    }
+
+    fn generate_check_id() -> [u8; 16] {
+        let mut id = [0u8; 16];
+        let mut rng = rand_os::OsRng::new().unwrap();
+        rng.fill_bytes(&mut id);
+        id
+    }
+
+    pub async fn calculate<T: NymTopology>(
+        topology: &T,
+        iterations: usize,
+        resolution_timeout: Duration,
+        connection_timeout: Duration,
+        identity_keys: &MixIdentityKeyPair,
+    ) -> Self {
+        // currently healthchecker supports only up to 255 iterations - if we somehow
+        // find we need more, it's relatively easy change
+        assert!(iterations <= 255);
+
+        let check_id = Self::generate_check_id();
+
+        let all_paths = match topology.all_paths() {
+            Ok(paths) => paths,
+            Err(_) => return Self::zero_score(topology),
+        };
+
+        // create entries for all nodes
+        let mut score_map = HashMap::new();
+        topology.mix_nodes().into_iter().for_each(|node| {
+            score_map.insert(node.get_pub_key_bytes(), NodeScore::from_mixnode(node));
+        });
+
+        topology.providers().into_iter().for_each(|node| {
+            score_map.insert(node.get_pub_key_bytes(), NodeScore::from_provider(node));
+        });
+
+        let providers = topology.providers();
+
+        let mut path_checker =
+            PathChecker::new(providers, identity_keys, connection_timeout, check_id).await;
+        for i in 0..iterations {
+            debug!("running healthcheck iteration {} / {}", i + 1, iterations);
+            for path in &all_paths {
+                path_checker.send_test_packet(&path, i as u8).await;
+                // increase sent count for each node
+                for node in path {
+                    let current_node_score = score_map.get_mut(&node.pub_key.0).unwrap();
+                    current_node_score.increase_sent_packet_count();
+                }
+            }
+        }
+
+        debug!(
+            "waiting {:?} for pending requests to resolve",
+            resolution_timeout
+        );
+        tokio::time::delay_for(resolution_timeout).await;
+        path_checker.resolve_pending_checks().await;
+
+        let all_statuses = path_checker.get_all_statuses();
+        for (path_key, status) in all_statuses.into_iter() {
+            let node_keys = PathChecker::path_key_to_node_keys(path_key);
+            for node in node_keys {
+                if status == PathStatus::Healthy {
+                    let current_node_score = score_map.get_mut(&node).unwrap();
+                    current_node_score.increase_received_packet_count();
+                }
+            }
+        }
+
+        HealthCheckResult(score_map.into_iter().map(|(_, v)| v).collect())
+    }
+}

common/healthcheck/src/score.rs

@@ -0,0 +1,167 @@
+use log::error;
+use sphinx::route::NodeAddressBytes;
+use std::cmp::Ordering;
+use std::fmt::Error;
+use std::fmt::Formatter;
+use std::net::SocketAddr;
+use topology::mix;
+use topology::provider;
+
+// TODO: should 'nodetype' really be part of healthcheck::score
+
+#[derive(Debug, Eq, PartialEq, PartialOrd, Ord, Clone, Copy)]
+pub(crate) enum NodeType {
+    Mix,
+    MixProvider,
+}
+
+impl std::fmt::Display for NodeType {
+    fn fmt(&self, f: &mut Formatter) -> Result<(), Error> {
+        match self {
+            NodeType::Mix => write!(f, "Mix"),
+            NodeType::MixProvider => write!(f, "MixProvider"),
+        }
+    }
+}
+
+#[derive(Debug, Eq)]
+pub(crate) struct NodeScore {
+    typ: NodeType,
+    pub_key: NodeAddressBytes,
+    addresses: Vec<SocketAddr>,
+    version: String,
+    layer: String,
+    packets_sent: u64,
+    packets_received: u64,
+}
+
+impl Ord for NodeScore {
+    // order by: version, layer, sent, received, pubkey; ignore addresses
+    #[allow(clippy::comparison_chain)]
+    fn cmp(&self, other: &Self) -> Ordering {
+        if self.typ > other.typ {
+            return Ordering::Greater;
+        } else if self.typ < other.typ {
+            return Ordering::Less;
+        }
+        if self.version > other.version {
+            return Ordering::Greater;
+        } else if self.version < other.version {
+            return Ordering::Less;
+        }
+        if self.layer > other.layer {
+            return Ordering::Greater;
+        } else if self.layer < other.layer {
+            return Ordering::Less;
+        }
+        if self.packets_sent > other.packets_sent {
+            return Ordering::Greater;
+        } else if self.packets_sent < other.packets_sent {
+            return Ordering::Less;
+        }
+        if self.packets_received > other.packets_received {
+            return Ordering::Greater;
+        } else if self.packets_received < other.packets_received {
+            return Ordering::Less;
+        }
+        if self.pub_key > other.pub_key {
+            return Ordering::Greater;
+        } else if self.pub_key < other.pub_key {
+            return Ordering::Less;
+        }
+
+        Ordering::Equal
+    }
+}
+
+impl PartialOrd for NodeScore {
+    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
+        Some(self.cmp(other))
+    }
+}
+
+impl PartialEq for NodeScore {
+    fn eq(&self, other: &Self) -> bool {
+        self.typ == other.typ
+            && self.pub_key == other.pub_key
+            && self.addresses == other.addresses
+            && self.version == other.version
+            && self.layer == other.layer
+            && self.packets_sent == other.packets_sent
+            && self.packets_received == other.packets_received
+    }
+}
+
+impl NodeScore {
+    pub(crate) fn from_mixnode(node: mix::Node) -> Self {
+        NodeScore {
+            typ: NodeType::Mix,
+            pub_key: NodeAddressBytes::from_base58_string(node.pub_key),
+            addresses: vec![node.host],
+            version: node.version,
+            layer: format!("layer {}", node.layer),
+            packets_sent: 0,
+            packets_received: 0,
+        }
+    }
+
+    pub(crate) fn from_provider(node: provider::Node) -> Self {
+        NodeScore {
+            typ: NodeType::MixProvider,
+            pub_key: NodeAddressBytes::from_base58_string(node.pub_key),
+            addresses: vec![node.mixnet_listener, node.client_listener],
+            version: node.version,
+            layer: "provider".to_string(),
+            packets_sent: 0,
+            packets_received: 0,
+        }
+    }
+
+    pub(crate) fn increase_sent_packet_count(&mut self) {
+        self.packets_sent += 1;
+    }
+
+    pub(crate) fn increase_received_packet_count(&mut self) {
+        self.packets_received += 1;
+    }
+
+    pub(crate) fn score(&self) -> f64 {
+        match self.packets_sent {
+            0 => 0.0,
+            _ => (self.packets_received as f64 / self.packets_sent as f64) * 100.0,
+        }
+    }
+
+    pub(crate) fn pub_key(&self) -> NodeAddressBytes {
+        self.pub_key.clone()
+    }
+}
+
+impl std::fmt::Display for NodeScore {
+    fn fmt(&self, f: &mut Formatter) -> Result<(), std::fmt::Error> {
+        let fmtd_addresses = match self.addresses.len() {
+            1 => format!("{}", self.addresses[0]),
+            2 => format!("{}, {}", self.addresses[0], self.addresses[1]),
+            n => {
+                error!(
+                    "could not format score - node has {} addresses while only 1 or 2 are allowed!",
+                    n
+                );
+                return Err(std::fmt::Error);
+            }
+        };
+        let stringified_key = self.pub_key.to_base58_string();
+        write!(
+            f,
+            "({})\t\t{}/{}\t({:.2}%) \t|| {}\tv{} <{}> - {}",
+            self.typ,
+            self.packets_received,
+            self.packets_sent,
+            self.score(),
+            self.layer,
+            self.version,
+            fmtd_addresses,
+            stringified_key,
+        )
+    }
+}

common/nymsphinx/Cargo.toml

@@ -0,0 +1,15 @@
+[package]
+name = "nymsphinx"
+version = "0.1.0"
+authors = ["Jedrzej Stuczynski <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+log = "0.4.8"
+rand = "0.7.3"
+
+## will be moved to proper dependencies once released
+sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" }
+

common/nymsphinx/src/addressing/mod.rs

@@ -0,0 +1 @@
+pub mod nodes;

common/nymsphinx/src/addressing/nodes.rs

@@ -0,0 +1,206 @@
+use sphinx::constants::NODE_ADDRESS_LENGTH;
+use sphinx::route::NodeAddressBytes;
+use std::convert::{TryFrom, TryInto};
+use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr};
+
+/// This module is responsible for encoding and decoding node routing information, so that
+/// they could be later put into an appropriate field in a sphinx header.
+/// Currently, that routing information is an IP address, but in principle it can be anything
+/// for as long as it's going to fit in the field.
+
+#[derive(Debug)]
+pub enum NymNodeRoutingAddressError {
+    InsufficientNumberOfBytesAvailableError,
+    InvalidIPVersion,
+}
+
+/// Current representation of Node routing information used in Nym system.
+/// At this point of time it is a simple `SocketAddr`.
+#[derive(Copy, Clone, PartialEq, Eq, Hash, Debug)]
+pub struct NymNodeRoutingAddress(SocketAddr);
+
+impl NymNodeRoutingAddress {
+    /// Minimum number of bytes that need to be available to represent self.
+    /// The value has no upper bound as when converted into bytes, it's always
+    /// padded with zeroes to be exactly NODE_ADDRESS_LENGTH long.
+    pub fn bytes_min_len(&self) -> usize {
+        match self.0 {
+            SocketAddr::V4(_) => 7,
+            SocketAddr::V6(_) => 19,
+        }
+    }
+
+    /// Converts self into a vector of bytes.
+    /// Note, this represents a generic bytes vector, not necessarily a NodeAddressBytes
+    /// and hence is not zero-padded.
+    pub fn as_bytes(&self) -> Vec<u8> {
+        let port_bytes = self.0.port().to_be_bytes();
+        let ip_octets_vec = match self.0.ip() {
+            IpAddr::V4(ip) => ip.octets().to_vec(),
+            IpAddr::V6(ip) => ip.octets().to_vec(),
+        };
+
+        std::iter::once(self.addr_type_as_u8())
+            .chain(port_bytes.iter().cloned())
+            .chain(ip_octets_vec.iter().cloned())
+            .collect()
+    }
+
+    /// Tries to recover `Self` from a bytes slice.
+    /// Does not care if it's zero-padded or not.
+    pub fn try_from_bytes(b: &[u8]) -> Result<Self, NymNodeRoutingAddressError> {
+        // the bare minimum to represent `Self` is 7 bytes (for the shorter V4 version)
+        if b.len() < 7 {
+            return Err(NymNodeRoutingAddressError::InsufficientNumberOfBytesAvailableError);
+        }
+
+        let ip_version = b[0];
+        let port: u16 = u16::from_be_bytes([b[1], b[2]]);
+        let ip = match ip_version {
+            4 => IpAddr::V4(Ipv4Addr::new(b[3], b[4], b[5], b[6])),
+            6 => {
+                if b.len() < 19 {
+                    return Err(
+                        NymNodeRoutingAddressError::InsufficientNumberOfBytesAvailableError,
+                    );
+                }
+                let mut address_octets = [0u8; 16];
+                address_octets.copy_from_slice(&b[3..19]);
+                IpAddr::V6(Ipv6Addr::from(address_octets))
+            }
+            _ => return Err(NymNodeRoutingAddressError::InvalidIPVersion),
+        };
+
+        Ok(Self(SocketAddr::new(ip, port)))
+    }
+
+    /// Single byte representation of self ip version.
+    pub fn addr_type_as_u8(&self) -> u8 {
+        match self.0 {
+            SocketAddr::V4(_) => 4,
+            SocketAddr::V6(_) => 6,
+        }
+    }
+}
+
+/// Considering `NymNodeRoutingAddress` is equivalent to a `SocketAddr` at this point,
+/// it makes perfect sense to allow the bilateral transformation.
+impl From<SocketAddr> for NymNodeRoutingAddress {
+    fn from(addr: SocketAddr) -> Self {
+        Self(addr)
+    }
+}
+
+/// Considering `NymNodeRoutingAddress` is equivalent to a `SocketAddr` at this point,
+/// it makes perfect sense to allow the bilateral transformation.
+impl Into<SocketAddr> for NymNodeRoutingAddress {
+    fn into(self) -> SocketAddr {
+        self.0
+    }
+}
+
+impl TryInto<NodeAddressBytes> for NymNodeRoutingAddress {
+    type Error = NymNodeRoutingAddressError;
+
+    /// `NymNodeRoutingAddress` (as a `SocketAddr`) is represented the following way:
+    /// VersionFlag || port || octets || zeropad
+    /// VersionFlag is one byte representing whether self is ipv4 or ipv6 address,
+    /// port is 16bit big endian representation of port value
+    /// octets is bytes representation of octets making up the ip address of the socket address
+    /// (either 4 bytes for ipv4 or 16 bytes for ipv6)
+    /// zeropad is padding of 0 for the `NymNodeRoutingAddress` to be
+    /// exactly `NODE_ADDRESS_LENGTH` long.
+    fn try_into(self) -> Result<NodeAddressBytes, Self::Error> {
+        // first check if we have enough bytes to represent `self`:
+        if self.bytes_min_len() > NODE_ADDRESS_LENGTH {
+            return Err(NymNodeRoutingAddressError::InsufficientNumberOfBytesAvailableError);
+        }
+
+        let unpadded_address = self.as_bytes();
+        let padded_address: Vec<_> = unpadded_address
+            .into_iter()
+            .chain(std::iter::repeat(0))
+            .take(NODE_ADDRESS_LENGTH)
+            .collect();
+
+        let mut node_address_bytes = [0u8; 32];
+        node_address_bytes.copy_from_slice(&padded_address);
+
+        Ok(NodeAddressBytes::from_bytes(node_address_bytes))
+    }
+}
+
+impl TryFrom<NodeAddressBytes> for NymNodeRoutingAddress {
+    type Error = NymNodeRoutingAddressError;
+
+    fn try_from(value: NodeAddressBytes) -> Result<Self, Self::Error> {
+        Self::try_from_bytes(value.as_bytes())
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn nym_node_routing_address_can_be_converted_to_and_from_bytes_for_v4_address() {
+        let address = NymNodeRoutingAddress(SocketAddr::new(IpAddr::from([1, 2, 3, 4]), 42));
+        let address_bytes = address.as_bytes();
+        assert_eq!(
+            address,
+            NymNodeRoutingAddress::try_from_bytes(&address_bytes).unwrap()
+        )
+    }
+
+    #[test]
+    fn nym_node_routing_address_can_be_converted_to_and_from_bytes_for_v6_address() {
+        let address = NymNodeRoutingAddress(SocketAddr::new(
+            IpAddr::from([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]),
+            42,
+        ));
+        let address_bytes = address.as_bytes();
+        assert_eq!(
+            address,
+            NymNodeRoutingAddress::try_from_bytes(&address_bytes).unwrap()
+        )
+    }
+
+    #[test]
+    fn nym_node_routing_address_can_be_converted_to_and_from_bytes_for_empty_v4_address() {
+        let address = NymNodeRoutingAddress(SocketAddr::new(IpAddr::from([0, 0, 0, 0]), 42));
+        let address_bytes = address.as_bytes();
+        assert_eq!(
+            address,
+            NymNodeRoutingAddress::try_from_bytes(&address_bytes).unwrap()
+        )
+    }
+
+    #[test]
+    fn nym_node_routing_address_can_be_converted_to_and_from_bytes_for_empty_v6_address() {
+        let address = NymNodeRoutingAddress(SocketAddr::new(
+            IpAddr::from([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]),
+            42,
+        ));
+        let address_bytes = address.as_bytes();
+        assert_eq!(
+            address,
+            NymNodeRoutingAddress::try_from_bytes(&address_bytes).unwrap()
+        )
+    }
+
+    #[test]
+    fn nym_node_routing_address_can_be_converted_to_and_from_node_address_bytes_with_no_data_loss()
+    {
+        let address_v4 = NymNodeRoutingAddress(SocketAddr::new(IpAddr::from([1, 2, 3, 4]), 42));
+        let address_v6 = NymNodeRoutingAddress(SocketAddr::new(
+            IpAddr::from([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]),
+            42,
+        ));
+
+        let node_address1: NodeAddressBytes = address_v4.try_into().unwrap();
+        let node_address2: NodeAddressBytes = address_v6.try_into().unwrap();
+
+        assert_eq!(address_v4, node_address1.try_into().unwrap());
+        assert_eq!(address_v6, node_address2.try_into().unwrap());
+    }
+}

common/nymsphinx/src/chunking/mod.rs

@@ -0,0 +1,56 @@
+use crate::chunking::set::split_into_sets;
+
+pub mod fragment;
+pub mod reconstruction;
+pub mod set;
+
+/// The idea behind the process of chunking is to incur as little data overhead as possible due
+/// to very computationally costly sphinx encapsulation procedure.
+///
+/// To achieve this, the underlying message is split into so-called "sets", which are further
+/// subdivided into the base unit of "fragment" that is directly encapsulated by a Sphinx packet.
+/// This allows to encapsulate messages of arbitrary length.
+///
+/// Each message, regardless of its size, consists of at least a single `Set` that has at least
+/// a single `Fragment`.
+///
+/// Each `Fragment` can have variable, yet fully deterministic, length,
+/// that depends on its position in the set as well as total number of sets. This is further
+/// explained in `fragment.rs` file.  
+///
+/// Similarly, each `Set` can have a variable number of `Fragment`s inside. However, that
+/// value is more restrictive: if it's the last set into which the message was split
+/// (or implicitly the only one), it has no lower bound on the number of `Fragment`s.
+/// (Apart from the restriction of containing at least a single one). If the set is located
+/// somewhere in the middle, *it must be* full. Finally, regardless of its position, it must also be
+/// true that it contains no more than `u8::max_value()`, i.e. 255 `Fragment`s.
+/// Again, the reasoning for this is further explained in `set.rs` file. However, you might
+/// also want to look at `fragment.rs` to understand the full context behind that design choice.
+///
+/// Both of those concepts as well as their structures, i.e. `Set` and `Fragment`
+/// are further explained in the respective files.
+
+#[derive(PartialEq, Debug)]
+pub enum ChunkingError {
+    InvalidPayloadLengthError,
+    TooBigMessageToSplit,
+    MalformedHeaderError,
+    NoValidProvidersError,
+    NoValidRoutesAvailableError,
+    InvalidTopologyError,
+    TooShortFragmentData,
+    MalformedFragmentData,
+    UnexpectedFragmentCount,
+}
+
+/// Takes the entire message and splits it into bytes chunks that will fit into sphinx packets
+/// directly. After receiving they can be combined using `reconstruction::MessageReconstructor`
+/// to obtain the original message back.
+pub fn split_and_prepare_payloads(message: &[u8]) -> Vec<Vec<u8>> {
+    let fragmented_messages = split_into_sets(message);
+    fragmented_messages
+        .into_iter()
+        .flat_map(|fragment_set| fragment_set.into_iter())
+        .map(|fragment| fragment.into_bytes())
+        .collect()
+}

common/nymsphinx/src/chunking/set.rs

@@ -0,0 +1,788 @@
+use crate::chunking::fragment::{
+    Fragment, LINKED_FRAGMENTED_HEADER_LEN, LINKED_FRAGMENTED_PAYLOAD_MAX_LEN,
+    UNFRAGMENTED_PAYLOAD_MAX_LEN, UNLINKED_FRAGMENTED_HEADER_LEN,
+    UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN,
+};
+use rand::Rng;
+
+/// In the simplest case of message being divided into a single set, the set has the upper bound
+/// on its payload length of the maximum number of `Fragment`s multiplied by their maximum,
+/// fragmented, length.
+pub const MAX_UNLINKED_SET_PAYLOAD_LENGTH: usize =
+    u8::max_value() as usize * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN;
+
+/// If the set is being linked to another one, by either being the very first set, or the very last,
+/// one of its `Fragment`s must be changed from "unlinked" into "linked" to compensate for a tiny
+/// bit extra data overhead: id of the other set.
+/// Note that the "MAX" prefix only applies to if the set is the last one as it does not have
+/// a lower bound on its length. If the set is one way linked and a first one, it *must have*
+/// this exact payload length instead.
+pub const MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH: usize = MAX_UNLINKED_SET_PAYLOAD_LENGTH
+    - (LINKED_FRAGMENTED_HEADER_LEN - UNLINKED_FRAGMENTED_HEADER_LEN);
+
+/// If the set is being linked two others sets by being stuck in the middle of divided message,
+/// two of its `Fragment`s (first and final one) must be changed from
+/// "unlinked" into "linked" to compensate for data overhead.
+/// Note that this constant no longer has a "MAX" prefix, this is because each set being stuck
+/// between different sets, *must* have this exact payload length.
+pub const TWO_WAY_LINKED_SET_PAYLOAD_LENGTH: usize = MAX_UNLINKED_SET_PAYLOAD_LENGTH
+    - 2 * (LINKED_FRAGMENTED_HEADER_LEN - UNLINKED_FRAGMENTED_HEADER_LEN);
+
+/// `FragmentSet` is an ordered collection of 1 to 255 `Fragment`s, each with the same ID
+/// that can be used to produce original message, assuming no linking took place.
+///
+/// Otherwise, if set linking took place, then first or last `Fragment` from the `FragmentSet`
+/// is used to determine preceding or succeeding other `FragmentSet`
+/// that should be used in tandem to reconstruct original message. The linking reconstruction
+/// is a recursive process as a message could have been divided into an arbitrary number of
+/// `FragmentSet`s with no upper bound at all.
+///
+/// For example if a message was divided into 300 `Fragment`s (i.e. 2 `FragmentSet`s,
+/// the structures might look as follows:
+///
+/// Set1: [f1 {id = 12345}, f2 {id = 12345},  ... f255 {id = 12345, next_id = 54321}]
+/// Set2: [f1 {id = 54321, previous_id = 12345}, f2 {id = 54321}, ... f45 {id = 54321}]
+pub(crate) type FragmentSet = Vec<Fragment>;
+
+/// Generate a pseudo-random id for a `FragmentSet`.
+/// Its value is restricted to (0, i32::max_value()].
+/// Note that it *excludes* 0, but *includes* i32::max_value().
+/// This particular range allows for the id to be represented using 31bits, rather than
+/// the full length of 32 while still providing more than enough variability to
+/// distinguish different `FragmentSet`s.
+/// The extra bit, as explained in `Fragment` definition is used to represents additional information,
+/// indicating how further bytes should be parsed.
+/// This approach saves whole byte per `Fragment`, which while may seem insignificant and
+/// introduces extra complexity, quickly adds up when faced with sphinx packet encapsulation for longer
+/// messages.
+/// Finally, the reason 0 id is not allowed is to explicitly distinguish it from unfragmented
+/// `Fragment`s thus allowing for some additional optimizations by letting it skip
+/// certain procedures when reconstructing.
+fn generate_set_id<R: Rng>(rng: &mut R) -> i32 {
+    let potential_id = rng.gen::<i32>().abs();
+    // make sure id is always non-zero, as we do not want to accidentally have weird
+    // reconstruction cases where unfragmented payload overwrites some part of set with id0
+    if potential_id == 0 {
+        generate_set_id(rng)
+    } else {
+        potential_id
+    }
+}
+
+/// The simplest case of splitting underlying message - when it fits into a single
+/// `Fragment` thus requiring no linking or even a set id.
+/// For obvious reasons the most efficient approach.
+fn prepare_unfragmented_set(message: &[u8]) -> FragmentSet {
+    vec![Fragment::try_new_unfragmented(&message).unwrap()]
+}
+
+/// Splits underlying message into multiple `Fragment`s while all of them fit in a single
+/// `Set` (number of `Fragment`s <= 255)
+fn prepare_unlinked_fragmented_set(message: &[u8], id: i32) -> FragmentSet {
+    let pre_casted_frags =
+        (message.len() as f64 / UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN as f64).ceil() as usize;
+
+    debug_assert!(pre_casted_frags <= u8::max_value() as usize);
+    let num_fragments = pre_casted_frags as u8;
+
+    let mut fragments = Vec::with_capacity(num_fragments as usize);
+
+    for i in 1..(pre_casted_frags + 1) {
+        // we can't use u8 directly here as upper (NON-INCLUSIVE, so it would always fit) bound could be u8::max_value() + 1
+        let lb = (i as usize - 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN;
+        let ub = usize::min(
+            message.len(),
+            i as usize * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN,
+        );
+        fragments.push(
+            Fragment::try_new_fragmented(&message[lb..ub], id, num_fragments, i as u8, None, None)
+                .unwrap(),
+        )
+    }
+
+    fragments
+}
+
+/// Similarly to `prepare_unlinked_fragmented_set`, splits part of underlying message into
+/// multiple `Fragment`s. The byte slice of the message *must* fit into a single linked set, however,
+/// the whole message itself is still longer than a single `Set` (number of `Fragment`s > 255).
+/// During the process of splitting message, this function is called multiple times.
+fn prepare_linked_fragment_set(
+    message: &[u8],
+    id: i32,
+    previous_link_id: Option<i32>,
+    next_link_id: Option<i32>,
+) -> FragmentSet {
+    // determine number of fragments in the set:
+    let num_frags_usize = if next_link_id.is_some() {
+        u8::max_value() as usize
+    } else {
+        // we know this set is linked, if it's not post-linked then it MUST BE pre-linked
+        let tail_len = if message.len() >= LINKED_FRAGMENTED_PAYLOAD_MAX_LEN {
+            message.len() - LINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+        } else {
+            0
+        };
+        let pre_casted_frags =
+            1 + (tail_len as f64 / UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN as f64).ceil() as usize;
+        if pre_casted_frags > u8::max_value() as usize {
+            panic!("message would produce too many fragments!")
+        };
+        pre_casted_frags
+    };
+
+    // determine bounds for the first fragment which depends on whether set is pre-linked
+    let mut lb = 0;
+    let mut ub = if previous_link_id.is_some() {
+        usize::min(message.len(), LINKED_FRAGMENTED_PAYLOAD_MAX_LEN)
+    } else {
+        // the set might be linked, but fragment itself is not (i.e. the set is linked at the tail)
+        UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+    };
+
+    let mut fragments = Vec::with_capacity(num_frags_usize);
+    for i in 1..(num_frags_usize + 1) {
+        // we can't use u8 directly here as upper (NON-INCLUSIVE, so i would always fit) bound could be u8::max_value() + 1
+        let fragment = Fragment::try_new_fragmented(
+            &message[lb..ub],
+            id,
+            num_frags_usize as u8,
+            i as u8,
+            if i == 1 { previous_link_id } else { None },
+            if i == num_frags_usize {
+                next_link_id
+            } else {
+                None
+            },
+        )
+        .unwrap();
+
+        fragments.push(fragment);
+        // update bounds for the next fragment
+        lb = ub;
+        ub = usize::min(message.len(), ub + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN);
+    }
+
+    fragments
+}
+
+/// Based on total message length, determines the number of sets into which it is going to be split.
+fn total_number_of_sets(message_len: usize) -> usize {
+    if message_len <= MAX_UNLINKED_SET_PAYLOAD_LENGTH {
+        1
+    } else if message_len > MAX_UNLINKED_SET_PAYLOAD_LENGTH
+        && message_len <= 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH
+    {
+        2
+    } else {
+        let len_without_edges = message_len - 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH;
+        // every set in between edges must be two way linked
+        (len_without_edges as f64 / TWO_WAY_LINKED_SET_PAYLOAD_LENGTH as f64).ceil() as usize + 2
+    }
+}
+
+/// Given part of the underlying message as well id of the set as well as its potential linked sets,
+/// correctly delegates to appropriate set constructor.
+fn prepare_fragment_set(
+    message: &[u8],
+    id: i32,
+    previous_link_id: Option<i32>,
+    next_link_id: Option<i32>,
+) -> FragmentSet {
+    if previous_link_id.is_some() || next_link_id.is_some() {
+        prepare_linked_fragment_set(message, id, previous_link_id, next_link_id)
+    } else if message.len() > UNFRAGMENTED_PAYLOAD_MAX_LEN {
+        // the bounds on whether the message fits in an unlinked set should have been done by the callee
+        // when determining ids of other sets
+        prepare_unlinked_fragmented_set(message, id)
+    } else {
+        prepare_unfragmented_set(message)
+    }
+}
+
+/// Entry point for splitting whole message into possibly multiple `Set`s.
+pub(crate) fn split_into_sets(message: &[u8]) -> Vec<FragmentSet> {
+    use rand::thread_rng;
+
+    let mut rng = thread_rng();
+    let num_of_sets = total_number_of_sets(message.len());
+    if num_of_sets == 1 {
+        let set_id = generate_set_id(&mut rng);
+        vec![prepare_fragment_set(message, set_id, None, None)]
+    } else {
+        let mut sets = Vec::with_capacity(num_of_sets);
+        // pre-generate all ids for the sets
+        let set_ids: Vec<_> = std::iter::repeat(())
+            .map(|_| generate_set_id(&mut rng))
+            .take(num_of_sets)
+            .collect();
+
+        // initial bounds for the set payloads
+        let mut lb = 0;
+        let mut ub = MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH;
+
+        for i in 0..num_of_sets {
+            let fragment_set = prepare_fragment_set(
+                &message[lb..ub],
+                set_ids[i],
+                if i == 0 { None } else { Some(set_ids[i - 1]) },
+                if i == (num_of_sets - 1) {
+                    None
+                } else {
+                    Some(set_ids[i + 1])
+                },
+            );
+
+            sets.push(fragment_set);
+            // update bounds for the next set
+            lb = ub;
+            ub = if i == num_of_sets - 2 {
+                // we're going to go into the last iteration now, hence the last set will be one-way linked
+                usize::min(message.len(), ub + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH)
+            } else {
+                usize::min(message.len(), ub + TWO_WAY_LINKED_SET_PAYLOAD_LENGTH)
+            }
+        }
+
+        sets
+    }
+}
+
+// reason for top level tests module is to be able to use the helper functions to verify sets payloads
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    fn verify_unlinked_set_payload(mut set: FragmentSet, payload: &[u8]) {
+        for i in (0..set.len()).rev() {
+            assert_eq!(
+                set.pop().unwrap().extract_payload(),
+                payload[i * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+                    ..usize::min(payload.len(), (i + 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN)]
+                    .to_vec()
+            )
+        }
+    }
+
+    fn verify_pre_linked_set_payload(mut set: FragmentSet, payload: &[u8]) {
+        for i in (0..set.len()).rev() {
+            let lb = if i == 0 {
+                0
+            } else {
+                (i - 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + LINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+            };
+            let ub = usize::min(
+                payload.len(),
+                i * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + LINKED_FRAGMENTED_PAYLOAD_MAX_LEN,
+            );
+
+            assert_eq!(
+                set.pop().unwrap().extract_payload(),
+                payload[lb..ub].to_vec()
+            )
+        }
+    }
+
+    fn verify_post_linked_set_payload(mut set: FragmentSet, payload: &[u8]) {
+        for i in (0..set.len()).rev() {
+            let lb = i * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN;
+            let ub = if i == (u8::max_value() as usize - 1) {
+                i * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + LINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+            } else {
+                (i + 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+            };
+
+            assert_eq!(
+                set.pop().unwrap().extract_payload(),
+                payload[lb..ub].to_vec(),
+            )
+        }
+    }
+
+    fn verify_two_way_linked_set_payload(mut set: FragmentSet, payload: &[u8]) {
+        for i in (0..set.len()).rev() {
+            let lb = if i == 0 {
+                0
+            } else {
+                (i - 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + LINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+            };
+            let ub = if i == (u8::max_value() as usize - 1) {
+                (i - 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+                    + 2 * LINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+            } else {
+                i * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + LINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+            };
+
+            assert_eq!(
+                set.pop().unwrap().extract_payload(),
+                payload[lb..ub].to_vec(),
+            )
+        }
+    }
+
+    fn verfiy_correct_link(left: &FragmentSet, right: &FragmentSet) {
+        let first_id = left[0].id();
+        let post_id = left[254].next_fragments_set_id().unwrap();
+
+        let second_id = right[0].id();
+        let pre_id = right[0].previous_fragments_set_id().unwrap();
+
+        assert_eq!(first_id, pre_id);
+        assert_eq!(second_id, post_id);
+    }
+
+    #[cfg(test)]
+    mod preparing_unfragmented_set {
+        use super::*;
+
+        #[test]
+        fn makes_set_with_single_unfragmented_element_for_valid_message_lengths() {
+            let mut set = prepare_unfragmented_set(&[1]);
+            assert_eq!(1, set.len());
+            assert_eq!(set.pop().unwrap().extract_payload(), [1].to_vec());
+
+            let mut set = prepare_unfragmented_set(&[1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN]);
+            assert_eq!(1, set.len());
+            assert_eq!(
+                set.pop().unwrap().extract_payload(),
+                [1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN].to_vec()
+            );
+        }
+
+        #[test]
+        #[should_panic]
+        fn panics_for_too_long_payload() {
+            prepare_unfragmented_set(&[1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN + 1]);
+        }
+    }
+
+    #[cfg(test)]
+    mod preparing_unlinked_set {
+        // remember this this is only called for a sole set with <= 255 fragments
+        use super::*;
+        use rand::{thread_rng, RngCore};
+
+        #[test]
+        fn makes_set_with_correctly_split_payload() {
+            let id = 12345;
+            let mut rng = thread_rng();
+
+            let mut two_element_set_payload = [0u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1];
+            rng.fill_bytes(&mut two_element_set_payload);
+            let two_element_set = prepare_unlinked_fragmented_set(&two_element_set_payload, id);
+            assert_eq!(2, two_element_set.len());
+            verify_unlinked_set_payload(two_element_set, &two_element_set_payload);
+
+            let mut forty_two_element_set_payload =
+                [0u8; 41 * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 42];
+            rng.fill_bytes(&mut forty_two_element_set_payload);
+            let forty_two_element_set =
+                prepare_unlinked_fragmented_set(&forty_two_element_set_payload, id);
+            assert_eq!(42, forty_two_element_set.len());
+            verify_unlinked_set_payload(forty_two_element_set, &forty_two_element_set_payload);
+
+            let mut max_fragments_set_payload =
+                [0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH - UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1]; // last fragment should have a single byte of data
+            rng.fill_bytes(&mut max_fragments_set_payload);
+            let max_fragment_set = prepare_unlinked_fragmented_set(&max_fragments_set_payload, id);
+            assert_eq!(u8::max_value() as usize, max_fragment_set.len());
+            verify_unlinked_set_payload(max_fragment_set, &max_fragments_set_payload);
+
+            let mut full_set_payload = [0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH];
+            rng.fill_bytes(&mut full_set_payload);
+            let full_fragment_set = prepare_unlinked_fragmented_set(&full_set_payload, id);
+            assert_eq!(u8::max_value() as usize, full_fragment_set.len());
+            verify_unlinked_set_payload(full_fragment_set, &full_set_payload);
+        }
+
+        #[test]
+        #[should_panic]
+        fn panics_for_too_long_payload() {
+            prepare_unlinked_fragmented_set(&[0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH + 1], 12345);
+        }
+    }
+
+    #[cfg(test)]
+    mod preparing_linked_set {
+        use super::*;
+        use rand::{thread_rng, RngCore};
+
+        #[test]
+        fn makes_set_with_correctly_split_payload_for_pre_linked_set() {
+            let id = 12345;
+            let link_id = 1234;
+            let mut rng = thread_rng();
+
+            let mut two_element_set_payload = [0u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1];
+            rng.fill_bytes(&mut two_element_set_payload);
+            let two_element_set =
+                prepare_linked_fragment_set(&two_element_set_payload, id, Some(link_id), None);
+            assert_eq!(2, two_element_set.len());
+            verify_pre_linked_set_payload(two_element_set, &two_element_set_payload);
+
+            let mut forty_two_element_set_payload = [0u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+                + 40 * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN
+                + 42];
+            rng.fill_bytes(&mut forty_two_element_set_payload);
+            let forty_two_element_set = prepare_linked_fragment_set(
+                &forty_two_element_set_payload,
+                id,
+                Some(link_id),
+                None,
+            );
+            assert_eq!(42, forty_two_element_set.len());
+            verify_pre_linked_set_payload(forty_two_element_set, &forty_two_element_set_payload);
+
+            let mut max_fragments_set_payload =
+                [0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH - LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1]; // last fragment should have a single byte of data
+            rng.fill_bytes(&mut max_fragments_set_payload);
+
+            let max_fragment_set =
+                prepare_linked_fragment_set(&max_fragments_set_payload, id, Some(link_id), None);
+            assert_eq!(u8::max_value() as usize, max_fragment_set.len());
+            verify_pre_linked_set_payload(max_fragment_set, &max_fragments_set_payload);
+
+            let mut full_set_payload = [0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH];
+            rng.fill_bytes(&mut full_set_payload);
+            let full_fragment_set =
+                prepare_linked_fragment_set(&full_set_payload, id, Some(link_id), None);
+            assert_eq!(u8::max_value() as usize, full_fragment_set.len());
+            verify_pre_linked_set_payload(full_fragment_set, &full_set_payload);
+        }
+
+        #[test]
+        #[should_panic]
+        fn panics_for_too_long_payload_for_pre_linked_set() {
+            prepare_linked_fragment_set(
+                &[0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 1],
+                12345,
+                Some(1234),
+                None,
+            );
+        }
+
+        #[test]
+        fn makes_set_with_correctly_split_payload_for_post_linked_set() {
+            let id = 12345;
+            let link_id = 1234;
+            let mut rng = thread_rng();
+
+            // if set is post-linked, there is only a single valid case - full length payload
+            let mut full_set_payload = [0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH];
+            rng.fill_bytes(&mut full_set_payload);
+            let full_fragment_set =
+                prepare_linked_fragment_set(&full_set_payload, id, None, Some(link_id));
+            assert_eq!(u8::max_value() as usize, full_fragment_set.len());
+            verify_post_linked_set_payload(full_fragment_set, &full_set_payload);
+        }
+
+        #[test]
+        #[should_panic]
+        fn panics_for_too_long_payload_for_post_linked_set() {
+            prepare_linked_fragment_set(
+                &[0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 1],
+                12345,
+                None,
+                Some(1234),
+            );
+        }
+
+        #[test]
+        #[should_panic]
+        fn panics_for_too_short_payload_for_post_linked_set() {
+            prepare_linked_fragment_set(
+                &[0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH - 1],
+                12345,
+                None,
+                Some(1234),
+            );
+        }
+
+        #[test]
+        fn makes_set_with_correctly_split_payload_for_two_way_linked_set() {
+            // again, relatively simple case -
+            // if set is two-way-linked, there is only a single valid case - full length payload
+            let id = 12345;
+            let pre_link_id = 1234;
+            let post_link_id = 123456;
+            let mut rng = thread_rng();
+
+            let mut full_set_payload = [0u8; TWO_WAY_LINKED_SET_PAYLOAD_LENGTH];
+            rng.fill_bytes(&mut full_set_payload);
+            let full_fragment_set = prepare_linked_fragment_set(
+                &full_set_payload,
+                id,
+                Some(pre_link_id),
+                Some(post_link_id),
+            );
+            assert_eq!(u8::max_value() as usize, full_fragment_set.len());
+            verify_two_way_linked_set_payload(full_fragment_set, &full_set_payload);
+        }
+
+        #[test]
+        #[should_panic]
+        fn panics_for_too_long_payload_for_two_way_linked_set() {
+            prepare_linked_fragment_set(
+                &[0u8; TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + 1],
+                12345,
+                Some(123456),
+                Some(1234),
+            );
+        }
+
+        #[test]
+        #[should_panic]
+        fn panics_for_too_short_payload_for_two_way_linked_set() {
+            prepare_linked_fragment_set(
+                &[0u8; TWO_WAY_LINKED_SET_PAYLOAD_LENGTH - 1],
+                12345,
+                Some(123456),
+                Some(1234),
+            );
+        }
+    }
+
+    #[cfg(test)]
+    mod splitting_into_sets {
+        use super::*;
+        use rand::{thread_rng, RngCore};
+
+        #[test]
+        fn correctly_creates_set_with_single_unfragmented_element_when_expected() {
+            let mut rng = thread_rng();
+            let tiny_message = [1, 2, 3, 4, 5];
+            let mut max_unfragmented_message = [0u8; UNFRAGMENTED_PAYLOAD_MAX_LEN];
+            rng.fill_bytes(&mut max_unfragmented_message);
+
+            let mut sets = split_into_sets(&tiny_message);
+            assert_eq!(1, sets.len());
+            assert_eq!(1, sets[0].len());
+            assert_eq!(
+                tiny_message.to_vec(),
+                sets.pop().unwrap().pop().unwrap().extract_payload()
+            );
+
+            let mut sets = split_into_sets(&max_unfragmented_message);
+            assert_eq!(1, sets.len());
+            assert_eq!(1, sets[0].len());
+            assert_eq!(
+                max_unfragmented_message.to_vec(),
+                sets.pop().unwrap().pop().unwrap().extract_payload()
+            );
+        }
+
+        #[test]
+        fn correctly_creates_single_fragmented_set_when_expected() {
+            let mut rng = thread_rng();
+            let mut message = [0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH - 2345];
+            rng.fill_bytes(&mut message);
+
+            let mut sets = split_into_sets(&message);
+            assert_eq!(1, sets.len());
+            verify_unlinked_set_payload(sets.pop().unwrap(), &message);
+        }
+
+        // a very specific test case that would have saved a lot of headache if was introduced
+        // earlier...
+        #[test]
+        fn correctly_creates_two_singly_linked_sets_with_second_set_containing_data_fitting_in_unfragmented_payload(
+        ) {
+            let mut rng = thread_rng();
+            let mut message = [0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 123];
+            rng.fill_bytes(&mut message);
+            let mut sets = split_into_sets(&message);
+            assert_eq!(2, sets.len());
+            verfiy_correct_link(&sets[0], &sets[1]);
+            verify_pre_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH..],
+            );
+            verify_post_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[..MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH],
+            );
+        }
+
+        #[test]
+        fn correctly_creates_two_singly_linked_sets_when_expected() {
+            let mut rng = thread_rng();
+            let mut message = [0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 2345];
+            rng.fill_bytes(&mut message);
+            let mut sets = split_into_sets(&message);
+            assert_eq!(2, sets.len());
+            verfiy_correct_link(&sets[0], &sets[1]);
+            verify_pre_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH..],
+            );
+            verify_post_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[..MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH],
+            );
+
+            let mut message = [0u8; 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH];
+            rng.fill_bytes(&mut message);
+
+            let mut sets = split_into_sets(&message);
+            assert_eq!(2, sets.len());
+            assert_eq!(sets[0].len(), u8::max_value() as usize);
+            assert_eq!(sets[1].len(), u8::max_value() as usize);
+            verfiy_correct_link(&sets[0], &sets[1]);
+            verify_pre_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH..],
+            );
+
+            verify_post_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[..MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH],
+            );
+        }
+
+        #[test]
+        fn correctly_creates_four_correctly_formed_sets_when_expected() {
+            let mut rng = thread_rng();
+            let mut message = [0u8; 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH
+                + 2345];
+            rng.fill_bytes(&mut message);
+            let mut sets = split_into_sets(&message);
+            assert_eq!(4, sets.len());
+            assert_eq!(sets[0].len(), u8::max_value() as usize);
+            assert_eq!(sets[1].len(), u8::max_value() as usize);
+            assert_eq!(sets[2].len(), u8::max_value() as usize);
+
+            verfiy_correct_link(&sets[0], &sets[1]);
+            verfiy_correct_link(&sets[1], &sets[2]);
+            verfiy_correct_link(&sets[2], &sets[3]);
+            verify_pre_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                    + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH..],
+            );
+            verify_two_way_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH
+                    ..2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                        + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH],
+            );
+            verify_two_way_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH
+                    ..TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH],
+            );
+            verify_post_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[..MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH],
+            );
+
+            let mut message = [0u8; 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH];
+            rng.fill_bytes(&mut message);
+
+            let mut sets = split_into_sets(&message);
+            assert_eq!(4, sets.len());
+            assert_eq!(sets[0].len(), u8::max_value() as usize);
+            assert_eq!(sets[1].len(), u8::max_value() as usize);
+            assert_eq!(sets[2].len(), u8::max_value() as usize);
+            assert_eq!(sets[3].len(), u8::max_value() as usize);
+
+            verfiy_correct_link(&sets[0], &sets[1]);
+            verfiy_correct_link(&sets[1], &sets[2]);
+            verfiy_correct_link(&sets[2], &sets[3]);
+            verify_pre_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                    + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH..],
+            );
+            verify_two_way_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH
+                    ..2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                        + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH],
+            );
+            verify_two_way_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH
+                    ..TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH],
+            );
+            verify_post_linked_set_payload(
+                sets.pop().unwrap(),
+                &message[..MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH],
+            );
+        }
+    }
+
+    #[cfg(test)]
+    mod helpers {
+        use super::*;
+
+        #[test]
+        fn total_number_of_sets() {
+            assert_eq!(
+                1,
+                super::total_number_of_sets(MAX_UNLINKED_SET_PAYLOAD_LENGTH - 1)
+            );
+            assert_eq!(
+                1,
+                super::total_number_of_sets(MAX_UNLINKED_SET_PAYLOAD_LENGTH)
+            );
+            assert_eq!(
+                2,
+                super::total_number_of_sets(MAX_UNLINKED_SET_PAYLOAD_LENGTH + 1)
+            );
+            assert_eq!(
+                2,
+                super::total_number_of_sets(2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH)
+            );
+            assert_eq!(
+                3,
+                super::total_number_of_sets(2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 1)
+            );
+            assert_eq!(
+                3,
+                super::total_number_of_sets(
+                    2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                        - 1
+                )
+            );
+            assert_eq!(
+                3,
+                super::total_number_of_sets(
+                    2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                )
+            );
+            assert_eq!(
+                4,
+                super::total_number_of_sets(
+                    2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH
+                        + TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                        + 1
+                )
+            );
+            assert_eq!(
+                4,
+                super::total_number_of_sets(
+                    2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH
+                        + 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                        - 1
+                )
+            );
+            assert_eq!(
+                4,
+                super::total_number_of_sets(
+                    2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH
+                        + 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                )
+            );
+            assert_eq!(
+                5,
+                super::total_number_of_sets(
+                    2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH
+                        + 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH
+                        + 1
+                )
+            );
+        }
+    }
+}

common/nymsphinx/src/lib.rs

@@ -0,0 +1,9 @@
+pub mod addressing;
+pub mod chunking;
+
+// Future consideration: currently in a lot of places, the payloads have randomised content
+// which is not a perfect testing strategy as it might not detect some edge cases I never would
+// have assumed could be possible. A better approach would be to research some Fuzz testing
+// library like: https://github.com/rust-fuzz/afl.rs and use that instead for the inputs.
+
+// perhaps it might be useful down the line for interaction testing between client,mixes,etc?

common/pemstore/Cargo.toml

@@ -0,0 +1,15 @@
+[package]
+name = "pemstore"
+version = "0.1.0"
+authors = ["Jedrzej Stuczynski <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+log = "0.4"
+pem = "0.7.0"
+pretty_env_logger = "0.3"
+
+## internal
+crypto = {path = "../crypto"}

common/pemstore/src/pathfinder.rs

@@ -0,0 +1,15 @@
+use std::path::PathBuf;
+
+pub trait PathFinder {
+    fn config_dir(&self) -> PathBuf;
+    fn private_identity_key(&self) -> PathBuf;
+    fn public_identity_key(&self) -> PathBuf;
+
+    // Optional:
+    fn private_encryption_key(&self) -> Option<PathBuf> {
+        None
+    }
+    fn public_encryption_key(&self) -> Option<PathBuf> {
+        None
+    }
+}

common/pemstore/src/pemstore.rs

@@ -0,0 +1,118 @@
+use crate::pathfinder::PathFinder;
+use crypto::identity::MixIdentityKeyPair;
+use crypto::PemStorableKey;
+use crypto::{encryption, PemStorableKeyPair};
+use log::info;
+use pem::{encode, parse, Pem};
+use std::fs::File;
+use std::io;
+use std::io::prelude::*;
+use std::path::PathBuf;
+
+pub struct PemStore {
+    #[allow(dead_code)]
+    config_dir: PathBuf,
+    private_mix_key_file: PathBuf,
+    public_mix_key_file: PathBuf,
+}
+
+impl PemStore {
+    pub fn new<P: PathFinder>(pathfinder: P) -> PemStore {
+        PemStore {
+            config_dir: pathfinder.config_dir(),
+            private_mix_key_file: pathfinder.private_identity_key(),
+            public_mix_key_file: pathfinder.public_identity_key(),
+        }
+    }
+
+    pub fn read_keypair<T: PemStorableKeyPair>(&self) -> io::Result<T> {
+        let private_pem = self.read_pem_file(self.private_mix_key_file.clone())?;
+        let public_pem = self.read_pem_file(self.public_mix_key_file.clone())?;
+
+        let key_pair = T::from_bytes(&private_pem.contents, &public_pem.contents);
+
+        if key_pair.private_key().pem_type() != private_pem.tag {
+            return Err(io::Error::new(
+                io::ErrorKind::Other,
+                "unexpected private key pem tag",
+            ));
+        }
+
+        if key_pair.public_key().pem_type() != public_pem.tag {
+            return Err(io::Error::new(
+                io::ErrorKind::Other,
+                "unexpected public key pem tag",
+            ));
+        }
+
+        Ok(key_pair)
+    }
+
+    pub fn read_encryption(&self) -> io::Result<encryption::KeyPair> {
+        self.read_keypair()
+    }
+
+    pub fn read_identity(&self) -> io::Result<MixIdentityKeyPair> {
+        self.read_keypair()
+    }
+
+    fn read_pem_file(&self, filepath: PathBuf) -> io::Result<Pem> {
+        let mut pem_bytes = File::open(filepath)?;
+        let mut buf = Vec::new();
+        pem_bytes.read_to_end(&mut buf)?;
+        parse(&buf).map_err(|e| io::Error::new(io::ErrorKind::Other, e))
+    }
+
+    fn write_keypair(&self, key_pair: impl PemStorableKeyPair) -> io::Result<()> {
+        let private_key = key_pair.private_key();
+        let public_key = key_pair.public_key();
+
+        self.write_pem_file(
+            self.private_mix_key_file.clone(),
+            private_key.to_bytes(),
+            private_key.pem_type(),
+        )?;
+        info!(
+            "Written private key to {:?}",
+            self.private_mix_key_file.clone()
+        );
+        self.write_pem_file(
+            self.public_mix_key_file.clone(),
+            public_key.to_bytes(),
+            public_key.pem_type(),
+        )?;
+        info!(
+            "Written public key to {:?}",
+            self.public_mix_key_file.clone()
+        );
+        Ok(())
+    }
+
+    // This should be refactored and made more generic for when we have other kinds of
+    // KeyPairs that we want to persist (e.g. validator keypairs, or keys for
+    // signing vs encryption). However, for the moment, it does the job.
+    pub fn write_identity(&self, key_pair: MixIdentityKeyPair) -> io::Result<()> {
+        self.write_keypair(key_pair)
+    }
+
+    pub fn write_encryption_keys(&self, key_pair: encryption::KeyPair) -> io::Result<()> {
+        self.write_keypair(key_pair)
+    }
+
+    fn write_pem_file(&self, filepath: PathBuf, data: Vec<u8>, tag: String) -> io::Result<()> {
+        // ensure the whole directory structure exists
+        if let Some(parent_dir) = filepath.parent() {
+            std::fs::create_dir_all(parent_dir)?;
+        }
+        let pem = Pem {
+            tag,
+            contents: data,
+        };
+        let key = encode(&pem);
+
+        let mut file = File::create(filepath)?;
+        file.write_all(key.as_bytes())?;
+
+        Ok(())
+    }
+}

common/topology/Cargo.toml

@@ -0,0 +1,23 @@
+[package]
+name = "topology"
+version = "0.1.0"
+authors = ["Jedrzej Stuczynski <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+bs58 = "0.3.0"
+itertools = "0.8.2"
+log = "0.4"
+pretty_env_logger = "0.3"
+rand = "0.7.2"
+serde = { version = "1.0.104", features = ["derive"] }
+
+## internal
+nymsphinx = {path = "../nymsphinx"}
+version-checker = {path = "../version-checker" }
+
+## will be moved to proper dependencies once released
+sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" }
+# sphinx = { path = "../../../sphinx"}

common/topology/src/coco.rs

@@ -0,0 +1,16 @@
+use crate::filter;
+
+#[derive(Debug, Clone)]
+pub struct Node {
+    pub location: String,
+    pub host: String,
+    pub pub_key: String,
+    pub last_seen: u64,
+    pub version: String,
+}
+
+impl filter::Versioned for Node {
+    fn version(&self) -> String {
+        self.version.clone()
+    }
+}

common/topology/src/filter.rs

@@ -0,0 +1,18 @@
+pub trait Versioned: Clone {
+    fn version(&self) -> String;
+}
+
+pub trait VersionFilterable<T> {
+    fn filter_by_version(&self, expected_version: &str) -> Self;
+}
+
+impl<T: Versioned> VersionFilterable<T> for Vec<T> {
+    fn filter_by_version(&self, expected_version: &str) -> Self {
+        self.iter()
+            .filter(|node| {
+                version_checker::is_minor_version_compatible(&node.version(), expected_version)
+            })
+            .cloned()
+            .collect()
+    }
+}

common/topology/src/lib.rs

@@ -0,0 +1,127 @@
+use crate::filter::VersionFilterable;
+use itertools::Itertools;
+use rand::seq::IteratorRandom;
+use sphinx::route::Node as SphinxNode;
+use std::cmp::max;
+use std::collections::HashMap;
+
+pub mod coco;
+mod filter;
+pub mod mix;
+pub mod provider;
+
+// TODO: Figure out why 'Clone' was required to have 'TopologyAccessor<T>' working
+// even though it only contains an Arc
+pub trait NymTopology: Sized + std::fmt::Debug + Send + Sync + Clone {
+    fn new(directory_server: String) -> Self;
+    fn new_from_nodes(
+        mix_nodes: Vec<mix::Node>,
+        mix_provider_nodes: Vec<provider::Node>,
+        coco_nodes: Vec<coco::Node>,
+    ) -> Self;
+    fn mix_nodes(&self) -> Vec<mix::Node>;
+    fn providers(&self) -> Vec<provider::Node>;
+    fn coco_nodes(&self) -> Vec<coco::Node>;
+    fn make_layered_topology(&self) -> Result<HashMap<u64, Vec<mix::Node>>, NymTopologyError> {
+        let mut layered_topology: HashMap<u64, Vec<mix::Node>> = HashMap::new();
+        let mut highest_layer = 0;
+        for mix in self.mix_nodes() {
+            // we need to have extra space for provider
+            if mix.layer > sphinx::constants::MAX_PATH_LENGTH as u64 {
+                return Err(NymTopologyError::InvalidMixLayerError);
+            }
+            highest_layer = max(highest_layer, mix.layer);
+
+            let layer_nodes = layered_topology.entry(mix.layer).or_insert_with(Vec::new);
+            layer_nodes.push(mix);
+        }
+
+        // verify the topology - make sure there are no gaps and there is at least one node per layer
+        let mut missing_layers = Vec::new();
+        for layer in 1..=highest_layer {
+            if !layered_topology.contains_key(&layer) {
+                missing_layers.push(layer);
+            }
+            if layered_topology[&layer].is_empty() {
+                missing_layers.push(layer);
+            }
+        }
+
+        if !missing_layers.is_empty() {
+            return Err(NymTopologyError::MissingLayerError(missing_layers));
+        }
+
+        Ok(layered_topology)
+    }
+
+    // Tries to get a route through the mix network
+    fn mix_route(&self) -> Result<Vec<SphinxNode>, NymTopologyError> {
+        let mut layered_topology = self.make_layered_topology()?;
+        let num_layers = layered_topology.len();
+        let route = (1..=num_layers as u64)
+            // unwrap is safe for 'remove' as it it failed, it implied the entry never existed
+            // in the map in the first place which would contradict what we've just done
+            .map(|layer| layered_topology.remove(&layer).unwrap()) // for each layer
+            .map(|nodes| nodes.into_iter().choose(&mut rand::thread_rng()).unwrap()) // choose random node
+            .map(|random_node| random_node.into()) // and convert it into sphinx specific node format
+            .collect();
+
+        Ok(route)
+    }
+
+    // Sets up a route to a specific provider
+    fn route_to(&self, provider_node: SphinxNode) -> Result<Vec<SphinxNode>, NymTopologyError> {
+        Ok(self
+            .mix_route()?
+            .into_iter()
+            .chain(std::iter::once(provider_node))
+            .collect())
+    }
+
+    fn all_paths(&self) -> Result<Vec<Vec<SphinxNode>>, NymTopologyError> {
+        let mut layered_topology = self.make_layered_topology()?;
+        let providers = self.providers();
+
+        let sorted_layers: Vec<Vec<SphinxNode>> = (1..=layered_topology.len() as u64)
+            .map(|layer| layered_topology.remove(&layer).unwrap()) // get all nodes per layer
+            .map(|layer_nodes| layer_nodes.into_iter().map(|node| node.into()).collect()) // convert them into 'proper' sphinx nodes
+            .chain(std::iter::once(
+                providers.into_iter().map(|node| node.into()).collect(),
+            )) // append all providers to the end
+            .collect();
+
+        let all_paths = sorted_layers
+            .into_iter()
+            .multi_cartesian_product() // create all possible paths through that
+            .collect();
+
+        Ok(all_paths)
+    }
+
+    fn filter_node_versions(
+        &self,
+        expected_mix_version: &str,
+        expected_provider_version: &str,
+        expected_coco_version: &str,
+    ) -> Self {
+        let mixes = self.mix_nodes().filter_by_version(expected_mix_version);
+        let providers = self
+            .providers()
+            .filter_by_version(expected_provider_version);
+        let cocos = self.coco_nodes().filter_by_version(expected_coco_version);
+
+        Self::new_from_nodes(mixes, providers, cocos)
+    }
+
+    fn can_construct_path_through(&self) -> bool {
+        !self.mix_nodes().is_empty()
+            && !self.providers().is_empty()
+            && self.make_layered_topology().is_ok()
+    }
+}
+
+#[derive(Debug)]
+pub enum NymTopologyError {
+    InvalidMixLayerError,
+    MissingLayerError(Vec<u64>),
+}

common/topology/src/mix.rs

@@ -0,0 +1,39 @@
+use crate::filter;
+use nymsphinx::addressing::nodes::NymNodeRoutingAddress;
+use sphinx::route::Node as SphinxNode;
+use std::convert::TryInto;
+use std::net::SocketAddr;
+
+#[derive(Debug, Clone)]
+pub struct Node {
+    pub location: String,
+    pub host: SocketAddr,
+    pub pub_key: String,
+    pub layer: u64,
+    pub last_seen: u64,
+    pub version: String,
+}
+
+impl Node {
+    pub fn get_pub_key_bytes(&self) -> [u8; 32] {
+        let mut key_bytes = [0; 32];
+        bs58::decode(&self.pub_key).into(&mut key_bytes).unwrap();
+        key_bytes
+    }
+}
+
+impl filter::Versioned for Node {
+    fn version(&self) -> String {
+        self.version.clone()
+    }
+}
+
+impl Into<SphinxNode> for Node {
+    fn into(self) -> SphinxNode {
+        let node_address_bytes = NymNodeRoutingAddress::from(self.host).try_into().unwrap();
+        let key_bytes = self.get_pub_key_bytes();
+        let key = sphinx::key::new(key_bytes);
+
+        SphinxNode::new(node_address_bytes, key)
+    }
+}

common/topology/src/provider.rs

@@ -0,0 +1,47 @@
+use crate::filter;
+use nymsphinx::addressing::nodes::NymNodeRoutingAddress;
+use sphinx::route::Node as SphinxNode;
+use std::convert::TryInto;
+use std::net::SocketAddr;
+
+#[derive(Debug, Clone)]
+pub struct Client {
+    pub pub_key: String,
+}
+
+#[derive(Debug, Clone)]
+pub struct Node {
+    pub location: String,
+    pub client_listener: SocketAddr,
+    pub mixnet_listener: SocketAddr,
+    pub pub_key: String,
+    pub registered_clients: Vec<Client>,
+    pub last_seen: u64,
+    pub version: String,
+}
+
+impl Node {
+    pub fn get_pub_key_bytes(&self) -> [u8; 32] {
+        let mut key_bytes = [0; 32];
+        bs58::decode(&self.pub_key).into(&mut key_bytes).unwrap();
+        key_bytes
+    }
+}
+
+impl filter::Versioned for Node {
+    fn version(&self) -> String {
+        self.version.clone()
+    }
+}
+
+impl Into<SphinxNode> for Node {
+    fn into(self) -> SphinxNode {
+        let node_address_bytes = NymNodeRoutingAddress::from(self.mixnet_listener)
+            .try_into()
+            .unwrap();
+        let key_bytes = self.get_pub_key_bytes();
+        let key = sphinx::key::new(key_bytes);
+
+        SphinxNode::new(node_address_bytes, key)
+    }
+}

common/version-checker/.gitignore

@@ -0,0 +1,3 @@
+/target
+**/*.rs.bk
+Cargo.lock

common/version-checker/Cargo.toml

@@ -0,0 +1,10 @@
+[package]
+name = "version-checker"
+version = "0.1.0"
+authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+semver = "0.9.0"

common/version-checker/src/lib.rs

@@ -0,0 +1,69 @@
+use semver::Version;
+
+/// Checks whether given `version` is compatible with a given semantic version requirement `req`
+/// according to major-minor semver rules. The semantic version requirement can be passed as a full,
+/// concrete version number, because that's what we'll have in our Cargo.toml files (e.g. 0.3.2).
+/// The patch number in the requirement gets dropped and replaced with a wildcard (0.3.*) as all
+/// minor versions should be compatible with each other.
+pub fn is_minor_version_compatible(version: &str, req: &str) -> bool {
+    let expected_version = match Version::parse(version) {
+        Ok(v) => v,
+        Err(_) => return false,
+    };
+    let req_version = match Version::parse(req) {
+        Ok(v) => v,
+        Err(_) => return false,
+    };
+
+    expected_version.major == req_version.major && expected_version.minor == req_version.minor
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn version_0_3_0_is_compatible_with_requirement_0_3_x() {
+        assert!(is_minor_version_compatible("0.3.0", "0.3.2"));
+    }
+
+    #[test]
+    fn version_0_3_1_is_compatible_with_minimum_requirement_0_3_x() {
+        assert!(is_minor_version_compatible("0.3.1", "0.3.2"));
+    }
+
+    #[test]
+    fn version_0_3_2_is_compatible_with_minimum_requirement_0_3_x() {
+        assert!(is_minor_version_compatible("0.3.2", "0.3.0"));
+    }
+
+    #[test]
+    fn version_0_2_0_is_not_compatible_with_requirement_0_3_x() {
+        assert!(!is_minor_version_compatible("0.2.0", "0.3.2"));
+    }
+
+    #[test]
+    fn version_0_4_0_is_not_compatible_with_requirement_0_3_x() {
+        assert!(!is_minor_version_compatible("0.4.0", "0.3.2"));
+    }
+
+    #[test]
+    fn version_1_3_2_is_not_compatible_with_requirement_0_3_x() {
+        assert!(!is_minor_version_compatible("1.3.2", "0.3.2"));
+    }
+
+    #[test]
+    fn version_0_4_0_rc_1_is_compatible_with_version_0_4_0_rc_1() {
+        assert!(is_minor_version_compatible("0.4.0-rc.1", "0.4.0-rc.1"));
+    }
+
+    #[test]
+    fn returns_false_on_foo_version() {
+        assert!(!is_minor_version_compatible("foo", "0.3.2"));
+    }
+
+    #[test]
+    fn returns_false_on_bar_version() {
+        assert!(!is_minor_version_compatible("0.3.2", "bar"));
+    }
+}

mixnode/Cargo.toml

@@ -0,0 +1,42 @@
+[package]
+build = "build.rs"
+name = "nym-mixnode"
+version = "0.6.0"
+authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+bs58 = "0.3.0"
+clap = "2.33.0"
+curve25519-dalek = "1.2.3"
+dirs = "2.0.2"
+dotenv = "0.15.0"
+futures = "0.3.1"
+log = "0.4"
+pretty_env_logger = "0.3"
+serde = { version = "1.0.104", features = ["derive"] }
+tokio = { version = "0.2", features = ["full"] }
+
+## internal
+config = {path = "../common/config"}
+crypto = {path = "../common/crypto"}
+directory-client = { path = "../common/clients/directory-client" }
+multi-tcp-client = { path = "../common/clients/multi-tcp-client" }
+nymsphinx = {path = "../common/nymsphinx" }
+pemstore = {path = "../common/pemstore"}
+topology = {path = "../common/topology"}
+
+## will be moved to proper dependencies once released
+sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" }
+
+[build-dependencies]
+built = "0.3.2"
+
+[dev-dependencies]
+tempfile = "3.1.0"
+
+[features]
+qa = []
+local = []

mixnode/README.md

@@ -0,0 +1,11 @@
+# Nym Mixnode
+
+A Rust mixnode implementation.
+
+## Usage
+
+* `nym-mixnode` prints a help message showing usage options
+* `nym-mixnode run --help` prints a help message showing usage options for the run command
+* `nym-mixnode run --layer 1 --host x.x.x.x` will start the mixnode in layer 1 and bind to the specified host IP address. Coordinate with other people in your network to find out which layer needs coverage.
+
+By default, the Nym Mixnode will start on port 1789. If desired, you can change the port using the `--port` option.

mixnode/src/built_info.rs

@@ -0,0 +1,2 @@
+// The file has been placed there by the build script.
+include!(concat!(env!("OUT_DIR"), "/built.rs"));

mixnode/src/commands/init.rs

@@ -0,0 +1,88 @@
+use crate::commands::override_config;
+use crate::config::persistence::pathfinder::MixNodePathfinder;
+use clap::{App, Arg, ArgMatches};
+use config::NymConfig;
+use crypto::encryption;
+use pemstore::pemstore::PemStore;
+
+pub fn command_args<'a, 'b>() -> clap::App<'a, 'b> {
+    App::new("init")
+        .about("Initialise the mixnode")
+        .arg(
+            Arg::with_name("id")
+                .long("id")
+                .help("Id of the nym-mixnode we want to create config for.")
+                .takes_value(true)
+                .required(true),
+        )
+        .arg(
+            Arg::with_name("location")
+                .long("location")
+                .help("Optional geographical location of this node")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("layer")
+                .long("layer")
+                .help("The mixnet layer of this particular node")
+                .takes_value(true)
+                .required(true),
+        )
+        .arg(
+            Arg::with_name("host")
+                .long("host")
+                .help("The host on which the mixnode will be running")
+                .takes_value(true)
+                .required(true),
+        )
+        .arg(
+            Arg::with_name("port")
+                .long("port")
+                .help("The port on which the mixnode will be listening")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("announce-host")
+                .long("announce-host")
+                .help("The custom host that will be reported to the directory server")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("announce-port")
+                .long("announce-port")
+                .help("The custom port that will be reported to the directory server")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("directory")
+                .long("directory")
+                .help("Address of the directory server the node is sending presence and metrics to")
+                .takes_value(true),
+        )
+}
+
+pub fn execute(matches: &ArgMatches) {
+    let id = matches.value_of("id").unwrap();
+    println!("Initialising mixnode {}...", id);
+
+    let layer = matches.value_of("layer").unwrap().parse().unwrap();
+    let mut config = crate::config::Config::new(id, layer);
+
+    config = override_config(config, matches);
+
+    let sphinx_keys = encryption::KeyPair::new();
+    let pathfinder = MixNodePathfinder::new_from_config(&config);
+    let pem_store = PemStore::new(pathfinder);
+    pem_store
+        .write_encryption_keys(sphinx_keys)
+        .expect("Failed to save sphinx keys");
+    println!("Saved mixnet sphinx keypair");
+
+    let config_save_location = config.get_config_file_save_location();
+    config
+        .save_to_file(None)
+        .expect("Failed to save the config file");
+    println!("Saved configuration file to {:?}", config_save_location);
+
+    println!("Mixnode configuration completed.\n\n\n")
+}

mixnode/src/commands/mod.rs

@@ -0,0 +1,49 @@
+use crate::config::Config;
+use clap::ArgMatches;
+
+pub mod init;
+pub mod run;
+
+pub(crate) fn override_config(mut config: Config, matches: &ArgMatches) -> Config {
+    let mut was_host_overridden = false;
+    if let Some(host) = matches.value_of("host") {
+        config = config.with_listening_host(host);
+        was_host_overridden = true;
+    }
+
+    if let Some(port) = matches.value_of("port").map(|port| port.parse::<u16>()) {
+        if let Err(err) = port {
+            // if port was overridden, it must be parsable
+            panic!("Invalid port value provided - {:?}", err);
+        }
+        config = config.with_listening_port(port.unwrap());
+    }
+
+    if let Some(directory) = matches.value_of("directory") {
+        config = config.with_custom_directory(directory);
+    }
+
+    if let Some(announce_host) = matches.value_of("announce-host") {
+        config = config.with_announce_host(announce_host);
+    } else if was_host_overridden {
+        // make sure our 'announce-host' always defaults to 'host'
+        config = config.announce_host_from_listening_host()
+    }
+
+    if let Some(announce_port) = matches
+        .value_of("announce-port")
+        .map(|port| port.parse::<u16>())
+    {
+        if let Err(err) = announce_port {
+            // if port was overridden, it must be parsable
+            panic!("Invalid port value provided - {:?}", err);
+        }
+        config = config.with_announce_port(announce_port.unwrap());
+    }
+
+    if let Some(location) = matches.value_of("location") {
+        config = config.with_location(location);
+    }
+
+    config
+}

mixnode/src/commands/run.rs

@@ -0,0 +1,118 @@
+use crate::commands::override_config;
+use crate::config::Config;
+use crate::node::MixNode;
+use clap::{App, Arg, ArgMatches};
+use config::NymConfig;
+
+pub fn command_args<'a, 'b>() -> App<'a, 'b> {
+    App::new("run")
+        .about("Starts the mixnode")
+        .arg(
+            Arg::with_name("id")
+                .long("id")
+                .help("Id of the nym-mixnode we want to run")
+                .takes_value(true)
+                .required(true),
+        )
+        // the rest of arguments are optional, they are used to override settings in config file
+        .arg(
+            Arg::with_name("location")
+                .long("location")
+                .help("Optional geographical location of this node")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("config")
+                .long("config")
+                .help("Custom path to the nym-mixnode configuration file")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("layer")
+                .long("layer")
+                .help("The mixnet layer of this particular node")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("host")
+                .long("host")
+                .help("The custom host on which the mixnode will be running")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("port")
+                .long("port")
+                .help("The port on which the mixnode will be listening")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("announce-host")
+                .long("announce-host")
+                .help("The host that will be reported to the directory server")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("announce-port")
+                .long("announce-port")
+                .help("The port that will be reported to the directory server")
+                .takes_value(true),
+        )
+        .arg(
+            Arg::with_name("directory")
+                .long("directory")
+                .help("Address of the directory server the node is sending presence and metrics to")
+                .takes_value(true),
+        )
+}
+
+fn show_binding_warning(address: String) {
+    println!("\n##### NOTE #####");
+    println!(
+        "\nYou are trying to bind to {} - you might not be accessible to other nodes\n\
+         You can ignore this note if you're running setup on a local network \n\
+         or have set a custom 'announce-host'",
+        address
+    );
+    println!("\n\n");
+}
+
+fn special_addresses() -> Vec<&'static str> {
+    vec!["localhost", "127.0.0.1", "0.0.0.0", "::1", "[::1]"]
+}
+
+pub fn execute(matches: &ArgMatches) {
+    let id = matches.value_of("id").unwrap();
+
+    println!("Starting mixnode {}...", id);
+
+    let mut config =
+        Config::load_from_file(matches.value_of("config").map(|path| path.into()), Some(id))
+            .expect("Failed to load config file");
+
+    config = override_config(config, matches);
+
+    let listening_ip_string = config.get_listening_address().ip().to_string();
+    if special_addresses().contains(&listening_ip_string.as_ref()) {
+        show_binding_warning(listening_ip_string);
+    }
+
+    println!(
+        "Directory server [presence]: {}",
+        config.get_presence_directory_server()
+    );
+    println!(
+        "Directory server [metrics]: {}",
+        config.get_metrics_directory_server()
+    );
+
+    println!(
+        "Listening for incoming packets on {}",
+        config.get_listening_address()
+    );
+    println!(
+        "Announcing the following socket address: {}",
+        config.get_announce_address()
+    );
+
+    MixNode::new(config).run();
+}

mixnode/src/config/mod.rs

@@ -0,0 +1,407 @@
+use crate::config::template::config_template;
+use config::NymConfig;
+use log::*;
+use serde::{Deserialize, Serialize};
+use std::net::{IpAddr, SocketAddr};
+use std::path::PathBuf;
+use std::str::FromStr;
+use std::time;
+
+pub mod persistence;
+mod template;
+
+// 'MIXNODE'
+const DEFAULT_LISTENING_PORT: u16 = 1789;
+
+// 'DEBUG'
+// where applicable, the below are defined in milliseconds
+const DEFAULT_PRESENCE_SENDING_DELAY: u64 = 1500;
+const DEFAULT_METRICS_SENDING_DELAY: u64 = 1000;
+const DEFAULT_METRICS_RUNNING_STATS_LOGGING_DELAY: u64 = 60_000; // 1min
+const DEFAULT_PACKET_FORWARDING_INITIAL_BACKOFF: u64 = 10_000; // 10s
+const DEFAULT_PACKET_FORWARDING_MAXIMUM_BACKOFF: u64 = 300_000; // 5min
+const DEFAULT_INITIAL_CONNECTION_TIMEOUT: u64 = 1_500; // 1.5s
+
+#[derive(Debug, Default, Deserialize, PartialEq, Serialize)]
+#[serde(deny_unknown_fields)]
+pub struct Config {
+    mixnode: MixNode,
+
+    #[serde(default)]
+    logging: Logging,
+    #[serde(default)]
+    debug: Debug,
+}
+
+impl NymConfig for Config {
+    fn template() -> &'static str {
+        config_template()
+    }
+
+    fn config_file_name() -> String {
+        "config.toml".to_string()
+    }
+
+    fn default_root_directory() -> PathBuf {
+        dirs::home_dir()
+            .expect("Failed to evaluate $HOME value")
+            .join(".nym")
+            .join("mixnodes")
+    }
+
+    fn root_directory(&self) -> PathBuf {
+        self.mixnode.nym_root_directory.clone()
+    }
+
+    fn config_directory(&self) -> PathBuf {
+        self.mixnode
+            .nym_root_directory
+            .join(&self.mixnode.id)
+            .join("config")
+    }
+
+    fn data_directory(&self) -> PathBuf {
+        self.mixnode
+            .nym_root_directory
+            .join(&self.mixnode.id)
+            .join("data")
+    }
+}
+
+impl Config {
+    pub fn new<S: Into<String>>(id: S, layer: u64) -> Self {
+        Config::default().with_id(id).with_layer(layer)
+    }
+
+    // builder methods
+    pub fn with_id<S: Into<String>>(mut self, id: S) -> Self {
+        let id = id.into();
+        if self.mixnode.private_sphinx_key_file.as_os_str().is_empty() {
+            self.mixnode.private_sphinx_key_file =
+                self::MixNode::default_private_sphinx_key_file(&id);
+        }
+        if self.mixnode.public_sphinx_key_file.as_os_str().is_empty() {
+            self.mixnode.public_sphinx_key_file =
+                self::MixNode::default_public_sphinx_key_file(&id);
+        }
+        self.mixnode.id = id;
+        self
+    }
+
+    pub fn with_layer(mut self, layer: u64) -> Self {
+        self.mixnode.layer = layer;
+        self
+    }
+
+    pub fn with_location<S: Into<String>>(mut self, location: S) -> Self {
+        self.mixnode.location = location.into();
+        self
+    }
+
+    // if you want to use distinct servers for metrics and presence
+    // you need to do so in the config.toml file.
+    pub fn with_custom_directory<S: Into<String>>(mut self, directory_server: S) -> Self {
+        let directory_server_string = directory_server.into();
+        self.debug.presence_directory_server = directory_server_string.clone();
+        self.debug.metrics_directory_server = directory_server_string;
+        self
+    }
+
+    pub fn with_listening_host<S: Into<String>>(mut self, host: S) -> Self {
+        // see if the provided `host` is just an ip address or ip:port
+        let host = host.into();
+
+        // is it ip:port?
+        match SocketAddr::from_str(host.as_ref()) {
+            Ok(socket_addr) => {
+                self.mixnode.listening_address = socket_addr;
+                self
+            }
+            // try just for ip
+            Err(_) => match IpAddr::from_str(host.as_ref()) {
+                Ok(ip_addr) => {
+                    self.mixnode.listening_address.set_ip(ip_addr);
+                    self
+                }
+                Err(_) => {
+                    error!(
+                        "failed to make any changes to config - invalid host {}",
+                        host
+                    );
+                    self
+                }
+            },
+        }
+    }
+
+    pub fn with_listening_port(mut self, port: u16) -> Self {
+        self.mixnode.listening_address.set_port(port);
+        self
+    }
+
+    pub fn with_announce_host<S: Into<String>>(mut self, host: S) -> Self {
+        // this is slightly more complicated as we store announce information as String,
+        // since it might not necessarily be a valid SocketAddr (say `nymtech.net:8080` is a valid
+        // announce address, yet invalid SocketAddr`
+
+        // first lets see if we received host:port or just host part of an address
+        let host = host.into();
+        let split_host: Vec<_> = host.split(':').collect();
+        match split_host.len() {
+            1 => {
+                // we provided only 'host' part so we are going to reuse existing port
+                self.mixnode.announce_address =
+                    format!("{}:{}", host, self.mixnode.listening_address.port());
+                self
+            }
+            2 => {
+                // we provided 'host:port' so just put the whole thing there
+                self.mixnode.announce_address = host;
+                self
+            }
+            _ => {
+                // we provided something completely invalid, so don't try to parse it
+                error!(
+                    "failed to make any changes to config - invalid announce host {}",
+                    host
+                );
+                self
+            }
+        }
+    }
+
+    pub fn announce_host_from_listening_host(mut self) -> Self {
+        self.mixnode.announce_address = self.mixnode.listening_address.to_string();
+        self
+    }
+
+    pub fn with_announce_port(mut self, port: u16) -> Self {
+        let current_host: Vec<_> = self.mixnode.announce_address.split(':').collect();
+        debug_assert_eq!(current_host.len(), 2);
+        self.mixnode.announce_address = format!("{}:{}", current_host[0], port);
+        self
+    }
+
+    // getters
+    pub fn get_config_file_save_location(&self) -> PathBuf {
+        self.config_directory().join(Self::config_file_name())
+    }
+
+    pub fn get_location(&self) -> String {
+        self.mixnode.location.clone()
+    }
+
+    pub fn get_private_sphinx_key_file(&self) -> PathBuf {
+        self.mixnode.private_sphinx_key_file.clone()
+    }
+
+    pub fn get_public_sphinx_key_file(&self) -> PathBuf {
+        self.mixnode.public_sphinx_key_file.clone()
+    }
+
+    pub fn get_presence_directory_server(&self) -> String {
+        self.debug.presence_directory_server.clone()
+    }
+
+    pub fn get_presence_sending_delay(&self) -> time::Duration {
+        time::Duration::from_millis(self.debug.presence_sending_delay)
+    }
+
+    pub fn get_metrics_directory_server(&self) -> String {
+        self.debug.metrics_directory_server.clone()
+    }
+
+    pub fn get_metrics_sending_delay(&self) -> time::Duration {
+        time::Duration::from_millis(self.debug.metrics_sending_delay)
+    }
+
+    pub fn get_metrics_running_stats_logging_delay(&self) -> time::Duration {
+        time::Duration::from_millis(self.debug.metrics_running_stats_logging_delay)
+    }
+
+    pub fn get_layer(&self) -> u64 {
+        self.mixnode.layer
+    }
+
+    pub fn get_listening_address(&self) -> SocketAddr {
+        self.mixnode.listening_address
+    }
+
+    pub fn get_announce_address(&self) -> String {
+        self.mixnode.announce_address.clone()
+    }
+
+    pub fn get_packet_forwarding_initial_backoff(&self) -> time::Duration {
+        time::Duration::from_millis(self.debug.packet_forwarding_initial_backoff)
+    }
+
+    pub fn get_packet_forwarding_maximum_backoff(&self) -> time::Duration {
+        time::Duration::from_millis(self.debug.packet_forwarding_maximum_backoff)
+    }
+
+    pub fn get_initial_connection_timeout(&self) -> time::Duration {
+        time::Duration::from_millis(self.debug.initial_connection_timeout)
+    }
+}
+
+#[derive(Debug, Deserialize, PartialEq, Serialize)]
+#[serde(deny_unknown_fields)]
+pub struct MixNode {
+    /// ID specifies the human readable ID of this particular mixnode.
+    id: String,
+
+    /// Completely optional value specifying geographical location of this particular node.
+    /// Currently it's used entirely for debug purposes, as there are no mechanisms implemented
+    /// to verify correctness of the information provided. However, feel free to fill in
+    /// this field with as much accuracy as you wish to share.
+    location: String,
+
+    /// Layer of this particular mixnode determining its position in the network.
+    layer: u64,
+
+    /// Socket address to which this mixnode will bind to and will be listening for packets.
+    listening_address: SocketAddr,
+
+    /// Optional address announced to the directory server for the clients to connect to.
+    /// It is useful, say, in NAT scenarios or wanting to more easily update actual IP address
+    /// later on by using name resolvable with a DNS query, such as `nymtech.net:8080`.
+    /// Additionally a custom port can be provided, so both `nymtech.net:8080` and `nymtech.net`
+    /// are valid announce addresses, while the later will default to whatever port is used for
+    /// `listening_address`.
+    announce_address: String,
+
+    /// Path to file containing private sphinx key.
+    private_sphinx_key_file: PathBuf,
+
+    /// Path to file containing public sphinx key.
+    public_sphinx_key_file: PathBuf,
+
+    /// nym_home_directory specifies absolute path to the home nym MixNodes directory.
+    /// It is expected to use default value and hence .toml file should not redefine this field.
+    nym_root_directory: PathBuf,
+}
+
+impl MixNode {
+    fn default_private_sphinx_key_file(id: &str) -> PathBuf {
+        Config::default_data_directory(Some(id)).join("private_sphinx.pem")
+    }
+
+    fn default_public_sphinx_key_file(id: &str) -> PathBuf {
+        Config::default_data_directory(Some(id)).join("public_sphinx.pem")
+    }
+
+    fn default_location() -> String {
+        "unknown".into()
+    }
+}
+
+impl Default for MixNode {
+    fn default() -> Self {
+        MixNode {
+            id: "".to_string(),
+            location: Self::default_location(),
+            layer: 0,
+            listening_address: format!("0.0.0.0:{}", DEFAULT_LISTENING_PORT)
+                .parse()
+                .unwrap(),
+            announce_address: format!("127.0.0.1:{}", DEFAULT_LISTENING_PORT),
+            private_sphinx_key_file: Default::default(),
+            public_sphinx_key_file: Default::default(),
+            nym_root_directory: Config::default_root_directory(),
+        }
+    }
+}
+
+#[derive(Debug, Deserialize, PartialEq, Serialize)]
+#[serde(deny_unknown_fields)]
+pub struct Logging {}
+
+impl Default for Logging {
+    fn default() -> Self {
+        Logging {}
+    }
+}
+
+#[derive(Debug, Deserialize, PartialEq, Serialize)]
+#[serde(default, deny_unknown_fields)]
+pub struct Debug {
+    // The idea of additional 'directory servers' is to let mixes report their presence
+    // and metrics to separate places
+    /// Directory server to which the server will be reporting their presence data.
+    presence_directory_server: String,
+
+    /// Delay between each subsequent presence data being sent.
+    /// The provided value is interpreted as milliseconds.
+    presence_sending_delay: u64,
+
+    /// Directory server to which the server will be reporting their metrics data.
+    metrics_directory_server: String,
+
+    /// Delay between each subsequent metrics data being sent.
+    /// The provided value is interpreted as milliseconds.
+    metrics_sending_delay: u64,
+
+    /// Delay between each subsequent running metrics statistics being logged.
+    /// The provided value is interpreted as milliseconds.
+    metrics_running_stats_logging_delay: u64,
+
+    /// Initial value of an exponential backoff to reconnect to dropped TCP connection when
+    /// forwarding sphinx packets.
+    /// The provided value is interpreted as milliseconds.
+    packet_forwarding_initial_backoff: u64,
+
+    /// Maximum value of an exponential backoff to reconnect to dropped TCP connection when
+    /// forwarding sphinx packets.
+    /// The provided value is interpreted as milliseconds.
+    packet_forwarding_maximum_backoff: u64,
+
+    /// Timeout for establishing initial connection when trying to forward a sphinx packet.
+    /// The provider value is interpreted as milliseconds.
+    initial_connection_timeout: u64,
+}
+
+impl Debug {
+    fn default_directory_server() -> String {
+        #[cfg(feature = "qa")]
+        return "https://qa-directory.nymtech.net".to_string();
+        #[cfg(feature = "local")]
+        return "http://localhost:8080".to_string();
+
+        "https://directory.nymtech.net".to_string()
+    }
+}
+
+impl Default for Debug {
+    fn default() -> Self {
+        Debug {
+            presence_directory_server: Self::default_directory_server(),
+            presence_sending_delay: DEFAULT_PRESENCE_SENDING_DELAY,
+            metrics_directory_server: Self::default_directory_server(),
+            metrics_sending_delay: DEFAULT_METRICS_SENDING_DELAY,
+            metrics_running_stats_logging_delay: DEFAULT_METRICS_RUNNING_STATS_LOGGING_DELAY,
+            packet_forwarding_initial_backoff: DEFAULT_PACKET_FORWARDING_INITIAL_BACKOFF,
+            packet_forwarding_maximum_backoff: DEFAULT_PACKET_FORWARDING_MAXIMUM_BACKOFF,
+            initial_connection_timeout: DEFAULT_INITIAL_CONNECTION_TIMEOUT,
+        }
+    }
+}
+
+#[cfg(test)]
+mod mixnode_config {
+    use super::*;
+
+    #[test]
+    fn after_saving_default_config_the_loaded_one_is_identical() {
+        // need to figure out how to do something similar but without touching the disk
+        // or the file system at all...
+        let temp_location = tempfile::tempdir().unwrap().path().join("config.toml");
+        let default_config = Config::default().with_id("foomp".to_string());
+        default_config
+            .save_to_file(Some(temp_location.clone()))
+            .unwrap();
+
+        let loaded_config = Config::load_from_file(Some(temp_location), None).unwrap();
+
+        assert_eq!(default_config, loaded_config);
+    }
+}

mixnode/src/config/persistence/mod.rs

@@ -0,0 +1 @@
+pub mod pathfinder;

mixnode/src/config/persistence/pathfinder.rs

@@ -0,0 +1,44 @@
+use crate::config::Config;
+use pemstore::pathfinder::PathFinder;
+use std::path::PathBuf;
+
+#[derive(Debug)]
+pub struct MixNodePathfinder {
+    pub config_dir: PathBuf,
+    pub private_sphinx_key: PathBuf,
+    pub public_sphinx_key: PathBuf,
+}
+
+impl MixNodePathfinder {
+    pub fn new_from_config(config: &Config) -> Self {
+        MixNodePathfinder {
+            config_dir: config.get_config_file_save_location(),
+            private_sphinx_key: config.get_private_sphinx_key_file(),
+            public_sphinx_key: config.get_public_sphinx_key_file(),
+        }
+    }
+}
+
+impl PathFinder for MixNodePathfinder {
+    fn config_dir(&self) -> PathBuf {
+        self.config_dir.clone()
+    }
+
+    fn private_identity_key(&self) -> PathBuf {
+        // TEMPORARILY USE SAME KEYS AS ENCRYPTION
+        self.private_sphinx_key.clone()
+    }
+
+    fn public_identity_key(&self) -> PathBuf {
+        // TEMPORARILY USE SAME KEYS AS ENCRYPTION
+        self.public_sphinx_key.clone()
+    }
+
+    fn private_encryption_key(&self) -> Option<PathBuf> {
+        Some(self.private_sphinx_key.clone())
+    }
+
+    fn public_encryption_key(&self) -> Option<PathBuf> {
+        Some(self.public_sphinx_key.clone())
+    }
+}

mixnode/src/config/template.rs

@@ -0,0 +1,57 @@
+pub(crate) fn config_template() -> &'static str {
+    // While using normal toml marshalling would have been way simpler with less overhead,
+    // I think it's useful to have comments attached to the saved config file to explain behaviour of
+    // particular fields.
+    // Note: any changes to the template must be reflected in the appropriate structs in mod.rs.
+    r#"
+# This is a TOML config file.
+# For more information, see https://github.com/toml-lang/toml
+
+##### main base mixnode config options #####
+
+[mixnode]
+# Human readable ID of this particular mixnode.
+id = '{{ mixnode.id }}'
+
+# Completely optional value specifying geographical location of this particular node.
+# Currently it's used entirely for debug purposes, as there are no mechanisms implemented
+# to verify correctness of the information provided. However, feel free to fill in
+# this field with as much accuracy as you wish to share.
+location = '{{ mixnode.location }}'
+    
+# Layer of this particular mixnode determining its position in the network.
+layer = {{ mixnode.layer }}
+
+# Socket address to which this mixnode will bind to and will be listening for packets.
+listening_address = '{{ mixnode.listening_address }}'
+
+# Path to file containing private identity key.
+private_sphinx_key_file = '{{ mixnode.private_sphinx_key_file }}'
+
+# Path to file containing public sphinx key.
+public_sphinx_key_file = '{{ mixnode.public_sphinx_key_file }}'
+
+##### additional mixnode config options #####
+
+# Optional address announced to the directory server for the clients to connect to.
+# It is useful, say, in NAT scenarios or wanting to more easily update actual IP address
+# later on by using name resolvable with a DNS query, such as `nymtech.net:8080`.
+# Additionally a custom port can be provided, so both `nymtech.net:8080` and `nymtech.net`
+# are valid announce addresses, while the later will default to whatever port is used for
+# `listening_address`.
+announce_address = '{{ mixnode.announce_address }}'
+
+##### advanced configuration options #####
+
+# Absolute path to the home Nym Clients directory.
+nym_root_directory = '{{ mixnode.nym_root_directory }}'
+
+
+##### logging configuration options #####
+
+[logging]
+
+# TODO
+
+"#
+}

mixnode/src/main.rs

@@ -0,0 +1,69 @@
+use clap::{App, ArgMatches};
+
+pub mod built_info;
+mod commands;
+mod config;
+mod node;
+
+fn main() {
+    dotenv::dotenv().ok();
+    setup_logging();
+    println!("{}", banner());
+
+    let arg_matches = App::new("Nym Mixnode")
+        .version(built_info::PKG_VERSION)
+        .author("Nymtech")
+        .about("Implementation of the Loopix-based Mixnode")
+        .subcommand(commands::init::command_args())
+        .subcommand(commands::run::command_args())
+        .get_matches();
+
+    execute(arg_matches);
+}
+
+fn execute(matches: ArgMatches) {
+    match matches.subcommand() {
+        ("init", Some(m)) => commands::init::execute(m),
+        ("run", Some(m)) => commands::run::execute(m),
+        _ => println!("{}", usage()),
+    }
+}
+
+fn usage() -> &'static str {
+    "usage: --help to see available options.\n\n"
+}
+
+fn banner() -> String {
+    format!(
+        r#"
+
+      _ __  _   _ _ __ ___
+     | '_ \| | | | '_ \ _ \
+     | | | | |_| | | | | | |
+     |_| |_|\__, |_| |_| |_|
+            |___/
+
+             (mixnode - version {:})
+
+    "#,
+        built_info::PKG_VERSION
+    )
+}
+
+fn setup_logging() {
+    let mut log_builder = pretty_env_logger::formatted_timed_builder();
+    if let Ok(s) = ::std::env::var("RUST_LOG") {
+        log_builder.parse_filters(&s);
+    } else {
+        // default to 'Info'
+        log_builder.filter(None, log::LevelFilter::Info);
+    }
+
+    log_builder
+        .filter_module("hyper", log::LevelFilter::Warn)
+        .filter_module("tokio_reactor", log::LevelFilter::Warn)
+        .filter_module("reqwest", log::LevelFilter::Warn)
+        .filter_module("mio", log::LevelFilter::Warn)
+        .filter_module("want", log::LevelFilter::Warn)
+        .init();
+}

mixnode/src/node/listener.rs

@@ -0,0 +1,108 @@
+use crate::node::packet_processing::{MixProcessingResult, PacketProcessor};
+use futures::channel::mpsc;
+use log::*;
+use std::io;
+use std::net::SocketAddr;
+use tokio::prelude::*;
+use tokio::runtime::Handle;
+use tokio::task::JoinHandle;
+
+async fn process_received_packet(
+    packet_data: [u8; sphinx::PACKET_SIZE],
+    packet_processor: PacketProcessor,
+    forwarding_channel: mpsc::UnboundedSender<(SocketAddr, Vec<u8>)>,
+) {
+    // all processing incl. delay was done, the only thing left is to forward it
+    match packet_processor.process_sphinx_packet(packet_data).await {
+        Err(e) => debug!("We failed to process received sphinx packet - {:?}", e),
+        Ok(res) => match res {
+            MixProcessingResult::ForwardHop(hop_address, hop_data) => {
+                // send our data to tcp client for forwarding. If forwarding fails, then it fails,
+                // it's not like we can do anything about it
+                //
+                // in unbounded_send() failed it means that the receiver channel was disconnected
+                // and hence something weird must have happened without a way of recovering
+                forwarding_channel
+                    .unbounded_send((hop_address, hop_data))
+                    .unwrap();
+                packet_processor.report_sent(hop_address);
+            }
+            MixProcessingResult::LoopMessage => {
+                warn!("Somehow processed a loop cover message that we haven't implemented yet!")
+            }
+        },
+    }
+}
+
+async fn process_socket_connection(
+    mut socket: tokio::net::TcpStream,
+    packet_processor: PacketProcessor,
+    forwarding_channel: mpsc::UnboundedSender<(SocketAddr, Vec<u8>)>,
+) {
+    let mut buf = [0u8; sphinx::PACKET_SIZE];
+    loop {
+        match socket.read_exact(&mut buf).await {
+            // socket closed
+            Ok(n) if n == 0 => {
+                trace!("Remote connection closed.");
+                return;
+            }
+            Ok(n) => {
+                // If I understand it correctly, this if should never be executed as if `read_exact`
+                // does not fill buffer, it will throw UnexpectedEof?
+                if n != sphinx::PACKET_SIZE {
+                    warn!("read data of different length than expected sphinx packet size - {} (expected {})", n, sphinx::PACKET_SIZE);
+                    continue;
+                }
+                // we must be able to handle multiple packets from same connection independently
+                tokio::spawn(process_received_packet(
+                    buf,
+                    // note: processing_data is relatively cheap (and safe) to clone -
+                    // it contains arc to private key and metrics reporter (which is just
+                    // a single mpsc unbounded sender)
+                    packet_processor.clone(),
+                    forwarding_channel.clone(),
+                ))
+            }
+            Err(e) => {
+                if e.kind() == io::ErrorKind::UnexpectedEof {
+                    debug!("Read buffer was not fully filled. Most likely the client ({:?}) closed the connection.\
+                    Also closing the connection on this end.", socket.peer_addr())
+                } else {
+                    warn!(
+                        "failed to read from socket (source: {:?}). Closing the connection; err = {:?}",
+                        socket.peer_addr(),
+                        e
+                    );
+                }
+                return;
+            }
+        };
+    }
+}
+
+pub(crate) fn run_socket_listener(
+    handle: &Handle,
+    addr: SocketAddr,
+    packet_processor: PacketProcessor,
+    forwarding_channel: mpsc::UnboundedSender<(SocketAddr, Vec<u8>)>,
+) -> JoinHandle<io::Result<()>> {
+    let handle_clone = handle.clone();
+    handle.spawn(async move {
+        let mut listener = tokio::net::TcpListener::bind(addr).await?;
+        loop {
+            let (socket, _) = listener.accept().await?;
+
+            let thread_packet_processor = packet_processor.clone();
+            let forwarding_channel_clone = forwarding_channel.clone();
+            handle_clone.spawn(async move {
+                process_socket_connection(
+                    socket,
+                    thread_packet_processor,
+                    forwarding_channel_clone,
+                )
+                .await;
+            });
+        }
+    })
+}

mixnode/src/node/metrics.rs

@@ -0,0 +1,275 @@
+use directory_client::metrics::MixMetric;
+use directory_client::requests::metrics_mixes_post::MetricsMixPoster;
+use directory_client::DirectoryClient;
+use futures::channel::mpsc;
+use futures::lock::Mutex;
+use futures::StreamExt;
+use log::*;
+use std::collections::HashMap;
+use std::sync::Arc;
+use std::time::{Duration, SystemTime};
+use tokio::runtime::Handle;
+use tokio::task::JoinHandle;
+
+type SentMetricsMap = HashMap<String, u64>;
+
+pub(crate) enum MetricEvent {
+    Sent(String),
+    Received,
+}
+
+#[derive(Debug, Clone)]
+// Note: you should NEVER create more than a single instance of this using 'new()'.
+// You should always use .clone() to create additional instances
+struct MixMetrics {
+    inner: Arc<Mutex<MixMetricsInner>>,
+}
+
+struct MixMetricsInner {
+    received: u64,
+    sent: SentMetricsMap,
+}
+
+impl MixMetrics {
+    pub(crate) fn new() -> Self {
+        MixMetrics {
+            inner: Arc::new(Mutex::new(MixMetricsInner {
+                received: 0,
+                sent: HashMap::new(),
+            })),
+        }
+    }
+
+    async fn increment_received_metrics(&mut self) {
+        let mut unlocked = self.inner.lock().await;
+        unlocked.received += 1;
+    }
+
+    async fn increment_sent_metrics(&mut self, destination: String) {
+        let mut unlocked = self.inner.lock().await;
+        let receiver_count = unlocked.sent.entry(destination).or_insert(0);
+        *receiver_count += 1;
+    }
+
+    async fn acquire_and_reset_metrics(&mut self) -> (u64, SentMetricsMap) {
+        let mut unlocked = self.inner.lock().await;
+        let received = unlocked.received;
+
+        let sent = std::mem::replace(&mut unlocked.sent, HashMap::new());
+        unlocked.received = 0;
+
+        (received, sent)
+    }
+}
+
+struct MetricsReceiver {
+    metrics: MixMetrics,
+    metrics_rx: mpsc::UnboundedReceiver<MetricEvent>,
+}
+
+impl MetricsReceiver {
+    fn new(metrics: MixMetrics, metrics_rx: mpsc::UnboundedReceiver<MetricEvent>) -> Self {
+        MetricsReceiver {
+            metrics,
+            metrics_rx,
+        }
+    }
+
+    fn start(mut self, handle: &Handle) -> JoinHandle<()> {
+        handle.spawn(async move {
+            while let Some(metrics_data) = self.metrics_rx.next().await {
+                match metrics_data {
+                    MetricEvent::Received => self.metrics.increment_received_metrics().await,
+                    MetricEvent::Sent(destination) => {
+                        self.metrics.increment_sent_metrics(destination).await
+                    }
+                }
+            }
+        })
+    }
+}
+
+struct MetricsSender {
+    metrics: MixMetrics,
+    directory_client: directory_client::Client,
+    pub_key_str: String,
+    sending_delay: Duration,
+    metrics_informer: MetricsInformer,
+}
+
+impl MetricsSender {
+    fn new(
+        metrics: MixMetrics,
+        directory_server: String,
+        pub_key_str: String,
+        sending_delay: Duration,
+        running_logging_delay: Duration,
+    ) -> Self {
+        MetricsSender {
+            metrics,
+            directory_client: directory_client::Client::new(directory_client::Config::new(
+                directory_server,
+            )),
+            pub_key_str,
+            sending_delay,
+            metrics_informer: MetricsInformer::new(running_logging_delay),
+        }
+    }
+
+    fn start(mut self, handle: &Handle) -> JoinHandle<()> {
+        handle.spawn(async move {
+            loop {
+                // set the deadline in the future
+                let sending_delay = tokio::time::delay_for(self.sending_delay);
+                let (received, sent) = self.metrics.acquire_and_reset_metrics().await;
+
+                self.metrics_informer.update_running_stats(received, &sent);
+                self.metrics_informer.log_report_stats(received, &sent);
+                self.metrics_informer.try_log_running_stats();
+
+                match self.directory_client.metrics_post.post(&MixMetric {
+                    pub_key: self.pub_key_str.clone(),
+                    received,
+                    sent,
+                }) {
+                    Err(err) => error!("failed to send metrics - {:?}", err),
+                    Ok(_) => debug!("sent metrics information"),
+                }
+
+                // wait for however much is left
+                sending_delay.await;
+            }
+        })
+    }
+}
+
+struct MetricsInformer {
+    total_received: u64,
+    sent_map: SentMetricsMap,
+
+    running_stats_logging_delay: Duration,
+    last_reported_stats: SystemTime,
+}
+
+impl MetricsInformer {
+    fn new(running_stats_logging_delay: Duration) -> Self {
+        MetricsInformer {
+            total_received: 0,
+            sent_map: HashMap::new(),
+            running_stats_logging_delay,
+            last_reported_stats: SystemTime::now(),
+        }
+    }
+
+    fn should_log_running_stats(&self) -> bool {
+        self.last_reported_stats + self.running_stats_logging_delay < SystemTime::now()
+    }
+
+    fn try_log_running_stats(&mut self) {
+        if self.should_log_running_stats() {
+            self.log_running_stats()
+        }
+    }
+
+    fn update_running_stats(&mut self, pre_reset_received: u64, pre_reset_sent: &SentMetricsMap) {
+        self.total_received += pre_reset_received;
+
+        for (mix, count) in pre_reset_sent.iter() {
+            *self.sent_map.entry(mix.clone()).or_insert(0) += *count;
+        }
+    }
+
+    fn log_report_stats(&self, pre_reset_received: u64, pre_reset_sent: &SentMetricsMap) {
+        debug!(
+            "Since last metrics report mixed {} packets!",
+            pre_reset_received
+        );
+        debug!(
+            "Since last metrics report received {} packets",
+            pre_reset_sent.values().sum::<u64>()
+        );
+        trace!(
+            "Since last metrics report sent packets to the following: \n{:#?}",
+            pre_reset_sent
+        );
+    }
+
+    fn log_running_stats(&mut self) {
+        info!(
+            "Since startup mixed {} packets!",
+            self.sent_map.values().sum::<u64>()
+        );
+        debug!("Since startup received {} packets", self.total_received);
+        trace!(
+            "Since startup sent packets to the following: \n{:#?}",
+            self.sent_map
+        );
+        self.last_reported_stats = SystemTime::now();
+    }
+}
+
+#[derive(Clone)]
+pub struct MetricsReporter {
+    metrics_tx: mpsc::UnboundedSender<MetricEvent>,
+}
+
+impl MetricsReporter {
+    pub(crate) fn new(metrics_tx: mpsc::UnboundedSender<MetricEvent>) -> Self {
+        MetricsReporter { metrics_tx }
+    }
+
+    pub(crate) fn report_sent(&self, destination: String) {
+        // in unbounded_send() failed it means that the receiver channel was disconnected
+        // and hence something weird must have happened without a way of recovering
+        self.metrics_tx
+            .unbounded_send(MetricEvent::Sent(destination))
+            .unwrap()
+    }
+
+    pub(crate) fn report_received(&self) {
+        // in unbounded_send() failed it means that the receiver channel was disconnected
+        // and hence something weird must have happened without a way of recovering
+        self.metrics_tx
+            .unbounded_send(MetricEvent::Received)
+            .unwrap()
+    }
+}
+
+// basically an easy single entry point to start all metrics related tasks
+pub struct MetricsController {
+    receiver: MetricsReceiver,
+    reporter: MetricsReporter,
+    sender: MetricsSender,
+}
+
+impl MetricsController {
+    pub(crate) fn new(
+        directory_server: String,
+        pub_key_str: String,
+        sending_delay: Duration,
+        running_stats_logging_delay: Duration,
+    ) -> Self {
+        let (metrics_tx, metrics_rx) = mpsc::unbounded();
+        let shared_metrics = MixMetrics::new();
+
+        MetricsController {
+            sender: MetricsSender::new(
+                shared_metrics.clone(),
+                directory_server,
+                pub_key_str,
+                sending_delay,
+                running_stats_logging_delay,
+            ),
+            receiver: MetricsReceiver::new(shared_metrics, metrics_rx),
+            reporter: MetricsReporter::new(metrics_tx),
+        }
+    }
+
+    // reporter is how node is going to be accessing the metrics data
+    pub(crate) fn start(self, handle: &Handle) -> MetricsReporter {
+        // TODO: should we do anything with JoinHandle(s) returned by start methods?
+        self.receiver.start(handle);
+        self.sender.start(handle);
+        self.reporter
+    }
+}

mixnode/src/node/mod.rs

@@ -0,0 +1,142 @@
+use crate::config::persistence::pathfinder::MixNodePathfinder;
+use crate::config::Config;
+use crate::node::packet_processing::PacketProcessor;
+use crypto::encryption;
+use directory_client::presence::Topology;
+use futures::channel::mpsc;
+use log::*;
+use pemstore::pemstore::PemStore;
+use std::net::SocketAddr;
+use tokio::runtime::Runtime;
+use topology::NymTopology;
+
+mod listener;
+mod metrics;
+mod packet_forwarding;
+pub(crate) mod packet_processing;
+mod presence;
+
+// the MixNode will live for whole duration of this program
+pub struct MixNode {
+    runtime: Runtime,
+    config: Config,
+    sphinx_keypair: encryption::KeyPair,
+}
+
+impl MixNode {
+    fn load_sphinx_keys(config_file: &Config) -> encryption::KeyPair {
+        let sphinx_keypair = PemStore::new(MixNodePathfinder::new_from_config(&config_file))
+            .read_encryption()
+            .expect("Failed to read stored sphinx key files");
+        println!(
+            "Public key: {}\n",
+            sphinx_keypair.public_key().to_base58_string()
+        );
+        sphinx_keypair
+    }
+
+    pub fn new(config: Config) -> Self {
+        let sphinx_keypair = Self::load_sphinx_keys(&config);
+
+        MixNode {
+            runtime: Runtime::new().unwrap(),
+            config,
+            sphinx_keypair,
+        }
+    }
+
+    fn start_presence_notifier(&self) {
+        info!("Starting presence notifier...");
+        let notifier_config = presence::NotifierConfig::new(
+            self.config.get_location(),
+            self.config.get_presence_directory_server(),
+            self.config.get_announce_address(),
+            self.sphinx_keypair.public_key().to_base58_string(),
+            self.config.get_layer(),
+            self.config.get_presence_sending_delay(),
+        );
+        presence::Notifier::new(notifier_config).start(self.runtime.handle());
+    }
+
+    fn start_metrics_reporter(&self) -> metrics::MetricsReporter {
+        info!("Starting metrics reporter...");
+        metrics::MetricsController::new(
+            self.config.get_metrics_directory_server(),
+            self.sphinx_keypair.public_key().to_base58_string(),
+            self.config.get_metrics_sending_delay(),
+            self.config.get_metrics_running_stats_logging_delay(),
+        )
+        .start(self.runtime.handle())
+    }
+
+    fn start_socket_listener(
+        &self,
+        metrics_reporter: metrics::MetricsReporter,
+        forwarding_channel: mpsc::UnboundedSender<(SocketAddr, Vec<u8>)>,
+    ) {
+        info!("Starting socket listener...");
+        // this is the only location where our private key is going to be copied
+        // it will be held in memory owned by `MixNode` and inside an Arc of `PacketProcessor`
+        let packet_processor =
+            PacketProcessor::new(self.sphinx_keypair.private_key().clone(), metrics_reporter);
+
+        listener::run_socket_listener(
+            self.runtime.handle(),
+            self.config.get_listening_address(),
+            packet_processor,
+            forwarding_channel,
+        );
+    }
+
+    fn start_packet_forwarder(&mut self) -> mpsc::UnboundedSender<(SocketAddr, Vec<u8>)> {
+        info!("Starting packet forwarder...");
+        self.runtime
+            .enter(|| {
+                packet_forwarding::PacketForwarder::new(
+                    self.config.get_packet_forwarding_initial_backoff(),
+                    self.config.get_packet_forwarding_maximum_backoff(),
+                    self.config.get_initial_connection_timeout(),
+                )
+            })
+            .start(self.runtime.handle())
+    }
+
+    fn check_if_same_ip_node_exists(&self) -> Option<String> {
+        // TODO: once we change to graph topology this here will need to be updated!
+        let topology = Topology::new(self.config.get_presence_directory_server());
+        let existing_mixes_presence = topology.mix_nodes;
+        existing_mixes_presence
+            .iter()
+            .find(|node| node.host == self.config.get_announce_address())
+            .map(|node| node.pub_key.clone())
+    }
+
+    pub fn run(&mut self) {
+        info!("Starting nym mixnode");
+
+        if let Some(duplicate_node_key) = self.check_if_same_ip_node_exists() {
+            error!(
+                "Our announce-host is identical to one of existing nodes! (its key is {:?}",
+                duplicate_node_key
+            );
+            return;
+        }
+        let forwarding_channel = self.start_packet_forwarder();
+        let metrics_reporter = self.start_metrics_reporter();
+        self.start_socket_listener(metrics_reporter, forwarding_channel);
+        self.start_presence_notifier();
+
+        info!("Finished nym mixnode startup procedure - it should now be able to receive mix traffic!");
+
+        if let Err(e) = self.runtime.block_on(tokio::signal::ctrl_c()) {
+            error!(
+                "There was an error while capturing SIGINT - {:?}. We will terminate regardless",
+                e
+            );
+        }
+
+        println!(
+            "Received SIGINT - the mixnode will terminate now (threads are not YET nicely stopped)"
+        );
+    }
+}

mixnode/src/node/packet_forwarding.rs

@@ -0,0 +1,46 @@
+use futures::channel::mpsc;
+use futures::StreamExt;
+use std::net::SocketAddr;
+use std::time::Duration;
+use tokio::runtime::Handle;
+
+pub(crate) struct PacketForwarder {
+    tcp_client: multi_tcp_client::Client,
+    conn_tx: mpsc::UnboundedSender<(SocketAddr, Vec<u8>)>,
+    conn_rx: mpsc::UnboundedReceiver<(SocketAddr, Vec<u8>)>,
+}
+
+impl PacketForwarder {
+    pub(crate) fn new(
+        initial_reconnection_backoff: Duration,
+        maximum_reconnection_backoff: Duration,
+        initial_connection_timeout: Duration,
+    ) -> PacketForwarder {
+        let tcp_client_config = multi_tcp_client::Config::new(
+            initial_reconnection_backoff,
+            maximum_reconnection_backoff,
+            initial_connection_timeout,
+        );
+
+        let (conn_tx, conn_rx) = mpsc::unbounded();
+
+        PacketForwarder {
+            tcp_client: multi_tcp_client::Client::new(tcp_client_config),
+            conn_tx,
+            conn_rx,
+        }
+    }
+
+    pub(crate) fn start(mut self, handle: &Handle) -> mpsc::UnboundedSender<(SocketAddr, Vec<u8>)> {
+        // TODO: what to do with the lost JoinHandle?
+        let sender_channel = self.conn_tx.clone();
+        handle.spawn(async move {
+            while let Some((address, packet)) = self.conn_rx.next().await {
+                // as a mix node we don't care about responses, we just want to fire packets
+                // as quickly as possible
+                self.tcp_client.send(address, packet, false).await.unwrap(); // if we're not waiting for response, we MUST get an Ok
+            }
+        });
+        sender_channel
+    }
+}

mixnode/src/node/packet_processing.rs

@@ -0,0 +1,109 @@
+use crate::node::metrics;
+use crypto::encryption;
+use log::*;
+use nymsphinx::addressing::nodes::{NymNodeRoutingAddress, NymNodeRoutingAddressError};
+use sphinx::header::delays::Delay as SphinxDelay;
+use sphinx::route::NodeAddressBytes;
+use sphinx::{ProcessedPacket, SphinxPacket};
+use std::convert::TryFrom;
+use std::net::SocketAddr;
+use std::ops::Deref;
+use std::sync::Arc;
+
+#[derive(Debug)]
+pub enum MixProcessingError {
+    SphinxRecoveryError,
+    ReceivedFinalHopError,
+    SphinxProcessingError,
+    InvalidHopAddress,
+}
+
+pub enum MixProcessingResult {
+    ForwardHop(SocketAddr, Vec<u8>),
+    #[allow(dead_code)]
+    LoopMessage,
+}
+
+impl From<sphinx::ProcessingError> for MixProcessingError {
+    // for time being just have a single error instance for all possible results of sphinx::ProcessingError
+    fn from(_: sphinx::ProcessingError) -> Self {
+        use MixProcessingError::*;
+
+        SphinxRecoveryError
+    }
+}
+
+impl From<NymNodeRoutingAddressError> for MixProcessingError {
+    fn from(_: NymNodeRoutingAddressError) -> Self {
+        use MixProcessingError::*;
+
+        InvalidHopAddress
+    }
+}
+
+// PacketProcessor contains all data required to correctly unwrap and forward sphinx packets
+#[derive(Clone)]
+pub struct PacketProcessor {
+    secret_key: Arc<encryption::PrivateKey>,
+    metrics_reporter: metrics::MetricsReporter,
+}
+
+impl PacketProcessor {
+    pub(crate) fn new(
+        secret_key: encryption::PrivateKey,
+        metrics_reporter: metrics::MetricsReporter,
+    ) -> Self {
+        PacketProcessor {
+            secret_key: Arc::new(secret_key),
+            metrics_reporter,
+        }
+    }
+
+    pub(crate) fn report_sent(&self, addr: SocketAddr) {
+        self.metrics_reporter.report_sent(addr.to_string())
+    }
+
+    async fn process_forward_hop(
+        &self,
+        packet: SphinxPacket,
+        forward_address: NodeAddressBytes,
+        delay: SphinxDelay,
+    ) -> Result<MixProcessingResult, MixProcessingError> {
+        let next_hop_address: SocketAddr = NymNodeRoutingAddress::try_from(forward_address)?.into();
+
+        // Delay packet for as long as required
+        tokio::time::delay_for(delay.to_duration()).await;
+
+        Ok(MixProcessingResult::ForwardHop(
+            next_hop_address,
+            packet.to_bytes(),
+        ))
+    }
+
+    pub(crate) async fn process_sphinx_packet(
+        &self,
+        raw_packet_data: [u8; sphinx::PACKET_SIZE],
+    ) -> Result<MixProcessingResult, MixProcessingError> {
+        // we received something resembling a sphinx packet, report it!
+        self.metrics_reporter.report_received();
+
+        let packet = SphinxPacket::from_bytes(&raw_packet_data)?;
+
+        match packet.process(self.secret_key.deref().inner()) {
+            Ok(ProcessedPacket::ProcessedPacketForwardHop(packet, address, delay)) => {
+                self.process_forward_hop(packet, address, delay).await
+            }
+            Ok(ProcessedPacket::ProcessedPacketFinalHop(_, _, _)) => {
+                warn!("Received a loop cover message that we haven't implemented yet!");
+                Err(MixProcessingError::ReceivedFinalHopError)
+            }
+            Err(e) => {
+                warn!("Failed to unwrap Sphinx packet: {:?}", e);
+                Err(MixProcessingError::SphinxProcessingError)
+            }
+        }
+    }
+}
+
+// TODO: the test that definitely needs to be written is as follows:
+// we are stuck trying to write to mix A, can we still forward just fine to mix B?

mixnode/src/node/presence.rs

@@ -0,0 +1,84 @@
+use crate::built_info;
+use directory_client::presence::mixnodes::MixNodePresence;
+use directory_client::requests::presence_mixnodes_post::PresenceMixNodesPoster;
+use directory_client::DirectoryClient;
+use log::{debug, error};
+use std::time::Duration;
+use tokio::runtime::Handle;
+use tokio::task::JoinHandle;
+
+pub struct NotifierConfig {
+    location: String,
+    directory_server: String,
+    announce_host: String,
+    pub_key_string: String,
+    layer: u64,
+    sending_delay: Duration,
+}
+
+impl NotifierConfig {
+    pub fn new(
+        location: String,
+        directory_server: String,
+        announce_host: String,
+        pub_key_string: String,
+        layer: u64,
+        sending_delay: Duration,
+    ) -> Self {
+        NotifierConfig {
+            location,
+            directory_server,
+            announce_host,
+            pub_key_string,
+            layer,
+            sending_delay,
+        }
+    }
+}
+
+pub struct Notifier {
+    net_client: directory_client::Client,
+    presence: MixNodePresence,
+    sending_delay: Duration,
+}
+
+impl Notifier {
+    pub fn new(config: NotifierConfig) -> Notifier {
+        let directory_client_cfg = directory_client::Config {
+            base_url: config.directory_server,
+        };
+        let net_client = directory_client::Client::new(directory_client_cfg);
+        let presence = MixNodePresence {
+            location: config.location,
+            host: config.announce_host,
+            pub_key: config.pub_key_string,
+            layer: config.layer,
+            last_seen: 0,
+            version: built_info::PKG_VERSION.to_string(),
+        };
+        Notifier {
+            net_client,
+            presence,
+            sending_delay: config.sending_delay,
+        }
+    }
+
+    fn notify(&self) {
+        match self.net_client.presence_mix_nodes_post.post(&self.presence) {
+            Err(err) => error!("failed to send presence - {:?}", err),
+            Ok(_) => debug!("sent presence information"),
+        }
+    }
+
+    pub fn start(self, handle: &Handle) -> JoinHandle<()> {
+        handle.spawn(async move {
+            loop {
+                // set the deadline in the future
+                let sending_delay = tokio::time::delay_for(self.sending_delay);
+                self.notify();
+                // wait for however much is left
+                sending_delay.await;
+            }
+        })
+    }
+}

nym-client/Cargo.toml

@@ -0,0 +1,55 @@
+[package]
+build = "build.rs"
+name = "nym-client"
+version = "0.6.0"
+authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
+edition = "2018"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[lib]
+name = "nym_client"
+path = "src/lib.rs"
+
+[dependencies]
+bs58 = "0.3.0"
+clap = "2.33.0"
+curve25519-dalek = "1.2.3"
+dirs = "2.0.2"
+dotenv = "0.15.0"
+futures = "0.3.1"
+log = "0.4"
+pem = "0.7.0"
+pretty_env_logger = "0.3"
+reqwest = "0.9.22"
+serde = { version = "1.0.104", features = ["derive"] }
+serde_json = "1.0.44"
+tokio = { version = "0.2", features = ["full"] }
+tokio-tungstenite = "0.10.1"
+
+## internal
+config = {path = "../common/config"}
+crypto = {path = "../common/crypto"}
+directory-client = { path = "../common/clients/directory-client" }
+healthcheck = { path = "../common/healthcheck" }
+mix-client = { path = "../common/clients/mix-client" }
+multi-tcp-client = { path = "../common/clients/multi-tcp-client" }
+nymsphinx = { path = "../common/nymsphinx" }
+pemstore = {path = "../common/pemstore"}
+provider-client = { path = "../common/clients/provider-client" }
+sfw-provider-requests = { path = "../sfw-provider/sfw-provider-requests" }
+topology = {path = "../common/topology" }
+
+## will be moved to proper dependencies once released
+sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" }
+# sphinx = { path = "../../sphinx"}
+
+[build-dependencies]
+built = "0.3.2"
+
+[dev-dependencies]
+tempfile = "3.1.0"
+
+[features]
+qa = []
+local = []

nym-client/README.md

@@ -0,0 +1,10 @@
+# Nym Client
+
+The Nym Client communicates with the remote, decentralised nodes which make up the Nym system as a whole. 
+
+It needs to handle communication with 3 types of remote nodes. Here's the development status of each:
+
+- [ ] Directory nodes
+- [ ] Mix nodes
+- [ ] Validator nodes
+

nym-client/build.rs

@@ -0,0 +1,5 @@
+use built;
+
+fn main() {
+    built::write_built_file().expect("Failed to acquire build-time information");
+}

nym-client/src/built_info.rs

@@ -0,0 +1,2 @@
+// The file has been placed there by the build script.
+include!(concat!(env!("OUT_DIR"), "/built.rs"));

nym-client/src/client/cover_traffic_stream.rs

@@ -0,0 +1,123 @@
+use crate::client::mix_traffic::{MixMessage, MixMessageSender};
+use crate::client::topology_control::TopologyAccessor;
+use futures::task::{Context, Poll};
+use futures::{Future, Stream, StreamExt};
+use log::*;
+use sphinx::route::Destination;
+use std::pin::Pin;
+use std::time::Duration;
+use tokio::runtime::Handle;
+use tokio::task::JoinHandle;
+use tokio::time;
+use topology::NymTopology;
+
+pub(crate) struct LoopCoverTrafficStream<T: NymTopology> {
+    average_packet_delay: Duration,
+    average_cover_message_sending_delay: Duration,
+    next_delay: time::Delay,
+    mix_tx: MixMessageSender,
+    our_info: Destination,
+    topology_access: TopologyAccessor<T>,
+}
+
+impl<T: NymTopology> Stream for LoopCoverTrafficStream<T> {
+    // Item is only used to indicate we should create a new message rather than actual cover message
+    // reason being to not introduce unnecessary complexity by having to keep state of topology
+    // mutex when trying to acquire it. So right now the Stream trait serves as a glorified timer.
+    // Perhaps this should be changed in the future.
+    type Item = ();
+
+    fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
+        // it is not yet time to return a message
+        if Pin::new(&mut self.next_delay).poll(cx).is_pending() {
+            return Poll::Pending;
+        };
+
+        // we know it's time to send a message, so let's prepare delay for the next one
+        // Get the `now` by looking at the current `delay` deadline
+        let now = self.next_delay.deadline();
+        let next_poisson_delay =
+            mix_client::poisson::sample(self.average_cover_message_sending_delay);
+
+        // The next interval value is `next_poisson_delay` after the one that just
+        // yielded.
+        let next = now + next_poisson_delay;
+        self.next_delay.reset(next);
+
+        Poll::Ready(Some(()))
+    }
+}
+
+impl<T: 'static + NymTopology> LoopCoverTrafficStream<T> {
+    pub(crate) fn new(
+        mix_tx: MixMessageSender,
+        our_info: Destination,
+        topology_access: TopologyAccessor<T>,
+        average_cover_message_sending_delay: time::Duration,
+        average_packet_delay: time::Duration,
+    ) -> Self {
+        LoopCoverTrafficStream {
+            average_packet_delay,
+            average_cover_message_sending_delay,
+            next_delay: time::delay_for(Default::default()),
+            mix_tx,
+            our_info,
+            topology_access,
+        }
+    }
+
+    async fn on_new_message(&mut self) {
+        trace!("next cover message!");
+        let route = match self.topology_access.random_route().await {
+            None => {
+                warn!("No valid topology detected - won't send any loop cover message this time");
+                return;
+            }
+            Some(route) => route,
+        };
+
+        let cover_message = match mix_client::packet::loop_cover_message_route(
+            self.our_info.address.clone(),
+            self.our_info.identifier,
+            route,
+            self.average_packet_delay,
+        ) {
+            Ok(message) => message,
+            Err(err) => {
+                error!(
+                    "Somehow we managed to create an invalid cover message - {:?}",
+                    err
+                );
+                return;
+            }
+        };
+
+        // if this one fails, there's no retrying because it means that either:
+        // - we run out of memory
+        // - the receiver channel is closed
+        // in either case there's no recovery and we can only panic
+        self.mix_tx
+            .unbounded_send(MixMessage::new(cover_message.0, cover_message.1))
+            .unwrap();
+        // JS: due to identical logical structure to OutQueueControl::on_message(), this is also
+        // presumably required to prevent bugs in the future. Exact reason is still unknown to me.
+        tokio::task::yield_now().await;
+    }
+
+    async fn run(&mut self) {
+        // we should set initial delay only when we actually start the stream
+        self.next_delay = time::delay_for(mix_client::poisson::sample(
+            self.average_cover_message_sending_delay,
+        ));
+
+        while let Some(_) = self.next().await {
+            self.on_new_message().await;
+        }
+    }
+
+    pub(crate) fn start(mut self, handle: &Handle) -> JoinHandle<()> {
+        handle.spawn(async move {
+            self.run().await;
+        })
+    }
+}

nym-client/src/client/mix_traffic.rs

@@ -0,0 +1,64 @@
+use futures::channel::mpsc;
+use futures::StreamExt;
+use log::*;
+use sphinx::SphinxPacket;
+use std::net::SocketAddr;
+use std::time::Duration;
+use tokio::runtime::Handle;
+use tokio::task::JoinHandle;
+
+pub(crate) struct MixMessage(SocketAddr, SphinxPacket);
+pub(crate) type MixMessageSender = mpsc::UnboundedSender<MixMessage>;
+pub(crate) type MixMessageReceiver = mpsc::UnboundedReceiver<MixMessage>;
+
+impl MixMessage {
+    pub(crate) fn new(address: SocketAddr, packet: SphinxPacket) -> Self {
+        MixMessage(address, packet)
+    }
+}
+
+pub(crate) struct MixTrafficController {
+    tcp_client: multi_tcp_client::Client,
+    mix_rx: MixMessageReceiver,
+}
+
+impl MixTrafficController {
+    pub(crate) fn new(
+        initial_reconnection_backoff: Duration,
+        maximum_reconnection_backoff: Duration,
+        initial_connection_timeout: Duration,
+        mix_rx: MixMessageReceiver,
+    ) -> Self {
+        let tcp_client_config = multi_tcp_client::Config::new(
+            initial_reconnection_backoff,
+            maximum_reconnection_backoff,
+            initial_connection_timeout,
+        );
+
+        MixTrafficController {
+            tcp_client: multi_tcp_client::Client::new(tcp_client_config),
+            mix_rx,
+        }
+    }
+
+    async fn on_message(&mut self, mix_message: MixMessage) {
+        debug!("Got a mix_message for {:?}", mix_message.0);
+        self.tcp_client
+            // TODO: possibly we might want to get an actual result here at some point
+            .send(mix_message.0, mix_message.1.to_bytes(), false)
+            .await
+            .unwrap(); // if we're not waiting for response, we MUST get an Ok
+    }
+
+    pub(crate) async fn run(&mut self) {
+        while let Some(mix_message) = self.mix_rx.next().await {
+            self.on_message(mix_message).await;
+        }
+    }
+
+    pub(crate) fn start(mut self, handle: &Handle) -> JoinHandle<()> {
+        handle.spawn(async move {
+            self.run().await;
+        })
+    }
+}

nym-client/src/client/mod.rs

@@ -0,0 +1,354 @@
+use crate::client::cover_traffic_stream::LoopCoverTrafficStream;
+use crate::client::mix_traffic::{MixMessageReceiver, MixMessageSender, MixTrafficController};
+use crate::client::provider_poller::{PolledMessagesReceiver, PolledMessagesSender};
+use crate::client::received_buffer::{
+    ReceivedBufferRequestReceiver, ReceivedBufferRequestSender, ReceivedMessagesBufferController,
+};
+use crate::client::topology_control::{
+    TopologyAccessor, TopologyRefresher, TopologyRefresherConfig,
+};
+use crate::config::persistence::pathfinder::ClientPathfinder;
+use crate::config::{Config, SocketType};
+use crate::sockets::{tcp, websocket};
+use crypto::identity::MixIdentityKeyPair;
+use directory_client::presence;
+use futures::channel::{mpsc, oneshot};
+use log::*;
+use nymsphinx::chunking::split_and_prepare_payloads;
+use pemstore::pemstore::PemStore;
+use sfw_provider_requests::AuthToken;
+use sphinx::route::Destination;
+use tokio::runtime::Runtime;
+use topology::NymTopology;
+
+mod cover_traffic_stream;
+mod mix_traffic;
+mod provider_poller;
+mod real_traffic_stream;
+pub(crate) mod received_buffer;
+pub(crate) mod topology_control;
+
+pub(crate) type InputMessageSender = mpsc::UnboundedSender<InputMessage>;
+pub(crate) type InputMessageReceiver = mpsc::UnboundedReceiver<InputMessage>;
+
+pub struct NymClient {
+    config: Config,
+    runtime: Runtime,
+    identity_keypair: MixIdentityKeyPair,
+
+    // to be used by "send" function or socket, etc
+    input_tx: Option<InputMessageSender>,
+
+    // to be used by "receive" function or socket, etc
+    receive_tx: Option<ReceivedBufferRequestSender>,
+}
+
+#[derive(Debug)]
+// TODO: make fields private
+pub(crate) struct InputMessage(pub Destination, pub Vec<u8>);
+
+impl NymClient {
+    fn load_identity_keys(config_file: &Config) -> MixIdentityKeyPair {
+        let identity_keypair = PemStore::new(ClientPathfinder::new_from_config(&config_file))
+            .read_identity()
+            .expect("Failed to read stored identity key files");
+        println!(
+            "Public identity key: {}\n",
+            identity_keypair.public_key.to_base58_string()
+        );
+        identity_keypair
+    }
+
+    pub fn new(config: Config) -> Self {
+        let identity_keypair = Self::load_identity_keys(&config);
+
+        NymClient {
+            runtime: Runtime::new().unwrap(),
+            config,
+            identity_keypair,
+            input_tx: None,
+            receive_tx: None,
+        }
+    }
+
+    pub fn as_mix_destination(&self) -> Destination {
+        Destination::new(
+            self.identity_keypair.public_key().derive_address(),
+            // TODO: what with SURBs?
+            Default::default(),
+        )
+    }
+
+    // future constantly pumping loop cover traffic at some specified average rate
+    // the pumped traffic goes to the MixTrafficController
+    fn start_cover_traffic_stream<T: 'static + NymTopology>(
+        &self,
+        topology_accessor: TopologyAccessor<T>,
+        mix_tx: MixMessageSender,
+    ) {
+        info!("Starting loop cover traffic stream...");
+        // we need to explicitly enter runtime due to "next_delay: time::delay_for(Default::default())"
+        // set in the constructor which HAS TO be called within context of a tokio runtime
+        self.runtime
+            .enter(|| {
+                LoopCoverTrafficStream::new(
+                    mix_tx,
+                    self.as_mix_destination(),
+                    topology_accessor,
+                    self.config.get_loop_cover_traffic_average_delay(),
+                    self.config.get_average_packet_delay(),
+                )
+            })
+            .start(self.runtime.handle());
+    }
+
+    fn start_real_traffic_stream<T: 'static + NymTopology>(
+        &self,
+        topology_accessor: TopologyAccessor<T>,
+        mix_tx: MixMessageSender,
+        input_rx: InputMessageReceiver,
+    ) {
+        info!("Starting real traffic stream...");
+        // we need to explicitly enter runtime due to "next_delay: time::delay_for(Default::default())"
+        // set in the constructor which HAS TO be called within context of a tokio runtime
+        self.runtime
+            .enter(|| {
+                real_traffic_stream::OutQueueControl::new(
+                    mix_tx,
+                    input_rx,
+                    self.as_mix_destination(),
+                    topology_accessor,
+                    self.config.get_average_packet_delay(),
+                    self.config.get_message_sending_average_delay(),
+                )
+            })
+            .start(self.runtime.handle());
+    }
+
+    // buffer controlling all messages fetched from provider
+    // required so that other components would be able to use them (say the websocket)
+    fn start_received_messages_buffer_controller(
+        &self,
+        query_receiver: ReceivedBufferRequestReceiver,
+        poller_receiver: PolledMessagesReceiver,
+    ) {
+        info!("Starting 'received messages buffer controller'...");
+        ReceivedMessagesBufferController::new(query_receiver, poller_receiver)
+            .start(self.runtime.handle())
+    }
+
+    // future constantly trying to fetch any received messages from the provider
+    // the received messages are sent to ReceivedMessagesBuffer to be available to rest of the system
+    fn start_provider_poller<T: NymTopology>(
+        &mut self,
+        mut topology_accessor: TopologyAccessor<T>,
+        poller_input_tx: PolledMessagesSender,
+    ) {
+        info!("Starting provider poller...");
+        // we already have our provider written in the config
+        let provider_id = self.config.get_provider_id();
+
+        // TODO: a slightly more graceful termination here
+        if !self.runtime.block_on(topology_accessor.is_routable()) {
+            panic!(
+                "The current network topology seem to be insufficient to route any packets through\
+             - check if enough nodes and a sfw-provider are online"
+            );
+        }
+
+        // TODO: a slightly more graceful termination here
+        let provider_client_listener_address = self.runtime.block_on(
+            topology_accessor.get_provider_socket_addr(&provider_id)
+        ).unwrap_or_else(|| panic!("Could not find provider with id {:?}. It does not seem to be present in the current network topology.\
+        Are you sure it is still online? Perhaps try to run `nym-client init` again to obtain a new provider", provider_id));
+
+        let mut provider_poller = provider_poller::ProviderPoller::new(
+            poller_input_tx,
+            provider_client_listener_address,
+            self.identity_keypair.public_key().derive_address(),
+            self.config
+                .get_provider_auth_token()
+                .map(|str_token| AuthToken::try_from_base58_string(str_token).ok())
+                .unwrap_or(None),
+            self.config.get_fetch_message_delay(),
+        );
+
+        if !provider_poller.is_registered() {
+            info!("Trying to perform initial provider registration...");
+            self.runtime
+                .block_on(provider_poller.perform_initial_registration())
+                .expect("Failed to perform initial provider registration");
+        }
+        provider_poller.start(self.runtime.handle());
+    }
+
+    // future responsible for periodically polling directory server and updating
+    // the current global view of topology
+    fn start_topology_refresher<T: 'static + NymTopology>(
+        &mut self,
+        topology_accessor: TopologyAccessor<T>,
+    ) {
+        let healthcheck_keys = MixIdentityKeyPair::new();
+
+        let topology_refresher_config = TopologyRefresherConfig::new(
+            self.config.get_directory_server(),
+            self.config.get_topology_refresh_rate(),
+            healthcheck_keys,
+            self.config.get_topology_resolution_timeout(),
+            self.config.get_healthcheck_connection_timeout(),
+            self.config.get_number_of_healthcheck_test_packets() as usize,
+            self.config.get_node_score_threshold(),
+        );
+        let mut topology_refresher =
+            TopologyRefresher::new(topology_refresher_config, topology_accessor);
+        // before returning, block entire runtime to refresh the current network view so that any
+        // components depending on topology would see a non-empty view
+        info!(
+            "Obtaining initial network topology from {}",
+            self.config.get_directory_server()
+        );
+        self.runtime.block_on(topology_refresher.refresh());
+        info!("Starting topology refresher...");
+        topology_refresher.start(self.runtime.handle());
+    }
+
+    // controller for sending sphinx packets to mixnet (either real traffic or cover traffic)
+    fn start_mix_traffic_controller(&mut self, mix_rx: MixMessageReceiver) {
+        info!("Starting mix trafic controller...");
+        self.runtime
+            .enter(|| {
+                MixTrafficController::new(
+                    self.config.get_packet_forwarding_initial_backoff(),
+                    self.config.get_packet_forwarding_maximum_backoff(),
+                    self.config.get_initial_connection_timeout(),
+                    mix_rx,
+                )
+            })
+            .start(self.runtime.handle());
+    }
+
+    fn start_socket_listener<T: 'static + NymTopology>(
+        &self,
+        topology_accessor: TopologyAccessor<T>,
+        received_messages_buffer_output_tx: ReceivedBufferRequestSender,
+        input_tx: InputMessageSender,
+    ) {
+        match self.config.get_socket_type() {
+            SocketType::WebSocket => {
+                websocket::listener::run(
+                    self.runtime.handle(),
+                    self.config.get_listening_port(),
+                    input_tx,
+                    received_messages_buffer_output_tx,
+                    self.identity_keypair.public_key().derive_address(),
+                    topology_accessor,
+                );
+            }
+            SocketType::TCP => {
+                tcp::start_tcpsocket(
+                    self.runtime.handle(),
+                    self.config.get_listening_port(),
+                    input_tx,
+                    received_messages_buffer_output_tx,
+                    self.identity_keypair.public_key().derive_address(),
+                    topology_accessor,
+                );
+            }
+            SocketType::None => (),
+        }
+    }
+
+    /// EXPERIMENTAL DIRECT RUST API
+    /// It's untested and there are absolutely no guarantees about it (but seems to have worked
+    /// well enough in local tests)
+    pub fn send_message(&mut self, destination: Destination, message: Vec<u8>) {
+        let split_message = split_and_prepare_payloads(&message);
+        debug!(
+            "Splitting message into {:?} fragments!",
+            split_message.len()
+        );
+        for message_fragment in split_message {
+            let input_msg = InputMessage(destination.clone(), message_fragment);
+            self.input_tx
+                .as_ref()
+                .expect("start method was not called before!")
+                .unbounded_send(input_msg)
+                .unwrap()
+        }
+    }
+
+    /// EXPERIMENTAL DIRECT RUST API
+    /// It's untested and there are absolutely no guarantees about it (but seems to have worked
+    /// well enough in local tests)
+    pub async fn check_for_messages_async(&self) -> Vec<Vec<u8>> {
+        let (res_tx, res_rx) = oneshot::channel();
+        self.receive_tx
+            .as_ref()
+            .expect("start method was not called before!")
+            .unbounded_send(res_tx)
+            .unwrap();
+
+        res_rx.await.unwrap()
+    }
+
+    /// blocking version of `start` method. Will run forever (or until SIGINT is sent)
+    pub fn run_forever(&mut self) {
+        self.start();
+        if let Err(e) = self.runtime.block_on(tokio::signal::ctrl_c()) {
+            error!(
+                "There was an error while capturing SIGINT - {:?}. We will terminate regardless",
+                e
+            );
+        }
+
+        println!(
+            "Received SIGINT - the client will terminate now (threads are not YET nicely stopped)"
+        );
+    }
+
+    pub fn start(&mut self) {
+        info!("Starting nym client");
+        // channels for inter-component communication
+
+        // mix_tx is the transmitter for any component generating sphinx packets that are to be sent to the mixnet
+        // they are used by cover traffic stream and real traffic stream
+        // mix_rx is the receiver used by MixTrafficController that sends the actual traffic
+        let (mix_tx, mix_rx) = mpsc::unbounded();
+
+        // poller_input_tx is the transmitter of messages fetched from the provider - used by ProviderPoller
+        // poller_input_rx is the receiver for said messages - used by ReceivedMessagesBuffer
+        let (poller_input_tx, poller_input_rx) = mpsc::unbounded();
+
+        // received_messages_buffer_output_tx is the transmitter for *REQUESTS* for messages contained in ReceivedMessagesBuffer - used by sockets
+        // the requests contain a oneshot channel to send a reply on
+        // received_messages_buffer_output_rx is the received for the said requests - used by ReceivedMessagesBuffer
+        let (received_messages_buffer_output_tx, received_messages_buffer_output_rx) =
+            mpsc::unbounded();
+
+        // channels responsible for controlling real messages
+        let (input_tx, input_rx) = mpsc::unbounded::<InputMessage>();
+
+        // TODO: when we switch to our graph topology, we need to remember to change 'presence::Topology' type
+        let shared_topology_accessor = TopologyAccessor::<presence::Topology>::new();
+        // the components are started in very specific order. Unless you know what you are doing,
+        // do not change that.
+        self.start_topology_refresher(shared_topology_accessor.clone());
+        self.start_received_messages_buffer_controller(
+            received_messages_buffer_output_rx,
+            poller_input_rx,
+        );
+        self.start_provider_poller(shared_topology_accessor.clone(), poller_input_tx);
+        self.start_mix_traffic_controller(mix_rx);
+        self.start_cover_traffic_stream(shared_topology_accessor.clone(), mix_tx.clone());
+        self.start_real_traffic_stream(shared_topology_accessor.clone(), mix_tx, input_rx);
+        self.start_socket_listener(
+            shared_topology_accessor,
+            received_messages_buffer_output_tx.clone(),
+            input_tx.clone(),
+        );
+        self.input_tx = Some(input_tx);
+        self.receive_tx = Some(received_messages_buffer_output_tx);
+
+        info!("Client startup finished!");
+    }
+}

nym-client/src/client/provider_poller.rs

@@ -0,0 +1,106 @@
+use futures::channel::mpsc;
+use log::*;
+use provider_client::ProviderClientError;
+use sfw_provider_requests::AuthToken;
+use sphinx::route::DestinationAddressBytes;
+use std::net::SocketAddr;
+use std::time;
+use tokio::runtime::Handle;
+use tokio::task::JoinHandle;
+
+pub(crate) type PolledMessagesSender = mpsc::UnboundedSender<Vec<Vec<u8>>>;
+pub(crate) type PolledMessagesReceiver = mpsc::UnboundedReceiver<Vec<Vec<u8>>>;
+
+pub(crate) struct ProviderPoller {
+    polling_rate: time::Duration,
+    provider_client: provider_client::ProviderClient,
+    poller_tx: mpsc::UnboundedSender<Vec<Vec<u8>>>,
+}
+
+impl ProviderPoller {
+    pub(crate) fn new(
+        poller_tx: mpsc::UnboundedSender<Vec<Vec<u8>>>,
+        provider_client_listener_address: SocketAddr,
+        client_address: DestinationAddressBytes,
+        auth_token: Option<AuthToken>,
+        polling_rate: time::Duration,
+    ) -> Self {
+        ProviderPoller {
+            provider_client: provider_client::ProviderClient::new(
+                provider_client_listener_address,
+                client_address,
+                auth_token,
+            ),
+            poller_tx,
+            polling_rate,
+        }
+    }
+
+    pub(crate) fn is_registered(&self) -> bool {
+        self.provider_client.is_registered()
+    }
+
+    // This method is only temporary until registration is moved to `client init`
+    pub(crate) async fn perform_initial_registration(&mut self) -> Result<(), ProviderClientError> {
+        debug!("performing initial provider registration");
+
+        if !self.is_registered() {
+            let auth_token = match self.provider_client.register().await {
+                // in this particular case we can ignore this error
+                Err(ProviderClientError::ClientAlreadyRegisteredError) => return Ok(()),
+                Err(err) => return Err(err),
+                Ok(token) => token,
+            };
+
+            self.provider_client.update_token(auth_token)
+        } else {
+            warn!("did not perform provider registration - we were already registered")
+        }
+
+        Ok(())
+    }
+
+    pub(crate) async fn start_provider_polling(self) {
+        let loop_message = &mix_client::packet::LOOP_COVER_MESSAGE_PAYLOAD.to_vec();
+        let dummy_message = &sfw_provider_requests::DUMMY_MESSAGE_CONTENT.to_vec();
+
+        let extended_delay_duration = self.polling_rate * 10;
+
+        loop {
+            debug!("Polling provider...");
+
+            let messages = match self.provider_client.retrieve_messages().await {
+                Err(err) => {
+                    error!("Failed to query the provider for messages: {:?}, ... Going to wait {:?} before retrying", err, extended_delay_duration);
+                    tokio::time::delay_for(extended_delay_duration).await;
+                    continue;
+                }
+                Ok(messages) => messages,
+            };
+
+            // todo: this will probably need to be updated at some point due to changed structure
+            // of nym-sphinx messages sent. However, for time being this is still compatible.
+            // Basically it is more of a personal note of if client keeps getting errors and is
+            // not getting messages, look at this filter here.
+            let good_messages: Vec<_> = messages
+                .into_iter()
+                .filter(|message| message != loop_message && message != dummy_message)
+                .collect();
+            trace!("Obtained the following messages: {:?}", good_messages);
+
+            // if this one fails, there's no retrying because it means that either:
+            // - we run out of memory
+            // - the receiver channel is closed
+            // in either case there's no recovery and we can only panic
+            if !good_messages.is_empty() {
+                self.poller_tx.unbounded_send(good_messages).unwrap();
+            }
+
+            tokio::time::delay_for(self.polling_rate).await;
+        }
+    }
+
+    pub(crate) fn start(self, handle: &Handle) -> JoinHandle<()> {
+        handle.spawn(async move { self.start_provider_polling().await })
+    }
+}

nym-client/src/client/real_traffic_stream.rs

@@ -0,0 +1,150 @@
+use crate::client::mix_traffic::MixMessage;
+use crate::client::topology_control::TopologyAccessor;
+use crate::client::InputMessage;
+use futures::channel::mpsc;
+use futures::task::{Context, Poll};
+use futures::{Future, Stream, StreamExt};
+use log::{error, info, trace, warn};
+use sphinx::route::Destination;
+use std::pin::Pin;
+use std::time::Duration;
+use tokio::runtime::Handle;
+use tokio::task::JoinHandle;
+use tokio::time;
+use topology::NymTopology;
+
+pub(crate) struct OutQueueControl<T: NymTopology> {
+    average_packet_delay: Duration,
+    average_message_sending_delay: Duration,
+    next_delay: time::Delay,
+    mix_tx: mpsc::UnboundedSender<MixMessage>,
+    input_rx: mpsc::UnboundedReceiver<InputMessage>,
+    our_info: Destination,
+    topology_access: TopologyAccessor<T>,
+}
+
+pub(crate) enum StreamMessage {
+    Cover,
+    Real(InputMessage),
+}
+
+impl<T: NymTopology> Stream for OutQueueControl<T> {
+    type Item = StreamMessage;
+
+    fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
+        // it is not yet time to return a message
+        if Pin::new(&mut self.next_delay).poll(cx).is_pending() {
+            return Poll::Pending;
+        };
+
+        // we know it's time to send a message, so let's prepare delay for the next one
+        // Get the `now` by looking at the current `delay` deadline
+        let now = self.next_delay.deadline();
+        let next_poisson_delay = mix_client::poisson::sample(self.average_message_sending_delay);
+
+        // The next interval value is `next_poisson_delay` after the one that just
+        // yielded.
+        let next = now + next_poisson_delay;
+        self.next_delay.reset(next);
+
+        // decide what kind of message to send
+        match Pin::new(&mut self.input_rx).poll_next(cx) {
+            // in the case our real message channel stream was closed, we should also indicate we are closed
+            // (and whoever is using the stream should panic)
+            Poll::Ready(None) => Poll::Ready(None),
+
+            // if there's an actual message - return it
+            Poll::Ready(Some(real_message)) => Poll::Ready(Some(StreamMessage::Real(real_message))),
+
+            // otherwise construct a dummy one
+            Poll::Pending => Poll::Ready(Some(StreamMessage::Cover)),
+        }
+    }
+}
+
+impl<T: 'static + NymTopology> OutQueueControl<T> {
+    pub(crate) fn new(
+        mix_tx: mpsc::UnboundedSender<MixMessage>,
+        input_rx: mpsc::UnboundedReceiver<InputMessage>,
+        our_info: Destination,
+        topology_access: TopologyAccessor<T>,
+        average_packet_delay: Duration,
+        average_message_sending_delay: Duration,
+    ) -> Self {
+        OutQueueControl {
+            average_packet_delay,
+            average_message_sending_delay,
+            next_delay: time::delay_for(Default::default()),
+            mix_tx,
+            input_rx,
+            our_info,
+            topology_access,
+        }
+    }
+
+    async fn on_message(&mut self, next_message: StreamMessage) {
+        trace!("created new message");
+        let route = match self.topology_access.random_route().await {
+            None => {
+                warn!("No valid topology detected - won't send any real or loop message this time");
+                // TODO: this creates a potential problem: we can lose real messages if we were
+                // unable to get topology, perhaps we should store them in some buffer?
+                return;
+            }
+            Some(route) => route,
+        };
+
+        let next_packet = match next_message {
+            StreamMessage::Cover => mix_client::packet::loop_cover_message_route(
+                self.our_info.address.clone(),
+                self.our_info.identifier,
+                route,
+                self.average_packet_delay,
+            ),
+            StreamMessage::Real(real_message) => mix_client::packet::encapsulate_message_route(
+                real_message.0,
+                real_message.1,
+                route,
+                self.average_packet_delay,
+            ),
+        };
+
+        let next_packet = match next_packet {
+            Ok(message) => message,
+            Err(err) => {
+                error!(
+                    "Somehow we managed to create an invalid traffic message - {:?}",
+                    err
+                );
+                return;
+            }
+        };
+
+        // if this one fails, there's no retrying because it means that either:
+        // - we run out of memory
+        // - the receiver channel is closed
+        // in either case there's no recovery and we can only panic
+        self.mix_tx
+            .unbounded_send(MixMessage::new(next_packet.0, next_packet.1))
+            .unwrap();
+        // JS: Not entirely sure why or how it fixes stuff, but without the yield call,
+        // the UnboundedReceiver [of mix_rx] will not get a chance to read anything
+        tokio::task::yield_now().await;
+    }
+
+    pub(crate) async fn run_out_queue_control(mut self) {
+        // we should set initial delay only when we actually start the stream
+        self.next_delay = time::delay_for(mix_client::poisson::sample(
+            self.average_message_sending_delay,
+        ));
+
+        info!("starting out queue controller");
+        while let Some(next_message) = self.next().await {
+            self.on_message(next_message).await;
+        }
+    }
+
+    pub(crate) fn start(self, handle: &Handle) -> JoinHandle<()> {
+        handle.spawn(async move { self.run_out_queue_control().await })
+    }
+}

nym-client/src/client/received_buffer.rs

@@ -0,0 +1,153 @@
+use crate::client::provider_poller::PolledMessagesReceiver;
+use futures::channel::{mpsc, oneshot};
+use futures::lock::Mutex;
+use futures::StreamExt;
+use log::*;
+use nymsphinx::chunking::reconstruction::MessageReconstructor;
+use std::sync::Arc;
+use tokio::runtime::Handle;
+use tokio::task::JoinHandle;
+
+pub(crate) type ReceivedBufferResponse = oneshot::Sender<Vec<Vec<u8>>>;
+pub(crate) type ReceivedBufferRequestSender = mpsc::UnboundedSender<ReceivedBufferResponse>;
+pub(crate) type ReceivedBufferRequestReceiver = mpsc::UnboundedReceiver<ReceivedBufferResponse>;
+
+struct ReceivedMessagesBufferInner {
+    messages: Vec<Vec<u8>>,
+    message_reconstructor: MessageReconstructor,
+}
+
+#[derive(Debug, Clone)]
+// Note: you should NEVER create more than a single instance of this using 'new()'.
+// You should always use .clone() to create additional instances
+struct ReceivedMessagesBuffer {
+    inner: Arc<Mutex<ReceivedMessagesBufferInner>>,
+}
+
+impl ReceivedMessagesBuffer {
+    fn new() -> Self {
+        ReceivedMessagesBuffer {
+            inner: Arc::new(Mutex::new(ReceivedMessagesBufferInner {
+                messages: Vec::new(),
+                message_reconstructor: MessageReconstructor::new(),
+            })),
+        }
+    }
+
+    async fn add_reconstructed_messages(&mut self, msgs: Vec<Vec<u8>>) {
+        debug!("Adding {:?} new messages to the buffer!", msgs.len());
+        trace!("Adding new messages to the buffer! {:?}", msgs);
+        self.inner.lock().await.messages.extend(msgs)
+    }
+
+    async fn add_new_message_fragments(&mut self, msgs: Vec<Vec<u8>>) {
+        debug!(
+            "Adding {:?} new message fragments to the buffer!",
+            msgs.len()
+        );
+        trace!("Adding new message fragments to the buffer! {:?}", msgs);
+
+        let mut completed_messages = Vec::new();
+        let mut inner_guard = self.inner.lock().await;
+        for msg_fragment in msgs {
+            if let Some(reconstructed_message) =
+                inner_guard.message_reconstructor.new_fragment(msg_fragment)
+            {
+                completed_messages.push(reconstructed_message);
+            }
+        }
+        // make sure to drop the lock to not deadlock
+        drop(inner_guard);
+        if !completed_messages.is_empty() {
+            self.add_reconstructed_messages(completed_messages).await;
+        }
+    }
+
+    async fn acquire_and_empty(&mut self) -> Vec<Vec<u8>> {
+        trace!("Emptying the buffer and returning all messages");
+        let mut mutex_guard = self.inner.lock().await;
+        std::mem::replace(&mut mutex_guard.messages, Vec::new())
+    }
+}
+
+struct RequestReceiver {
+    received_buffer: ReceivedMessagesBuffer,
+    query_receiver: ReceivedBufferRequestReceiver,
+}
+
+impl RequestReceiver {
+    fn new(
+        received_buffer: ReceivedMessagesBuffer,
+        query_receiver: ReceivedBufferRequestReceiver,
+    ) -> Self {
+        RequestReceiver {
+            received_buffer,
+            query_receiver,
+        }
+    }
+
+    fn start(mut self, handle: &Handle) -> JoinHandle<()> {
+        handle.spawn(async move {
+            while let Some(request) = self.query_receiver.next().await {
+                let messages = self.received_buffer.acquire_and_empty().await;
+                if let Err(failed_messages) = request.send(messages) {
+                    error!(
+                        "Failed to send the messages to the requester. Adding them back to the buffer"
+                    );
+                    self.received_buffer.add_reconstructed_messages(failed_messages).await;
+                }
+            }
+        })
+    }
+}
+
+struct MessageReceiver {
+    received_buffer: ReceivedMessagesBuffer,
+    poller_receiver: PolledMessagesReceiver,
+}
+
+impl MessageReceiver {
+    fn new(
+        received_buffer: ReceivedMessagesBuffer,
+        poller_receiver: PolledMessagesReceiver,
+    ) -> Self {
+        MessageReceiver {
+            received_buffer,
+            poller_receiver,
+        }
+    }
+    fn start(mut self, handle: &Handle) -> JoinHandle<()> {
+        handle.spawn(async move {
+            while let Some(new_messages) = self.poller_receiver.next().await {
+                self.received_buffer
+                    .add_new_message_fragments(new_messages)
+                    .await;
+            }
+        })
+    }
+}
+
+pub(crate) struct ReceivedMessagesBufferController {
+    messsage_receiver: MessageReceiver,
+    request_receiver: RequestReceiver,
+}
+
+impl ReceivedMessagesBufferController {
+    pub(crate) fn new(
+        query_receiver: ReceivedBufferRequestReceiver,
+        poller_receiver: PolledMessagesReceiver,
+    ) -> Self {
+        let received_buffer = ReceivedMessagesBuffer::new();
+
+        ReceivedMessagesBufferController {
+            messsage_receiver: MessageReceiver::new(received_buffer.clone(), poller_receiver),
+            request_receiver: RequestReceiver::new(received_buffer, query_receiver),
+        }
+    }
+
+    pub(crate) fn start(self, handle: &Handle) {
+        // TODO: should we do anything with JoinHandle(s) returned by start methods?
+        self.messsage_receiver.start(handle);
+        self.request_receiver.start(handle);
+    }
+}

nym-client/src/client/topology_control.rs

@@ -0,0 +1,226 @@
+use crate::built_info;
+use crypto::identity::MixIdentityKeyPair;
+use futures::lock::Mutex;
+use healthcheck::HealthChecker;
+use log::*;
+use std::sync::Arc;
+use std::time;
+use std::time::Duration;
+use tokio::runtime::Handle;
+// use tokio::sync::RwLock;
+use std::net::SocketAddr;
+use tokio::task::JoinHandle;
+use topology::{provider, NymTopology};
+
+struct TopologyAccessorInner<T: NymTopology>(Option<T>);
+
+impl<T: NymTopology> TopologyAccessorInner<T> {
+    fn new() -> Self {
+        TopologyAccessorInner(None)
+    }
+
+    fn update(&mut self, new: Option<T>) {
+        self.0 = new;
+    }
+}
+
+#[derive(Clone, Debug)]
+pub(crate) struct TopologyAccessor<T: NymTopology> {
+    // TODO: this requires some actual benchmarking to determine if obtaining mutex is not going
+    // to cause some bottlenecking and whether perhaps RwLock would be better
+    inner: Arc<Mutex<TopologyAccessorInner<T>>>,
+}
+
+impl<T: NymTopology> TopologyAccessor<T> {
+    pub(crate) fn new() -> Self {
+        TopologyAccessor {
+            inner: Arc::new(Mutex::new(TopologyAccessorInner::new())),
+        }
+    }
+
+    async fn update_global_topology(&mut self, new_topology: Option<T>) {
+        self.inner.lock().await.update(new_topology);
+    }
+
+    pub(crate) async fn get_provider_socket_addr(&mut self, id: &str) -> Option<SocketAddr> {
+        match &self.inner.lock().await.0 {
+            None => None,
+            Some(ref topology) => topology
+                .providers()
+                .iter()
+                .find(|provider| provider.pub_key == id)
+                .map(|provider| provider.client_listener),
+        }
+    }
+
+    // only used by the client at startup to get a slightly more reasonable error message
+    pub(crate) async fn is_routable(&self) -> bool {
+        match &self.inner.lock().await.0 {
+            None => false,
+            Some(ref topology) => topology.can_construct_path_through(),
+        }
+    }
+
+    // Unless you absolutely need the entire topology, use `random_route` instead
+    pub(crate) async fn get_current_topology_clone(&mut self) -> Option<T> {
+        self.inner.lock().await.0.clone()
+    }
+
+    pub(crate) async fn get_all_clients(&mut self) -> Option<Vec<provider::Client>> {
+        // TODO: this will need to be modified to instead return pairs (provider, client)
+        match &self.inner.lock().await.0 {
+            None => None,
+            Some(ref topology) => Some(
+                topology
+                    .providers()
+                    .iter()
+                    .flat_map(|provider| provider.registered_clients.iter())
+                    .cloned()
+                    .collect::<Vec<_>>(),
+            ),
+        }
+    }
+
+    // this is a rather temporary solution as each client will have an associated provider
+    // currently that is not implemented yet and there only exists one provider in the network
+    pub(crate) async fn random_route(&mut self) -> Option<Vec<sphinx::route::Node>> {
+        match &self.inner.lock().await.0 {
+            None => None,
+            Some(ref topology) => {
+                let mut providers = topology.providers();
+                if providers.is_empty() {
+                    return None;
+                }
+                // unwrap is fine here as we asserted there is at least single provider
+                let provider = providers.pop().unwrap().into();
+                topology.route_to(provider).ok()
+            }
+        }
+    }
+}
+
+#[derive(Debug)]
+enum TopologyError {
+    HealthCheckError,
+    NoValidPathsError,
+}
+
+pub(crate) struct TopologyRefresherConfig {
+    directory_server: String,
+    refresh_rate: time::Duration,
+    identity_keypair: MixIdentityKeyPair,
+    resolution_timeout: time::Duration,
+    connection_timeout: time::Duration,
+    number_test_packets: usize,
+    node_score_threshold: f64,
+}
+
+impl TopologyRefresherConfig {
+    pub(crate) fn new(
+        directory_server: String,
+        refresh_rate: time::Duration,
+        identity_keypair: MixIdentityKeyPair,
+        resolution_timeout: time::Duration,
+        connection_timeout: time::Duration,
+        number_test_packets: usize,
+        node_score_threshold: f64,
+    ) -> Self {
+        TopologyRefresherConfig {
+            directory_server,
+            refresh_rate,
+            identity_keypair,
+            resolution_timeout,
+            connection_timeout,
+            number_test_packets,
+            node_score_threshold,
+        }
+    }
+}
+
+pub(crate) struct TopologyRefresher<T: NymTopology> {
+    directory_server: String,
+    topology_accessor: TopologyAccessor<T>,
+    health_checker: HealthChecker,
+    refresh_rate: Duration,
+    node_score_threshold: f64,
+}
+
+impl<T: 'static + NymTopology> TopologyRefresher<T> {
+    pub(crate) fn new(
+        cfg: TopologyRefresherConfig,
+        topology_accessor: TopologyAccessor<T>,
+    ) -> Self {
+        // this is a temporary solution as the healthcheck will eventually be moved to validators
+        let health_checker = healthcheck::HealthChecker::new(
+            cfg.resolution_timeout,
+            cfg.connection_timeout,
+            cfg.number_test_packets,
+            cfg.identity_keypair,
+        );
+
+        TopologyRefresher {
+            directory_server: cfg.directory_server,
+            topology_accessor,
+            health_checker,
+            refresh_rate: cfg.refresh_rate,
+            node_score_threshold: cfg.node_score_threshold,
+        }
+    }
+
+    async fn get_current_compatible_topology(&self) -> Result<T, TopologyError> {
+        let full_topology = T::new(self.directory_server.clone());
+        let version_filtered_topology = full_topology.filter_node_versions(
+            built_info::PKG_VERSION,
+            built_info::PKG_VERSION,
+            built_info::PKG_VERSION,
+        );
+
+        let healthcheck_result = self
+            .health_checker
+            .do_check(&version_filtered_topology)
+            .await;
+        let healthcheck_scores = match healthcheck_result {
+            Err(err) => {
+                error!("Error while performing the healthcheck: {:?}", err);
+                return Err(TopologyError::HealthCheckError);
+            }
+            Ok(scores) => scores,
+        };
+
+        debug!("{}", healthcheck_scores);
+
+        let healthy_topology = healthcheck_scores
+            .filter_topology_by_score(&version_filtered_topology, self.node_score_threshold);
+
+        // make sure you can still send a packet through the network:
+        if !healthy_topology.can_construct_path_through() {
+            return Err(TopologyError::NoValidPathsError);
+        }
+
+        Ok(healthy_topology)
+    }
+
+    pub(crate) async fn refresh(&mut self) {
+        trace!("Refreshing the topology");
+        let new_topology = match self.get_current_compatible_topology().await {
+            Ok(topology) => Some(topology),
+            Err(err) => {
+                warn!("the obtained topology seems to be invalid - {:?}, it will be impossible to send packets through", err);
+                None
+            }
+        };
+
+        self.topology_accessor
+            .update_global_topology(new_topology)
+            .await;
+    }
+
+    pub(crate) fn start(mut self, handle: &Handle) -> JoinHandle<()> {
+        handle.spawn(async move {
+            loop {
+                self.refresh().await;
+                tokio::time::delay_for(self.refresh_rate).await;
+            }
+        })
+    }
+}