GRIN/nym
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'release/v0.8.0'

Browse Source
This commit is contained in:
Jedrzej Stuczynski
2020-09-10 14:17:13 +01:00
parent 6aec202938 68dac6266e
commit ea7da83d97
433 changed files with 46128 additions and 18443 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+7 -1
View File
@@ -8,4 +8,10 @@ target
/.vscode/settings.json
validator/.vscode
sample-configs/validator-config.toml
.vscode
.vscode
scripts/deploy_qa.sh
scripts/run_gate.sh
scripts/run_mix.sh
scripts/start_local_tmux_network.sh
/.floo
/.flooignore
CHANGELOG.md
+462 -376
View File
@@ -1,376 +1,462 @@
# Changelog
## [v0.7.0](https://github.com/nymtech/nym/tree/HEAD)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.6.0...HEAD)
The main features of this release are:
* the addition of gateway nodes
* the retiring of the store-and-forward providers in favour of gateway nodes
* got rid of TCP connections for clients, everything now happens through websockets
* a new [Nym webassembly client](https://www.npmjs.com/package/@nymproject/nym-client-wasm), making it possible interact with Nym easily in browser-based runtimes
* reliability and performance improvements for mixnode networking
* initial validator code running (little functionality yet though)
See the [changelog](https://github.com/nymtech/nym/blob/develop/CHANGELOG.md) for detailed release notes.
**Implemented enhancements:**
- Use tokio codecs for multi\_tcp\_client [\#207](https://github.com/nymtech/nym/issues/207)
- Consider rewriting sfw\_provider\_requests using tokio Framed + Codec [\#181](https://github.com/nymtech/nym/issues/181)
**Fixed bugs:**
- Unexplained traffic increase in presence of unroutable node [\#232](https://github.com/nymtech/nym/issues/232)
- Gateway won't send to restarted layer1 nodes [\#231](https://github.com/nymtech/nym/issues/231)
**Closed issues:**
- Move to userpubkey@gatewaypubkey addresses. [\#235](https://github.com/nymtech/nym/issues/235)
- Get `start\_local\_network.sh` working with the js example [\#227](https://github.com/nymtech/nym/issues/227)
- Fix indeterminate test failure [\#218](https://github.com/nymtech/nym/issues/218)
- Remove 'fetch' mechanism from desktop client's client in favour of push [\#211](https://github.com/nymtech/nym/issues/211)
- Mixnode - load Sphinx keys like Gateway [\#209](https://github.com/nymtech/nym/issues/209)
- Publish NPM package for WebAssembly client [\#206](https://github.com/nymtech/nym/issues/206)
- Change --sockettype option on desktop client [\#203](https://github.com/nymtech/nym/issues/203)
- Remove TCP sockets from desktop client [\#202](https://github.com/nymtech/nym/issues/202)
- Desktop client currently hard-codes first provider [\#198](https://github.com/nymtech/nym/issues/198)
- Webassembly client currently hard-codes first provider [\#197](https://github.com/nymtech/nym/issues/197)
- Add Rust-based route construction to wasm client [\#196](https://github.com/nymtech/nym/issues/196)
- Remove fetch event [\#195](https://github.com/nymtech/nym/issues/195)
- Control messages should all be JSON [\#194](https://github.com/nymtech/nym/issues/194)
- Desktop Client should attach to gateway websocket [\#193](https://github.com/nymtech/nym/issues/193)
- Merge gateway and provider nodes [\#192](https://github.com/nymtech/nym/issues/192)
- Remove direct Sphinx dependencies [\#184](https://github.com/nymtech/nym/issues/184)
- \[Windows\] Presence notification fill OS socket queue [\#170](https://github.com/nymtech/nym/issues/170)
- Figure out connection hiccups between client and provider [\#162](https://github.com/nymtech/nym/issues/162)
- Improve the healthchecker [\#160](https://github.com/nymtech/nym/issues/160)
- Rethink client addressability [\#135](https://github.com/nymtech/nym/issues/135)
- Give some love to the service provider client ledger [\#116](https://github.com/nymtech/nym/issues/116)
- Start Gateway node type [\#80](https://github.com/nymtech/nym/issues/80)
- Bring health-checker into validator mix-mining [\#78](https://github.com/nymtech/nym/issues/78)
- Solidify TCPSocket on client [\#72](https://github.com/nymtech/nym/issues/72)
- scripts: run\_local\_network.sh doesn't die nicely [\#45](https://github.com/nymtech/nym/issues/45)
- WASM version of the Sphinx packet [\#19](https://github.com/nymtech/nym/issues/19)
- Persistent socket connection \(TCP Socket with provider\) [\#18](https://github.com/nymtech/nym/issues/18)
- Persistent socket connection \(Websocket with client\) [\#17](https://github.com/nymtech/nym/issues/17)
- Persistent socket connection \(TCP Socket with client\) [\#13](https://github.com/nymtech/nym/issues/13)
- Persistent socket connection \(Websocket with client\) [\#12](https://github.com/nymtech/nym/issues/12)
- Persistently store ledger with registered clients and their auth tokens [\#6](https://github.com/nymtech/nym/issues/6)
**Merged pull requests:**
- renaming desktop to native client [\#251](https://github.com/nymtech/nym/pull/251) ([futurechimp](https://github.com/futurechimp))
- Adding a pipenv dependencies file to the python client example [\#250](https://github.com/nymtech/nym/pull/250) ([futurechimp](https://github.com/futurechimp))
- Cleaning up startup messages in native client [\#249](https://github.com/nymtech/nym/pull/249) ([futurechimp](https://github.com/futurechimp))
- fixing up readme, bumping version number [\#246](https://github.com/nymtech/nym/pull/246) ([futurechimp](https://github.com/futurechimp))
- Feature/sphinx socket packet encoder [\#245](https://github.com/nymtech/nym/pull/245) ([jstuczyn](https://github.com/jstuczyn))
- Adding some documentation to the webassembly client [\#244](https://github.com/nymtech/nym/pull/244) ([futurechimp](https://github.com/futurechimp))
- Simplified some names and used the published npm package [\#242](https://github.com/nymtech/nym/pull/242) ([futurechimp](https://github.com/futurechimp))
- Feature/make andrew happy [\#241](https://github.com/nymtech/nym/pull/241) ([futurechimp](https://github.com/futurechimp))
- Removed redundant console.log [\#240](https://github.com/nymtech/nym/pull/240) ([jstuczyn](https://github.com/jstuczyn))
- Feature/explicit gateway addressing [\#239](https://github.com/nymtech/nym/pull/239) ([jstuczyn](https://github.com/jstuczyn))
- Feature/clean up [\#238](https://github.com/nymtech/nym/pull/238) ([futurechimp](https://github.com/futurechimp))
- Feature/addressing update [\#237](https://github.com/nymtech/nym/pull/237) ([jstuczyn](https://github.com/jstuczyn))
- Added hidden init flag to increase default traffic volume [\#234](https://github.com/nymtech/nym/pull/234) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/issue\#231 [\#233](https://github.com/nymtech/nym/pull/233) ([jstuczyn](https://github.com/jstuczyn))
- Fixed unwrap on none value [\#230](https://github.com/nymtech/nym/pull/230) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/gateway crash on incomplete ws handshake [\#229](https://github.com/nymtech/nym/pull/229) ([jstuczyn](https://github.com/jstuczyn))
- Feature/start local network improvements [\#228](https://github.com/nymtech/nym/pull/228) ([jstuczyn](https://github.com/jstuczyn))
- Updated directory\_client reqwest to 0.10 [\#226](https://github.com/nymtech/nym/pull/226) ([jstuczyn](https://github.com/jstuczyn))
- Updated js-example to get gateway from topology [\#225](https://github.com/nymtech/nym/pull/225) ([jstuczyn](https://github.com/jstuczyn))
- Requiring explicit timestamp when converting from rest to service mix… [\#224](https://github.com/nymtech/nym/pull/224) ([jstuczyn](https://github.com/jstuczyn))
- Feature/minor docs fixes [\#223](https://github.com/nymtech/nym/pull/223) ([futurechimp](https://github.com/futurechimp))
- Removed having to care about SURB\_ID [\#222](https://github.com/nymtech/nym/pull/222) ([jstuczyn](https://github.com/jstuczyn))
- Moved relevant parts of old mix-client to nymsphinx [\#221](https://github.com/nymtech/nym/pull/221) ([jstuczyn](https://github.com/jstuczyn))
- Feature/load keys on run [\#220](https://github.com/nymtech/nym/pull/220) ([jstuczyn](https://github.com/jstuczyn))
- Updated wasm code to work with new gateway and updated the example [\#219](https://github.com/nymtech/nym/pull/219) ([jstuczyn](https://github.com/jstuczyn))
- validator: removing health checker [\#217](https://github.com/nymtech/nym/pull/217) ([futurechimp](https://github.com/futurechimp))
- The great sfw-provider purge of 2020 [\#216](https://github.com/nymtech/nym/pull/216) ([jstuczyn](https://github.com/jstuczyn))
- Fixed compilation warnings on unreachable code when compiling with fe… [\#215](https://github.com/nymtech/nym/pull/215) ([jstuczyn](https://github.com/jstuczyn))
- Feature/healthchecker removal [\#214](https://github.com/nymtech/nym/pull/214) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/send to correct gateway [\#213](https://github.com/nymtech/nym/pull/213) ([jstuczyn](https://github.com/jstuczyn))
- Feature/client socket adjustments [\#212](https://github.com/nymtech/nym/pull/212) ([jstuczyn](https://github.com/jstuczyn))
- Sending sphinx packet independent of the receiver task [\#210](https://github.com/nymtech/nym/pull/210) ([jstuczyn](https://github.com/jstuczyn))
- Feature/gateway provider merge [\#208](https://github.com/nymtech/nym/pull/208) ([jstuczyn](https://github.com/jstuczyn))
- Feature/route from topology [\#201](https://github.com/nymtech/nym/pull/201) ([futurechimp](https://github.com/futurechimp))
- Intermediate gateway-heart surgery checkpoint [\#199](https://github.com/nymtech/nym/pull/199) ([jstuczyn](https://github.com/jstuczyn))
- Feature/wasm js demo [\#191](https://github.com/nymtech/nym/pull/191) ([futurechimp](https://github.com/futurechimp))
- Feature/improve js example [\#190](https://github.com/nymtech/nym/pull/190) ([futurechimp](https://github.com/futurechimp))
- Feature/limit direct sphinx dependency + remove direct curve25519 dependency from wasm client [\#189](https://github.com/nymtech/nym/pull/189) ([jstuczyn](https://github.com/jstuczyn))
- Feature/very minor refactoring [\#188](https://github.com/nymtech/nym/pull/188) ([jstuczyn](https://github.com/jstuczyn))
- Feature/persistent ledger [\#187](https://github.com/nymtech/nym/pull/187) ([jstuczyn](https://github.com/jstuczyn))
- Optimising wasm build size, shaves about 10% size off our wasm output. [\#186](https://github.com/nymtech/nym/pull/186) ([futurechimp](https://github.com/futurechimp))
- Ran `npm audit fix` on the wasm demo directory. [\#185](https://github.com/nymtech/nym/pull/185) ([futurechimp](https://github.com/futurechimp))
- Feature/nym sphinx wasm [\#183](https://github.com/nymtech/nym/pull/183) ([futurechimp](https://github.com/futurechimp))
- Improvements to sfw-provider - client communcation [\#180](https://github.com/nymtech/nym/pull/180) ([jstuczyn](https://github.com/jstuczyn))
- Adding Apache 2 license headers to all files [\#178](https://github.com/nymtech/nym/pull/178) ([futurechimp](https://github.com/futurechimp))
- Feature/service persistence [\#171](https://github.com/nymtech/nym/pull/171) ([futurechimp](https://github.com/futurechimp))
## [v0.6.0](https://github.com/nymtech/nym/tree/v0.6.0) (2020-04-07)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.5.0...v0.6.0)
This release fixes bugs in v0.5.0. All testnet node operators are advised to upgrade from v0.5.0.
* fixed premature EOFs on socket connections by using the new multi-TCP client
* fixed a bug causing client and mixnode connection hangs for misconfigured nodes
* by default 'Debug' section of saved configs is now empty and default values are used unless explicitly overridden
* introduced packet chunking allowing clients to send messages of arbitrary length. Note that packet retransmission is not implemented yet, so for longer messages, you might not get anything
* mixnodes now periodically log stats regarding number of packets mixed
* fixed possible client hang ups when sending high rates of traffic
* preventing mixes from starting with same announce-host as an existing node
* fixed overflow multiplication if connection backoff was set to a high value
**Closed issues:**
- Periodic activity summary [\#172](https://github.com/nymtech/nym/issues/172)
- Move contents of 'common/addressing' into 'common/nymsphinx' [\#161](https://github.com/nymtech/nym/issues/161)
- Make builds simpler for node operators [\#114](https://github.com/nymtech/nym/issues/114)
- Chunking in `nym-client` \(receive\) [\#83](https://github.com/nymtech/nym/issues/83)
- Chunking in `nym-client` \(send\) [\#82](https://github.com/nymtech/nym/issues/82)
**Merged pull requests:**
- Feature/tcp client connection timeout [\#176](https://github.com/nymtech/nym/pull/176) ([jstuczyn](https://github.com/jstuczyn))
- Feature/mixing stats logging [\#175](https://github.com/nymtech/nym/pull/175) ([jstuczyn](https://github.com/jstuczyn))
- Preventing multiplication overflow for reconnection backoff [\#174](https://github.com/nymtech/nym/pull/174) ([jstuczyn](https://github.com/jstuczyn))
- Feature/non mandatory debug config [\#173](https://github.com/nymtech/nym/pull/173) ([jstuczyn](https://github.com/jstuczyn))
- Feature/addressing move [\#169](https://github.com/nymtech/nym/pull/169) ([jstuczyn](https://github.com/jstuczyn))
- Checking if any other node is already announcing the same host [\#168](https://github.com/nymtech/nym/pull/168) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/closing tcp client connections on drop [\#167](https://github.com/nymtech/nym/pull/167) ([jstuczyn](https://github.com/jstuczyn))
- Yielding tokio task upon creating loop/real traffic message [\#166](https://github.com/nymtech/nym/pull/166) ([jstuczyn](https://github.com/jstuczyn))
- Feature/minor healthchecker improvements [\#165](https://github.com/nymtech/nym/pull/165) ([jstuczyn](https://github.com/jstuczyn))
- Feature/packet chunking [\#158](https://github.com/nymtech/nym/pull/158) ([jstuczyn](https://github.com/jstuczyn))
## [v0.5.0](https://github.com/nymtech/nym/tree/v0.5.0) (2020-03-23)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.5.0-rc.1...v0.5.0)
1. Introduced proper configuration options for mixnodes, clients and providers. Everything is initialised with the `init` command that creates a saved config.toml file. To run the binary you now use `nym-<binary-name> run`, for example `nym-mixnode run`. Each flag can be overwritten at any stage with the following priority: run flags, data in config.toml and finally init flags.
2. Made mixnet TCP connections persistent. When sending a Sphinx packet, it should no longer go through the lengthy process of establishing a TCP connection only to immediately tear it down after sending a single packet. This significantly boosts throughput.
3. A lot of work on code clean up and refactoring including some performance fixes.
4. Client now determines its default nym-sfw-provider at startup and should always try to connect to the same one. Note: we still can't reliably run more than a single provider on the network.
5. Logging messages now have timestamps and when running at more aggressive log mode (like debug or even trace) we should no longer be overwhelmed with messages from external crates.
6. Initial compatibility with Windows. Please let us know if you have problems.
7. More work on validator, including initial Tendermint integration in Rust, and the start of the mixmining system.
**Closed issues:**
- Introduce timestamps to log messages [\#124](https://github.com/nymtech/nym/issues/124)
**Merged pull requests:**
- removing spooky startup warning message [\#155](https://github.com/nymtech/nym/pull/155) ([futurechimp](https://github.com/futurechimp))
- Some more startup fixes [\#154](https://github.com/nymtech/nym/pull/154) ([futurechimp](https://github.com/futurechimp))
- Entering runtime context when creating mix traffic controller [\#153](https://github.com/nymtech/nym/pull/153) ([jstuczyn](https://github.com/jstuczyn))
- Friendlification of startup messages [\#151](https://github.com/nymtech/nym/pull/151) ([futurechimp](https://github.com/futurechimp))
- Entering runtime context when creating packet forwarder [\#150](https://github.com/nymtech/nym/pull/150) ([jstuczyn](https://github.com/jstuczyn))
- Feature/add topology to validator [\#149](https://github.com/nymtech/nym/pull/149) ([futurechimp](https://github.com/futurechimp))
- Making code work on windows machines [\#148](https://github.com/nymtech/nym/pull/148) ([jstuczyn](https://github.com/jstuczyn))
- validator: adding HTTP interface [\#146](https://github.com/nymtech/nym/pull/146) ([futurechimp](https://github.com/futurechimp))
- Extracting the log setup [\#145](https://github.com/nymtech/nym/pull/145) ([futurechimp](https://github.com/futurechimp))
- Feature/optional location in configs [\#144](https://github.com/nymtech/nym/pull/144) ([jstuczyn](https://github.com/jstuczyn))
- Feature/concurrent connection managers [\#142](https://github.com/nymtech/nym/pull/142) ([jstuczyn](https://github.com/jstuczyn))
- Defaulting for global 'Info' logging level if not set in .env [\#140](https://github.com/nymtech/nym/pull/140) ([jstuczyn](https://github.com/jstuczyn))
- Provider not storing loop cover messages [\#139](https://github.com/nymtech/nym/pull/139) ([jstuczyn](https://github.com/jstuczyn))
- Using log builder to include timestamps + filters [\#138](https://github.com/nymtech/nym/pull/138) ([jstuczyn](https://github.com/jstuczyn))
- Feature/client ws refactoring [\#134](https://github.com/nymtech/nym/pull/134) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/metrics presence delay fix [\#133](https://github.com/nymtech/nym/pull/133) ([jstuczyn](https://github.com/jstuczyn))
- Removed outdated and redundant sample-configs [\#131](https://github.com/nymtech/nym/pull/131) ([jstuczyn](https://github.com/jstuczyn))
- If not overridden, 'announce-host' should default to 'host' [\#130](https://github.com/nymtech/nym/pull/130) ([jstuczyn](https://github.com/jstuczyn))
- Nice to know who we're talking to at startup... [\#129](https://github.com/nymtech/nym/pull/129) ([futurechimp](https://github.com/futurechimp))
## [v0.5.0-rc.1](https://github.com/nymtech/nym/tree/v0.5.0-rc.1) (2020-03-06)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.1...v0.5.0-rc.1)
**Closed issues:**
- COMPILE: Could not compile project using Cargo [\#118](https://github.com/nymtech/nym/issues/118)
- Wherever unbounded mpsc channel is used, prefer unbounded\_send\(\) over send\(\).await [\#90](https://github.com/nymtech/nym/issues/90)
- Add a `Send` method in nym-client [\#81](https://github.com/nymtech/nym/issues/81)
- Start on Tendermint integration [\#79](https://github.com/nymtech/nym/issues/79)
- Ditch DummyKeyPair [\#75](https://github.com/nymtech/nym/issues/75)
- Replace args with proper config files [\#69](https://github.com/nymtech/nym/issues/69)
- Fix incorrectly used Arcs [\#47](https://github.com/nymtech/nym/issues/47)
- nym-mixnode mandatory host option [\#26](https://github.com/nymtech/nym/issues/26)
- Create config struct for mixnode \(possibly also for client\) [\#21](https://github.com/nymtech/nym/issues/21)
- Reuse TCP socket connection between client and mixnodes [\#20](https://github.com/nymtech/nym/issues/20)
- Once implementation is available, wherever appropriate, replace `futures::lock::Mutex` with `futures::lock::RwLock` [\#9](https://github.com/nymtech/nym/issues/9)
- Check if RwLock on MixProcessingData is still needed [\#8](https://github.com/nymtech/nym/issues/8)
- Reuse TCP socket connection between mixnodes and providers [\#3](https://github.com/nymtech/nym/issues/3)
- Persistent socket connection with other mixes [\#2](https://github.com/nymtech/nym/issues/2)
**Merged pull requests:**
- Feature/client refactoring [\#128](https://github.com/nymtech/nym/pull/128) ([jstuczyn](https://github.com/jstuczyn))
- Feature/provider refactoring [\#125](https://github.com/nymtech/nym/pull/125) ([jstuczyn](https://github.com/jstuczyn))
- all: fixing mis-spelling [\#123](https://github.com/nymtech/nym/pull/123) ([futurechimp](https://github.com/futurechimp))
- Feature/further clippy fixes [\#121](https://github.com/nymtech/nym/pull/121) ([jstuczyn](https://github.com/jstuczyn))
- Feature/tokio tungstenite dependency fix [\#120](https://github.com/nymtech/nym/pull/120) ([jstuczyn](https://github.com/jstuczyn))
- Feature/config files cleanup [\#119](https://github.com/nymtech/nym/pull/119) ([jstuczyn](https://github.com/jstuczyn))
- Feature/config files [\#117](https://github.com/nymtech/nym/pull/117) ([jstuczyn](https://github.com/jstuczyn))
- Feature/un genericize keys [\#111](https://github.com/nymtech/nym/pull/111) ([futurechimp](https://github.com/futurechimp))
- Feature/abci [\#110](https://github.com/nymtech/nym/pull/110) ([futurechimp](https://github.com/futurechimp))
- Simplified the use of generics on identity keypair by using output types [\#109](https://github.com/nymtech/nym/pull/109) ([jstuczyn](https://github.com/jstuczyn))
## [v0.4.1](https://github.com/nymtech/nym/tree/v0.4.1) (2020-01-29)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0...v0.4.1)
**Closed issues:**
- Change healthcheck to run on provided topology rather than pull one itself [\#95](https://github.com/nymtech/nym/issues/95)
**Merged pull requests:**
- Bugfix/healthcheck on provided topology [\#108](https://github.com/nymtech/nym/pull/108) ([jstuczyn](https://github.com/jstuczyn))
## [v0.4.0](https://github.com/nymtech/nym/tree/v0.4.0) (2020-01-28)
[Full Changelog](https://github.com/nymtech/nym/compare/0.4.0-rc.2...v0.4.0)
Nym 0.4.0 Platform
In this release, we're taking a lot more care with version numbers, so that we can ensure upgrade compatibility for mixnodes, providers, clients, and validators more easily.
This release also integrates a health-checker and network topology refresh into the Nym client, so that the client can intelligently choose paths which route around any non-functional or incompatible nodes.
## [0.4.0-rc.2](https://github.com/nymtech/nym/tree/0.4.0-rc.2) (2020-01-28)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0-rc.2...0.4.0-rc.2)
## [v0.4.0-rc.2](https://github.com/nymtech/nym/tree/v0.4.0-rc.2) (2020-01-28)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0-rc.1...v0.4.0-rc.2)
**Merged pull requests:**
- Hotfix/semver compatibility [\#106](https://github.com/nymtech/nym/pull/106) ([jstuczyn](https://github.com/jstuczyn))
## [v0.4.0-rc.1](https://github.com/nymtech/nym/tree/v0.4.0-rc.1) (2020-01-28)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.3...v0.4.0-rc.1)
**Closed issues:**
- Check Sphinx packet length in client [\#98](https://github.com/nymtech/nym/issues/98)
- workflow test [\#97](https://github.com/nymtech/nym/issues/97)
- Client SemVer [\#85](https://github.com/nymtech/nym/issues/85)
- Move PemStore [\#74](https://github.com/nymtech/nym/issues/74)
- Periodic client refresh [\#70](https://github.com/nymtech/nym/issues/70)
- Logging [\#68](https://github.com/nymtech/nym/issues/68)
- Nym-client refactor [\#67](https://github.com/nymtech/nym/issues/67)
- Stop panicking! [\#66](https://github.com/nymtech/nym/issues/66)
- Fix Mixnode Panic on Sphinx packet replay [\#65](https://github.com/nymtech/nym/issues/65)
- Convert older code to start using logging [\#35](https://github.com/nymtech/nym/issues/35)
- Convert to non-url-safe base64 \(everywhere\) [\#28](https://github.com/nymtech/nym/issues/28)
- If a thread blows at startup, panic the entire application [\#15](https://github.com/nymtech/nym/issues/15)
- Split provider/mod.rs [\#10](https://github.com/nymtech/nym/issues/10)
**Merged pull requests:**
- Feature/health checker with existing keys [\#105](https://github.com/nymtech/nym/pull/105) ([jstuczyn](https://github.com/jstuczyn))
- Feature/remove topology equality check [\#104](https://github.com/nymtech/nym/pull/104) ([futurechimp](https://github.com/futurechimp))
- Feature/base58 [\#102](https://github.com/nymtech/nym/pull/102) ([futurechimp](https://github.com/futurechimp))
- Feature/panic improvements [\#101](https://github.com/nymtech/nym/pull/101) ([jstuczyn](https://github.com/jstuczyn))
- Feature/fix sphinx unwraps [\#100](https://github.com/nymtech/nym/pull/100) ([futurechimp](https://github.com/futurechimp))
- Feature/check packet length [\#99](https://github.com/nymtech/nym/pull/99) ([futurechimp](https://github.com/futurechimp))
- Feature/version filtering improvements [\#96](https://github.com/nymtech/nym/pull/96) ([futurechimp](https://github.com/futurechimp))
- Feature/refreshing topology [\#94](https://github.com/nymtech/nym/pull/94) ([jstuczyn](https://github.com/jstuczyn))
- Feature/consistent logging [\#93](https://github.com/nymtech/nym/pull/93) ([futurechimp](https://github.com/futurechimp))
- Feature/semver client [\#92](https://github.com/nymtech/nym/pull/92) ([futurechimp](https://github.com/futurechimp))
- Feature/client refactor [\#91](https://github.com/nymtech/nym/pull/91) ([jstuczyn](https://github.com/jstuczyn))
- Release builds should no longer silently fail - everything will be im… [\#89](https://github.com/nymtech/nym/pull/89) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.3](https://github.com/nymtech/nym/tree/v0.3.3) (2020-01-20)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.2...v0.3.3)
**Fixed bugs:**
- Nym client crashing and disconnecting local websocket with complaint about binary data [\#56](https://github.com/nymtech/nym/issues/56)
**Closed issues:**
- Websocket text fix [\#64](https://github.com/nymtech/nym/issues/64)
- Restore nym-client lib in crate [\#63](https://github.com/nymtech/nym/issues/63)
- Make websocket not crash on ping or pong messages [\#62](https://github.com/nymtech/nym/issues/62)
- Messages returned by fetch are base64 encoded [\#55](https://github.com/nymtech/nym/issues/55)
- Check layer 1 connectivity at client start [\#38](https://github.com/nymtech/nym/issues/38)
- Check required sfw-provider args [\#27](https://github.com/nymtech/nym/issues/27)
- Make Electron app work with new Rust mixnet client [\#16](https://github.com/nymtech/nym/issues/16)
- Take version numbers into account when picking routes [\#14](https://github.com/nymtech/nym/issues/14)
**Merged pull requests:**
- Feature/websocket improvements [\#88](https://github.com/nymtech/nym/pull/88) ([jstuczyn](https://github.com/jstuczyn))
- Using println rather than log for startup banner, it's not an error [\#87](https://github.com/nymtech/nym/pull/87) ([futurechimp](https://github.com/futurechimp))
- Feature/nym client lib [\#61](https://github.com/nymtech/nym/pull/61) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.2](https://github.com/nymtech/nym/tree/v0.3.2) (2020-01-17)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.1...v0.3.2)
**Merged pull requests:**
- Feature/separate presence address [\#59](https://github.com/nymtech/nym/pull/59) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.1](https://github.com/nymtech/nym/tree/v0.3.1) (2020-01-16)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.0...v0.3.1)
**Merged pull requests:**
- Bugfix/presence client crash [\#58](https://github.com/nymtech/nym/pull/58) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.0](https://github.com/nymtech/nym/tree/v0.3.0) (2020-01-14)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.2.0...v0.3.0)
**Closed issues:**
- Error referring to mismatched types caused by topology [\#46](https://github.com/nymtech/nym/issues/46)
- Provider needs to announce two of its addresses \(+ issue of hardcoded port\) [\#32](https://github.com/nymtech/nym/issues/32)
- Port run\_network.sh from old Go mixnet [\#30](https://github.com/nymtech/nym/issues/30)
- Health Checker inside Validator [\#29](https://github.com/nymtech/nym/issues/29)
- Combine Rust repositories into a single master repo containing multiple crates [\#24](https://github.com/nymtech/nym/issues/24)
- Fix the version numbers on provider and mixnode [\#23](https://github.com/nymtech/nym/issues/23)
- Travis releases for Nym [\#22](https://github.com/nymtech/nym/issues/22)
- Add version number to presence [\#4](https://github.com/nymtech/nym/issues/4)
- Add version number to presence [\#1](https://github.com/nymtech/nym/issues/1)
**Merged pull requests:**
- Feature/client topology filtering [\#54](https://github.com/nymtech/nym/pull/54) ([jstuczyn](https://github.com/jstuczyn))
- print public key for nym client tools [\#53](https://github.com/nymtech/nym/pull/53) ([mileschet](https://github.com/mileschet))
- Showing binding warning on binding to localhost, 0.0.0.0 or 127.0.0.1 [\#52](https://github.com/nymtech/nym/pull/52) ([jstuczyn](https://github.com/jstuczyn))
- validator: moving sample config files into sample configs directory [\#51](https://github.com/nymtech/nym/pull/51) ([futurechimp](https://github.com/futurechimp))
- Feature/validator health checker [\#50](https://github.com/nymtech/nym/pull/50) ([jstuczyn](https://github.com/jstuczyn))
- Improving websocket connection errors [\#49](https://github.com/nymtech/nym/pull/49) ([futurechimp](https://github.com/futurechimp))
- Feature/crypto crate [\#48](https://github.com/nymtech/nym/pull/48) ([jstuczyn](https://github.com/jstuczyn))
- Feature/chill log messages [\#43](https://github.com/nymtech/nym/pull/43) ([futurechimp](https://github.com/futurechimp))
- persistence: improving PEM file reading and parsing failure messages [\#42](https://github.com/nymtech/nym/pull/42) ([futurechimp](https://github.com/futurechimp))
- Providing a nicer error than "failed on unwrap\(\)" when topology retri… [\#41](https://github.com/nymtech/nym/pull/41) ([futurechimp](https://github.com/futurechimp))
- Prettying up sfw-provider start sequence a bit. [\#40](https://github.com/nymtech/nym/pull/40) ([futurechimp](https://github.com/futurechimp))
- Removing the run command from code and documentation [\#39](https://github.com/nymtech/nym/pull/39) ([futurechimp](https://github.com/futurechimp))
- Feature/script/localnet [\#37](https://github.com/nymtech/nym/pull/37) ([futurechimp](https://github.com/futurechimp))
- Feature/arguments fix [\#36](https://github.com/nymtech/nym/pull/36) ([jstuczyn](https://github.com/jstuczyn))
- Feature/crates cleanup [\#34](https://github.com/nymtech/nym/pull/34) ([jstuczyn](https://github.com/jstuczyn))
- Features/version number to presence [\#25](https://github.com/nymtech/nym/pull/25) ([aniampio](https://github.com/aniampio))
- Feature/minor client changes [\#7](https://github.com/nymtech/nym/pull/7) ([jstuczyn](https://github.com/jstuczyn))
## [v0.2.0](https://github.com/nymtech/nym/tree/v0.2.0) (2020-01-07)
[Full Changelog](https://github.com/nymtech/nym/compare/0.2.0...v0.2.0)
## [0.2.0](https://github.com/nymtech/nym/tree/0.2.0) (2020-01-06)
[Full Changelog](https://github.com/nymtech/nym/compare/0.1.0...0.2.0)
\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*
# Changelog
## [v0.8.0](https://github.com/nymtech/nym/tree/v0.8.0)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.7.0...v0.8.0)
This release introduces, among other things, the following improvements:
- SURB-acks for significant boost to the mixnet messaging reliability,
- SURB-replies for allowing for anonymous replies,
- SOCKS5 proxying capabilities,
- replacing the `AuthToken` with a shared key derived between client and its gateway,
- encryption and tagging of mix messages exchanged between client and its gateway,
- end-to-end encryption of traffic between clients,
- general performance and reliability improvements.
**Implemented enhancements:**
- Change how un-ack'd packets are retransmitted [\#307](https://github.com/nymtech/nym/issues/307)
- Feature/socks5 sequencing [\#318](https://github.com/nymtech/nym/pull/318) ([jstuczyn](https://github.com/jstuczyn))
- Feature/socks client config [\#316](https://github.com/nymtech/nym/pull/316) ([jstuczyn](https://github.com/jstuczyn))
- Explicit proxy runner + closing local connection if remote is over [\#314](https://github.com/nymtech/nym/pull/314) ([jstuczyn](https://github.com/jstuczyn))
- Feature/ack messing [\#313](https://github.com/nymtech/nym/pull/313) ([jstuczyn](https://github.com/jstuczyn))
- Removed client list from topology [\#301](https://github.com/nymtech/nym/pull/301) ([jstuczyn](https://github.com/jstuczyn))
- Feature/reply surbs [\#299](https://github.com/nymtech/nym/pull/299) ([jstuczyn](https://github.com/jstuczyn))
**Fixed bugs:**
- Socks client no longer logging dns resolved addresses [\#329](https://github.com/nymtech/nym/pull/329) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/remove packet buffering [\#300](https://github.com/nymtech/nym/pull/300) ([jstuczyn](https://github.com/jstuczyn))
**Closed issues:**
- Do not buffer packets for mixes we are reconnecting to [\#291](https://github.com/nymtech/nym/issues/291)
- Loop cover messages need to be encrypted! [\#287](https://github.com/nymtech/nym/issues/287)
- Get rid of instances of Deref polymorphism antipattern [\#283](https://github.com/nymtech/nym/issues/283)
- Remove client list from topology [\#279](https://github.com/nymtech/nym/issues/279)
- The messages pushed from gateway should be encrypted. [\#276](https://github.com/nymtech/nym/issues/276)
- The shared key between client and gateway should be stored in a file. [\#273](https://github.com/nymtech/nym/issues/273)
- Refactor topology, NymTopology trait, and related code [\#200](https://github.com/nymtech/nym/issues/200)
- Fragment retransmission for split messages [\#164](https://github.com/nymtech/nym/issues/164)
- Clean up common/clients/mix-clients [\#126](https://github.com/nymtech/nym/issues/126)
- Reliable chunk transmission [\#84](https://github.com/nymtech/nym/issues/84)
- Change how topology is obtained [\#44](https://github.com/nymtech/nym/issues/44)
- More secured auth token - some signature on request [\#5](https://github.com/nymtech/nym/issues/5)
**Merged pull requests:**
- Recommended testnet gateway [\#335](https://github.com/nymtech/nym/pull/335) ([jstuczyn](https://github.com/jstuczyn))
- placeholder contact form url [\#334](https://github.com/nymtech/nym/pull/334) ([jstuczyn](https://github.com/jstuczyn))
- Knocking down delay on message sending default [\#332](https://github.com/nymtech/nym/pull/332) ([futurechimp](https://github.com/futurechimp))
- Made gateway mandatory during init [\#331](https://github.com/nymtech/nym/pull/331) ([jstuczyn](https://github.com/jstuczyn))
- Renaming client binary [\#330](https://github.com/nymtech/nym/pull/330) ([futurechimp](https://github.com/futurechimp))
- Feature/sphinx socks [\#326](https://github.com/nymtech/nym/pull/326) ([futurechimp](https://github.com/futurechimp))
- Feature/print client address on startup [\#325](https://github.com/nymtech/nym/pull/325) ([jstuczyn](https://github.com/jstuczyn))
- Feature/temp wasm example removal [\#324](https://github.com/nymtech/nym/pull/324) ([jstuczyn](https://github.com/jstuczyn))
- Feature/websocket js example dependency update [\#323](https://github.com/nymtech/nym/pull/323) ([jstuczyn](https://github.com/jstuczyn))
- snake\_cased replySURBs [\#322](https://github.com/nymtech/nym/pull/322) ([jstuczyn](https://github.com/jstuczyn))
- Feature/outbound request lists [\#321](https://github.com/nymtech/nym/pull/321) ([futurechimp](https://github.com/futurechimp))
- Feature/temp fix for ipv6 [\#317](https://github.com/nymtech/nym/pull/317) ([futurechimp](https://github.com/futurechimp))
- Removed unused dependencies [\#315](https://github.com/nymtech/nym/pull/315) ([jstuczyn](https://github.com/jstuczyn))
- Feature/perf messing [\#311](https://github.com/nymtech/nym/pull/311) ([futurechimp](https://github.com/futurechimp))
- Upgrades tungstenite libraries to new versions with 64MB message sizes. [\#310](https://github.com/nymtech/nym/pull/310) ([futurechimp](https://github.com/futurechimp))
- Assigning connection shared key post registration [\#309](https://github.com/nymtech/nym/pull/309) ([jstuczyn](https://github.com/jstuczyn))
- Assigning connection shared key post registration [\#308](https://github.com/nymtech/nym/pull/308) ([jstuczyn](https://github.com/jstuczyn))
- Feature/client binary api update [\#306](https://github.com/nymtech/nym/pull/306) ([jstuczyn](https://github.com/jstuczyn))
- Removes unused Cargo dependencies so we stay slim and trim. [\#305](https://github.com/nymtech/nym/pull/305) ([futurechimp](https://github.com/futurechimp))
- Removes unused code from the socks client implementation [\#304](https://github.com/nymtech/nym/pull/304) ([futurechimp](https://github.com/futurechimp))
- Feature/client core [\#303](https://github.com/nymtech/nym/pull/303) ([futurechimp](https://github.com/futurechimp))
- Feature/socks5 [\#302](https://github.com/nymtech/nym/pull/302) ([futurechimp](https://github.com/futurechimp))
- Updated blake3 dependency to 0.3.5 [\#281](https://github.com/nymtech/nym/pull/281) ([jstuczyn](https://github.com/jstuczyn))
- Feature/ws send confirmation removal [\#280](https://github.com/nymtech/nym/pull/280) ([jstuczyn](https://github.com/jstuczyn))
- Added simple react example [\#275](https://github.com/nymtech/nym/pull/275) ([keviinfoes](https://github.com/keviinfoes))
- Feature/topology refactor [\#274](https://github.com/nymtech/nym/pull/274) ([jstuczyn](https://github.com/jstuczyn))
- Feature/gateway shared key generation [\#272](https://github.com/nymtech/nym/pull/272) ([jstuczyn](https://github.com/jstuczyn))
- Removed the healthcheck module, it's no longer in use. [\#271](https://github.com/nymtech/nym/pull/271) ([futurechimp](https://github.com/futurechimp))
- Adding description field to wasm client to kill warning [\#270](https://github.com/nymtech/nym/pull/270) ([futurechimp](https://github.com/futurechimp))
- Running `npm audit fix` on js examples [\#269](https://github.com/nymtech/nym/pull/269) ([futurechimp](https://github.com/futurechimp))
- Feature/constant length packet payloads [\#268](https://github.com/nymtech/nym/pull/268) ([jstuczyn](https://github.com/jstuczyn))
- Feature/wasm topology duplication [\#265](https://github.com/nymtech/nym/pull/265) ([jstuczyn](https://github.com/jstuczyn))
- Removed misplaced WorkingDirectory parameter [\#264](https://github.com/nymtech/nym/pull/264) ([ststefa](https://github.com/ststefa))
- Feature/packet retransmission [\#263](https://github.com/nymtech/nym/pull/263) ([jstuczyn](https://github.com/jstuczyn))
- could not count to ten properly [\#262](https://github.com/nymtech/nym/pull/262) ([ststefa](https://github.com/ststefa))
- build\(deps\): bump websocket-extensions from 0.1.3 to 0.1.4 in /clients/webassembly/js-example [\#261](https://github.com/nymtech/nym/pull/261) ([dependabot[bot]](https://github.com/apps/dependabot))
- add disabling feature 'offline-test' for network-dependent tests [\#260](https://github.com/nymtech/nym/pull/260) ([hyperfekt](https://github.com/hyperfekt))
## [v0.7.0](https://github.com/nymtech/nym/tree/v0.7.0) (2020-06-08)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.6.0...v0.7.0)
The main features of this release are:
* the addition of gateway nodes
* the retiring of the store-and-forward providers in favour of gateway nodes
* got rid of TCP connections for clients, everything now happens through websockets
* a new [Nym webassembly client](https://www.npmjs.com/package/@nymproject/nym-client-wasm), making it possible interact with Nym easily in browser-based runtimes
* reliability and performance improvements for mixnode networking
* initial validator code running (little functionality yet though)
See the [changelog](https://github.com/nymtech/nym/blob/develop/CHANGELOG.md) for detailed release notes.
**Implemented enhancements:**
- Use tokio codecs for multi\_tcp\_client [\#207](https://github.com/nymtech/nym/issues/207)
- Consider rewriting sfw\_provider\_requests using tokio Framed + Codec [\#181](https://github.com/nymtech/nym/issues/181)
**Fixed bugs:**
- Unexplained traffic increase in presence of unroutable node [\#232](https://github.com/nymtech/nym/issues/232)
- Gateway won't send to restarted layer1 nodes [\#231](https://github.com/nymtech/nym/issues/231)
**Closed issues:**
- Move to userpubkey@gatewaypubkey addresses. [\#235](https://github.com/nymtech/nym/issues/235)
- Get `start\_local\_network.sh` working with the js example [\#227](https://github.com/nymtech/nym/issues/227)
- Fix indeterminate test failure [\#218](https://github.com/nymtech/nym/issues/218)
- Remove 'fetch' mechanism from desktop client's client in favour of push [\#211](https://github.com/nymtech/nym/issues/211)
- Mixnode - load Sphinx keys like Gateway [\#209](https://github.com/nymtech/nym/issues/209)
- Publish NPM package for WebAssembly client [\#206](https://github.com/nymtech/nym/issues/206)
- Change --sockettype option on desktop client [\#203](https://github.com/nymtech/nym/issues/203)
- Remove TCP sockets from desktop client [\#202](https://github.com/nymtech/nym/issues/202)
- Desktop client currently hard-codes first provider [\#198](https://github.com/nymtech/nym/issues/198)
- Webassembly client currently hard-codes first provider [\#197](https://github.com/nymtech/nym/issues/197)
- Add Rust-based route construction to wasm client [\#196](https://github.com/nymtech/nym/issues/196)
- Remove fetch event [\#195](https://github.com/nymtech/nym/issues/195)
- Control messages should all be JSON [\#194](https://github.com/nymtech/nym/issues/194)
- Desktop Client should attach to gateway websocket [\#193](https://github.com/nymtech/nym/issues/193)
- Merge gateway and provider nodes [\#192](https://github.com/nymtech/nym/issues/192)
- Remove direct Sphinx dependencies [\#184](https://github.com/nymtech/nym/issues/184)
- tests::client\_reconnects\_to\_server\_after\_it\_went\_down fails on aarch64-linux [\#179](https://github.com/nymtech/nym/issues/179)
- \[Windows\] Presence notification fill OS socket queue [\#170](https://github.com/nymtech/nym/issues/170)
- Figure out connection hiccups between client and provider [\#162](https://github.com/nymtech/nym/issues/162)
- Improve the healthchecker [\#160](https://github.com/nymtech/nym/issues/160)
- Rethink client addressability [\#135](https://github.com/nymtech/nym/issues/135)
- Give some love to the service provider client ledger [\#116](https://github.com/nymtech/nym/issues/116)
- Start Gateway node type [\#80](https://github.com/nymtech/nym/issues/80)
- Bring health-checker into validator mix-mining [\#78](https://github.com/nymtech/nym/issues/78)
- Solidify TCPSocket on client [\#72](https://github.com/nymtech/nym/issues/72)
- scripts: run\_local\_network.sh doesn't die nicely [\#45](https://github.com/nymtech/nym/issues/45)
- WASM version of the Sphinx packet [\#19](https://github.com/nymtech/nym/issues/19)
- Persistent socket connection \(TCP Socket with provider\) [\#18](https://github.com/nymtech/nym/issues/18)
- Persistent socket connection \(Websocket with client\) [\#17](https://github.com/nymtech/nym/issues/17)
- Persistent socket connection \(TCP Socket with client\) [\#13](https://github.com/nymtech/nym/issues/13)
- Persistent socket connection \(Websocket with client\) [\#12](https://github.com/nymtech/nym/issues/12)
- Persistently store ledger with registered clients and their auth tokens [\#6](https://github.com/nymtech/nym/issues/6)
**Merged pull requests:**
- Filtering compatible node versions [\#259](https://github.com/nymtech/nym/pull/259) ([jstuczyn](https://github.com/jstuczyn))
- systemd service unit example [\#257](https://github.com/nymtech/nym/pull/257) ([ststefa](https://github.com/ststefa))
- renaming desktop to native client [\#251](https://github.com/nymtech/nym/pull/251) ([futurechimp](https://github.com/futurechimp))
- Adding a pipenv dependencies file to the python client example [\#250](https://github.com/nymtech/nym/pull/250) ([futurechimp](https://github.com/futurechimp))
- Cleaning up startup messages in native client [\#249](https://github.com/nymtech/nym/pull/249) ([futurechimp](https://github.com/futurechimp))
- fixing up readme, bumping version number [\#246](https://github.com/nymtech/nym/pull/246) ([futurechimp](https://github.com/futurechimp))
- Feature/sphinx socket packet encoder [\#245](https://github.com/nymtech/nym/pull/245) ([jstuczyn](https://github.com/jstuczyn))
- Adding some documentation to the webassembly client [\#244](https://github.com/nymtech/nym/pull/244) ([futurechimp](https://github.com/futurechimp))
- Simplified some names and used the published npm package [\#242](https://github.com/nymtech/nym/pull/242) ([futurechimp](https://github.com/futurechimp))
- Feature/make andrew happy [\#241](https://github.com/nymtech/nym/pull/241) ([futurechimp](https://github.com/futurechimp))
- Removed redundant console.log [\#240](https://github.com/nymtech/nym/pull/240) ([jstuczyn](https://github.com/jstuczyn))
- Feature/explicit gateway addressing [\#239](https://github.com/nymtech/nym/pull/239) ([jstuczyn](https://github.com/jstuczyn))
- Feature/clean up [\#238](https://github.com/nymtech/nym/pull/238) ([futurechimp](https://github.com/futurechimp))
- Feature/addressing update [\#237](https://github.com/nymtech/nym/pull/237) ([jstuczyn](https://github.com/jstuczyn))
- Added hidden init flag to increase default traffic volume [\#234](https://github.com/nymtech/nym/pull/234) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/issue\#231 [\#233](https://github.com/nymtech/nym/pull/233) ([jstuczyn](https://github.com/jstuczyn))
- Fixed unwrap on none value [\#230](https://github.com/nymtech/nym/pull/230) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/gateway crash on incomplete ws handshake [\#229](https://github.com/nymtech/nym/pull/229) ([jstuczyn](https://github.com/jstuczyn))
- Feature/start local network improvements [\#228](https://github.com/nymtech/nym/pull/228) ([jstuczyn](https://github.com/jstuczyn))
- Updated directory\_client reqwest to 0.10 [\#226](https://github.com/nymtech/nym/pull/226) ([jstuczyn](https://github.com/jstuczyn))
- Updated js-example to get gateway from topology [\#225](https://github.com/nymtech/nym/pull/225) ([jstuczyn](https://github.com/jstuczyn))
- Requiring explicit timestamp when converting from rest to service mix… [\#224](https://github.com/nymtech/nym/pull/224) ([jstuczyn](https://github.com/jstuczyn))
- Feature/minor docs fixes [\#223](https://github.com/nymtech/nym/pull/223) ([futurechimp](https://github.com/futurechimp))
- Removed having to care about SURB\_ID [\#222](https://github.com/nymtech/nym/pull/222) ([jstuczyn](https://github.com/jstuczyn))
- Moved relevant parts of old mix-client to nymsphinx [\#221](https://github.com/nymtech/nym/pull/221) ([jstuczyn](https://github.com/jstuczyn))
- Feature/load keys on run [\#220](https://github.com/nymtech/nym/pull/220) ([jstuczyn](https://github.com/jstuczyn))
- Updated wasm code to work with new gateway and updated the example [\#219](https://github.com/nymtech/nym/pull/219) ([jstuczyn](https://github.com/jstuczyn))
- validator: removing health checker [\#217](https://github.com/nymtech/nym/pull/217) ([futurechimp](https://github.com/futurechimp))
- The great sfw-provider purge of 2020 [\#216](https://github.com/nymtech/nym/pull/216) ([jstuczyn](https://github.com/jstuczyn))
- Fixed compilation warnings on unreachable code when compiling with fe… [\#215](https://github.com/nymtech/nym/pull/215) ([jstuczyn](https://github.com/jstuczyn))
- Feature/healthchecker removal [\#214](https://github.com/nymtech/nym/pull/214) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/send to correct gateway [\#213](https://github.com/nymtech/nym/pull/213) ([jstuczyn](https://github.com/jstuczyn))
- Feature/client socket adjustments [\#212](https://github.com/nymtech/nym/pull/212) ([jstuczyn](https://github.com/jstuczyn))
- Sending sphinx packet independent of the receiver task [\#210](https://github.com/nymtech/nym/pull/210) ([jstuczyn](https://github.com/jstuczyn))
- Feature/gateway provider merge [\#208](https://github.com/nymtech/nym/pull/208) ([jstuczyn](https://github.com/jstuczyn))
- Feature/route from topology [\#201](https://github.com/nymtech/nym/pull/201) ([futurechimp](https://github.com/futurechimp))
- Intermediate gateway-heart surgery checkpoint [\#199](https://github.com/nymtech/nym/pull/199) ([jstuczyn](https://github.com/jstuczyn))
- Feature/wasm js demo [\#191](https://github.com/nymtech/nym/pull/191) ([futurechimp](https://github.com/futurechimp))
- Feature/improve js example [\#190](https://github.com/nymtech/nym/pull/190) ([futurechimp](https://github.com/futurechimp))
- Feature/limit direct sphinx dependency + remove direct curve25519 dependency from wasm client [\#189](https://github.com/nymtech/nym/pull/189) ([jstuczyn](https://github.com/jstuczyn))
- Feature/very minor refactoring [\#188](https://github.com/nymtech/nym/pull/188) ([jstuczyn](https://github.com/jstuczyn))
- Feature/persistent ledger [\#187](https://github.com/nymtech/nym/pull/187) ([jstuczyn](https://github.com/jstuczyn))
- Optimising wasm build size, shaves about 10% size off our wasm output. [\#186](https://github.com/nymtech/nym/pull/186) ([futurechimp](https://github.com/futurechimp))
- Ran `npm audit fix` on the wasm demo directory. [\#185](https://github.com/nymtech/nym/pull/185) ([futurechimp](https://github.com/futurechimp))
- Feature/nym sphinx wasm [\#183](https://github.com/nymtech/nym/pull/183) ([futurechimp](https://github.com/futurechimp))
- Improvements to sfw-provider - client communcation [\#180](https://github.com/nymtech/nym/pull/180) ([jstuczyn](https://github.com/jstuczyn))
- Adding Apache 2 license headers to all files [\#178](https://github.com/nymtech/nym/pull/178) ([futurechimp](https://github.com/futurechimp))
- Feature/tcp client connection timeout [\#176](https://github.com/nymtech/nym/pull/176) ([jstuczyn](https://github.com/jstuczyn))
- Feature/service persistence [\#171](https://github.com/nymtech/nym/pull/171) ([futurechimp](https://github.com/futurechimp))
## [v0.6.0](https://github.com/nymtech/nym/tree/v0.6.0) (2020-04-07)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.5.0...v0.6.0)
This release fixes bugs in v0.5.0. All testnet node operators are advised to upgrade from v0.5.0.
* fixed premature EOFs on socket connections by using the new multi-TCP client
* fixed a bug causing client and mixnode connection hangs for misconfigured nodes
* by default 'Debug' section of saved configs is now empty and default values are used unless explicitly overridden
* introduced packet chunking allowing clients to send messages of arbitrary length. Note that packet retransmission is not implemented yet, so for longer messages, you might not get anything
* mixnodes now periodically log stats regarding number of packets mixed
* fixed possible client hang ups when sending high rates of traffic
* preventing mixes from starting with same announce-host as an existing node
* fixed overflow multiplication if connection backoff was set to a high value
**Closed issues:**
- Periodic activity summary [\#172](https://github.com/nymtech/nym/issues/172)
- Move contents of 'common/addressing' into 'common/nymsphinx' [\#161](https://github.com/nymtech/nym/issues/161)
- Make builds simpler for node operators [\#114](https://github.com/nymtech/nym/issues/114)
- Chunking in `nym-client` \(receive\) [\#83](https://github.com/nymtech/nym/issues/83)
- Chunking in `nym-client` \(send\) [\#82](https://github.com/nymtech/nym/issues/82)
**Merged pull requests:**
- Feature/mixing stats logging [\#175](https://github.com/nymtech/nym/pull/175) ([jstuczyn](https://github.com/jstuczyn))
- Preventing multiplication overflow for reconnection backoff [\#174](https://github.com/nymtech/nym/pull/174) ([jstuczyn](https://github.com/jstuczyn))
- Feature/non mandatory debug config [\#173](https://github.com/nymtech/nym/pull/173) ([jstuczyn](https://github.com/jstuczyn))
- Feature/addressing move [\#169](https://github.com/nymtech/nym/pull/169) ([jstuczyn](https://github.com/jstuczyn))
- Checking if any other node is already announcing the same host [\#168](https://github.com/nymtech/nym/pull/168) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/closing tcp client connections on drop [\#167](https://github.com/nymtech/nym/pull/167) ([jstuczyn](https://github.com/jstuczyn))
- Yielding tokio task upon creating loop/real traffic message [\#166](https://github.com/nymtech/nym/pull/166) ([jstuczyn](https://github.com/jstuczyn))
- Feature/minor healthchecker improvements [\#165](https://github.com/nymtech/nym/pull/165) ([jstuczyn](https://github.com/jstuczyn))
- Feature/packet chunking [\#158](https://github.com/nymtech/nym/pull/158) ([jstuczyn](https://github.com/jstuczyn))
## [v0.5.0](https://github.com/nymtech/nym/tree/v0.5.0) (2020-03-23)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.5.0-rc.1...v0.5.0)
1. Introduced proper configuration options for mixnodes, clients and providers. Everything is initialised with the `init` command that creates a saved config.toml file. To run the binary you now use `nym-<binary-name> run`, for example `nym-mixnode run`. Each flag can be overwritten at any stage with the following priority: run flags, data in config.toml and finally init flags.
2. Made mixnet TCP connections persistent. When sending a Sphinx packet, it should no longer go through the lengthy process of establishing a TCP connection only to immediately tear it down after sending a single packet. This significantly boosts throughput.
3. A lot of work on code clean up and refactoring including some performance fixes.
4. Client now determines its default nym-sfw-provider at startup and should always try to connect to the same one. Note: we still can't reliably run more than a single provider on the network.
5. Logging messages now have timestamps and when running at more aggressive log mode (like debug or even trace) we should no longer be overwhelmed with messages from external crates.
6. Initial compatibility with Windows. Please let us know if you have problems.
7. More work on validator, including initial Tendermint integration in Rust, and the start of the mixmining system.
**Closed issues:**
- Introduce timestamps to log messages [\#124](https://github.com/nymtech/nym/issues/124)
**Merged pull requests:**
- removing spooky startup warning message [\#155](https://github.com/nymtech/nym/pull/155) ([futurechimp](https://github.com/futurechimp))
- Some more startup fixes [\#154](https://github.com/nymtech/nym/pull/154) ([futurechimp](https://github.com/futurechimp))
- Entering runtime context when creating mix traffic controller [\#153](https://github.com/nymtech/nym/pull/153) ([jstuczyn](https://github.com/jstuczyn))
- Friendlification of startup messages [\#151](https://github.com/nymtech/nym/pull/151) ([futurechimp](https://github.com/futurechimp))
- Entering runtime context when creating packet forwarder [\#150](https://github.com/nymtech/nym/pull/150) ([jstuczyn](https://github.com/jstuczyn))
- Feature/add topology to validator [\#149](https://github.com/nymtech/nym/pull/149) ([futurechimp](https://github.com/futurechimp))
- Making code work on windows machines [\#148](https://github.com/nymtech/nym/pull/148) ([jstuczyn](https://github.com/jstuczyn))
- validator: adding HTTP interface [\#146](https://github.com/nymtech/nym/pull/146) ([futurechimp](https://github.com/futurechimp))
- Extracting the log setup [\#145](https://github.com/nymtech/nym/pull/145) ([futurechimp](https://github.com/futurechimp))
- Feature/optional location in configs [\#144](https://github.com/nymtech/nym/pull/144) ([jstuczyn](https://github.com/jstuczyn))
- Feature/concurrent connection managers [\#142](https://github.com/nymtech/nym/pull/142) ([jstuczyn](https://github.com/jstuczyn))
- Defaulting for global 'Info' logging level if not set in .env [\#140](https://github.com/nymtech/nym/pull/140) ([jstuczyn](https://github.com/jstuczyn))
- Provider not storing loop cover messages [\#139](https://github.com/nymtech/nym/pull/139) ([jstuczyn](https://github.com/jstuczyn))
- Using log builder to include timestamps + filters [\#138](https://github.com/nymtech/nym/pull/138) ([jstuczyn](https://github.com/jstuczyn))
- Feature/client ws refactoring [\#134](https://github.com/nymtech/nym/pull/134) ([jstuczyn](https://github.com/jstuczyn))
- Bugfix/metrics presence delay fix [\#133](https://github.com/nymtech/nym/pull/133) ([jstuczyn](https://github.com/jstuczyn))
- Removed outdated and redundant sample-configs [\#131](https://github.com/nymtech/nym/pull/131) ([jstuczyn](https://github.com/jstuczyn))
- If not overridden, 'announce-host' should default to 'host' [\#130](https://github.com/nymtech/nym/pull/130) ([jstuczyn](https://github.com/jstuczyn))
- Nice to know who we're talking to at startup... [\#129](https://github.com/nymtech/nym/pull/129) ([futurechimp](https://github.com/futurechimp))
## [v0.5.0-rc.1](https://github.com/nymtech/nym/tree/v0.5.0-rc.1) (2020-03-06)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.1...v0.5.0-rc.1)
**Closed issues:**
- COMPILE: Could not compile project using Cargo [\#118](https://github.com/nymtech/nym/issues/118)
- Wherever unbounded mpsc channel is used, prefer unbounded\_send\(\) over send\(\).await [\#90](https://github.com/nymtech/nym/issues/90)
- Add a `Send` method in nym-client [\#81](https://github.com/nymtech/nym/issues/81)
- Start on Tendermint integration [\#79](https://github.com/nymtech/nym/issues/79)
- Ditch DummyKeyPair [\#75](https://github.com/nymtech/nym/issues/75)
- Replace args with proper config files [\#69](https://github.com/nymtech/nym/issues/69)
- Fix incorrectly used Arcs [\#47](https://github.com/nymtech/nym/issues/47)
- nym-mixnode mandatory host option [\#26](https://github.com/nymtech/nym/issues/26)
- Create config struct for mixnode \(possibly also for client\) [\#21](https://github.com/nymtech/nym/issues/21)
- Reuse TCP socket connection between client and mixnodes [\#20](https://github.com/nymtech/nym/issues/20)
- Once implementation is available, wherever appropriate, replace `futures::lock::Mutex` with `futures::lock::RwLock` [\#9](https://github.com/nymtech/nym/issues/9)
- Check if RwLock on MixProcessingData is still needed [\#8](https://github.com/nymtech/nym/issues/8)
- Reuse TCP socket connection between mixnodes and providers [\#3](https://github.com/nymtech/nym/issues/3)
- Persistent socket connection with other mixes [\#2](https://github.com/nymtech/nym/issues/2)
**Merged pull requests:**
- Feature/client refactoring [\#128](https://github.com/nymtech/nym/pull/128) ([jstuczyn](https://github.com/jstuczyn))
- Feature/provider refactoring [\#125](https://github.com/nymtech/nym/pull/125) ([jstuczyn](https://github.com/jstuczyn))
- all: fixing mis-spelling [\#123](https://github.com/nymtech/nym/pull/123) ([futurechimp](https://github.com/futurechimp))
- Feature/further clippy fixes [\#121](https://github.com/nymtech/nym/pull/121) ([jstuczyn](https://github.com/jstuczyn))
- Feature/tokio tungstenite dependency fix [\#120](https://github.com/nymtech/nym/pull/120) ([jstuczyn](https://github.com/jstuczyn))
- Feature/config files cleanup [\#119](https://github.com/nymtech/nym/pull/119) ([jstuczyn](https://github.com/jstuczyn))
- Feature/config files [\#117](https://github.com/nymtech/nym/pull/117) ([jstuczyn](https://github.com/jstuczyn))
- Feature/un genericize keys [\#111](https://github.com/nymtech/nym/pull/111) ([futurechimp](https://github.com/futurechimp))
- Feature/abci [\#110](https://github.com/nymtech/nym/pull/110) ([futurechimp](https://github.com/futurechimp))
- Simplified the use of generics on identity keypair by using output types [\#109](https://github.com/nymtech/nym/pull/109) ([jstuczyn](https://github.com/jstuczyn))
## [v0.4.1](https://github.com/nymtech/nym/tree/v0.4.1) (2020-01-29)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0...v0.4.1)
**Closed issues:**
- Change healthcheck to run on provided topology rather than pull one itself [\#95](https://github.com/nymtech/nym/issues/95)
**Merged pull requests:**
- Bugfix/healthcheck on provided topology [\#108](https://github.com/nymtech/nym/pull/108) ([jstuczyn](https://github.com/jstuczyn))
## [v0.4.0](https://github.com/nymtech/nym/tree/v0.4.0) (2020-01-28)
[Full Changelog](https://github.com/nymtech/nym/compare/0.4.0-rc.2...v0.4.0)
Nym 0.4.0 Platform
In this release, we're taking a lot more care with version numbers, so that we can ensure upgrade compatibility for mixnodes, providers, clients, and validators more easily.
This release also integrates a health-checker and network topology refresh into the Nym client, so that the client can intelligently choose paths which route around any non-functional or incompatible nodes.
## [0.4.0-rc.2](https://github.com/nymtech/nym/tree/0.4.0-rc.2) (2020-01-28)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0-rc.2...0.4.0-rc.2)
## [v0.4.0-rc.2](https://github.com/nymtech/nym/tree/v0.4.0-rc.2) (2020-01-28)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0-rc.1...v0.4.0-rc.2)
**Merged pull requests:**
- Hotfix/semver compatibility [\#106](https://github.com/nymtech/nym/pull/106) ([jstuczyn](https://github.com/jstuczyn))
## [v0.4.0-rc.1](https://github.com/nymtech/nym/tree/v0.4.0-rc.1) (2020-01-28)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.3...v0.4.0-rc.1)
**Closed issues:**
- Check Sphinx packet length in client [\#98](https://github.com/nymtech/nym/issues/98)
- workflow test [\#97](https://github.com/nymtech/nym/issues/97)
- Client SemVer [\#85](https://github.com/nymtech/nym/issues/85)
- Move PemStore [\#74](https://github.com/nymtech/nym/issues/74)
- Periodic client refresh [\#70](https://github.com/nymtech/nym/issues/70)
- Logging [\#68](https://github.com/nymtech/nym/issues/68)
- Nym-client refactor [\#67](https://github.com/nymtech/nym/issues/67)
- Stop panicking! [\#66](https://github.com/nymtech/nym/issues/66)
- Fix Mixnode Panic on Sphinx packet replay [\#65](https://github.com/nymtech/nym/issues/65)
- Convert older code to start using logging [\#35](https://github.com/nymtech/nym/issues/35)
- Convert to non-url-safe base64 \(everywhere\) [\#28](https://github.com/nymtech/nym/issues/28)
- If a thread blows at startup, panic the entire application [\#15](https://github.com/nymtech/nym/issues/15)
- Split provider/mod.rs [\#10](https://github.com/nymtech/nym/issues/10)
**Merged pull requests:**
- Feature/health checker with existing keys [\#105](https://github.com/nymtech/nym/pull/105) ([jstuczyn](https://github.com/jstuczyn))
- Feature/remove topology equality check [\#104](https://github.com/nymtech/nym/pull/104) ([futurechimp](https://github.com/futurechimp))
- Feature/base58 [\#102](https://github.com/nymtech/nym/pull/102) ([futurechimp](https://github.com/futurechimp))
- Feature/panic improvements [\#101](https://github.com/nymtech/nym/pull/101) ([jstuczyn](https://github.com/jstuczyn))
- Feature/fix sphinx unwraps [\#100](https://github.com/nymtech/nym/pull/100) ([futurechimp](https://github.com/futurechimp))
- Feature/check packet length [\#99](https://github.com/nymtech/nym/pull/99) ([futurechimp](https://github.com/futurechimp))
- Feature/version filtering improvements [\#96](https://github.com/nymtech/nym/pull/96) ([futurechimp](https://github.com/futurechimp))
- Feature/refreshing topology [\#94](https://github.com/nymtech/nym/pull/94) ([jstuczyn](https://github.com/jstuczyn))
- Feature/consistent logging [\#93](https://github.com/nymtech/nym/pull/93) ([futurechimp](https://github.com/futurechimp))
- Feature/semver client [\#92](https://github.com/nymtech/nym/pull/92) ([futurechimp](https://github.com/futurechimp))
- Feature/client refactor [\#91](https://github.com/nymtech/nym/pull/91) ([jstuczyn](https://github.com/jstuczyn))
- Release builds should no longer silently fail - everything will be im… [\#89](https://github.com/nymtech/nym/pull/89) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.3](https://github.com/nymtech/nym/tree/v0.3.3) (2020-01-20)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.2...v0.3.3)
**Fixed bugs:**
- Nym client crashing and disconnecting local websocket with complaint about binary data [\#56](https://github.com/nymtech/nym/issues/56)
**Closed issues:**
- Websocket text fix [\#64](https://github.com/nymtech/nym/issues/64)
- Restore nym-client lib in crate [\#63](https://github.com/nymtech/nym/issues/63)
- Make websocket not crash on ping or pong messages [\#62](https://github.com/nymtech/nym/issues/62)
- Messages returned by fetch are base64 encoded [\#55](https://github.com/nymtech/nym/issues/55)
- Check layer 1 connectivity at client start [\#38](https://github.com/nymtech/nym/issues/38)
- Check required sfw-provider args [\#27](https://github.com/nymtech/nym/issues/27)
- Make Electron app work with new Rust mixnet client [\#16](https://github.com/nymtech/nym/issues/16)
- Take version numbers into account when picking routes [\#14](https://github.com/nymtech/nym/issues/14)
**Merged pull requests:**
- Feature/websocket improvements [\#88](https://github.com/nymtech/nym/pull/88) ([jstuczyn](https://github.com/jstuczyn))
- Using println rather than log for startup banner, it's not an error [\#87](https://github.com/nymtech/nym/pull/87) ([futurechimp](https://github.com/futurechimp))
- Feature/nym client lib [\#61](https://github.com/nymtech/nym/pull/61) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.2](https://github.com/nymtech/nym/tree/v0.3.2) (2020-01-17)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.1...v0.3.2)
**Merged pull requests:**
- Feature/separate presence address [\#59](https://github.com/nymtech/nym/pull/59) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.1](https://github.com/nymtech/nym/tree/v0.3.1) (2020-01-16)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.0...v0.3.1)
**Merged pull requests:**
- Bugfix/presence client crash [\#58](https://github.com/nymtech/nym/pull/58) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.0](https://github.com/nymtech/nym/tree/v0.3.0) (2020-01-14)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.2.0...v0.3.0)
**Closed issues:**
- Error referring to mismatched types caused by topology [\#46](https://github.com/nymtech/nym/issues/46)
- Provider needs to announce two of its addresses \(+ issue of hardcoded port\) [\#32](https://github.com/nymtech/nym/issues/32)
- Port run\_network.sh from old Go mixnet [\#30](https://github.com/nymtech/nym/issues/30)
- Health Checker inside Validator [\#29](https://github.com/nymtech/nym/issues/29)
- Combine Rust repositories into a single master repo containing multiple crates [\#24](https://github.com/nymtech/nym/issues/24)
- Fix the version numbers on provider and mixnode [\#23](https://github.com/nymtech/nym/issues/23)
- Travis releases for Nym [\#22](https://github.com/nymtech/nym/issues/22)
- Add version number to presence [\#4](https://github.com/nymtech/nym/issues/4)
- Add version number to presence [\#1](https://github.com/nymtech/nym/issues/1)
**Merged pull requests:**
- Feature/client topology filtering [\#54](https://github.com/nymtech/nym/pull/54) ([jstuczyn](https://github.com/jstuczyn))
- print public key for nym client tools [\#53](https://github.com/nymtech/nym/pull/53) ([mileschet](https://github.com/mileschet))
- Showing binding warning on binding to localhost, 0.0.0.0 or 127.0.0.1 [\#52](https://github.com/nymtech/nym/pull/52) ([jstuczyn](https://github.com/jstuczyn))
- validator: moving sample config files into sample configs directory [\#51](https://github.com/nymtech/nym/pull/51) ([futurechimp](https://github.com/futurechimp))
- Feature/validator health checker [\#50](https://github.com/nymtech/nym/pull/50) ([jstuczyn](https://github.com/jstuczyn))
- Improving websocket connection errors [\#49](https://github.com/nymtech/nym/pull/49) ([futurechimp](https://github.com/futurechimp))
- Feature/crypto crate [\#48](https://github.com/nymtech/nym/pull/48) ([jstuczyn](https://github.com/jstuczyn))
- Feature/chill log messages [\#43](https://github.com/nymtech/nym/pull/43) ([futurechimp](https://github.com/futurechimp))
- persistence: improving PEM file reading and parsing failure messages [\#42](https://github.com/nymtech/nym/pull/42) ([futurechimp](https://github.com/futurechimp))
- Providing a nicer error than "failed on unwrap\(\)" when topology retri… [\#41](https://github.com/nymtech/nym/pull/41) ([futurechimp](https://github.com/futurechimp))
- Prettying up sfw-provider start sequence a bit. [\#40](https://github.com/nymtech/nym/pull/40) ([futurechimp](https://github.com/futurechimp))
- Removing the run command from code and documentation [\#39](https://github.com/nymtech/nym/pull/39) ([futurechimp](https://github.com/futurechimp))
- Feature/script/localnet [\#37](https://github.com/nymtech/nym/pull/37) ([futurechimp](https://github.com/futurechimp))
- Feature/arguments fix [\#36](https://github.com/nymtech/nym/pull/36) ([jstuczyn](https://github.com/jstuczyn))
- Feature/crates cleanup [\#34](https://github.com/nymtech/nym/pull/34) ([jstuczyn](https://github.com/jstuczyn))
- Features/version number to presence [\#25](https://github.com/nymtech/nym/pull/25) ([aniampio](https://github.com/aniampio))
- Feature/minor client changes [\#7](https://github.com/nymtech/nym/pull/7) ([jstuczyn](https://github.com/jstuczyn))
## [v0.2.0](https://github.com/nymtech/nym/tree/v0.2.0) (2020-01-07)
[Full Changelog](https://github.com/nymtech/nym/compare/0.2.0...v0.2.0)
## [0.2.0](https://github.com/nymtech/nym/tree/0.2.0) (2020-01-06)
[Full Changelog](https://github.com/nymtech/nym/compare/0.1.0...0.2.0)
\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*
Cargo.lock
Generated
+774 -558
View File
File diff suppressed because it is too large Load Diff
Cargo.toml
+18 -1
View File
@@ -2,23 +2,40 @@
panic = "abort"
opt-level = "s"
[profile.dev]
panic = "abort"
[workspace]
members = [
"clients/native",
"clients/native/websocket-requests",
"clients/socks5",
"clients/webassembly",
"clients/client-core",
"common/client-libs/directory-client",
"common/client-libs/directory-client/models",
"common/client-libs/gateway-client",
"common/client-libs/mixnet-client",
"common/client-libs/validator-client",
"common/config",
"common/crypto",
"common/healthcheck",
"common/nymsphinx",
"common/nymsphinx/acknowledgements",
"common/nymsphinx/addressing",
"common/nymsphinx/anonymous-replies",
"common/nymsphinx/chunking",
"common/nymsphinx/cover",
"common/nymsphinx/framing",
"common/nymsphinx/params",
"common/nymsphinx/types",
"common/pemstore",
"common/socks5/requests",
"common/socks5/proxy-helpers",
"common/topology",
"gateway",
"gateway/gateway-requests",
"service-providers/sphinx-socks",
"mixnode",
"validator",
]
clients/client-core/Cargo.toml
+34
View File
@@ -0,0 +1,34 @@
[package]
build = "build.rs"
name = "client-core"
version = "0.8.0-dev"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
dirs = "2.0.2"
futures = "0.3.1"
log = "0.4"
rand = { version = "0.7.3", features = ["wasm-bindgen"] }
serde = { version = "1.0.104", features = ["derive"] }
sled = "0.33"
tokio = { version = "0.2", features = ["full"] }
url = "2.1"
# internal
config = { path = "../../common/config" }
crypto = { path = "../../common/crypto" }
directory-client = { path = "../../common/client-libs/directory-client" }
gateway-client = { path = "../../common/client-libs/gateway-client" }
gateway-requests = { path = "../../gateway/gateway-requests" }
nymsphinx = { path = "../../common/nymsphinx" }
pemstore = { path = "../../common/pemstore" }
topology = { path = "../../common/topology" }
[build-dependencies]
built = "0.4.3"
[dev-dependencies]
tempfile = "3.1.0"
common/pemstore/src/pathfinder.rs → clients/client-core/build.rs
+3 -13
View File
@@ -12,18 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use std::path::PathBuf;
use built;
pub trait PathFinder {
fn config_dir(&self) -> PathBuf;
fn private_identity_key(&self) -> PathBuf;
fn public_identity_key(&self) -> PathBuf;
// Optional:
fn private_encryption_key(&self) -> Option<PathBuf> {
None
}
fn public_encryption_key(&self) -> Option<PathBuf> {
None
}
fn main() {
built::write_built_file().expect("Failed to acquire build-time information");
}
clients/client-core/src/built_info.rs
+16
View File
@@ -0,0 +1,16 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// The file has been placed there by the build script.
include!(concat!(env!("OUT_DIR"), "/built.rs"));
clients/native/src/client/cover_traffic_stream.rs → clients/client-core/src/client/cover_traffic_stream.rs
+88 -41
View File
@@ -17,25 +17,55 @@ use crate::client::topology_control::TopologyAccessor;
use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt};
use log::*;
use nymsphinx::acknowledgements::AckKey;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::utils::{encapsulation, poisson};
use nymsphinx::cover::generate_loop_cover_packet;
use nymsphinx::utils::sample_poisson_duration;
use rand::{rngs::OsRng, CryptoRng, Rng};
use std::pin::Pin;
use std::time::Duration;
use std::sync::Arc;
use tokio::runtime::Handle;
use tokio::task::JoinHandle;
use tokio::time;
use topology::NymTopology;
pub(crate) struct LoopCoverTrafficStream<T: NymTopology> {
average_packet_delay: Duration,
average_cover_message_sending_delay: Duration,
pub struct LoopCoverTrafficStream<R>
where
R: CryptoRng + Rng,
{
/// Key used to encrypt and decrypt content of an ACK packet.
ack_key: Arc<AckKey>,
/// Average delay an acknowledgement packet is going to get delay at a single mixnode.
average_ack_delay: time::Duration,
/// Average delay a data packet is going to get delay at a single mixnode.
average_packet_delay: time::Duration,
/// Average delay between sending subsequent cover packets.
average_cover_message_sending_delay: time::Duration,
/// Internal state, determined by `average_message_sending_delay`,
/// used to keep track of when a next packet should be sent out.
next_delay: time::Delay,
/// Channel used for sending prepared sphinx packets to `MixTrafficController` that sends them
/// out to the network without any further delays.
mix_tx: MixMessageSender,
/// Represents full address of this client.
our_full_destination: Recipient,
topology_access: TopologyAccessor<T>,
/// Instance of a cryptographically secure random number generator.
rng: R,
/// Accessor to the common instance of network topology.
topology_access: TopologyAccessor,
}
impl<T: NymTopology> Stream for LoopCoverTrafficStream<T> {
impl<R> Stream for LoopCoverTrafficStream<R>
where
R: CryptoRng + Rng + Unpin,
{
// Item is only used to indicate we should create a new message rather than actual cover message
// reason being to not introduce unnecessary complexity by having to keep state of topology
// mutex when trying to acquire it. So right now the Stream trait serves as a glorified timer.
@@ -50,8 +80,9 @@ impl<T: NymTopology> Stream for LoopCoverTrafficStream<T> {
// we know it's time to send a message, so let's prepare delay for the next one
// Get the `now` by looking at the current `delay` deadline
let avg_delay = self.average_cover_message_sending_delay;
let now = self.next_delay.deadline();
let next_poisson_delay = poisson::sample(self.average_cover_message_sending_delay);
let next_poisson_delay = sample_poisson_duration(&mut self.rng, avg_delay);
// The next interval value is `next_poisson_delay` after the one that just
// yielded.
@@ -62,52 +93,60 @@ impl<T: NymTopology> Stream for LoopCoverTrafficStream<T> {
}
}
impl<T: 'static + NymTopology> LoopCoverTrafficStream<T> {
pub(crate) fn new(
// obviously when we finally make shared rng that is on 'higher' level, this should become
// generic `R`
impl LoopCoverTrafficStream<OsRng> {
pub fn new(
ack_key: Arc<AckKey>,
average_ack_delay: time::Duration,
average_packet_delay: time::Duration,
average_cover_message_sending_delay: time::Duration,
mix_tx: MixMessageSender,
our_full_destination: Recipient,
topology_access: TopologyAccessor<T>,
average_cover_message_sending_delay: time::Duration,
average_packet_delay: time::Duration,
topology_access: TopologyAccessor,
) -> Self {
let rng = OsRng;
LoopCoverTrafficStream {
ack_key,
average_ack_delay,
average_packet_delay,
average_cover_message_sending_delay,
next_delay: time::delay_for(Default::default()),
mix_tx,
our_full_destination,
rng,
topology_access,
}
}
async fn on_new_message(&mut self) {
trace!("next cover message!");
let route = match self
.topology_access
.random_route_to_gateway(&self.our_full_destination.gateway())
.await
{
None => {
warn!("No valid topology detected - won't send any loop cover message this time");
return;
}
Some(route) => route,
};
let cover_message = match encapsulation::loop_cover_message_route(
self.our_full_destination.destination().clone(),
route,
// TODO for way down the line: in very rare cases (during topology update) we might have
// to wait a really tiny bit before actually obtaining the permit hence messing with our
// poisson delay, but is it really a problem?
let topology_permit = self.topology_access.get_read_permit().await;
// the ack is sent back to ourselves (and then ignored)
let topology_ref_option = topology_permit.try_get_valid_topology_ref(
&self.our_full_destination,
Some(&self.our_full_destination),
);
if topology_ref_option.is_none() {
warn!("No valid topology detected - won't send any loop cover message this time");
return;
}
let topology_ref = topology_ref_option.unwrap();
let cover_message = generate_loop_cover_packet(
&mut self.rng,
topology_ref,
&*self.ack_key,
&self.our_full_destination,
self.average_ack_delay,
self.average_packet_delay,
) {
Ok(message) => message,
Err(err) => {
error!(
"Somehow we managed to create an invalid cover message - {:?}",
err
);
return;
}
};
)
.expect("Somehow failed to generate a loop cover message with a valid topology");
// if this one fails, there's no retrying because it means that either:
// - we run out of memory
@@ -116,6 +155,12 @@ impl<T: 'static + NymTopology> LoopCoverTrafficStream<T> {
self.mix_tx
.unbounded_send(MixMessage::new(cover_message.0, cover_message.1))
.unwrap();
// TODO: I'm not entirely sure whether this is really required, because I'm not 100%
// sure how `yield_now()` works - whether it just notifies the scheduler or whether it
// properly blocks. So to play it on the safe side, just explicitly drop the read permit
drop(topology_permit);
// JS: due to identical logical structure to OutQueueControl::on_message(), this is also
// presumably required to prevent bugs in the future. Exact reason is still unknown to me.
tokio::task::yield_now().await;
@@ -123,15 +168,17 @@ impl<T: 'static + NymTopology> LoopCoverTrafficStream<T> {
async fn run(&mut self) {
// we should set initial delay only when we actually start the stream
self.next_delay =
time::delay_for(poisson::sample(self.average_cover_message_sending_delay));
self.next_delay = time::delay_for(sample_poisson_duration(
&mut self.rng,
self.average_cover_message_sending_delay,
));
while let Some(_) = self.next().await {
self.on_new_message().await;
}
}
pub(crate) fn start(mut self, handle: &Handle) -> JoinHandle<()> {
pub fn start(mut self, handle: &Handle) -> JoinHandle<()> {
handle.spawn(async move {
self.run().await;
})
clients/client-core/src/client/inbound_messages.rs
+33
View File
@@ -0,0 +1,33 @@
use futures::channel::mpsc;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::anonymous_replies::ReplySURB;
pub type InputMessageSender = mpsc::UnboundedSender<InputMessage>;
pub type InputMessageReceiver = mpsc::UnboundedReceiver<InputMessage>;
#[derive(Debug)]
pub enum InputMessage {
Fresh {
recipient: Recipient,
data: Vec<u8>,
with_reply_surb: bool,
},
Reply {
reply_surb: ReplySURB,
data: Vec<u8>,
},
}
impl InputMessage {
pub fn new_fresh(recipient: Recipient, data: Vec<u8>, with_reply_surb: bool) -> Self {
InputMessage::Fresh {
recipient,
data,
with_reply_surb,
}
}
pub fn new_reply(reply_surb: ReplySURB, data: Vec<u8>) -> Self {
InputMessage::Reply { reply_surb, data }
}
}
clients/client-core/src/client/key_manager.rs
+168
View File
@@ -0,0 +1,168 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::config::persistence::key_pathfinder::ClientKeyPathfinder;
use crypto::asymmetric::{encryption, identity};
use gateway_requests::registration::handshake::SharedKeys;
use log::*;
use nymsphinx::acknowledgements::AckKey;
use rand::{CryptoRng, RngCore};
use std::io;
use std::sync::Arc;
// Note: to support key rotation in the future, all keys will require adding an extra smart pointer,
// most likely an AtomicCell, or if it doesn't work as I think it does, a Mutex. Although I think
// AtomicCell includes a Mutex implicitly if the underlying type does not work atomically.
// And I guess there will need to be some mechanism for a grace period when you can still
// use the old key after new one was issued.
// Remember that Arc<T> has Deref implementation for T
pub struct KeyManager {
/// identity key associated with the client instance.
identity_keypair: Arc<identity::KeyPair>,
/// encryption key associated with the client instance.
encryption_keypair: Arc<encryption::KeyPair>,
/// shared key derived with the gateway during "registration handshake"
gateway_shared_key: Option<Arc<SharedKeys>>,
/// key used for producing and processing acknowledgement packets.
ack_key: Arc<AckKey>,
}
// The expected flow of a KeyManager "lifetime" is as follows:
/*
1. ::new() is called during client-init
2. after gateway registration is completed [in init] ::insert_gateway_shared_key() is called
3. ::store_keys() is called before init finishes execution.
4. ::load_keys() is called at the beginning of each subsequent client-run
5. [not implemented] ::rotate_keys() is called periodically during client-run I presume?
*/
impl KeyManager {
// this is actually **NOT** dead code
// I have absolutely no idea why the compiler insists it's unused. The call happens during client::init::execute
#[allow(dead_code)]
/// Creates new instance of a [`KeyManager`]
pub fn new<R>(rng: &mut R) -> Self
where
R: RngCore + CryptoRng,
{
KeyManager {
identity_keypair: Arc::new(identity::KeyPair::new_with_rng(rng)),
encryption_keypair: Arc::new(encryption::KeyPair::new_with_rng(rng)),
gateway_shared_key: None,
ack_key: Arc::new(AckKey::new(rng)),
}
}
// this is actually **NOT** dead code
// I have absolutely no idea why the compiler insists it's unused. The call happens during client::init::execute
#[allow(dead_code)]
/// After shared key with the gateway is derived, puts its ownership to this instance of a [`KeyManager`].
pub fn insert_gateway_shared_key(&mut self, gateway_shared_key: SharedKeys) {
self.gateway_shared_key = Some(Arc::new(gateway_shared_key))
}
/// Loads previously stored keys from the disk.
pub fn load_keys(client_pathfinder: &ClientKeyPathfinder) -> io::Result<Self> {
let identity_keypair: identity::KeyPair =
pemstore::load_keypair(&pemstore::KeyPairPath::new(
client_pathfinder.private_identity_key().to_owned(),
client_pathfinder.public_identity_key().to_owned(),
))?;
let encryption_keypair: encryption::KeyPair =
pemstore::load_keypair(&pemstore::KeyPairPath::new(
client_pathfinder.private_encryption_key().to_owned(),
client_pathfinder.public_encryption_key().to_owned(),
))?;
let gateway_shared_key: SharedKeys =
pemstore::load_key(&client_pathfinder.gateway_shared_key().to_owned())?;
let ack_key: AckKey = pemstore::load_key(&client_pathfinder.ack_key().to_owned())?;
// TODO: ack key is never stored so it is generated now. But perhaps it should be stored
// after all for consistency sake?
Ok(KeyManager {
identity_keypair: Arc::new(identity_keypair),
encryption_keypair: Arc::new(encryption_keypair),
gateway_shared_key: Some(Arc::new(gateway_shared_key)),
ack_key: Arc::new(ack_key),
})
}
// this is actually **NOT** dead code
// I have absolutely no idea why the compiler insists it's unused. The call happens during client::init::execute
#[allow(dead_code)]
/// Stores all available keys on the disk.
// While perhaps there is no much point in storing the `AckKey` on the disk,
// it is done so for the consistency sake so that you wouldn't require an rng instance
// during `load_keys` to generate the said key.
pub fn store_keys(&self, client_pathfinder: &ClientKeyPathfinder) -> io::Result<()> {
pemstore::store_keypair(
self.identity_keypair.as_ref(),
&pemstore::KeyPairPath::new(
client_pathfinder.private_identity_key().to_owned(),
client_pathfinder.public_identity_key().to_owned(),
),
)?;
pemstore::store_keypair(
self.encryption_keypair.as_ref(),
&pemstore::KeyPairPath::new(
client_pathfinder.private_encryption_key().to_owned(),
client_pathfinder.public_encryption_key().to_owned(),
),
)?;
pemstore::store_key(self.ack_key.as_ref(), &client_pathfinder.ack_key())?;
match self.gateway_shared_key.as_ref() {
None => warn!("No gateway shared key available to store!"),
Some(gate_key) => {
pemstore::store_key(gate_key.as_ref(), &client_pathfinder.gateway_shared_key())?
}
}
Ok(())
}
/// Gets an atomically reference counted pointer to [`identity::KeyPair`].
pub fn identity_keypair(&self) -> Arc<identity::KeyPair> {
Arc::clone(&self.identity_keypair)
}
/// Gets an atomically reference counted pointer to [`encryption::KeyPair`].
pub fn encryption_keypair(&self) -> Arc<encryption::KeyPair> {
Arc::clone(&self.encryption_keypair)
}
/// Gets an atomically reference counted pointer to [`SharedKey`].
// since this function is not fully public, it is not expected to be used externally and
// hence it's up to us to ensure it's called in correct context
pub fn gateway_shared_key(&self) -> Arc<SharedKeys> {
Arc::clone(
&self
.gateway_shared_key
.as_ref()
.expect("tried to unwrap empty gateway key!"),
)
}
/// Gets an atomically reference counted pointer to [`AckKey`].
pub fn ack_key(&self) -> Arc<AckKey> {
Arc::clone(&self.ack_key)
}
}
clients/native/src/client/mix_traffic.rs → clients/client-core/src/client/mix_traffic.rs
+28 -12
View File
@@ -16,36 +16,42 @@ use futures::channel::mpsc;
use futures::StreamExt;
use gateway_client::GatewayClient;
use log::*;
use nymsphinx::SphinxPacket;
use std::net::SocketAddr;
use nymsphinx::{addressing::nodes::NymNodeRoutingAddress, SphinxPacket};
use tokio::runtime::Handle;
use tokio::task::JoinHandle;
pub(crate) struct MixMessage(SocketAddr, SphinxPacket);
pub(crate) type MixMessageSender = mpsc::UnboundedSender<MixMessage>;
pub(crate) type MixMessageReceiver = mpsc::UnboundedReceiver<MixMessage>;
pub struct MixMessage(NymNodeRoutingAddress, SphinxPacket);
pub type MixMessageSender = mpsc::UnboundedSender<MixMessage>;
pub type MixMessageReceiver = mpsc::UnboundedReceiver<MixMessage>;
impl MixMessage {
pub(crate) fn new(address: SocketAddr, packet: SphinxPacket) -> Self {
pub fn new(address: NymNodeRoutingAddress, packet: SphinxPacket) -> Self {
MixMessage(address, packet)
}
}
pub(crate) struct MixTrafficController<'a> {
const MAX_FAILURE_COUNT: usize = 100;
pub struct MixTrafficController<'a> {
// TODO: most likely to be replaced by some higher level construct as
// later on gateway_client will need to be accessible by other entities
gateway_client: GatewayClient<'a, url::Url>,
mix_rx: MixMessageReceiver,
// TODO: this is temporary work-around.
// in long run `gateway_client` will be moved away from `MixTrafficController` anyway.
consecutive_gateway_failure_count: usize,
}
impl<'a> MixTrafficController<'static> {
pub(crate) fn new(
pub fn new(
mix_rx: MixMessageReceiver,
gateway_client: GatewayClient<'a, url::Url>,
) -> MixTrafficController<'a> {
MixTrafficController {
gateway_client,
mix_rx,
consecutive_gateway_failure_count: 0,
}
}
@@ -56,18 +62,28 @@ impl<'a> MixTrafficController<'static> {
.send_sphinx_packet(mix_message.0, mix_message.1)
.await
{
Err(e) => error!("Failed to send sphinx packet to the gateway! - {:?}", e),
Ok(_) => trace!("We *might* have managed to forward sphinx packet to the gateway!"),
Err(e) => {
error!("Failed to send sphinx packet to the gateway! - {:?}", e);
self.consecutive_gateway_failure_count += 1;
if self.consecutive_gateway_failure_count == MAX_FAILURE_COUNT {
// todo: in the future this should initiate a 'graceful' shutdown
panic!("failed to send sphinx packet to the gateway {} times in a row - assuming the gateway is dead. Can't do anything about it yet :(", MAX_FAILURE_COUNT)
}
}
Ok(_) => {
trace!("We *might* have managed to forward sphinx packet to the gateway!");
self.consecutive_gateway_failure_count = 0;
}
}
}
pub(crate) async fn run(&mut self) {
pub async fn run(&mut self) {
while let Some(mix_message) = self.mix_rx.next().await {
self.on_message(mix_message).await;
}
}
pub(crate) fn start(mut self, handle: &Handle) -> JoinHandle<()> {
pub fn start(mut self, handle: &Handle) -> JoinHandle<()> {
handle.spawn(async move {
self.run().await;
})
clients/client-core/src/client/mod.rs
+8
View File
@@ -0,0 +1,8 @@
pub mod cover_traffic_stream;
pub mod inbound_messages;
pub mod key_manager;
pub mod mix_traffic;
pub mod real_messages_control;
pub mod received_buffer;
pub mod reply_key_storage;
pub mod topology_control;
clients/client-core/src/client/real_messages_control/acknowledgement_control/ack_delay_queue.rs
+68
View File
@@ -0,0 +1,68 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use futures::Stream;
use std::pin::Pin;
use std::task::{Context, Poll, Waker};
use std::time::Duration;
use tokio::time::{
delay_queue::{self, Expired},
DelayQueue,
};
// works under assumption that it will be used inside a loop, where we never want a `None`
// TODO: perhaps this should/could be renamed and moved to common/utils (and expose all inner methods?)
pub struct AckDelayQueue<T> {
inner: DelayQueue<T>,
waker: Option<Waker>,
}
// more methods of underlying DelayQueue will get exposed as we need them
impl<T> AckDelayQueue<T> {
pub fn new() -> Self {
AckDelayQueue {
inner: DelayQueue::new(),
waker: None,
}
}
pub fn insert(&mut self, value: T, timeout: Duration) -> delay_queue::Key {
let key = self.inner.insert(value, timeout);
if let Some(waker) = self.waker.take() {
// we were waiting for an item - wake the executor!
waker.wake()
}
key
}
pub fn remove(&mut self, key: &delay_queue::Key) -> Expired<T> {
self.inner.remove(key)
}
}
impl<T> Stream for AckDelayQueue<T> {
type Item = <DelayQueue<T> as Stream>::Item;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
match Pin::new(&mut self.inner).poll_next(cx) {
Poll::Pending => Poll::Pending,
Poll::Ready(Some(item)) => Poll::Ready(Some(item)),
Poll::Ready(None) => {
// we'll need to keep the waker to notify the executor once we get new item
self.waker = Some(cx.waker().clone());
Poll::Pending
}
}
}
}
clients/client-core/src/client/real_messages_control/acknowledgement_control/acknowledgement_listener.rs
+87
View File
@@ -0,0 +1,87 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::action_controller::{Action, ActionSender};
use futures::StreamExt;
use gateway_client::AcknowledgementReceiver;
use log::*;
use nymsphinx::{
acknowledgements::{identifier::recover_identifier, AckKey},
chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID},
};
use std::sync::Arc;
/// Module responsible for listening for any data resembling acknowledgements from the network
/// and firing actions to remove them from the 'Pending' state.
pub(super) struct AcknowledgementListener {
ack_key: Arc<AckKey>,
ack_receiver: AcknowledgementReceiver,
action_sender: ActionSender,
}
impl AcknowledgementListener {
pub(super) fn new(
ack_key: Arc<AckKey>,
ack_receiver: AcknowledgementReceiver,
action_sender: ActionSender,
) -> Self {
AcknowledgementListener {
ack_key,
ack_receiver,
action_sender,
}
}
async fn on_ack(&mut self, ack_content: Vec<u8>) {
debug!("Received an ack");
let frag_id = match recover_identifier(&self.ack_key, &ack_content)
.map(FragmentIdentifier::try_from_bytes)
{
Some(Ok(frag_id)) => frag_id,
_ => {
warn!("Received invalid ACK!"); // should we do anything else about that?
return;
}
};
// if we received an ack for cover message or a reply there will be nothing to remove,
// because nothing was inserted in the first place
if frag_id == COVER_FRAG_ID {
trace!("Received an ack for a cover message - no need to do anything");
return;
} else if frag_id.is_reply() {
info!("Received an ack for a reply message - no need to do anything! (don't know what to do!)");
// TODO: probably there will need to be some extra procedure here, something to notify
// user that his reply reached the recipient (since we got an ack)
return;
}
trace!("Received {} from the mix network", frag_id);
self.action_sender
.unbounded_send(Action::new_remove(frag_id))
.unwrap();
}
pub(super) async fn run(&mut self) {
debug!("Started AcknowledgementListener");
while let Some(acks) = self.ack_receiver.next().await {
// realistically we would only be getting one ack at the time
for ack in acks {
self.on_ack(ack).await;
}
}
error!("TODO: error msg. Or maybe panic?")
}
}
clients/client-core/src/client/real_messages_control/acknowledgement_control/action_controller.rs
+277
View File
@@ -0,0 +1,277 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::PendingAcknowledgement;
use crate::client::real_messages_control::acknowledgement_control::ack_delay_queue::AckDelayQueue;
use crate::client::real_messages_control::acknowledgement_control::RetransmissionRequestSender;
use futures::channel::mpsc::{self, UnboundedReceiver, UnboundedSender};
use log::*;
use nymsphinx::chunking::fragment::FragmentIdentifier;
use nymsphinx::Delay as SphinxDelay;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use tokio::stream::StreamExt;
use tokio::time::delay_queue::{self, Expired};
use tokio::time::Error as TimerError;
pub(crate) type ActionSender = UnboundedSender<Action>;
// The actual data being sent off as well as potential key to the delay queue
type PendingAckEntry = (Arc<PendingAcknowledgement>, Option<delay_queue::Key>);
// we can either:
// - have a completely new set of packets we just sent and need to create entries for
// - received an ack so we want to remove an entry
// - start a retransmission timer for sending the packet into the network (on either first try or retransmission)
// - update the internal sphinx delay of an expired packet
pub(crate) enum Action {
/// Inserts new `PendingAcknowledgement`s into the 'shared' state.
/// Initiated by `InputMessageListener`
InsertPending(Vec<PendingAcknowledgement>),
/// Removes given `PendingAcknowledgement` from the 'shared' state. Also cancels the retransmission timer.
/// Initiated by `AcknowledgementListener`
RemovePending(FragmentIdentifier),
/// Starts the retransmission timer on given `PendingAcknowledgement` with the `Duration` based on
/// its internal data.
/// Initiated by `SentNotificationListener`
/// Can also be initiated by `RetransmissionRequestListener` in the rare cases of invalid Topology.
StartTimer(FragmentIdentifier),
/// Updates the expected delay of given `PendingAcknowledgement` with the new provided `SphinxDelay`.
/// Initiated by `RetransmissionRequestListener`
UpdateDelay(FragmentIdentifier, SphinxDelay),
}
impl Action {
pub(crate) fn new_insert(pending_acks: Vec<PendingAcknowledgement>) -> Self {
Action::InsertPending(pending_acks)
}
pub(crate) fn new_remove(frag_id: FragmentIdentifier) -> Self {
Action::RemovePending(frag_id)
}
pub(crate) fn new_start_timer(frag_id: FragmentIdentifier) -> Self {
Action::StartTimer(frag_id)
}
pub(crate) fn new_update_delay(frag_id: FragmentIdentifier, delay: SphinxDelay) -> Self {
Action::UpdateDelay(frag_id, delay)
}
}
/// Configurable parameters of the `ActionController`
pub(super) struct Config {
/// Given ack timeout in the form a * BASE_DELAY + b, it specifies the additive part `b`
ack_wait_addition: Duration,
/// Given ack timeout in the form a * BASE_DELAY + b, it specifies the multiplier `a`
ack_wait_multiplier: f64,
}
impl Config {
pub(super) fn new(ack_wait_addition: Duration, ack_wait_multiplier: f64) -> Self {
Config {
ack_wait_addition,
ack_wait_multiplier,
}
}
}
pub(super) struct ActionController {
/// Configurable parameters of the `ActionController`
config: Config,
/// Contains a map between `FragmentIdentifier` and its full `PendingAcknowledgement` as well as
/// key to its `AckDelayQueue` entry if it was started.
pending_acks_data: HashMap<FragmentIdentifier, PendingAckEntry>,
// This structure ensures that we will EITHER handle expired timer or a received action and NEVER both
// at the same time hence getting rid of one possible race condition that we suffered from in the
// previous version.
/// DelayQueue with all `PendingAcknowledgement` that are waiting to be either received or
/// retransmitted if their timer fires up.
pending_acks_timers: AckDelayQueue<FragmentIdentifier>,
/// Channel for receiving `Action`s from other modules.
incoming_actions: UnboundedReceiver<Action>,
/// Channel for notifying `RetransmissionRequestListener` about expired acknowledgements.
retransmission_sender: RetransmissionRequestSender,
}
impl ActionController {
pub(super) fn new(
config: Config,
retransmission_sender: RetransmissionRequestSender,
) -> (Self, ActionSender) {
let (sender, receiver) = mpsc::unbounded();
(
ActionController {
config,
pending_acks_data: HashMap::new(),
pending_acks_timers: AckDelayQueue::new(),
incoming_actions: receiver,
retransmission_sender,
},
sender,
)
}
fn handle_insert(&mut self, pending_acks: Vec<PendingAcknowledgement>) {
for pending_ack in pending_acks {
let frag_id = pending_ack.message_chunk.fragment_identifier();
trace!("{} is inserted", frag_id);
if self
.pending_acks_data
.insert(frag_id, (Arc::new(pending_ack), None))
.is_some()
{
panic!("Tried to insert duplicate pending ack")
}
}
}
fn handle_start_timer(&mut self, frag_id: FragmentIdentifier) {
trace!("{} is starting its timer", frag_id);
if let Some((pending_ack_data, queue_key)) = self.pending_acks_data.get_mut(&frag_id) {
if queue_key.is_some() {
// this branch should be IMPOSSIBLE under ANY condition. It would imply starting
// timer TWICE for the SAME PendingAcknowledgement
panic!("Tried to start an already started ack timer!")
}
let timeout = (pending_ack_data.delay.clone() * self.config.ack_wait_multiplier)
.to_duration()
+ self.config.ack_wait_addition;
let new_queue_key = self.pending_acks_timers.insert(frag_id, timeout);
*queue_key = Some(new_queue_key)
} else {
debug!(
"Tried to START TIMER on pending ack that is already gone! - {}",
frag_id
);
}
}
fn handle_remove(&mut self, frag_id: FragmentIdentifier) {
trace!("{} is getting removed", frag_id);
match self.pending_acks_data.remove(&frag_id) {
None => {
debug!(
"Tried to REMOVE pending ack that is already gone! - {}",
frag_id
);
}
Some((_, queue_key)) => {
if let Some(queue_key) = queue_key {
// there are no possible checks here, we must GUARANTEE that we NEVER try
// to remove an entry that doesn't exist (and we MUST GUARANTEE that
// we do not have a stale key)
self.pending_acks_timers.remove(&queue_key);
// remove timer
} else {
// I'm not 100% sure if having a `None` key is even possible here
// (REMOVE would have to be called before START TIMER),
debug!(
"Tried to REMOVE pending ack without TIMER active - {}",
frag_id
);
}
}
}
}
// initiated basically as a first step of retransmission. At first data has its delay updated
// (as new sphinx packet was created with new expected delivery time)
fn handle_update_delay(&mut self, frag_id: FragmentIdentifier, delay: SphinxDelay) {
trace!("{} is updating its delay", frag_id);
// TODO: is it possible to solve this without either locking or temporarily removing the value?
if let Some((pending_ack_data, queue_key)) = self.pending_acks_data.remove(&frag_id) {
// this Action is triggered by `RetransmissionRequestListener` which held the other potential
// reference to this Arc. HOWEVER, before the Action was pushed onto the queue, the reference
// was dropped hence this unwrap is safe.
let mut inner_data = Arc::try_unwrap(pending_ack_data).unwrap();
inner_data.update_delay(delay);
self.pending_acks_data
.insert(frag_id, (Arc::new(inner_data), queue_key));
} else {
debug!(
"Tried to UPDATE TIMER on pending ack that is already gone! - {}",
frag_id
);
}
}
// note: when the entry expires it's automatically removed from pending_acks_timers
fn handle_expired_ack_timer(
&mut self,
expired_ack: Result<Expired<FragmentIdentifier>, TimerError>,
) {
// I'm honestly not sure how to handle it, because getting it means other things in our
// system are already misbehaving. If we ever see this panic, then I guess we should worry
// about it. Perhaps just reschedule it at later point?
let frag_id = expired_ack
.expect("Tokio timer returned an error!")
.into_inner();
trace!("{} has expired", frag_id);
if let Some((pending_ack_data, queue_key)) = self.pending_acks_data.get_mut(&frag_id) {
if queue_key.is_none() {
// this branch should be IMPOSSIBLE under ANY condition. It would imply the timeout
// happened before it even started.
panic!("Ack expired before it was even scheduled!")
}
*queue_key = None;
// downgrading an arc and then upgrading vs cloning is difference of 30ns vs 15ns
// so it's literally a NO difference while it might prevent us from unnecessarily
// resending data (in maybe 1 in 1 million cases, but it's something)
self.retransmission_sender
.unbounded_send(Arc::downgrade(pending_ack_data))
.unwrap()
} else {
// this shouldn't cause any issues but shouldn't have happened to begin with!
error!("An already removed pending ack has expired")
}
}
fn process_action(&mut self, action: Action) {
match action {
Action::InsertPending(pending_acks) => self.handle_insert(pending_acks),
Action::RemovePending(frag_id) => self.handle_remove(frag_id),
Action::StartTimer(frag_id) => self.handle_start_timer(frag_id),
Action::UpdateDelay(frag_id, delay) => self.handle_update_delay(frag_id, delay),
}
}
pub(super) async fn run(&mut self) {
loop {
// at some point there will be a global shutdown signal here as the third option
tokio::select! {
// we NEVER expect for ANY sender to get dropped so unwrap here is fine
action = self.incoming_actions.next() => self.process_action(action.unwrap()),
// pending ack queue Stream CANNOT return a `None` so unwrap here is fine
expired_ack = self.pending_acks_timers.next() => self.handle_expired_ack_timer(expired_ack.unwrap())
}
}
}
}
clients/client-core/src/client/real_messages_control/acknowledgement_control/input_message_listener.rs
+200
View File
@@ -0,0 +1,200 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::action_controller::{Action, ActionSender};
use super::PendingAcknowledgement;
use crate::client::reply_key_storage::ReplyKeyStorage;
use crate::client::{
inbound_messages::{InputMessage, InputMessageReceiver},
real_messages_control::real_traffic_stream::{RealMessage, RealMessageSender},
topology_control::TopologyAccessor,
};
use futures::StreamExt;
use log::*;
use nymsphinx::anonymous_replies::ReplySURB;
use nymsphinx::preparer::MessagePreparer;
use nymsphinx::{acknowledgements::AckKey, addressing::clients::Recipient};
use rand::{CryptoRng, Rng};
use std::sync::Arc;
/// Module responsible for dealing with the received messages: splitting them, creating acknowledgements,
/// putting everything into sphinx packets, etc.
/// It also makes an initial sending attempt for said messages.
pub(super) struct InputMessageListener<R>
where
R: CryptoRng + Rng,
{
ack_key: Arc<AckKey>,
ack_recipient: Recipient,
input_receiver: InputMessageReceiver,
message_preparer: MessagePreparer<R>,
action_sender: ActionSender,
real_message_sender: RealMessageSender,
topology_access: TopologyAccessor,
reply_key_storage: ReplyKeyStorage,
}
impl<R> InputMessageListener<R>
where
R: CryptoRng + Rng,
{
pub(super) fn new(
ack_key: Arc<AckKey>,
ack_recipient: Recipient,
input_receiver: InputMessageReceiver,
message_preparer: MessagePreparer<R>,
action_sender: ActionSender,
real_message_sender: RealMessageSender,
topology_access: TopologyAccessor,
reply_key_storage: ReplyKeyStorage,
) -> Self {
InputMessageListener {
ack_key,
ack_recipient,
input_receiver,
message_preparer,
action_sender,
real_message_sender,
topology_access,
reply_key_storage,
}
}
// we require topology for replies to generate surb_acks
async fn handle_reply(&mut self, reply_surb: ReplySURB, data: Vec<u8>) -> Option<RealMessage> {
let topology_permit = self.topology_access.get_read_permit().await;
let topology = match topology_permit.try_get_valid_topology_ref(&self.ack_recipient, None) {
Some(topology_ref) => topology_ref,
None => {
warn!("Could not process the message - the network topology is invalid");
return None;
}
};
match self
.message_preparer
.prepare_reply_for_use(data, reply_surb, topology, &self.ack_key)
{
Ok((reply_id, sphinx_packet, first_hop)) => {
// TODO: later probably write pending ack here
// and deal with them....
// ... somehow
Some(RealMessage::new(first_hop, sphinx_packet, reply_id))
}
Err(err) => {
// TODO: should we have some mechanism to indicate to the user that the `reply_surb`
// could be reused since technically it wasn't used up here?
warn!("failed to deal with received reply surb - {:?}", err);
None
}
}
}
async fn handle_fresh_message(
&mut self,
recipient: Recipient,
content: Vec<u8>,
with_reply_surb: bool,
) -> Vec<RealMessage> {
let topology_permit = self.topology_access.get_read_permit().await;
let topology = match topology_permit
.try_get_valid_topology_ref(&self.ack_recipient, Some(&recipient))
{
Some(topology_ref) => topology_ref,
None => {
warn!("Could not process the message - the network topology is invalid");
return Vec::new();
}
};
// split the message, attach optional reply surb
let (split_message, reply_key) = self
.message_preparer
.prepare_and_split_message(content, with_reply_surb, topology)
.expect("somehow the topology was invalid after all!");
if let Some(reply_key) = reply_key {
self.reply_key_storage
.insert_encryption_key(reply_key)
.expect("Failed to insert surb reply key to the store!")
}
// encrypt chunks, put them inside sphinx packets and generate acks
let mut pending_acks = Vec::with_capacity(split_message.len());
let mut real_messages = Vec::with_capacity(split_message.len());
for message_chunk in split_message {
// we need to clone it because we need to keep it in memory in case we had to retransmit
// it. And then we'd need to recreate entire ACK again.
let chunk_clone = message_chunk.clone();
let prepared_fragment = self
.message_preparer
.prepare_chunk_for_sending(chunk_clone, topology, &self.ack_key, &recipient)
.unwrap();
real_messages.push(RealMessage::new(
prepared_fragment.first_hop_address,
prepared_fragment.sphinx_packet,
message_chunk.fragment_identifier(),
));
pending_acks.push(PendingAcknowledgement::new(
message_chunk,
prepared_fragment.total_delay,
recipient.clone(),
));
}
// tells the controller to put this into the hashmap
self.action_sender
.unbounded_send(Action::new_insert(pending_acks))
.unwrap();
real_messages
}
async fn on_input_message(&mut self, msg: InputMessage) {
let real_messages = match msg {
InputMessage::Fresh {
recipient,
data,
with_reply_surb,
} => {
self.handle_fresh_message(recipient, data, with_reply_surb)
.await
}
InputMessage::Reply { reply_surb, data } => {
if let Some(real_message) = self.handle_reply(reply_surb, data).await {
vec![real_message]
} else {
return;
}
}
};
// tells real message sender (with the poisson timer) to send this to the mix network
for real_message in real_messages {
self.real_message_sender
.unbounded_send(real_message)
.unwrap();
}
}
pub(super) async fn run(&mut self) {
debug!("Started InputMessageListener");
while let Some(input_msg) = self.input_receiver.next().await {
self.on_input_message(input_msg).await;
}
error!("TODO: error msg. Or maybe panic?")
}
}
clients/client-core/src/client/real_messages_control/acknowledgement_control/mod.rs
+288
View File
@@ -0,0 +1,288 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use self::{
acknowledgement_listener::AcknowledgementListener, action_controller::ActionController,
input_message_listener::InputMessageListener,
retransmission_request_listener::RetransmissionRequestListener,
sent_notification_listener::SentNotificationListener,
};
use super::real_traffic_stream::RealMessageSender;
use crate::client::reply_key_storage::ReplyKeyStorage;
use crate::client::{inbound_messages::InputMessageReceiver, topology_control::TopologyAccessor};
use futures::channel::mpsc;
use gateway_client::AcknowledgementReceiver;
use log::*;
use nymsphinx::{
acknowledgements::AckKey,
addressing::clients::Recipient,
chunking::fragment::{Fragment, FragmentIdentifier},
preparer::MessagePreparer,
Delay as SphinxDelay,
};
use rand::{CryptoRng, Rng};
use std::{
sync::{Arc, Weak},
time::Duration,
};
use tokio::task::JoinHandle;
mod ack_delay_queue;
mod acknowledgement_listener;
mod action_controller;
mod input_message_listener;
mod retransmission_request_listener;
mod sent_notification_listener;
/// Channel used for indicating that the particular `Fragment` should be retransmitted.
type RetransmissionRequestSender = mpsc::UnboundedSender<Weak<PendingAcknowledgement>>;
/// Channel used for receiving data about particular `Fragment` that should be retransmitted.
type RetransmissionRequestReceiver = mpsc::UnboundedReceiver<Weak<PendingAcknowledgement>>;
/// Channel used for signalling that the particular `Fragment` (associated with the `FragmentIdentifier`)
/// is done being delayed and is about to be sent to the mix network.
pub(super) type SentPacketNotificationSender = mpsc::UnboundedSender<FragmentIdentifier>;
/// Channel used for receiving signals about the particular `Fragment` (associated with the `FragmentIdentifier`)
/// that it is about to be sent to tbe mix network and its timeout timer should be started.
type SentPacketNotificationReceiver = mpsc::UnboundedReceiver<FragmentIdentifier>;
/// Structure representing a data `Fragment` that is on-route to the specified `Recipient`
#[derive(Debug)]
pub(crate) struct PendingAcknowledgement {
message_chunk: Fragment,
delay: SphinxDelay,
recipient: Recipient,
}
impl PendingAcknowledgement {
/// Creates new instance of `PendingAcknowledgement` using the provided data.
fn new(message_chunk: Fragment, delay: SphinxDelay, recipient: Recipient) -> Self {
PendingAcknowledgement {
message_chunk,
delay,
recipient,
}
}
fn update_delay(&mut self, new_delay: SphinxDelay) {
self.delay = new_delay;
}
}
/// AcknowledgementControllerConnectors represents set of channels for communication with
/// other parts of the system in order to support acknowledgements and retransmission.
pub(super) struct AcknowledgementControllerConnectors {
/// Channel used for forwarding prepared sphinx messages into the poisson sender
/// to be sent to the mix network.
real_message_sender: RealMessageSender,
/// Channel used for receiving raw messages from a client. The messages need to be put
/// into sphinx packets first.
input_receiver: InputMessageReceiver,
/// Channel used for receiving notification about particular packet being sent off to the
/// mix network (i.e. it was done being delayed by whatever value was determined in the poisson
/// sender)
sent_notifier: SentPacketNotificationReceiver,
/// Channel used for receiving acknowledgements from the mix network.
ack_receiver: AcknowledgementReceiver,
}
impl AcknowledgementControllerConnectors {
pub(super) fn new(
real_message_sender: RealMessageSender,
input_receiver: InputMessageReceiver,
sent_notifier: SentPacketNotificationReceiver,
ack_receiver: AcknowledgementReceiver,
) -> Self {
AcknowledgementControllerConnectors {
real_message_sender,
input_receiver,
sent_notifier,
ack_receiver,
}
}
}
/// Configurable parameters of the `AcknowledgementController`
pub(super) struct Config {
/// Given ack timeout in the form a * BASE_DELAY + b, it specifies the additive part `b`
ack_wait_addition: Duration,
/// Given ack timeout in the form a * BASE_DELAY + b, it specifies the multiplier `a`
ack_wait_multiplier: f64,
/// Average delay an acknowledgement packet is going to get delayed at a single mixnode.
average_ack_delay: Duration,
/// Average delay a data packet is going to get delayed at a single mixnode.
average_packet_delay: Duration,
}
impl Config {
pub(super) fn new(
ack_wait_addition: Duration,
ack_wait_multiplier: f64,
average_ack_delay: Duration,
average_packet_delay: Duration,
) -> Self {
Config {
ack_wait_addition,
ack_wait_multiplier,
average_ack_delay,
average_packet_delay,
}
}
}
pub(super) struct AcknowledgementController<R>
where
R: CryptoRng + Rng,
{
acknowledgement_listener: Option<AcknowledgementListener>,
input_message_listener: Option<InputMessageListener<R>>,
retransmission_request_listener: Option<RetransmissionRequestListener<R>>,
sent_notification_listener: Option<SentNotificationListener>,
action_controller: Option<ActionController>,
}
impl<R> AcknowledgementController<R>
where
R: 'static + CryptoRng + Rng + Clone + Send,
{
pub(super) fn new(
config: Config,
rng: R,
topology_access: TopologyAccessor,
ack_key: Arc<AckKey>,
ack_recipient: Recipient,
reply_key_storage: ReplyKeyStorage,
connectors: AcknowledgementControllerConnectors,
) -> Self {
let (retransmission_tx, retransmission_rx) = mpsc::unbounded();
let action_config =
action_controller::Config::new(config.ack_wait_addition, config.ack_wait_multiplier);
let (action_controller, action_sender) =
ActionController::new(action_config, retransmission_tx);
let message_preparer = MessagePreparer::new(
rng,
ack_recipient.clone(),
config.average_packet_delay,
config.average_ack_delay,
);
// will listen for any acks coming from the network
let acknowledgement_listener = AcknowledgementListener::new(
Arc::clone(&ack_key),
connectors.ack_receiver,
action_sender.clone(),
);
// will listen for any new messages from the client
let input_message_listener = InputMessageListener::new(
Arc::clone(&ack_key),
ack_recipient.clone(),
connectors.input_receiver,
message_preparer.clone(),
action_sender.clone(),
connectors.real_message_sender.clone(),
topology_access.clone(),
reply_key_storage,
);
// will listen for any ack timeouts and trigger retransmission
let retransmission_request_listener = RetransmissionRequestListener::new(
Arc::clone(&ack_key),
ack_recipient,
message_preparer,
action_sender.clone(),
connectors.real_message_sender,
retransmission_rx,
topology_access,
);
// will listen for events indicating the packet was sent through the network so that
// the retransmission timer should be started.
let sent_notification_listener =
SentNotificationListener::new(connectors.sent_notifier, action_sender);
AcknowledgementController {
acknowledgement_listener: Some(acknowledgement_listener),
input_message_listener: Some(input_message_listener),
retransmission_request_listener: Some(retransmission_request_listener),
sent_notification_listener: Some(sent_notification_listener),
action_controller: Some(action_controller),
}
}
pub(super) async fn run(&mut self) {
let mut acknowledgement_listener = self.acknowledgement_listener.take().unwrap();
let mut input_message_listener = self.input_message_listener.take().unwrap();
let mut retransmission_request_listener =
self.retransmission_request_listener.take().unwrap();
let mut sent_notification_listener = self.sent_notification_listener.take().unwrap();
let mut action_controller = self.action_controller.take().unwrap();
// the below are log messages are errors as at the current stage we do not expect any of
// the task to ever finish. This will of course change once we introduce
// graceful shutdowns.
let ack_listener_fut = tokio::spawn(async move {
acknowledgement_listener.run().await;
error!("The acknowledgement listener has finished execution!");
acknowledgement_listener
});
let input_listener_fut = tokio::spawn(async move {
input_message_listener.run().await;
error!("The input listener has finished execution!");
input_message_listener
});
let retransmission_req_fut = tokio::spawn(async move {
retransmission_request_listener.run().await;
error!("The retransmission request listener has finished execution!");
retransmission_request_listener
});
let sent_notification_fut = tokio::spawn(async move {
sent_notification_listener.run().await;
error!("The sent notification listener has finished execution!");
sent_notification_listener
});
let action_controller_fut = tokio::spawn(async move {
action_controller.run().await;
error!("The controller has finished execution!");
action_controller
});
// technically we don't have to bring `AcknowledgementController` back to a valid state
// but we can do it, so why not? Perhaps it might be useful if we wanted to allow
// for restarts of certain modules without killing the entire process.
self.acknowledgement_listener = Some(ack_listener_fut.await.unwrap());
self.input_message_listener = Some(input_listener_fut.await.unwrap());
self.retransmission_request_listener = Some(retransmission_req_fut.await.unwrap());
self.sent_notification_listener = Some(sent_notification_fut.await.unwrap());
self.action_controller = Some(action_controller_fut.await.unwrap());
}
#[allow(dead_code)]
pub(super) fn start(mut self) -> JoinHandle<Self> {
tokio::spawn(async move {
self.run().await;
self
})
}
}
clients/client-core/src/client/real_messages_control/acknowledgement_control/retransmission_request_listener.rs
+141
View File
@@ -0,0 +1,141 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::action_controller::{Action, ActionSender};
use super::PendingAcknowledgement;
use super::RetransmissionRequestReceiver;
use crate::client::{
real_messages_control::real_traffic_stream::{RealMessage, RealMessageSender},
topology_control::TopologyAccessor,
};
use futures::StreamExt;
use log::*;
use nymsphinx::preparer::MessagePreparer;
use nymsphinx::{acknowledgements::AckKey, addressing::clients::Recipient};
use rand::{CryptoRng, Rng};
use std::sync::{Arc, Weak};
// responsible for packet retransmission upon fired timer
pub(super) struct RetransmissionRequestListener<R>
where
R: CryptoRng + Rng,
{
ack_key: Arc<AckKey>,
ack_recipient: Recipient,
message_preparer: MessagePreparer<R>,
action_sender: ActionSender,
real_message_sender: RealMessageSender,
request_receiver: RetransmissionRequestReceiver,
topology_access: TopologyAccessor,
}
impl<R> RetransmissionRequestListener<R>
where
R: CryptoRng + Rng,
{
pub(super) fn new(
ack_key: Arc<AckKey>,
ack_recipient: Recipient,
message_preparer: MessagePreparer<R>,
action_sender: ActionSender,
real_message_sender: RealMessageSender,
request_receiver: RetransmissionRequestReceiver,
topology_access: TopologyAccessor,
) -> Self {
RetransmissionRequestListener {
ack_key,
ack_recipient,
message_preparer,
action_sender,
real_message_sender,
request_receiver,
topology_access,
}
}
async fn on_retransmission_request(&mut self, timed_out_ack: Weak<PendingAcknowledgement>) {
let timed_out_ack = match timed_out_ack.upgrade() {
Some(timed_out_ack) => timed_out_ack,
None => {
debug!("We received an ack JUST as we were about to retransmit [1]");
return;
}
};
let packet_recipient = &timed_out_ack.recipient;
let chunk_clone = timed_out_ack.message_chunk.clone();
let frag_id = chunk_clone.fragment_identifier();
let topology_permit = self.topology_access.get_read_permit().await;
let topology_ref = match topology_permit
.try_get_valid_topology_ref(&self.ack_recipient, Some(packet_recipient))
{
Some(topology_ref) => topology_ref,
None => {
warn!("Could not retransmit the packet - the network topology is invalid");
// we NEED to start timer here otherwise we will have this guy permanently stuck in memory
self.action_sender
.unbounded_send(Action::new_start_timer(frag_id))
.unwrap();
return;
}
};
let prepared_fragment = self
.message_preparer
.prepare_chunk_for_sending(chunk_clone, topology_ref, &self.ack_key, packet_recipient)
.unwrap();
// if we have the ONLY strong reference to the ack data, it means it was removed from the
// pending acks
if Arc::strong_count(&timed_out_ack) == 1 {
// while we were messing with topology, wrapping data in sphinx, etc. we actually received
// this ack after all! no need to retransmit then
debug!("We received an ack JUST as we were about to retransmit [2]");
return;
}
// we no longer need the reference - let's drop it so that if somehow `UpdateTimer` action
// reached the controller before this function terminated, the controller would not panic.
drop(timed_out_ack);
let new_delay = prepared_fragment.total_delay;
// We know this update will be reflected by the `StartTimer` Action performed when this
// message is sent through the mix network.
// Reason being: UpdateTimer is now pushed onto the Action queue and `StartTimer` will
// only be pushed when the below `RealMessage` (which we are about to create)
// is sent to the `OutQueueControl` and has gone through its internal queue
// with the additional poisson delay.
// And since Actions are executed in order `UpdateTimer` will HAVE TO be executed before `StartTimer`
self.action_sender
.unbounded_send(Action::new_update_delay(frag_id, new_delay))
.unwrap();
// send to `OutQueueControl` to eventually send to the mix network
self.real_message_sender
.unbounded_send(RealMessage::new(
prepared_fragment.first_hop_address,
prepared_fragment.sphinx_packet,
frag_id,
))
.unwrap();
}
pub(super) async fn run(&mut self) {
debug!("Started RetransmissionRequestListener");
while let Some(timed_out_ack) = self.request_receiver.next().await {
self.on_retransmission_request(timed_out_ack).await;
}
error!("TODO: error msg. Or maybe panic?")
}
}
clients/client-core/src/client/real_messages_control/acknowledgement_control/sent_notification_listener.rs
+63
View File
@@ -0,0 +1,63 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use super::action_controller::{Action, ActionSender};
use super::SentPacketNotificationReceiver;
use futures::StreamExt;
use log::*;
use nymsphinx::chunking::fragment::{FragmentIdentifier, COVER_FRAG_ID};
/// Module responsible for starting up retransmission timers.
/// It is required because when we send our packet to the `real traffic stream` controlled
/// by a poisson timer, there's no guarantee the message will be sent immediately, so we might
/// accidentally fire retransmission way quicker than we should have.
pub(super) struct SentNotificationListener {
sent_notifier: SentPacketNotificationReceiver,
action_sender: ActionSender,
}
impl SentNotificationListener {
pub(super) fn new(
sent_notifier: SentPacketNotificationReceiver,
action_sender: ActionSender,
) -> Self {
SentNotificationListener {
sent_notifier,
action_sender,
}
}
async fn on_sent_message(&mut self, frag_id: FragmentIdentifier) {
if frag_id == COVER_FRAG_ID {
trace!("sent off a cover message - no need to start retransmission timer!");
return;
} else if frag_id.is_reply() {
debug!("sent off a reply message - no need to start retransmission timer!");
// TODO: probably there will need to be some extra procedure here, like it would
// be nice to know that our reply actually reached the recipient (i.e. we got the ack)
return;
}
self.action_sender
.unbounded_send(Action::new_start_timer(frag_id))
.unwrap();
}
pub(super) async fn run(&mut self) {
debug!("Started SentNotificationListener");
while let Some(frag_id) = self.sent_notifier.next().await {
self.on_sent_message(frag_id).await;
}
error!("TODO: error msg. Or maybe panic?")
}
}
clients/client-core/src/client/real_messages_control/mod.rs
+179
View File
@@ -0,0 +1,179 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// INPUT: InputMessage from user
// INPUT2: Acks from mix
// OUTPUT: MixMessage to mix traffic
use self::{
acknowledgement_control::AcknowledgementController, real_traffic_stream::OutQueueControl,
};
use crate::client::real_messages_control::acknowledgement_control::AcknowledgementControllerConnectors;
use crate::client::reply_key_storage::ReplyKeyStorage;
use crate::client::{
inbound_messages::InputMessageReceiver, mix_traffic::MixMessageSender,
topology_control::TopologyAccessor,
};
use futures::channel::mpsc;
use gateway_client::AcknowledgementReceiver;
use log::*;
use nymsphinx::acknowledgements::AckKey;
use nymsphinx::addressing::clients::Recipient;
use rand::{rngs::OsRng, CryptoRng, Rng};
use std::sync::Arc;
use std::time::Duration;
use tokio::runtime::Handle;
use tokio::task::JoinHandle;
mod acknowledgement_control;
mod real_traffic_stream;
// TODO: ack_key and self_recipient shouldn't really be part of this config
pub struct Config {
ack_key: Arc<AckKey>,
ack_wait_multiplier: f64,
ack_wait_addition: Duration,
self_recipient: Recipient,
average_packet_delay_duration: Duration,
average_ack_delay_duration: Duration,
average_message_sending_delay: Duration,
}
impl Config {
pub fn new(
ack_key: Arc<AckKey>,
ack_wait_multiplier: f64,
ack_wait_addition: Duration,
average_ack_delay_duration: Duration,
average_message_sending_delay: Duration,
average_packet_delay_duration: Duration,
self_recipient: Recipient,
) -> Self {
Config {
ack_key,
self_recipient,
average_packet_delay_duration,
average_ack_delay_duration,
average_message_sending_delay,
ack_wait_multiplier,
ack_wait_addition,
}
}
}
pub struct RealMessagesController<R>
where
R: CryptoRng + Rng,
{
out_queue_control: Option<OutQueueControl<R>>,
ack_control: Option<AcknowledgementController<R>>,
}
// obviously when we finally make shared rng that is on 'higher' level, this should become
// generic `R`
impl RealMessagesController<OsRng> {
pub fn new(
config: Config,
ack_receiver: AcknowledgementReceiver,
input_receiver: InputMessageReceiver,
mix_sender: MixMessageSender,
topology_access: TopologyAccessor,
reply_key_storage: ReplyKeyStorage,
) -> Self {
let rng = OsRng;
let (real_message_sender, real_message_receiver) = mpsc::unbounded();
let (sent_notifier_tx, sent_notifier_rx) = mpsc::unbounded();
let ack_controller_connectors = AcknowledgementControllerConnectors::new(
real_message_sender,
input_receiver,
sent_notifier_rx,
ack_receiver,
);
let ack_control_config = acknowledgement_control::Config::new(
config.ack_wait_addition,
config.ack_wait_multiplier,
config.average_ack_delay_duration,
config.average_packet_delay_duration,
);
let ack_control = AcknowledgementController::new(
ack_control_config,
rng,
topology_access.clone(),
Arc::clone(&config.ack_key),
config.self_recipient.clone(),
reply_key_storage,
ack_controller_connectors,
);
let out_queue_config = real_traffic_stream::Config::new(
config.average_ack_delay_duration,
config.average_packet_delay_duration,
config.average_message_sending_delay,
);
let out_queue_control = OutQueueControl::new(
out_queue_config,
Arc::clone(&config.ack_key),
sent_notifier_tx,
mix_sender,
real_message_receiver,
rng,
config.self_recipient,
topology_access,
);
RealMessagesController {
out_queue_control: Some(out_queue_control),
ack_control: Some(ack_control),
}
}
pub(super) async fn run(&mut self) {
let mut out_queue_control = self.out_queue_control.take().unwrap();
let mut ack_control = self.ack_control.take().unwrap();
// the below are log messages are errors as at the current stage we do not expect any of
// the task to ever finish. This will of course change once we introduce
// graceful shutdowns.
let out_queue_control_fut = tokio::spawn(async move {
out_queue_control.run_out_queue_control().await;
error!("The out queue controller has finished execution!");
out_queue_control
});
let ack_control_fut = tokio::spawn(async move {
ack_control.run().await;
error!("The acknowledgement controller has finished execution!");
ack_control
});
// technically we don't have to bring `RealMessagesController` back to a valid state
// but we can do it, so why not? Perhaps it might be useful if we wanted to allow
// for restarts of certain modules without killing the entire process.
self.out_queue_control = Some(out_queue_control_fut.await.unwrap());
self.ack_control = Some(ack_control_fut.await.unwrap());
}
// &Handle is only passed for consistency sake with other client modules, but I think
// when we get to refactoring, we should apply gateway approach and make it implicit
pub fn start(mut self, handle: &Handle) -> JoinHandle<Self> {
handle.spawn(async move {
self.run().await;
self
})
}
}
clients/client-core/src/client/real_messages_control/real_traffic_stream.rs
+265
View File
@@ -0,0 +1,265 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::client::mix_traffic::{MixMessage, MixMessageSender};
use crate::client::real_messages_control::acknowledgement_control::SentPacketNotificationSender;
use crate::client::topology_control::TopologyAccessor;
use futures::channel::mpsc;
use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt};
use log::*;
use nymsphinx::acknowledgements::AckKey;
use nymsphinx::addressing::{clients::Recipient, nodes::NymNodeRoutingAddress};
use nymsphinx::chunking::fragment::FragmentIdentifier;
use nymsphinx::cover::generate_loop_cover_packet;
use nymsphinx::utils::sample_poisson_duration;
use nymsphinx::SphinxPacket;
use rand::{CryptoRng, Rng};
use std::pin::Pin;
use std::sync::Arc;
use std::time::Duration;
use tokio::time;
/// Configurable parameters of the `OutQueueControl`
pub(crate) struct Config {
/// Average delay an acknowledgement packet is going to get delay at a single mixnode.
average_ack_delay: Duration,
/// Average delay a data packet is going to get delay at a single mixnode.
average_packet_delay: Duration,
/// Average delay between sending subsequent packets.
average_message_sending_delay: Duration,
}
impl Config {
pub(crate) fn new(
average_ack_delay: Duration,
average_packet_delay: Duration,
average_message_sending_delay: Duration,
) -> Self {
Config {
average_ack_delay,
average_packet_delay,
average_message_sending_delay,
}
}
}
pub(crate) struct OutQueueControl<R>
where
R: CryptoRng + Rng,
{
/// Configurable parameters of the `ActionController`
config: Config,
/// Key used to encrypt and decrypt content of an ACK packet.
ack_key: Arc<AckKey>,
/// Channel used for notifying of a real packet being sent out. Used to start up retransmission timer.
sent_notifier: SentPacketNotificationSender,
/// Internal state, determined by `average_message_sending_delay`,
/// used to keep track of when a next packet should be sent out.
next_delay: time::Delay,
/// Channel used for sending prepared sphinx packets to `MixTrafficController` that sends them
/// out to the network without any further delays.
mix_tx: MixMessageSender,
/// Channel used for receiving real, prepared, messages that must be first sufficiently delayed
/// before being sent out into the network.
real_receiver: RealMessageReceiver,
/// Represents full address of this client.
our_full_destination: Recipient,
/// Instance of a cryptographically secure random number generator.
rng: R,
/// Accessor to the common instance of network topology.
topology_access: TopologyAccessor,
}
pub(crate) struct RealMessage {
first_hop_address: NymNodeRoutingAddress,
packet: SphinxPacket,
fragment_id: FragmentIdentifier,
}
impl RealMessage {
pub(crate) fn new(
first_hop_address: NymNodeRoutingAddress,
packet: SphinxPacket,
fragment_id: FragmentIdentifier,
) -> Self {
RealMessage {
first_hop_address,
packet,
fragment_id,
}
}
}
// messages are already prepared, etc. the real point of it is to forward it to mix_traffic
// after sufficient delay
pub(crate) type RealMessageSender = mpsc::UnboundedSender<RealMessage>;
type RealMessageReceiver = mpsc::UnboundedReceiver<RealMessage>;
pub(crate) enum StreamMessage {
Cover,
Real(RealMessage),
}
impl<R> Stream for OutQueueControl<R>
where
R: CryptoRng + Rng + Unpin,
{
type Item = StreamMessage;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
// it is not yet time to return a message
if Pin::new(&mut self.next_delay).poll(cx).is_pending() {
return Poll::Pending;
};
// we know it's time to send a message, so let's prepare delay for the next one
// Get the `now` by looking at the current `delay` deadline
let avg_delay = self.config.average_message_sending_delay;
let now = self.next_delay.deadline();
let next_poisson_delay = sample_poisson_duration(&mut self.rng, avg_delay);
// The next interval value is `next_poisson_delay` after the one that just
// yielded.
let next = now + next_poisson_delay;
self.next_delay.reset(next);
// decide what kind of message to send
match Pin::new(&mut self.real_receiver).poll_next(cx) {
// in the case our real message channel stream was closed, we should also indicate we are closed
// (and whoever is using the stream should panic)
Poll::Ready(None) => Poll::Ready(None),
// if there's an actual message - return it
Poll::Ready(Some(real_message)) => Poll::Ready(Some(StreamMessage::Real(real_message))),
// otherwise construct a dummy one
Poll::Pending => Poll::Ready(Some(StreamMessage::Cover)),
}
}
}
impl<R> OutQueueControl<R>
where
R: CryptoRng + Rng + Unpin,
{
pub(crate) fn new(
config: Config,
ack_key: Arc<AckKey>,
sent_notifier: SentPacketNotificationSender,
mix_tx: MixMessageSender,
real_receiver: RealMessageReceiver,
rng: R,
our_full_destination: Recipient,
topology_access: TopologyAccessor,
) -> Self {
OutQueueControl {
config,
ack_key,
sent_notifier,
next_delay: time::delay_for(Default::default()),
mix_tx,
real_receiver,
our_full_destination,
rng,
topology_access,
}
}
async fn on_message(&mut self, next_message: StreamMessage) {
trace!("created new message");
let next_message = match next_message {
StreamMessage::Cover => {
// TODO for way down the line: in very rare cases (during topology update) we might have
// to wait a really tiny bit before actually obtaining the permit hence messing with our
// poisson delay, but is it really a problem?
let topology_permit = self.topology_access.get_read_permit().await;
// the ack is sent back to ourselves (and then ignored)
let topology_ref_option = topology_permit.try_get_valid_topology_ref(
&self.our_full_destination,
Some(&self.our_full_destination),
);
if topology_ref_option.is_none() {
warn!(
"No valid topology detected - won't send any loop cover message this time"
);
return;
}
let topology_ref = topology_ref_option.unwrap();
let cover_message = generate_loop_cover_packet(
&mut self.rng,
topology_ref,
&*self.ack_key,
&self.our_full_destination,
self.config.average_ack_delay,
self.config.average_packet_delay,
)
.expect("Somehow failed to generate a loop cover message with a valid topology");
MixMessage::new(cover_message.0, cover_message.1)
}
StreamMessage::Real(real_message) => {
// well technically the message was not sent just yet, but now it's up to internal
// queues and client load rather than the required delay. So realistically we can treat
// whatever is about to happen as negligible additional delay.
trace!(
"{} is about to get sent to the mixnet",
real_message.fragment_id
);
self.sent_notifier
.unbounded_send(real_message.fragment_id)
.unwrap();
MixMessage::new(real_message.first_hop_address, real_message.packet)
}
};
// if this one fails, there's no retrying because it means that either:
// - we run out of memory
// - the receiver channel is closed
// in either case there's no recovery and we can only panic
self.mix_tx.unbounded_send(next_message).unwrap();
// JS: Not entirely sure why or how it fixes stuff, but without the yield call,
// the UnboundedReceiver [of mix_rx] will not get a chance to read anything
// JS2: Basically it was the case that with high enough rate, the stream had already a next value
// ready and hence was immediately re-scheduled causing other tasks to be starved;
// yield makes it go back the scheduling queue regardless of its value availability
tokio::task::yield_now().await;
}
pub(crate) async fn run_out_queue_control(&mut self) {
// we should set initial delay only when we actually start the stream
self.next_delay = time::delay_for(sample_poisson_duration(
&mut self.rng,
self.config.average_message_sending_delay,
));
debug!("Starting out queue controller...");
while let Some(next_message) = self.next().await {
self.on_message(next_message).await;
}
}
}
clients/client-core/src/client/received_buffer.rs
+374
View File
@@ -0,0 +1,374 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::client::reply_key_storage::ReplyKeyStorage;
use crypto::asymmetric::encryption;
use crypto::symmetric::stream_cipher;
use crypto::Digest;
use futures::channel::mpsc;
use futures::lock::Mutex;
use futures::StreamExt;
use gateway_client::MixnetMessageReceiver;
use log::*;
use nymsphinx::anonymous_replies::{encryption_key::EncryptionKeyDigest, SURBEncryptionKey};
use nymsphinx::params::{ReplySURBEncryptionAlgorithm, ReplySURBKeyDigestAlgorithm};
use nymsphinx::receiver::{MessageReceiver, MessageRecoveryError, ReconstructedMessage};
use std::collections::HashSet;
use std::sync::Arc;
use tokio::runtime::Handle;
use tokio::task::JoinHandle;
// Buffer Requests to say "hey, send any reconstructed messages to this channel"
// or to say "hey, I'm going offline, don't send anything more to me. Just buffer them instead"
pub type ReceivedBufferRequestSender = mpsc::UnboundedSender<ReceivedBufferMessage>;
pub type ReceivedBufferRequestReceiver = mpsc::UnboundedReceiver<ReceivedBufferMessage>;
// The channel set for the above
pub type ReconstructedMessagesSender = mpsc::UnboundedSender<Vec<ReconstructedMessage>>;
pub type ReconstructedMessagesReceiver = mpsc::UnboundedReceiver<Vec<ReconstructedMessage>>;
struct ReceivedMessagesBufferInner {
messages: Vec<ReconstructedMessage>,
local_encryption_keypair: Arc<encryption::KeyPair>,
// TODO: looking how it 'looks' here, perhaps `MessageReceiver` should be renamed to something
// else instead.
message_receiver: MessageReceiver,
message_sender: Option<ReconstructedMessagesSender>,
// TODO: this will get cleared upon re-running the client
// but perhaps it should be changed to include timestamps of when the message was reconstructed
// and every now and then remove ids older than X
recently_reconstructed: HashSet<i32>,
}
impl ReceivedMessagesBufferInner {
fn process_received_fragment(&mut self, raw_fragment: Vec<u8>) -> Option<ReconstructedMessage> {
let fragment_data = match self
.message_receiver
.recover_plaintext(self.local_encryption_keypair.private_key(), raw_fragment)
{
Err(e) => {
warn!("failed to recover fragment data: {:?}. The whole underlying message might be corrupted and unrecoverable!", e);
return None;
}
Ok(frag_data) => frag_data,
};
if nymsphinx::cover::is_cover(&fragment_data) {
trace!("The message was a loop cover message! Skipping it");
return None;
}
let fragment = match self.message_receiver.recover_fragment(&fragment_data) {
Err(e) => {
warn!("failed to recover fragment from raw data: {:?}. The whole underlying message might be corrupted and unrecoverable!", e);
return None;
}
Ok(frag) => frag,
};
if self.recently_reconstructed.contains(&fragment.id()) {
debug!("Received a chunk of already re-assembled message ({:?})! It probably got here because the ack got lost", fragment.id());
return None;
}
// if we returned an error the underlying message is malformed in some way
match self.message_receiver.insert_new_fragment(fragment) {
Err(err) => match err {
MessageRecoveryError::MalformedReconstructedMessage(message_sets) => {
// TODO: should we really insert reconstructed sets? could this be abused for some attack?
for set_id in message_sets {
if !self.recently_reconstructed.insert(set_id) {
// or perhaps we should even panic at this point?
error!("Reconstructed another message containing already used set id!")
}
}
None
}
_ => unreachable!(
"no other error kind should have been returned here! If so, it's a bug!"
),
},
Ok(reconstruction_result) => match reconstruction_result {
Some((reconstructed_message, used_sets)) => {
for set_id in used_sets {
if !self.recently_reconstructed.insert(set_id) {
// or perhaps we should even panic at this point?
error!("Reconstructed another message containing already used set id!")
}
}
Some(reconstructed_message)
}
None => None,
},
}
}
}
#[derive(Debug, Clone)]
// Note: you should NEVER create more than a single instance of this using 'new()'.
// You should always use .clone() to create additional instances
struct ReceivedMessagesBuffer {
inner: Arc<Mutex<ReceivedMessagesBufferInner>>,
/// Storage containing keys to all [`ReplySURB`]s ever sent out that we did not receive back.
// There's no need to put it behind a Mutex since it's already properly concurrent
reply_key_storage: ReplyKeyStorage,
}
impl ReceivedMessagesBuffer {
fn new(
local_encryption_keypair: Arc<encryption::KeyPair>,
reply_key_storage: ReplyKeyStorage,
) -> Self {
ReceivedMessagesBuffer {
inner: Arc::new(Mutex::new(ReceivedMessagesBufferInner {
messages: Vec::new(),
local_encryption_keypair,
message_receiver: MessageReceiver::new(),
message_sender: None,
recently_reconstructed: HashSet::new(),
})),
reply_key_storage,
}
}
async fn disconnect_sender(&mut self) {
let mut guard = self.inner.lock().await;
if guard.message_sender.is_none() {
// in theory we could just ignore it, but that situation should have never happened
// in the first place, so this way we at least know we have an important bug to fix
panic!("trying to disconnect non-existent sender!")
}
guard.message_sender = None;
}
async fn connect_sender(&mut self, sender: ReconstructedMessagesSender) {
let mut guard = self.inner.lock().await;
if guard.message_sender.is_some() {
// in theory we could just ignore it, but that situation should have never happened
// in the first place, so this way we at least know we have an important bug to fix
panic!("trying overwrite an existing sender!")
}
// while we're at it, also empty the buffer if we happened to receive anything while
// no sender was connected
let stored_messages = std::mem::replace(&mut guard.messages, Vec::new());
if !stored_messages.is_empty() {
if let Err(err) = sender.unbounded_send(stored_messages) {
error!(
"The sender channel we just received is already invalidated - {:?}",
err
);
// put the values back to the buffer
// the returned error has two fields: err: SendError and val: T,
// where val is the value that was failed to get sent;
// it's returned by the `into_inner` call
guard.messages = err.into_inner();
return;
}
}
guard.message_sender = Some(sender);
}
async fn add_reconstructed_messages(&mut self, msgs: Vec<ReconstructedMessage>) {
debug!("Adding {:?} new messages to the buffer!", msgs.len());
trace!("Adding new messages to the buffer! {:?}", msgs);
self.inner.lock().await.messages.extend(msgs)
}
fn process_received_reply(
reply_ciphertext: &[u8],
reply_key: SURBEncryptionKey,
) -> Option<ReconstructedMessage> {
let zero_iv = stream_cipher::zero_iv::<ReplySURBEncryptionAlgorithm>();
let mut reply_msg = stream_cipher::decrypt::<ReplySURBEncryptionAlgorithm>(
&reply_key.inner(),
&zero_iv,
reply_ciphertext,
);
if let Err(err) = MessageReceiver::remove_padding(&mut reply_msg) {
warn!("Received reply had malformed padding! - {:?}", err);
None
} else {
// TODO: perhaps having to say it doesn't have a surb an indication the type should be changed?
Some(ReconstructedMessage {
message: reply_msg,
reply_surb: None,
})
}
}
async fn handle_new_received(&mut self, msgs: Vec<Vec<u8>>) {
debug!(
"Processing {:?} new message that might get added to the buffer!",
msgs.len()
);
let mut completed_messages = Vec::new();
let mut inner_guard = self.inner.lock().await;
let reply_surb_digest_size = ReplySURBKeyDigestAlgorithm::output_size();
// first check if this is a reply or a chunked message
// TODO: verify with @AP if this way of doing it is safe or whether it could
// cause some attacks due to, I don't know, stupid edge case collisions?
// Update: this DOES introduce a possible leakage: https://github.com/nymtech/nym/issues/296
for msg in msgs {
let possible_key_digest =
EncryptionKeyDigest::clone_from_slice(&msg[..reply_surb_digest_size]);
// check first `HasherOutputSize` bytes if they correspond to known encryption key
// if yes - this is a reply message
// TODO: this might be a bottleneck - since the keys are stored on disk we, presumably,
// are doing a disk operation every single received fragment
if let Some(reply_encryption_key) = self
.reply_key_storage
.get_and_remove_encryption_key(possible_key_digest)
.expect("storage operation failed!")
{
if let Some(completed_message) = Self::process_received_reply(
&msg[reply_surb_digest_size..],
reply_encryption_key,
) {
completed_messages.push(completed_message)
}
} else {
// otherwise - it's a 'normal' message
if let Some(completed_message) = inner_guard.process_received_fragment(msg) {
completed_messages.push(completed_message)
}
}
}
if !completed_messages.is_empty() {
if let Some(sender) = &inner_guard.message_sender {
trace!("Sending reconstructed messages to announced sender");
if let Err(err) = sender.unbounded_send(completed_messages) {
warn!("The reconstructed message receiver went offline without explicit notification (relevant error: - {:?})", err);
// make sure to drop the lock to not deadlock
// (it is required by `add_reconstructed_messages`)
inner_guard.message_sender = None;
drop(inner_guard);
self.add_reconstructed_messages(err.into_inner()).await;
}
} else {
// make sure to drop the lock to not deadlock
// (it is required by `add_reconstructed_messages`)
drop(inner_guard);
trace!("No sender available - buffering reconstructed messages");
self.add_reconstructed_messages(completed_messages).await;
}
}
}
}
pub enum ReceivedBufferMessage {
// Signals a websocket connection (or a native implementation) was established and we should stop buffering messages,
// and instead send them directly to the received channel
ReceiverAnnounce(ReconstructedMessagesSender),
// Explicit signal that Receiver connection will no longer accept messages
ReceiverDisconnect,
}
struct RequestReceiver {
received_buffer: ReceivedMessagesBuffer,
query_receiver: ReceivedBufferRequestReceiver,
}
impl RequestReceiver {
fn new(
received_buffer: ReceivedMessagesBuffer,
query_receiver: ReceivedBufferRequestReceiver,
) -> Self {
RequestReceiver {
received_buffer,
query_receiver,
}
}
fn start(mut self, handle: &Handle) -> JoinHandle<()> {
handle.spawn(async move {
while let Some(request) = self.query_receiver.next().await {
match request {
ReceivedBufferMessage::ReceiverAnnounce(sender) => {
self.received_buffer.connect_sender(sender).await;
}
ReceivedBufferMessage::ReceiverDisconnect => {
self.received_buffer.disconnect_sender().await
}
}
}
})
}
}
struct FragmentedMessageReceiver {
received_buffer: ReceivedMessagesBuffer,
mixnet_packet_receiver: MixnetMessageReceiver,
}
impl FragmentedMessageReceiver {
fn new(
received_buffer: ReceivedMessagesBuffer,
mixnet_packet_receiver: MixnetMessageReceiver,
) -> Self {
FragmentedMessageReceiver {
received_buffer,
mixnet_packet_receiver,
}
}
fn start(mut self, handle: &Handle) -> JoinHandle<()> {
handle.spawn(async move {
while let Some(new_messages) = self.mixnet_packet_receiver.next().await {
self.received_buffer.handle_new_received(new_messages).await;
}
})
}
}
pub struct ReceivedMessagesBufferController {
fragmented_message_receiver: FragmentedMessageReceiver,
request_receiver: RequestReceiver,
}
impl ReceivedMessagesBufferController {
pub fn new(
local_encryption_keypair: Arc<encryption::KeyPair>,
query_receiver: ReceivedBufferRequestReceiver,
mixnet_packet_receiver: MixnetMessageReceiver,
reply_key_storage: ReplyKeyStorage,
) -> Self {
let received_buffer =
ReceivedMessagesBuffer::new(local_encryption_keypair, reply_key_storage);
ReceivedMessagesBufferController {
fragmented_message_receiver: FragmentedMessageReceiver::new(
received_buffer.clone(),
mixnet_packet_receiver,
),
request_receiver: RequestReceiver::new(received_buffer, query_receiver),
}
}
pub fn start(self, handle: &Handle) {
// TODO: should we do anything with JoinHandle(s) returned by start methods?
self.fragmented_message_receiver.start(handle);
self.request_receiver.start(handle);
}
}
clients/client-core/src/client/reply_key_storage.rs
+105
View File
@@ -0,0 +1,105 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use log::*;
use nymsphinx::anonymous_replies::{
encryption_key::EncryptionKeyDigest, encryption_key::Unsigned, SURBEncryptionKey,
SURBEncryptionKeySize,
};
use std::path::Path;
#[derive(Debug)]
pub enum ReplyKeyStorageError {
DbReadError(sled::Error),
DbWriteError(sled::Error),
DbOpenError(sled::Error),
}
/// Permanent storage for keys in all sent [`ReplySURB`]
///
/// Each sent out [`ReplySURB`] has a new key associated with it that is going to be used for
/// payload encryption. In order to decrypt whatever reply we receive, we need to know which
/// key to use for that purpose. We do it based on received `H(t)` which has to be included
/// with each reply.
/// Moreover, there is no restriction when the [`ReplySURB`] might get used so we need to
/// have a permanent storage for all the keys that we might ever see in the future.
#[derive(Debug, Clone)]
pub struct ReplyKeyStorage {
db: sled::Db,
}
impl ReplyKeyStorage {
pub fn load<P: AsRef<Path>>(path: P) -> Result<Self, ReplyKeyStorageError> {
let db = match sled::open(path) {
Err(e) => return Err(ReplyKeyStorageError::DbOpenError(e)),
Ok(db) => db,
};
Ok(ReplyKeyStorage { db })
}
fn read_encryption_key(&self, raw_key: sled::IVec) -> SURBEncryptionKey {
let key_bytes_ref = raw_key.as_ref();
// if this fails it means we have some database corruption and we
// absolutely can't continue
if key_bytes_ref.len() != SURBEncryptionKeySize::to_usize() {
error!("REPLY KEY STORAGE DATA CORRUPTION - ENCRYPTION KEY HAS INVALID LENGTH");
panic!("REPLY KEY STORAGE DATA CORRUPTION - ENCRYPTION KEY HAS INVALID LENGTH");
}
// this can only fail if the bytes have invalid length but we already asserted it
SURBEncryptionKey::try_from_bytes(key_bytes_ref).unwrap()
}
// TOOD: perhaps we could also store some part of original message here too?
pub fn insert_encryption_key(
&mut self,
encryption_key: SURBEncryptionKey,
) -> Result<(), ReplyKeyStorageError> {
let digest = encryption_key.compute_digest();
let insertion_result = match self.db.insert(digest.to_vec(), encryption_key.to_bytes()) {
Err(e) => Err(ReplyKeyStorageError::DbWriteError(e)),
Ok(existing_key) => {
if existing_key.is_some() {
panic!("HASH COLLISION DETECTED")
};
Ok(())
}
};
// TODO: perhaps we could implement some batching mechanism to avoid frequent flushes?
self.db.flush().unwrap();
insertion_result
}
// Once we use key once, we do not expect to use it again
pub fn get_and_remove_encryption_key(
&self,
key_digest: EncryptionKeyDigest,
) -> Result<Option<SURBEncryptionKey>, ReplyKeyStorageError> {
let removal_result = match self.db.remove(&key_digest.to_vec()) {
Err(e) => Err(ReplyKeyStorageError::DbReadError(e)),
Ok(existing_key) => {
Ok(existing_key.map(|existing_key| self.read_encryption_key(existing_key)))
}
};
// TODO: not sure how to feel about flushing it every single time here...
// same with insertion
self.db.flush().unwrap();
removal_result
}
}
clients/client-core/src/client/topology_control.rs
+204
View File
@@ -0,0 +1,204 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::built_info;
use directory_client::DirectoryClient;
use log::*;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::params::DEFAULT_NUM_MIX_HOPS;
use std::convert::TryInto;
use std::ops::Deref;
use std::sync::Arc;
use std::time;
use std::time::Duration;
use tokio::runtime::Handle;
use tokio::sync::{RwLock, RwLockReadGuard};
use tokio::task::JoinHandle;
use topology::NymTopology;
// I'm extremely curious why compiler NEVER complained about lack of Debug here before
#[derive(Debug)]
pub struct TopologyAccessorInner(Option<NymTopology>);
impl AsRef<Option<NymTopology>> for TopologyAccessorInner {
fn as_ref(&self) -> &Option<NymTopology> {
&self.0
}
}
impl TopologyAccessorInner {
fn new() -> Self {
TopologyAccessorInner(None)
}
fn update(&mut self, new: Option<NymTopology>) {
self.0 = new;
}
}
pub struct TopologyReadPermit<'a> {
permit: RwLockReadGuard<'a, TopologyAccessorInner>,
}
impl<'a> Deref for TopologyReadPermit<'a> {
type Target = TopologyAccessorInner;
fn deref(&self) -> &Self::Target {
&self.permit
}
}
impl<'a> TopologyReadPermit<'a> {
/// Using provided topology read permit, tries to get an immutable reference to the underlying
/// topology. For obvious reasons the lifetime of the topology reference is bound to the permit.
pub(super) fn try_get_valid_topology_ref(
&'a self,
ack_recipient: &Recipient,
packet_recipient: Option<&Recipient>,
) -> Option<&'a NymTopology> {
// Note: implicit deref with Deref for TopologyReadPermit is happening here
let topology_ref_option = self.permit.as_ref();
match topology_ref_option {
None => None,
Some(topology_ref) => {
// see if it's possible to route the packet to both gateways
if !topology_ref.can_construct_path_through(DEFAULT_NUM_MIX_HOPS)
|| !topology_ref.gateway_exists(&ack_recipient.gateway())
|| if let Some(packet_recipient) = packet_recipient {
!topology_ref.gateway_exists(&packet_recipient.gateway())
} else {
false
}
{
None
} else {
Some(topology_ref)
}
}
}
}
}
impl<'a> From<RwLockReadGuard<'a, TopologyAccessorInner>> for TopologyReadPermit<'a> {
fn from(read_permit: RwLockReadGuard<'a, TopologyAccessorInner>) -> Self {
TopologyReadPermit {
permit: read_permit,
}
}
}
#[derive(Clone, Debug)]
pub struct TopologyAccessor {
// `RwLock` *seems to* be the better approach for this as write access is only requested every
// few seconds, while reads are needed every single packet generated.
// However, proper benchmarks will be needed to determine if `RwLock` is indeed a better
// approach than a `Mutex`
inner: Arc<RwLock<TopologyAccessorInner>>,
}
impl TopologyAccessor {
pub fn new() -> Self {
TopologyAccessor {
inner: Arc::new(RwLock::new(TopologyAccessorInner::new())),
}
}
pub async fn get_read_permit(&self) -> TopologyReadPermit<'_> {
self.inner.read().await.into()
}
async fn update_global_topology(&mut self, new_topology: Option<NymTopology>) {
self.inner.write().await.update(new_topology);
}
// only used by the client at startup to get a slightly more reasonable error message
// (currently displays as unused because health checker is disabled due to required changes)
pub async fn is_routable(&self) -> bool {
match &self.inner.read().await.0 {
None => false,
Some(ref topology) => topology.can_construct_path_through(DEFAULT_NUM_MIX_HOPS),
}
}
}
pub struct TopologyRefresherConfig {
directory_server: String,
refresh_rate: time::Duration,
}
impl TopologyRefresherConfig {
pub fn new(directory_server: String, refresh_rate: time::Duration) -> Self {
TopologyRefresherConfig {
directory_server,
refresh_rate,
}
}
}
pub struct TopologyRefresher {
directory_client: directory_client::Client,
topology_accessor: TopologyAccessor,
refresh_rate: Duration,
}
impl TopologyRefresher {
pub fn new_directory_client(
cfg: TopologyRefresherConfig,
topology_accessor: TopologyAccessor,
) -> Self {
let directory_client_config = directory_client::Config::new(cfg.directory_server);
let directory_client = directory_client::Client::new(directory_client_config);
TopologyRefresher {
directory_client,
topology_accessor,
refresh_rate: cfg.refresh_rate,
}
}
async fn get_current_compatible_topology(&self) -> Option<NymTopology> {
match self.directory_client.get_topology().await {
Err(err) => {
error!("failed to get network topology! - {:?}", err);
None
}
Ok(topology) => {
let nym_topology: NymTopology = topology.try_into().ok()?;
Some(nym_topology.filter_system_version(built_info::PKG_VERSION))
}
}
}
pub async fn refresh(&mut self) {
trace!("Refreshing the topology");
let new_topology = self.get_current_compatible_topology().await;
self.topology_accessor
.update_global_topology(new_topology)
.await;
}
pub async fn is_topology_routable(&self) -> bool {
self.topology_accessor.is_routable().await
}
pub fn start(mut self, handle: &Handle) -> JoinHandle<()> {
handle.spawn(async move {
loop {
tokio::time::delay_for(self.refresh_rate).await;
self.refresh().await;
}
})
}
}
clients/native/src/config/mod.rs → clients/client-core/src/config/mod.rs
+191 -159
View File
@@ -12,51 +12,33 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::config::template::config_template;
use config::NymConfig;
use serde::{Deserialize, Deserializer, Serialize};
use serde::{Deserialize, Serialize};
use std::marker::PhantomData;
use std::path::PathBuf;
use std::time;
pub mod persistence;
mod template;
// 'CLIENT'
const DEFAULT_LISTENING_PORT: u16 = 1977;
const DEFAULT_DIRECTORY_SERVER: &str = "https://directory.nymtech.net";
// 'DEBUG'
// where applicable, the below are defined in milliseconds
const DEFAULT_LOOP_COVER_STREAM_AVERAGE_DELAY: u64 = 1000; // 1s
const DEFAULT_MESSAGE_STREAM_AVERAGE_DELAY: u64 = 500; // 0.5s
const DEFAULT_AVERAGE_PACKET_DELAY: u64 = 200; // 0.2s
const DEFAULT_TOPOLOGY_REFRESH_RATE: u64 = 30_000; // 30s
const DEFAULT_TOPOLOGY_RESOLUTION_TIMEOUT: u64 = 5_000; // 5s
const DEFAULT_ACK_WAIT_MULTIPLIER: f64 = 1.5;
const DEFAULT_GATEWAY_RESPONSE_TIMEOUT: u64 = 1_500; // 1.5s
// all delays are in milliseconds
const DEFAULT_ACK_WAIT_ADDITION: u64 = 800;
const DEFAULT_LOOP_COVER_STREAM_AVERAGE_DELAY: u64 = 1000;
const DEFAULT_MESSAGE_STREAM_AVERAGE_DELAY: u64 = 100;
const DEFAULT_AVERAGE_PACKET_DELAY: u64 = 100;
const DEFAULT_TOPOLOGY_REFRESH_RATE: u64 = 30_000;
const DEFAULT_TOPOLOGY_RESOLUTION_TIMEOUT: u64 = 5_000;
const DEFAULT_GATEWAY_RESPONSE_TIMEOUT: u64 = 1_500;
#[derive(Debug, Deserialize, PartialEq, Serialize, Clone, Copy)]
#[derive(Debug, Deserialize, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub enum SocketType {
WebSocket,
None,
}
impl SocketType {
pub fn from_string<S: Into<String>>(val: S) -> Self {
let mut upper = val.into();
upper.make_ascii_uppercase();
match upper.as_ref() {
"WEBSOCKET" | "WS" => SocketType::WebSocket,
_ => SocketType::None,
}
}
}
#[derive(Debug, Default, Deserialize, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Config {
client: Client,
socket: Socket,
pub struct Config<T> {
client: Client<T>,
#[serde(default)]
logging: Logging,
@@ -64,96 +46,94 @@ pub struct Config {
debug: Debug,
}
impl NymConfig for Config {
fn template() -> &'static str {
config_template()
}
fn config_file_name() -> String {
"config.toml".to_string()
}
fn default_root_directory() -> PathBuf {
dirs::home_dir()
.expect("Failed to evaluate $HOME value")
.join(".nym")
.join("clients")
}
fn root_directory(&self) -> PathBuf {
self.client.nym_root_directory.clone()
}
fn config_directory(&self) -> PathBuf {
self.client
.nym_root_directory
.join(&self.client.id)
.join("config")
}
fn data_directory(&self) -> PathBuf {
self.client
.nym_root_directory
.join(&self.client.id)
.join("data")
}
}
impl Config {
impl<T: NymConfig> Config<T> {
pub fn new<S: Into<String>>(id: S) -> Self {
Config::default().with_id(id)
let mut cfg = Config::default();
cfg.with_id(id);
cfg
}
// builder methods
pub fn with_id<S: Into<String>>(mut self, id: S) -> Self {
pub fn with_id<S: Into<String>>(&mut self, id: S) {
let id = id.into();
// identity key setting
if self.client.private_identity_key_file.as_os_str().is_empty() {
self.client.private_identity_key_file =
self::Client::default_private_identity_key_file(&id);
self::Client::<T>::default_private_identity_key_file(&id);
}
if self.client.public_identity_key_file.as_os_str().is_empty() {
self.client.public_identity_key_file =
self::Client::default_public_identity_key_file(&id);
self::Client::<T>::default_public_identity_key_file(&id);
}
// encryption key setting
if self
.client
.private_encryption_key_file
.as_os_str()
.is_empty()
{
self.client.private_encryption_key_file =
self::Client::<T>::default_private_encryption_key_file(&id);
}
if self
.client
.public_encryption_key_file
.as_os_str()
.is_empty()
{
self.client.public_encryption_key_file =
self::Client::<T>::default_public_encryption_key_file(&id);
}
// shared gateway key setting
if self.client.gateway_shared_key_file.as_os_str().is_empty() {
self.client.gateway_shared_key_file =
self::Client::<T>::default_gateway_shared_key_file(&id);
}
// ack key setting
if self.client.ack_key_file.as_os_str().is_empty() {
self.client.ack_key_file = self::Client::<T>::default_ack_key_file(&id);
}
if self
.client
.reply_encryption_key_store_path
.as_os_str()
.is_empty()
{
self.client.reply_encryption_key_store_path =
self::Client::<T>::default_reply_encryption_key_store_path(&id);
}
self.client.id = id;
self
}
pub fn with_gateway_id<S: Into<String>>(mut self, id: S) -> Self {
pub fn with_gateway_id<S: Into<String>>(&mut self, id: S) {
self.client.gateway_id = id.into();
self
}
pub fn with_gateway_auth_token<S: Into<String>>(mut self, token: S) -> Self {
self.client.gateway_authtoken = Some(token.into());
self
pub fn with_gateway_listener<S: Into<String>>(&mut self, gateway_listener: S) {
self.client.gateway_listener = gateway_listener.into();
}
pub fn with_custom_directory<S: Into<String>>(mut self, directory_server: S) -> Self {
pub fn with_custom_directory<S: Into<String>>(&mut self, directory_server: S) {
self.client.directory_server = directory_server.into();
self
}
pub fn with_socket(mut self, socket_type: SocketType) -> Self {
self.socket.socket_type = socket_type;
self
}
pub fn with_port(mut self, port: u16) -> Self {
self.socket.listening_port = port;
self
}
pub fn set_high_default_traffic_volume(mut self) -> Self {
pub fn set_high_default_traffic_volume(&mut self) {
self.debug.average_packet_delay = 10;
self.debug.loop_cover_traffic_average_delay = 20; // 50 cover messages / s
self.debug.message_sending_average_delay = 5; // 200 "real" messages / s
self
}
// getters
pub fn get_config_file_save_location(&self) -> PathBuf {
self.config_directory().join(Self::config_file_name())
pub fn get_id(&self) -> String {
self.client.id.clone()
}
pub fn get_nym_root_directory(&self) -> PathBuf {
self.client.nym_root_directory.clone()
}
pub fn get_private_identity_key_file(&self) -> PathBuf {
@@ -164,6 +144,26 @@ impl Config {
self.client.public_identity_key_file.clone()
}
pub fn get_private_encryption_key_file(&self) -> PathBuf {
self.client.private_encryption_key_file.clone()
}
pub fn get_public_encryption_key_file(&self) -> PathBuf {
self.client.public_encryption_key_file.clone()
}
pub fn get_gateway_shared_key_file(&self) -> PathBuf {
self.client.gateway_shared_key_file.clone()
}
pub fn get_reply_encryption_key_store_path(&self) -> PathBuf {
self.client.reply_encryption_key_store_path.clone()
}
pub fn get_ack_key_file(&self) -> PathBuf {
self.client.ack_key_file.clone()
}
pub fn get_directory_server(&self) -> String {
self.client.directory_server.clone()
}
@@ -172,16 +172,8 @@ impl Config {
self.client.gateway_id.clone()
}
pub fn get_gateway_auth_token(&self) -> Option<String> {
self.client.gateway_authtoken.clone()
}
pub fn get_socket_type(&self) -> SocketType {
self.socket.socket_type
}
pub fn get_listening_port(&self) -> u16 {
self.socket.listening_port
pub fn get_gateway_listener(&self) -> String {
self.client.gateway_listener.clone()
}
// Debug getters
@@ -189,6 +181,18 @@ impl Config {
time::Duration::from_millis(self.debug.average_packet_delay)
}
pub fn get_average_ack_delay(&self) -> time::Duration {
time::Duration::from_millis(self.debug.average_ack_delay)
}
pub fn get_ack_wait_multiplier(&self) -> f64 {
self.debug.ack_wait_multiplier
}
pub fn get_ack_wait_addition(&self) -> time::Duration {
time::Duration::from_millis(self.debug.ack_wait_addition)
}
pub fn get_loop_cover_traffic_average_delay(&self) -> time::Duration {
time::Duration::from_millis(self.debug.loop_cover_traffic_average_delay)
}
@@ -210,21 +214,19 @@ impl Config {
}
}
fn de_option_string<'de, D>(deserializer: D) -> Result<Option<String>, D::Error>
where
D: Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
if s.is_empty() {
Ok(None)
} else {
Ok(Some(s))
impl<T: NymConfig> Default for Config<T> {
fn default() -> Self {
Config {
client: Client::<T>::default(),
logging: Default::default(),
debug: Default::default(),
}
}
}
#[derive(Debug, Deserialize, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Client {
pub struct Client<T> {
/// ID specifies the human readable ID of this particular client.
id: String,
@@ -237,21 +239,40 @@ pub struct Client {
/// Path to file containing public identity key.
public_identity_key_file: PathBuf,
/// Path to file containing private encryption key.
private_encryption_key_file: PathBuf,
/// Path to file containing public encryption key.
public_encryption_key_file: PathBuf,
/// Path to file containing shared key derived with the specified gateway that is used
/// for all communication with it.
gateway_shared_key_file: PathBuf,
/// Path to file containing key used for encrypting and decrypting the content of an
/// acknowledgement so that nobody besides the client knows which packet it refers to.
ack_key_file: PathBuf,
/// Full path to file containing reply encryption keys of all reply-SURBs we have ever
/// sent but not received back.
reply_encryption_key_store_path: PathBuf,
/// gateway_id specifies ID of the gateway to which the client should send messages.
/// If initially omitted, a random gateway will be chosen from the available topology.
gateway_id: String,
/// A gateway specific, optional, base58 stringified authentication token used for
/// communication with particular gateway.
#[serde(deserialize_with = "de_option_string")]
gateway_authtoken: Option<String>,
/// Address of the gateway listener to which all client requests should be sent.
gateway_listener: String,
/// nym_home_directory specifies absolute path to the home nym Clients directory.
/// It is expected to use default value and hence .toml file should not redefine this field.
nym_root_directory: PathBuf,
#[serde(skip)]
super_struct: PhantomData<T>,
}
impl Default for Client {
impl<T: NymConfig> Default for Client<T> {
fn default() -> Self {
// there must be explicit checks for whether id is not empty later
Client {
@@ -259,36 +280,46 @@ impl Default for Client {
directory_server: DEFAULT_DIRECTORY_SERVER.to_string(),
private_identity_key_file: Default::default(),
public_identity_key_file: Default::default(),
private_encryption_key_file: Default::default(),
public_encryption_key_file: Default::default(),
gateway_shared_key_file: Default::default(),
ack_key_file: Default::default(),
reply_encryption_key_store_path: Default::default(),
gateway_id: "".to_string(),
gateway_authtoken: None,
nym_root_directory: Config::default_root_directory(),
gateway_listener: "".to_string(),
nym_root_directory: T::default_root_directory(),
super_struct: Default::default(),
}
}
}
impl Client {
impl<T: NymConfig> Client<T> {
fn default_private_identity_key_file(id: &str) -> PathBuf {
Config::default_data_directory(Some(id)).join("private_identity.pem")
T::default_data_directory(Some(id)).join("private_identity.pem")
}
fn default_public_identity_key_file(id: &str) -> PathBuf {
Config::default_data_directory(Some(id)).join("public_identity.pem")
T::default_data_directory(Some(id)).join("public_identity.pem")
}
}
#[derive(Debug, Deserialize, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Socket {
socket_type: SocketType,
listening_port: u16,
}
fn default_private_encryption_key_file(id: &str) -> PathBuf {
T::default_data_directory(Some(id)).join("private_encryption.pem")
}
impl Default for Socket {
fn default() -> Self {
Socket {
socket_type: SocketType::WebSocket,
listening_port: DEFAULT_LISTENING_PORT,
}
fn default_public_encryption_key_file(id: &str) -> PathBuf {
T::default_data_directory(Some(id)).join("public_encryption.pem")
}
fn default_gateway_shared_key_file(id: &str) -> PathBuf {
T::default_data_directory(Some(id)).join("gateway_shared.pem")
}
fn default_ack_key_file(id: &str) -> PathBuf {
T::default_data_directory(Some(id)).join("ack_key.pem")
}
fn default_reply_encryption_key_store_path(id: &str) -> PathBuf {
T::default_data_directory(Some(id)).join("reply_key_store")
}
}
@@ -312,6 +343,24 @@ pub struct Debug {
/// The provided value is interpreted as milliseconds.
average_packet_delay: u64,
/// The parameter of Poisson distribution determining how long, on average,
/// sent acknowledgement is going to be delayed at any given mix node.
/// So for an ack going through three mix nodes, on average, it will take three times this value
/// until the packet reaches its destination.
/// The provided value is interpreted as milliseconds.
average_ack_delay: u64,
/// Value multiplied with the expected round trip time of an acknowledgement packet before
/// it is assumed it was lost and retransmission of the data packet happens.
/// In an ideal network with 0 latency, this value would have been 1.
ack_wait_multiplier: f64,
/// Value added to the expected round trip time of an acknowledgement packet before
/// it is assumed it was lost and retransmission of the data packet happens.
/// In an ideal network with 0 latency, this value would have been 0.
/// The provided value is interpreted as milliseconds.
ack_wait_addition: u64,
/// The parameter of Poisson distribution determining how long, on average,
/// it is going to take for another loop cover traffic message to be sent.
/// The provided value is interpreted as milliseconds.
@@ -345,6 +394,9 @@ impl Default for Debug {
fn default() -> Self {
Debug {
average_packet_delay: DEFAULT_AVERAGE_PACKET_DELAY,
average_ack_delay: DEFAULT_AVERAGE_PACKET_DELAY,
ack_wait_multiplier: DEFAULT_ACK_WAIT_MULTIPLIER,
ack_wait_addition: DEFAULT_ACK_WAIT_ADDITION,
loop_cover_traffic_average_delay: DEFAULT_LOOP_COVER_STREAM_AVERAGE_DELAY,
message_sending_average_delay: DEFAULT_MESSAGE_STREAM_AVERAGE_DELAY,
gateway_response_timeout: DEFAULT_GATEWAY_RESPONSE_TIMEOUT,
@@ -353,23 +405,3 @@ impl Default for Debug {
}
}
}
#[cfg(test)]
mod client_config {
use super::*;
#[test]
fn after_saving_default_config_the_loaded_one_is_identical() {
// need to figure out how to do something similar but without touching the disk
// or the file system at all...
let temp_location = tempfile::tempdir().unwrap().path().join("config.toml");
let default_config = Config::default().with_id("foomp".to_string());
default_config
.save_to_file(Some(temp_location.clone()))
.unwrap();
let loaded_config = Config::load_from_file(Some(temp_location), None).unwrap();
assert_eq!(default_config, loaded_config);
}
}
clients/client-core/src/config/persistence/key_pathfinder.rs
+77
View File
@@ -0,0 +1,77 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::config::Config;
use config::NymConfig;
use std::path::{Path, PathBuf};
#[derive(Debug)]
pub struct ClientKeyPathfinder {
identity_private_key: PathBuf,
identity_public_key: PathBuf,
encryption_private_key: PathBuf,
encryption_public_key: PathBuf,
gateway_shared_key: PathBuf,
ack_key: PathBuf,
}
impl ClientKeyPathfinder {
pub fn new(id: String) -> Self {
let os_config_dir = dirs::config_dir().expect("no config directory known for this OS"); // grabs the OS default config dir
let config_dir = os_config_dir.join("nym").join("clients").join(id);
ClientKeyPathfinder {
identity_private_key: config_dir.join("private_identity.pem"),
identity_public_key: config_dir.join("public_identity.pem"),
encryption_private_key: config_dir.join("public_encryption.pem"),
encryption_public_key: config_dir.join("private_encryption.pem"),
gateway_shared_key: config_dir.join("gateway_shared.pem"),
ack_key: config_dir.join("ack_key.pem"),
}
}
pub fn new_from_config<T: NymConfig>(config: &Config<T>) -> Self {
ClientKeyPathfinder {
identity_private_key: config.get_private_identity_key_file(),
identity_public_key: config.get_public_identity_key_file(),
encryption_private_key: config.get_private_encryption_key_file(),
encryption_public_key: config.get_public_encryption_key_file(),
gateway_shared_key: config.get_gateway_shared_key_file(),
ack_key: config.get_ack_key_file(),
}
}
pub fn private_identity_key(&self) -> &Path {
&self.identity_private_key
}
pub fn public_identity_key(&self) -> &Path {
&self.identity_public_key
}
pub fn private_encryption_key(&self) -> &Path {
&self.encryption_private_key
}
pub fn public_encryption_key(&self) -> &Path {
&self.encryption_public_key
}
pub fn gateway_shared_key(&self) -> &Path {
&self.gateway_shared_key
}
pub fn ack_key(&self) -> &Path {
&self.ack_key
}
}
clients/client-core/src/config/persistence/mod.rs
+15
View File
@@ -0,0 +1,15 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub mod key_pathfinder;
clients/native/src/config/template.rs → clients/client-core/src/config/template.rs
+20 -2
View File
@@ -36,14 +36,31 @@ private_identity_key_file = '{{ client.private_identity_key_file }}'
# Path to file containing public identity key.
public_identity_key_file = '{{ client.public_identity_key_file }}'
# Path to file containing private encryption key.
private_encryption_key_file = '{{ client.private_encryption_key_file }}'
# Path to file containing public encryption key.
public_encryption_key_file = '{{ client.public_encryption_key_file }}'
# Full path to file containing reply encryption keys of all reply-SURBs we have ever
# sent but not received back.
reply_encryption_key_store_path = '{{ client.reply_encryption_key_store_path }}'
##### additional client config options #####
# ID of the gateway from which the client should be fetching messages.
gateway_id = '{{ client.gateway_id }}'
# A gateway specific, optional, base58 stringified authentication token used for
# Address of the gateway listener to which all client requests should be sent.
gateway_listener = '{{ client.gateway_listener }}'
# A gateway specific, optional, base58 stringified shared key used for
# communication with particular gateway.
gateway_authtoken = '{{ client.gateway_authtoken }}'
gateway_shared_key_file = '{{ client.gateway_shared_key_file }}'
# Path to file containing key used for encrypting and decrypting the content of an
# acknowledgement so that nobody besides the client knows which packet it refers to.
ack_key_file = '{{ client.ack_key_file }}'
##### advanced configuration options #####
@@ -77,6 +94,7 @@ listening_port = {{ socket.listening_port }}
[debug]
average_packet_delay = {{ debug.average_packet_delay }}
average_ack_delay = {{ debug.average_ack_delay }}
loop_cover_traffic_average_delay = {{ debug.loop_cover_traffic_average_delay }}
message_sending_average_delay = {{ debug.message_sending_average_delay }}
clients/client-core/src/lib.rs
+3
View File
@@ -0,0 +1,3 @@
pub mod built_info;
pub mod client;
pub mod config;
clients/native/Cargo.toml
+27 -23
View File
@@ -1,7 +1,7 @@
[package]
build = "build.rs"
name = "nym-client"
version = "0.7.0"
version = "0.8.0"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
edition = "2018"
@@ -12,35 +12,39 @@ name = "nym_client"
path = "src/lib.rs"
[dependencies]
bs58 = "0.3.0"
clap = "2.33.0"
curve25519-dalek = "2.0.0"
dirs = "2.0.2"
dotenv = "0.15.0"
futures = "0.3.1"
log = "0.4"
pem = "0.7.0"
pretty_env_logger = "0.3"
reqwest = "0.9.22"
serde = { version = "1.0.104", features = ["derive"] }
serde_json = "1.0.44"
tokio = { version = "0.2", features = ["full"] }
tokio-tungstenite = "0.10.1"
url = "2.1"
# dependencies to review:
futures = "0.3" # bunch of futures stuff, however, now that I think about it, it could perhaps be completely removed
# the AsyncRead, AsyncWrite, Stream, Sink, etc. traits could be used from tokio
# channels should really be replaced with crossbeam due to that implementation being more efficient
# and the single instance of abortable we have should really be refactored anyway
url = "2.1" # I think we should just replace it with String
clap = "2.33.0" # for the command line arguments
dirs = "3.0" # for determining default store directories in config
dotenv = "0.15.0" # for obtaining environmental variables (only used for RUST_LOG for time being)
log = "0.4" # self explanatory
pretty_env_logger = "0.3" # for formatting log messages
rand = {version = "0.7.3", features = ["wasm-bindgen"]} # rng-related traits + some rng implementation to use
serde = { version = "1.0.104", features = ["derive"] } # for config serialization/deserialization
sled = "0.33" # for storage of replySURB decryption keys
tokio = { version = "0.2", features = ["full"] } # async runtime
tokio-tungstenite = "0.11.0" # websocket
## internal
config = {path = "../../common/config"}
crypto = {path = "../../common/crypto"}
client-core = { path = "../client-core" }
config = { path = "../../common/config" }
crypto = { path = "../../common/crypto" }
directory-client = { path = "../../common/client-libs/directory-client" }
gateway-client = { path = "../../common/client-libs/gateway-client" }
gateway-requests = { path = "../../gateway/gateway-requests" }
mixnet-client = { path = "../../common/client-libs/mixnet-client" }
nymsphinx = { path = "../../common/nymsphinx" }
pemstore = {path = "../../common/pemstore"}
topology = {path = "../../common/topology" }
pemstore = { path = "../../common/pemstore" }
topology = { path = "../../common/topology" }
websocket-requests = { path = "websocket-requests" }
[build-dependencies]
built = "0.3.2"
built = "0.4.3"
[dev-dependencies]
tempfile = "3.1.0"
tempfile = "3.1.0"
serde_json = "1.0" # for the "textsend" example
clients/native/examples/go-examples/websocket/binarysend.go
+217
View File
@@ -0,0 +1,217 @@
package main
import (
"encoding/binary"
"fmt"
"io/ioutil"
"github.com/gorilla/websocket"
)
// request tags
const SEND_REQUEST_TAG = 0x00
const REPLY_REQUEST_TAG = 0x01
const SELF_ADDRESS_REQUEST_TAG = 0x02
// response tags
const ERROR_RESPONSE_TAG = 0x00
const RECEIVED_RESPONSE_TAG = 0x01
const SELF_ADDRESS_RESPONSE_TAG = 0x02
func makeSelfAddressRequest() []byte {
return []byte{SELF_ADDRESS_REQUEST_TAG}
}
func parseSelfAddressResponse(rawResponse []byte) []byte {
if len(rawResponse) != 97 || rawResponse[0] != SELF_ADDRESS_RESPONSE_TAG {
panic("Received invalid response")
}
return rawResponse[1:]
}
func makeSendRequest(recipient []byte, message []byte, withReplySurb bool) []byte {
messageLen := make([]byte, 8)
binary.BigEndian.PutUint64(messageLen, uint64(len(message)))
surbByte := byte(0)
if withReplySurb {
surbByte = 1
}
out := []byte{SEND_REQUEST_TAG, surbByte}
out = append(out, recipient...)
out = append(out, messageLen...)
out = append(out, message...)
return out
}
func makeReplyRequest(message []byte, replySURB []byte) []byte {
messageLen := make([]byte, 8)
binary.BigEndian.PutUint64(messageLen, uint64(len(message)))
surbLen := make([]byte, 8)
binary.BigEndian.PutUint64(surbLen, uint64(len(replySURB)))
out := []byte{REPLY_REQUEST_TAG}
out = append(out, surbLen...)
out = append(out, replySURB...)
out = append(out, messageLen...)
out = append(out, message...)
return out
}
func parseReceived(rawResponse []byte) ([]byte, []byte) {
if rawResponse[0] != RECEIVED_RESPONSE_TAG {
panic("Received invalid response!")
}
hasSurb := false
if rawResponse[1] == 1 {
hasSurb = true
} else if rawResponse[1] == 0 {
hasSurb = false
} else {
panic("malformed received response!")
}
data := rawResponse[2:]
if hasSurb {
surbLen := binary.BigEndian.Uint64(data[:8])
other := data[8:]
surb := other[:surbLen]
msgLen := binary.BigEndian.Uint64(other[surbLen : surbLen+8])
if len(other[surbLen+8:]) != int(msgLen) {
panic("invalid msg len")
}
msg := other[surbLen+8:]
return msg, surb
} else {
msgLen := binary.BigEndian.Uint64(data[:8])
other := data[8:]
if len(other) != int(msgLen) {
panic("invalid msg len")
}
msg := other[:msgLen]
return msg, nil
}
}
func sendWithoutReply() {
uri := "ws://localhost:1977"
conn, _, err := websocket.DefaultDialer.Dial(uri, nil)
if err != nil {
panic(err)
}
defer conn.Close()
selfAddressRequest := makeSelfAddressRequest()
if err = conn.WriteMessage(websocket.BinaryMessage, selfAddressRequest); err != nil {
panic(err)
}
_, receivedResponse, err := conn.ReadMessage()
if err != nil {
panic(err)
}
selfAddress := parseSelfAddressResponse(receivedResponse)
read_data, err := ioutil.ReadFile("dummy_file")
if err != nil {
panic(err)
}
sendRequest := makeSendRequest(selfAddress, read_data, false)
fmt.Printf("sending content of 'dummy file' over the mix network...\n")
if err = conn.WriteMessage(websocket.BinaryMessage, sendRequest); err != nil {
panic(err)
}
fmt.Printf("waiting to receive a message from the mix network...\n")
_, receivedResponse, err = conn.ReadMessage()
if err != nil {
panic(err)
}
fileData, replySURB := parseReceived(receivedResponse)
if replySURB != nil {
panic("did not expect a replySURB!")
}
fmt.Printf("writing the file back to the disk!\n")
ioutil.WriteFile("received_file_noreply", fileData, 0644)
}
func sendWithReply() {
uri := "ws://localhost:1977"
conn, _, err := websocket.DefaultDialer.Dial(uri, nil)
if err != nil {
panic(err)
}
defer conn.Close()
selfAddressRequest := makeSelfAddressRequest()
if err = conn.WriteMessage(websocket.BinaryMessage, selfAddressRequest); err != nil {
panic(err)
}
_, receivedResponse, err := conn.ReadMessage()
if err != nil {
panic(err)
}
selfAddress := parseSelfAddressResponse(receivedResponse)
readData, err := ioutil.ReadFile("dummy_file")
if err != nil {
panic(err)
}
sendRequest := makeSendRequest(selfAddress, readData, true)
fmt.Printf("sending content of 'dummy file' over the mix network...\n")
if err = conn.WriteMessage(websocket.BinaryMessage, sendRequest); err != nil {
panic(err)
}
fmt.Printf("waiting to receive a message from the mix network...\n")
_, receivedResponse, err = conn.ReadMessage()
if err != nil {
panic(err)
}
fileData, replySURB := parseReceived(receivedResponse)
fmt.Printf("writing the file back to the disk!\n")
ioutil.WriteFile("received_file_withreply", fileData, 0644)
replyMessage := []byte("hello from reply SURB! - thanks for sending me the file!")
replyRequest := makeReplyRequest(replyMessage, replySURB)
fmt.Printf("sending '%v' (using reply SURB) over the mix network...\n", string(replyMessage))
if err = conn.WriteMessage(websocket.BinaryMessage, replyRequest); err != nil {
panic(err)
}
fmt.Printf("waiting to receive a message from the mix network...\n")
_, receivedResponse, err = conn.ReadMessage()
if err != nil {
panic(err)
}
receivedMessage, replySURB := parseReceived(receivedResponse)
if replySURB != nil {
panic("did not expect a replySURB!")
}
fmt.Printf("received %v from the mix network!\n", string(receivedMessage))
}
func main() {
// sendWithoutReply()
sendWithReply()
}
clients/native/examples/go-examples/websocket/filesend.go
-76
View File
@@ -1,76 +0,0 @@
package main
import (
"encoding/json"
"fmt"
"github.com/btcsuite/btcutil/base58"
"github.com/gorilla/websocket"
"io/ioutil"
"strings"
)
func getSelfAddress(conn *websocket.Conn) string {
selfAddressRequest, err := json.Marshal(map[string]string{"type": "selfAddress"})
if err != nil {
panic(err)
}
if err = conn.WriteMessage(websocket.TextMessage, []byte(selfAddressRequest)); err != nil {
panic(err)
}
responseJSON := make(map[string]interface{})
err = conn.ReadJSON(&responseJSON)
if err != nil {
panic(err)
}
return responseJSON["address"].(string)
}
func main() {
uri := "ws://localhost:1977"
conn, _, err := websocket.DefaultDialer.Dial(uri, nil)
if err != nil {
panic(err)
}
defer conn.Close()
selfAddress := getSelfAddress(conn)
fmt.Printf("our address is: %v\n", selfAddress)
// we receive our address in string format of OUR_PUB_KEY @ OUR_GATE_PUB_KEY
// both keys are 32 bytes and we need to encode them as binary without the '@' sign
splitAddress := strings.Split(selfAddress, "@");
decodedDestination := base58.Decode(splitAddress[0])
decodedGateway := base58.Decode(splitAddress[1])
read_data, err := ioutil.ReadFile("dummy_file")
if err != nil {
panic(err)
}
payload := append(decodedDestination[:], append(decodedGateway[:], read_data[:]...)...)
fmt.Printf("sending content of 'dummy file' over the mix network...\n")
if err = conn.WriteMessage(websocket.BinaryMessage, payload); err != nil {
panic(err)
}
sendConfirmationJSON := make(map[string]interface{})
err = conn.ReadJSON(&sendConfirmationJSON)
if err != nil {
panic(err)
}
if sendConfirmationJSON["type"].(string) != "send" {
panic("invalid send confirmation")
}
fmt.Printf("waiting to receive a message from the mix network...\n")
_, receivedMessage, err := conn.ReadMessage()
if err != nil {
panic(err)
}
fmt.Printf("writing the file back to the disk!\n")
ioutil.WriteFile("received_file", receivedMessage, 0644)
}
clients/native/examples/go-examples/websocket/go.mod
+8
View File
@@ -0,0 +1,8 @@
module github.com/nymtech/nym/clients/native/examples/go
go 1.14
require (
github.com/btcsuite/btcutil v1.0.2 // indirect
github.com/gorilla/websocket v1.4.2 // indirect
)
clients/native/examples/go-examples/websocket/go.sum
+37
View File
@@ -0,0 +1,37 @@
github.com/aead/siphash v1.0.1/go.mod h1:Nywa3cDsYNNK3gaciGTWPwHt0wlpNV15vwmswBAUSII=
github.com/btcsuite/btcd v0.20.1-beta/go.mod h1:wVuoA8VJLEcwgqHBwHmzLRazpKxTv13Px/pDuV7OomQ=
github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f/go.mod h1:TdznJufoqS23FtqVCzL0ZqgP5MqXbb4fg/WgDys70nA=
github.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d/go.mod h1:+5NJ2+qvTyV9exUAL/rxXi3DcLg2Ts+ymUAY5y4NvMg=
github.com/btcsuite/btcutil v1.0.2 h1:9iZ1Terx9fMIOtq1VrwdqfsATL9MC2l8ZrUY6YZ2uts=
github.com/btcsuite/btcutil v1.0.2/go.mod h1:j9HUFwoQRsZL3V4n+qG+CUnEGHOarIxfC3Le2Yhbcts=
github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd/go.mod h1:HHNXQzUsZCxOoE+CPiyCTO6x34Zs86zZUiwtpXoGdtg=
github.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd/go.mod h1:F+uVaaLLH7j4eDXPRvw78tMflu7Ie2bzYOH4Y8rRKBY=
github.com/btcsuite/snappy-go v0.0.0-20151229074030-0bdef8d06723/go.mod h1:8woku9dyThutzjeg+3xrA5iCpBRH8XEEg3lh6TiUghc=
github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792/go.mod h1:ghJtEyQwv5/p4Mg4C0fgbePVuGr935/5ddU9Z3TmDRY=
github.com/btcsuite/winsvc v1.0.0/go.mod h1:jsenWakMcC0zFBFurPLEAyrnc/teJEM1O46fmI40EZs=
github.com/davecgh/go-spew v0.0.0-20171005155431-ecdeabc65495/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/gorilla/websocket v1.4.2 h1:+/TMaTYc4QFitKJxsQ7Yye35DkWvkdLcvGKqM+x0Ufc=
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/jessevdk/go-flags v0.0.0-20141203071132-1679536dcc89/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
github.com/jrick/logrotate v1.0.0/go.mod h1:LNinyqDIJnpAur+b8yyulnQw/wDuN1+BYKlTRt3OuAQ=
github.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23/go.mod h1:J+Gs4SYgM6CZQHDETBtE9HaSEkGmuNXF86RwHhHUvq4=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
golang.org/x/crypto v0.0.0-20170930174604-9419663f5a44/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200115085410-6d4e4cb37c7d/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
clients/native/examples/go-examples/websocket/textsend.go
+79 -12
View File
@@ -26,7 +26,7 @@ func getSelfAddress(conn *websocket.Conn) string {
return responseJSON["address"].(string)
}
func main() {
func sendWithoutReply() {
message := "Hello Nym!"
uri := "ws://localhost:1977"
@@ -39,25 +39,21 @@ func main() {
selfAddress := getSelfAddress(conn)
fmt.Printf("our address is: %v\n", selfAddress)
sendRequest, err := json.Marshal(map[string]string{"type": "send", "recipient": selfAddress, "message": message})
sendRequest, err := json.Marshal(map[string]interface{}{
"type": "send",
"recipient": selfAddress,
"message": message,
"withReplySurb": false,
})
if err != nil {
panic(err)
}
fmt.Printf("sending '%v' over the mix network...\n", message)
fmt.Printf("sending '%v' (*without* reply SURB) over the mix network...\n", message)
if err = conn.WriteMessage(websocket.TextMessage, []byte(sendRequest)); err != nil {
panic(err)
}
sendConfirmationJSON := make(map[string]interface{})
err = conn.ReadJSON(&sendConfirmationJSON)
if err != nil {
panic(err)
}
if sendConfirmationJSON["type"].(string) != "send" {
panic("invalid send confirmation")
}
fmt.Printf("waiting to receive a message from the mix network...\n")
_, receivedMessage, err := conn.ReadMessage()
if err != nil {
@@ -65,3 +61,74 @@ func main() {
}
fmt.Printf("received %v from the mix network!\n", string(receivedMessage))
}
func sendWithReply() {
message := "Hello Nym!"
uri := "ws://localhost:1977"
conn, _, err := websocket.DefaultDialer.Dial(uri, nil)
if err != nil {
panic(err)
}
defer conn.Close()
selfAddress := getSelfAddress(conn)
fmt.Printf("our address is: %v\n", selfAddress)
sendRequest, err := json.Marshal(map[string]interface{}{
"type": "send",
"recipient": selfAddress,
"message": message,
"withReplySurb": true,
})
if err != nil {
panic(err)
}
fmt.Printf("sending '%v' (*with* reply SURB) over the mix network...\n", message)
if err = conn.WriteMessage(websocket.TextMessage, []byte(sendRequest)); err != nil {
panic(err)
}
fmt.Printf("waiting to receive a message from the mix network...\n")
_, receivedMessage, err := conn.ReadMessage()
if err != nil {
panic(err)
}
fmt.Printf("received %v from the mix network!\n", string(receivedMessage))
receivedMessageJson := make(map[string]interface{})
if err := json.Unmarshal(receivedMessage, &receivedMessageJson); err != nil {
panic(err)
}
// use the received surb to send an anonymous reply!
replySurb := receivedMessageJson["replySurb"]
replyMessage := "hello from reply SURB!"
reply, err := json.Marshal(map[string]interface{}{
"type": "reply",
"message": replyMessage,
"replySurb": replySurb,
})
if err != nil {
panic(err)
}
fmt.Printf("sending '%v' (using reply SURB) over the mix network...\n", replyMessage)
if err = conn.WriteMessage(websocket.TextMessage, []byte(reply)); err != nil {
panic(err)
}
fmt.Printf("waiting to receive a message from the mix network...\n")
_, receivedMessage, err = conn.ReadMessage()
if err != nil {
panic(err)
}
fmt.Printf("received %v from the mix network!\n", string(receivedMessage))
}
func main() {
// sendWithoutReply()
sendWithReply()
}
clients/native/examples/js-examples/websocket/package-lock.json
Generated
+1600 -2047
View File
File diff suppressed because it is too large Load Diff
clients/native/examples/js-examples/websocket/package.json
+1 -1
View File
@@ -22,7 +22,7 @@
"clean-webpack-plugin": "^3.0.0",
"webpack": "^4.42.1",
"webpack-cli": "^3.3.11",
"webpack-dev-server": "^3.10.3"
"webpack-dev-server": "^3.11.0"
},
"dependencies": {
"core-js": "^3.6.5",
clients/native/examples/js-examples/websocket/src/index.html
+3
View File
@@ -9,6 +9,9 @@
<body>
<label for="fname">Text to send: </label><input type="text" id="sendtext" name="sendtext" value="Hello mixnet!">
<input type="checkbox" id="with-surb" name="with-surb">
<label for="with-surb"> Attach Reply SURB </label><br>
<button id="send-button">Send</button>
<p>Send messages to the mixnet using the "send" button.</p>
clients/native/examples/js-examples/websocket/src/index.js
+89 -15
View File
@@ -1,11 +1,15 @@
var ourAddress;
var connection;
// Please note that this javascript is extremely bad, it's only purpose is to show some basic API calls, not how
// a proper application should have been written.
async function main() {
var port = '1977' // client websocket listens on 1977 by default, change if yours is different
var localClientUrl = "ws://127.0.0.1:" + port;
// Set up and handle websocket connection to our desktop client.
var connection = await connectWebsocket(localClientUrl).then(function (c) {
connection = await connectWebsocket(localClientUrl).then(function (c) {
return c;
}).catch(function (err) {
display("Websocket connection error. Is the client running with <pre>--connection-type WebSocket</pre> on port " + port + "?");
@@ -14,12 +18,12 @@ async function main() {
handleResponse(e);
};
sendSelfAddressRequest(connection);
sendSelfAddressRequest();
// Set up the send button
const sendButton = document.querySelector('#send-button');
sendButton.onclick = function () {
sendMessageToMixnet(connection);
sendMessageToMixnet();
}
}
@@ -31,28 +35,57 @@ function handleResponse(resp) {
try {
let response = JSON.parse(resp.data);
if (response.type == "error") {
displayJsonResponse("Server responded with error: " + response.message);
displayJsonResponseWithoutReply("Server responded with error: " + response.message);
} else if (response.type == "selfAddress") {
displayJsonResponse(response);
displayJsonResponseWithoutReply(response);
ourAddress = response.address;
display("Our address is: " + ourAddress + ", we will now send messages to ourself.");
} else if (response.type == "received") {
handleReceivedTextMessage(response)
}
} catch (_) {
displayJsonResponse(resp.data)
displayJsonResponseWithoutReply(resp.data)
}
}
function handleReceivedTextMessage(message) {
console.log("received a message!")
const text = message.message
const replySurb = message.replySurb
if (replySurb != null) {
displayJsonResponseWithReply(text, replySurb)
} else {
displayJsonResponseWithoutReply(text)
}
}
// Send a message to the mixnet.
function sendMessageToMixnet(connection) {
var sendText = document.getElementById("sendtext").value;
var message = {
function sendMessageToMixnet() {
const sendText = document.getElementById("sendtext").value;
const surbCheckbox = document.querySelector('#with-surb');
const attachReplySURB = surbCheckbox.checked;
const message = {
type: "send",
message: sendText,
recipient: ourAddress
recipient: ourAddress,
withReplySurb: attachReplySURB,
}
displayJsonSend(message);
displayJsonResponseWithoutReply(message);
connection.send(JSON.stringify(message));
}
function sendReplyMessageToMixnet(messageContent, replySurb) {
const message = {
type: "reply",
message: messageContent,
replySurb: replySurb,
}
displayJsonResponseWithoutReply(message);
connection.send(JSON.stringify(message));
}
@@ -61,7 +94,7 @@ function sendMessageToMixnet(connection) {
//
// In a real application, you might want to ensure that somebody else got your
// address so that they could send messages to you.
function sendSelfAddressRequest(connection) {
function sendSelfAddressRequest() {
var selfAddress = {
type: "selfAddress"
}
@@ -74,11 +107,52 @@ function display(message) {
}
function displayJsonSend(message) {
document.getElementById("output").innerHTML += "<p style='color: blue;'>sent >>> " + JSON.stringify(message) + "</p >";
let sendDiv = document.createElement("div")
let paragraph = document.createElement("p")
paragraph.setAttribute('style', 'color: blue')
let paragraphContent = document.createTextNode("sent >>> " + JSON.stringify(message))
paragraph.appendChild(paragraphContent)
sendDiv.appendChild(paragraph)
document.getElementById("output").appendChild(sendDiv)
}
function displayJsonResponse(message) {
document.getElementById("output").innerHTML += "<p style='color: green;'>received <<<" + JSON.stringify(message) + "</p >";
function displayJsonResponseWithoutReply(message) {
let receivedDiv = document.createElement("div")
let paragraph = document.createElement("p")
paragraph.setAttribute('style', 'color: green')
let paragraphContent = document.createTextNode("received >>> " + JSON.stringify(message) + "(NO REPLY AVAILABLE)")
paragraph.appendChild(paragraphContent)
receivedDiv.appendChild(paragraph)
document.getElementById("output").appendChild(receivedDiv)
}
function displayJsonResponseWithReply(message, replySurb) {
let replyBox = document.createElement("input")
replyBox.setAttribute('type', 'text');
replyBox.setAttribute('value', 'type your anonymous reply here!');
replyBox.setAttribute('size', 50);
let sendButton = document.createElement("button")
let buttonText = document.createTextNode("Send")
sendButton.appendChild(buttonText)
sendButton.onclick = () => {
sendReplyMessageToMixnet(replyBox.value, replySurb)
}
let receivedDiv = document.createElement("div")
let paragraph = document.createElement("p")
paragraph.setAttribute('style', 'color: green')
let paragraphContent = document.createTextNode("received >>> " + JSON.stringify(message) + "(HERE BE SURB)")
paragraph.appendChild(paragraphContent)
receivedDiv.appendChild(paragraph)
receivedDiv.appendChild(replyBox)
receivedDiv.appendChild(sendButton)
document.getElementById("output").appendChild(receivedDiv)
}
// Connect to a websocket.
clients/native/examples/js-examples/websocket/yarn.lock
-4330
View File
File diff suppressed because it is too large Load Diff
clients/native/examples/python-examples/websocket/binarysend.py
+139
View File
@@ -0,0 +1,139 @@
import asyncio
import websockets
from pathlib import Path
import struct
# request tags
SEND_REQUEST_TAG = 0x00
REPLY_REQUEST_TAG = 0x01
SELF_ADDRESS_REQUEST_TAG = 0x02
# response tags
ERROR_RESPONSE_TAG = 0x00
RECEIVED_RESPONSE_TAG = 0x01
SELF_ADDRESS_RESPONSE_TAG = 0x02
def make_self_address_request() -> bytes:
return bytes([SELF_ADDRESS_REQUEST_TAG])
def parse_self_address_response(raw_response: bytes) -> bytes:
if len(raw_response) != 97 or raw_response[0] != SELF_ADDRESS_RESPONSE_TAG:
print('Received invalid response!')
raise
return raw_response[1:]
def make_send_request(recipient: bytes, message: bytes, with_reply_surb: bool) -> bytes:
# a big endian uint64
message_len = len(message).to_bytes(length=8, byteorder='big', signed=False)
return bytes([SEND_REQUEST_TAG]) + bytes([with_reply_surb]) + recipient + message_len + message
def make_reply_request(message: bytes, reply_surb: bytes) -> bytes:
message_len = len(message).to_bytes(length=8, byteorder='big', signed=False)
surb_len = len(reply_surb).to_bytes(length=8, byteorder='big', signed=False)
return bytes([REPLY_REQUEST_TAG]) + surb_len + reply_surb + message_len + message
# it should have structure of RECEIVED_RESPONSE_TAG || with_reply || (surb_len || surb) || msg_len || msg
# where surb_len || surb is only present if 'with_reply' is true
def parse_received(raw_response: bytes) -> (bytes, bytes):
if raw_response[0] != RECEIVED_RESPONSE_TAG:
print('Received invalid response!')
raise
if raw_response[1] == 1:
has_surb = True
elif raw_response[1] == 0:
has_surb = False
else:
print("malformed received response!")
raise
data = raw_response[2:]
if has_surb:
(surb_len,), other = struct.unpack(">Q", data[:8]), data[8:]
surb = other[:surb_len]
(msg_len,) = struct.unpack(">Q", other[surb_len:surb_len + 8])
if len(other[surb_len + 8:]) != msg_len:
print("invalid msg len")
raise
msg = other[surb_len + 8:]
return msg, surb
else:
(msg_len,), other = struct.unpack(">Q", data[:8]), data[8:]
if len(other) != msg_len:
print("invalid msg len")
raise
msg = other[:msg_len]
return msg, None
async def send_file_with_reply():
uri = "ws://localhost:1977"
async with websockets.connect(uri) as websocket:
self_address_req = make_self_address_request()
await websocket.send(self_address_req)
self_address = parse_self_address_response(await websocket.recv())
file_data = Path('dummy_file').read_bytes()
send_request = make_send_request(self_address, file_data, True)
print("sending content of 'dummy_file' over the mix network...")
await websocket.send(send_request)
print("waiting to receive the 'dummy_file' from the mix network...")
received_response = await websocket.recv()
received_file, surb = parse_received(received_response)
with open("received_file_withreply", "wb") as output_file:
print("writing the file back to the disk!")
output_file.write(received_file)
reply_message = b"hello from reply SURB! - thanks for sending me the file!"
reply_request = make_reply_request(reply_message, surb)
print("sending '{}' (using reply SURB!) over the mix network...".format(reply_message))
await websocket.send(reply_request)
print("waiting to receive a message from the mix network...")
received_response = await websocket.recv()
received_msg, surb = parse_received(received_response)
assert surb is None # no surbs in replies!
print("received '{}' from the mix network".format(received_msg))
async def send_file_without_reply():
uri = "ws://localhost:1977"
async with websockets.connect(uri) as websocket:
self_address_req = make_self_address_request()
await websocket.send(self_address_req)
self_address = parse_self_address_response(await websocket.recv())
file_data = Path('dummy_file').read_bytes()
send_request = make_send_request(self_address, file_data, False)
print("sending content of 'dummy_file' over the mix network...")
await websocket.send(send_request)
print("waiting to receive the 'dummy_file' from the mix network...")
received_response = await websocket.recv()
received_file, surb = parse_received(received_response)
assert surb is None # we didn't attach a surb so we expect a None here!
with open("received_file_noreply", "wb") as output_file:
print("writing the file back to the disk!")
output_file.write(received_file)
# asyncio.get_event_loop().run_until_complete(send_file_without_reply())
asyncio.get_event_loop().run_until_complete(send_file_with_reply())
clients/native/examples/python-examples/websocket/filesend.py
-38
View File
@@ -1,38 +0,0 @@
import asyncio
import base58
import json
import websockets
self_address_request = json.dumps({
"type": "selfAddress"
})
async def send_file():
uri = "ws://localhost:1977"
async with websockets.connect(uri) as websocket:
await websocket.send(self_address_request)
self_address = json.loads(await websocket.recv())
print("our address is: {}".format(self_address["address"]))
# we receive our address in string format of OUR_PUB_KEY @ OUR_GATE_PUB_KEY
# both keys are 32 bytes and we need to encode them as binary without the '@' sign
split_address = self_address["address"].split("@")
bin_payload = bytearray(base58.b58decode(split_address[0]))
bin_payload += base58.b58decode(split_address[1])
with open("dummy_file", "rb") as input_file:
read_data = input_file.read()
bin_payload += read_data
print("sending content of 'dummy_file' over the mix network...")
await websocket.send(bin_payload)
msg_send_confirmation = json.loads(await websocket.recv())
assert msg_send_confirmation["type"], "send"
print("waiting to receive the 'dummy_file' from the mix network...")
received_data = await websocket.recv()
with open("received_file", "wb") as output_file:
print("writing the file back to the disk!")
output_file.write(received_data)
asyncio.get_event_loop().run_until_complete(send_file())
clients/native/examples/python-examples/websocket/textsend.py
+49 -6
View File
@@ -7,7 +7,7 @@ self_address_request = json.dumps({
})
async def send_text():
async def send_text_without_reply():
message = "Hello Nym!"
uri = "ws://localhost:1977"
@@ -19,16 +19,59 @@ async def send_text():
text_send = json.dumps({
"type": "send",
"message": message,
"recipient": self_address["address"]
"recipient": self_address["address"],
"withReplySurb": False,
})
print("sending '{}' over the mix network...".format(message))
print("sending '{}' (*without* reply SURB) over the mix network...".format(message))
await websocket.send(text_send)
msg_send_confirmation = json.loads(await websocket.recv())
assert msg_send_confirmation["type"], "send"
print("waiting to receive a message from the mix network...")
received_message = await websocket.recv()
print("received '{}' from the mix network".format(received_message))
asyncio.get_event_loop().run_until_complete(send_text())
async def send_text_with_reply():
message = "Hello Nym!"
uri = "ws://localhost:1977"
async with websockets.connect(uri) as websocket:
await websocket.send(self_address_request)
self_address = json.loads(await websocket.recv())
print("our address is: {}".format(self_address["address"]))
text_send = json.dumps({
"type": "send",
"message": message,
"recipient": self_address["address"],
"withReplySurb": True,
})
print("sending '{}' (*with* reply SURB) over the mix network...".format(message))
await websocket.send(text_send)
print("waiting to receive a message from the mix network...")
received_message = json.loads(await websocket.recv())
print("received '{}' from the mix network".format(received_message))
# use the received surb to send an anonymous reply!
reply_surb = received_message["replySurb"]
reply_message = "hello from reply SURB!"
reply = json.dumps({
"type": "reply",
"message": reply_message,
"replySurb": reply_surb
})
print("sending '{}' (using reply SURB!) over the mix network...".format(reply_message))
await websocket.send(reply)
print("waiting to receive a message from the mix network...")
received_message = await websocket.recv()
print("received '{}' from the mix network".format(received_message))
# asyncio.get_event_loop().run_until_complete(send_text_without_reply())
asyncio.get_event_loop().run_until_complete(send_text_with_reply())
clients/native/examples/websocket_binarysend.rs
+117
View File
@@ -0,0 +1,117 @@
use futures::{SinkExt, StreamExt};
use nymsphinx::addressing::clients::Recipient;
use tokio::net::TcpStream;
use tokio_tungstenite::{connect_async, tungstenite::protocol::Message, WebSocketStream};
use websocket_requests::{requests::ClientRequest, responses::ServerResponse};
// just helpers functions that work in this very particular context because we are sending to ourselves
// and hence will always get a response back (i.e. the message we sent)
async fn send_message_and_get_response(
ws_stream: &mut WebSocketStream<TcpStream>,
req: Vec<u8>,
) -> ServerResponse {
ws_stream.send(Message::Binary(req)).await.unwrap();
let raw_message = ws_stream.next().await.unwrap().unwrap();
match raw_message {
Message::Binary(bin_payload) => ServerResponse::deserialize(&bin_payload).unwrap(),
_ => panic!("received an unexpected response type!"),
}
}
async fn get_self_address(ws_stream: &mut WebSocketStream<TcpStream>) -> Recipient {
let self_address_request = ClientRequest::SelfAddress.serialize();
let response = send_message_and_get_response(ws_stream, self_address_request).await;
match response {
ServerResponse::SelfAddress(recipient) => recipient,
_ => panic!("received an unexpected response!"),
}
}
async fn send_file_with_reply() {
let uri = "ws://localhost:1977";
let (mut ws_stream, _) = connect_async(uri).await.unwrap();
let recipient = get_self_address(&mut ws_stream).await;
println!("our full address is: {}", recipient.to_string());
let read_data = std::fs::read("examples/dummy_file").unwrap();
let send_request = ClientRequest::Send {
recipient,
message: read_data,
with_reply_surb: true,
};
println!("sending content of 'dummy_file' over the mix network...");
let response = send_message_and_get_response(&mut ws_stream, send_request.serialize()).await;
let received = match response {
ServerResponse::Received(received) => received,
_ => panic!("received an unexpected response!"),
};
println!("writing the file back to the disk!");
std::fs::write("examples/received_file_withreply", received.message).unwrap();
let reply_message = b"hello from reply SURB! - thanks for sending me the file!".to_vec();
let reply_request = ClientRequest::Reply {
message: reply_message.clone(),
reply_surb: received.reply_surb.unwrap(),
};
println!(
"sending {:?} (using reply SURB!) over the mix network...",
String::from_utf8(reply_message).unwrap()
);
let response = send_message_and_get_response(&mut ws_stream, reply_request.serialize()).await;
let received = match response {
ServerResponse::Received(received) => received,
_ => panic!("received an unexpected response!"),
};
println!(
"received {:#?} from the mix network!",
String::from_utf8(received.message).unwrap()
);
}
async fn send_file_without_reply() {
let uri = "ws://localhost:1977";
let (mut ws_stream, _) = connect_async(uri).await.unwrap();
let recipient = get_self_address(&mut ws_stream).await;
println!("our full address is: {}", recipient.to_string());
let read_data = std::fs::read("examples/dummy_file").unwrap();
let send_request = ClientRequest::Send {
recipient,
message: read_data,
with_reply_surb: false,
};
println!("sending content of 'dummy_file' over the mix network...");
let response = send_message_and_get_response(&mut ws_stream, send_request.serialize()).await;
let received = match response {
ServerResponse::Received(received) => received,
_ => panic!("received an unexpected response!"),
};
println!("writing the file back to the disk!");
std::fs::write("examples/received_file_noreply", received.message).unwrap();
}
#[tokio::main]
async fn main() {
println!("#############################");
println!("Example without using replies");
send_file_without_reply().await;
println!("\n\n#############################");
println!("Example using replies");
send_file_with_reply().await;
}
clients/native/examples/websocket_filesend.rs
-57
View File
@@ -1,57 +0,0 @@
use futures::{SinkExt, StreamExt};
use nym_client::websocket::{BinaryClientRequest, ClientRequest, ServerResponse};
use nymsphinx::addressing::clients::Recipient;
use std::convert::TryFrom;
use tokio_tungstenite::{connect_async, tungstenite::protocol::Message};
#[tokio::main]
async fn main() {
let uri = "ws://localhost:1977";
let (mut ws_stream, _) = connect_async(uri).await.unwrap();
let self_address_request = ClientRequest::SelfAddress;
ws_stream.send(self_address_request.into()).await.unwrap();
let raw_response = ws_stream.next().await.unwrap().unwrap();
// what we received now is just a json, but we know it's exact format
// so might as well use that
let response = match raw_response {
Message::Text(txt_msg) => ServerResponse::try_from(txt_msg).unwrap(),
_ => panic!("received an unexpected response type!"),
};
let recipient = match response {
ServerResponse::SelfAddress { address } => Recipient::try_from_string(address).unwrap(),
_ => panic!("received an unexpected response type!"),
};
println!("our full address is: {}", recipient.to_string());
let read_data = std::fs::read("examples/dummy_file").unwrap();
let send_request = BinaryClientRequest::Send {
recipient,
data: read_data,
};
println!("sending content of 'dummy_file' over the mix network...");
ws_stream.send(send_request.into()).await.unwrap();
let raw_send_confirmation = ws_stream.next().await.unwrap().unwrap();
let _send_confirmation = match raw_send_confirmation {
Message::Text(txt_msg) => match ServerResponse::try_from(txt_msg).unwrap() {
ServerResponse::Send => (),
_ => panic!("received an unexpected response type!"),
},
_ => panic!("received an unexpected response type!"),
};
println!("waiting to receive the 'dummy_file' from the mix network...");
let raw_message = ws_stream.next().await.unwrap().unwrap();
let message = match raw_message {
Message::Binary(bin_payload) => bin_payload,
_ => panic!("received an unexpected response type!"),
};
println!("writing the file back to the disk!");
std::fs::write("examples/received_file", message).unwrap();
}
clients/native/examples/websocket_textsend.rs
+94 -41
View File
@@ -1,54 +1,107 @@
use futures::{SinkExt, StreamExt};
use nym_client::websocket::{ClientRequest, ServerResponse};
use std::convert::TryFrom;
use tokio_tungstenite::{connect_async, tungstenite::protocol::Message};
use serde_json::json;
use tokio::net::TcpStream;
use tokio_tungstenite::{connect_async, tungstenite::protocol::Message, WebSocketStream};
#[tokio::main]
async fn main() {
// PREFACE: in practice I don't see why you would ever want to use text api while in Rust, but example
// is here for the completion sake
// just helpers functions that work in this very particular context because we are sending to ourselves
// and hence will always get a response back (i.e. the message we sent)
async fn send_message_and_get_json_response(
ws_stream: &mut WebSocketStream<TcpStream>,
text_req: String,
) -> serde_json::Value {
ws_stream.send(Message::Text(text_req)).await.unwrap();
let raw_message = ws_stream.next().await.unwrap().unwrap();
match raw_message {
Message::Text(txt_msg) => serde_json::from_str(&txt_msg).unwrap(),
_ => panic!("received an unexpected response type!"),
}
}
async fn get_self_address(ws_stream: &mut WebSocketStream<TcpStream>) -> String {
let self_address_request = json!({ "type": "selfAddress" }).to_string();
let response = send_message_and_get_json_response(ws_stream, self_address_request).await;
response["address"].as_str().unwrap().to_string()
}
async fn send_text_with_reply() {
let message = "Hello Nym!".to_string();
let uri = "ws://localhost:1977";
let (mut ws_stream, _) = connect_async(uri).await.unwrap();
let self_address_request = ClientRequest::SelfAddress;
ws_stream.send(self_address_request.into()).await.unwrap();
let recipient = get_self_address(&mut ws_stream).await;
println!("our full address is: {}", recipient.to_string());
let raw_response = ws_stream.next().await.unwrap().unwrap();
// what we received now is just a json, but we know it's exact format
// so might as well use that
let response = match raw_response {
Message::Text(txt_msg) => ServerResponse::try_from(txt_msg).unwrap(),
_ => panic!("received an unexpected response type!"),
};
let send_request = json!({
"type" : "send",
"recipient": recipient,
"message": message,
"withReplySurb": true,
});
let self_address = match response {
ServerResponse::SelfAddress { address } => address,
_ => panic!("received an unexpected response type!"),
};
println!("our address is: {}", self_address.clone());
println!(
"sending {:?} (*with* reply SURB) over the mix network...",
message
);
let response =
send_message_and_get_json_response(&mut ws_stream, send_request.to_string()).await;
let send_request = ClientRequest::Send {
message: message.clone(),
recipient: self_address,
};
println!("sending {:?} over the mix network...", message);
ws_stream.send(send_request.into()).await.unwrap();
let reply_message = "hello from reply SURB!";
let reply_request = json!({
"type": "reply",
"message": reply_message,
"replySurb": response["replySurb"]
});
let raw_send_confirmation = ws_stream.next().await.unwrap().unwrap();
let _send_confirmation = match raw_send_confirmation {
Message::Text(txt_msg) => match ServerResponse::try_from(txt_msg).unwrap() {
ServerResponse::Send => (),
_ => panic!("received an unexpected response type!"),
},
_ => panic!("received an unexpected response type!"),
};
println!(
"sending {:?} (using reply SURB!) over the mix network...",
reply_message
);
println!("waiting to receive a message from the mix network...");
let raw_message = ws_stream.next().await.unwrap().unwrap();
let message = match raw_message {
Message::Text(txt_msg) => txt_msg,
_ => panic!("received an unexpected response type!"),
};
println!("received {:?} from the mix network!", message);
let response =
send_message_and_get_json_response(&mut ws_stream, reply_request.to_string()).await;
println!("received {:#?} from the mix network!", response.to_string());
}
async fn send_text_without_reply() {
let message = "Hello Nym!".to_string();
let uri = "ws://localhost:1977";
let (mut ws_stream, _) = connect_async(uri).await.unwrap();
let recipient = get_self_address(&mut ws_stream).await;
println!("our full address is: {}", recipient.to_string());
let send_request = json!({
"type" : "send",
"recipient": recipient,
"message": message,
"withReplySurb": false,
});
println!(
"sending {:?} (*without* reply SURB) over the mix network...",
message
);
let response =
send_message_and_get_json_response(&mut ws_stream, send_request.to_string()).await;
println!("received {:#?} from the mix network!", response.to_string());
}
#[tokio::main]
async fn main() {
println!("#############################");
println!("Example without using replies");
send_text_without_reply().await;
println!("\n\n#############################");
println!("Example using replies");
send_text_with_reply().await;
}
clients/native/src/client/config/mod.rs
+157
View File
@@ -0,0 +1,157 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::client::config::template::config_template;
use client_core::config::Config as BaseConfig;
use config::NymConfig;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
mod template;
const DEFAULT_LISTENING_PORT: u16 = 1977;
#[derive(Debug, Deserialize, PartialEq, Serialize, Clone, Copy)]
#[serde(deny_unknown_fields)]
pub enum SocketType {
WebSocket,
None,
}
impl SocketType {
pub fn from_string<S: Into<String>>(val: S) -> Self {
let mut upper = val.into();
upper.make_ascii_uppercase();
match upper.as_ref() {
"WEBSOCKET" | "WS" => SocketType::WebSocket,
_ => SocketType::None,
}
}
}
#[derive(Debug, Default, Deserialize, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Config {
#[serde(flatten)]
base: BaseConfig<Config>,
socket: Socket,
}
impl NymConfig for Config {
fn template() -> &'static str {
config_template()
}
fn config_file_name() -> String {
"config.toml".to_string()
}
fn default_root_directory() -> PathBuf {
dirs::home_dir()
.expect("Failed to evaluate $HOME value")
.join(".nym")
.join("clients")
}
fn root_directory(&self) -> PathBuf {
self.base.get_nym_root_directory()
}
fn config_directory(&self) -> PathBuf {
self.root_directory()
.join(self.base.get_id())
.join("config")
}
fn data_directory(&self) -> PathBuf {
self.root_directory().join(self.base.get_id()).join("data")
}
}
impl Config {
pub fn new<S: Into<String>>(id: S) -> Self {
Config {
base: BaseConfig::new(id),
socket: Default::default(),
}
}
pub fn with_socket(mut self, socket_type: SocketType) -> Self {
self.socket.socket_type = socket_type;
self
}
pub fn with_port(mut self, port: u16) -> Self {
self.socket.listening_port = port;
self
}
// getters
pub fn get_config_file_save_location(&self) -> PathBuf {
self.config_directory().join(Self::config_file_name())
}
pub fn get_base(&self) -> &BaseConfig<Self> {
&self.base
}
pub fn get_base_mut(&mut self) -> &mut BaseConfig<Self> {
&mut self.base
}
pub fn get_socket_type(&self) -> SocketType {
self.socket.socket_type
}
pub fn get_listening_port(&self) -> u16 {
self.socket.listening_port
}
}
#[derive(Debug, Deserialize, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Socket {
socket_type: SocketType,
listening_port: u16,
}
impl Default for Socket {
fn default() -> Self {
Socket {
socket_type: SocketType::WebSocket,
listening_port: DEFAULT_LISTENING_PORT,
}
}
}
#[cfg(test)]
mod client_config {
use super::*;
#[test]
fn after_saving_default_config_the_loaded_one_is_identical() {
// need to figure out how to do something similar but without touching the disk
// or the file system at all...
let temp_location = tempfile::tempdir().unwrap().path().join("config.toml");
let default_config = Config::new("foomp".to_string());
default_config
.save_to_file(Some(temp_location.clone()))
.unwrap();
let loaded_config = Config::load_from_file(Some(temp_location), None).unwrap();
assert_eq!(default_config, loaded_config);
}
}
clients/native/src/client/config/template.rs
+102
View File
@@ -0,0 +1,102 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) fn config_template() -> &'static str {
// While using normal toml marshalling would have been way simpler with less overhead,
// I think it's useful to have comments attached to the saved config file to explain behaviour of
// particular fields.
// Note: any changes to the template must be reflected in the appropriate structs in mod.rs.
r#"
# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml
##### main base client config options #####
[client]
# Human readable ID of this particular client.
id = '{{ client.id }}'
# URL to the directory server.
directory_server = '{{ client.directory_server }}'
# Path to file containing private identity key.
private_identity_key_file = '{{ client.private_identity_key_file }}'
# Path to file containing public identity key.
public_identity_key_file = '{{ client.public_identity_key_file }}'
# Path to file containing private encryption key.
private_encryption_key_file = '{{ client.private_encryption_key_file }}'
# Path to file containing public encryption key.
public_encryption_key_file = '{{ client.public_encryption_key_file }}'
# Full path to file containing reply encryption keys of all reply-SURBs we have ever
# sent but not received back.
reply_encryption_key_store_path = '{{ client.reply_encryption_key_store_path }}'
##### additional client config options #####
# ID of the gateway from which the client should be fetching messages.
gateway_id = '{{ client.gateway_id }}'
# Address of the gateway listener to which all client requests should be sent.
gateway_listener = '{{ client.gateway_listener }}'
# A gateway specific, optional, base58 stringified shared key used for
# communication with particular gateway.
gateway_shared_key_file = '{{ client.gateway_shared_key_file }}'
# Path to file containing key used for encrypting and decrypting the content of an
# acknowledgement so that nobody besides the client knows which packet it refers to.
ack_key_file = '{{ client.ack_key_file }}'
##### advanced configuration options #####
# Absolute path to the home Nym Clients directory.
nym_root_directory = '{{ client.nym_root_directory }}'
##### socket config options #####
[socket]
# allowed values are 'WebSocket' or 'None'
socket_type = '{{ socket.socket_type }}'
# if applicable (for the case of 'WebSocket'), the port on which the client
# will be listening for incoming requests
listening_port = {{ socket.listening_port }}
##### logging configuration options #####
[logging]
# TODO
##### debug configuration options #####
# The following options should not be modified unless you know EXACTLY what you are doing
# as if set incorrectly, they may impact your anonymity.
[debug]
average_packet_delay = {{ debug.average_packet_delay }}
average_ack_delay = {{ debug.average_ack_delay }}
loop_cover_traffic_average_delay = {{ debug.loop_cover_traffic_average_delay }}
message_sending_average_delay = {{ debug.message_sending_average_delay }}
"#
}
clients/native/src/client/inbound_messages.rs
-22
View File
@@ -1,22 +0,0 @@
use futures::channel::mpsc;
use nymsphinx::addressing::clients::Recipient;
pub(crate) type InputMessageSender = mpsc::UnboundedSender<InputMessage>;
pub(crate) type InputMessageReceiver = mpsc::UnboundedReceiver<InputMessage>;
#[derive(Debug)]
pub(crate) struct InputMessage {
recipient: Recipient,
data: Vec<u8>,
}
impl InputMessage {
pub(crate) fn new(recipient: Recipient, data: Vec<u8>) -> Self {
InputMessage { recipient, data }
}
// I'm open to suggestions on how to rename this.
pub(crate) fn destruct(self) -> (Recipient, Vec<u8>) {
(self.recipient, self.data)
}
}
clients/native/src/client/mod.rs
+176 -152
View File
@@ -12,55 +12,74 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::client::cover_traffic_stream::LoopCoverTrafficStream;
use crate::client::inbound_messages::{InputMessage, InputMessageReceiver, InputMessageSender};
use crate::client::mix_traffic::{MixMessageReceiver, MixMessageSender, MixTrafficController};
use crate::client::received_buffer::{
ReceivedBufferRequestReceiver, ReceivedBufferRequestSender, ReceivedMessagesBufferController,
use crate::client::config::{Config, SocketType};
use crate::websocket;
use client_core::client::cover_traffic_stream::LoopCoverTrafficStream;
use client_core::client::inbound_messages::{
InputMessage, InputMessageReceiver, InputMessageSender,
};
use crate::client::topology_control::{
use client_core::client::key_manager::KeyManager;
use client_core::client::mix_traffic::{
MixMessageReceiver, MixMessageSender, MixTrafficController,
};
use client_core::client::real_messages_control;
use client_core::client::real_messages_control::RealMessagesController;
use client_core::client::received_buffer::{
ReceivedBufferMessage, ReceivedBufferRequestReceiver, ReceivedBufferRequestSender,
ReceivedMessagesBufferController, ReconstructedMessagesReceiver,
};
use client_core::client::reply_key_storage::ReplyKeyStorage;
use client_core::client::topology_control::{
TopologyAccessor, TopologyRefresher, TopologyRefresherConfig,
};
use crate::config::{Config, SocketType};
use crate::websocket;
use crypto::identity::MixIdentityKeyPair;
use directory_client::presence;
use client_core::config::persistence::key_pathfinder::ClientKeyPathfinder;
use crypto::asymmetric::identity;
use futures::channel::mpsc;
use gateway_client::{GatewayClient, SphinxPacketReceiver, SphinxPacketSender};
use gateway_requests::auth_token::AuthToken;
use gateway_client::{
AcknowledgementReceiver, AcknowledgementSender, GatewayClient, MixnetMessageReceiver,
MixnetMessageSender,
};
use log::*;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::chunking::split_and_prepare_payloads;
use nymsphinx::NodeAddressBytes;
use received_buffer::{ReceivedBufferMessage, ReconstructedMessagesReceiver};
use nymsphinx::addressing::nodes::NodeIdentity;
use nymsphinx::anonymous_replies::ReplySURB;
use nymsphinx::receiver::ReconstructedMessage;
use tokio::runtime::Runtime;
use topology::NymTopology;
mod cover_traffic_stream;
pub(crate) mod inbound_messages;
mod mix_traffic;
mod real_traffic_stream;
pub(crate) mod received_buffer;
pub(crate) mod topology_control;
pub(crate) mod config;
pub struct NymClient {
/// Client configuration options, including, among other things, packet sending rates,
/// key filepaths, etc.
config: Config,
runtime: Runtime,
identity_keypair: MixIdentityKeyPair,
// to be used by "send" function or socket, etc
/// Tokio runtime used for futures execution.
// TODO: JS: Personally I think I prefer the implicit way of using it that we've done with the
// gateway.
runtime: Runtime,
/// KeyManager object containing smart pointers to all relevant keys used by the client.
key_manager: KeyManager,
/// Channel used for transforming 'raw' messages into sphinx packets and sending them
/// through the mix network.
/// It is only available if the client started with the websocket listener disabled.
input_tx: Option<InputMessageSender>,
// to be used by "receive" function or socket, etc
/// Channel used for obtaining reconstructed messages received from the mix network.
/// It is only available if the client started with the websocket listener disabled.
receive_tx: Option<ReconstructedMessagesReceiver>,
}
impl NymClient {
pub fn new(config: Config, identity_keypair: MixIdentityKeyPair) -> Self {
pub fn new(config: Config) -> Self {
let pathfinder = ClientKeyPathfinder::new_from_config(config.get_base());
let key_manager = KeyManager::load_keys(&pathfinder).expect("failed to load stored keys");
NymClient {
runtime: Runtime::new().unwrap(),
config,
identity_keypair,
key_manager,
input_tx: None,
receive_tx: None,
}
@@ -68,18 +87,19 @@ impl NymClient {
pub fn as_mix_recipient(&self) -> Recipient {
Recipient::new(
self.identity_keypair.public_key.derive_address(),
*self.key_manager.identity_keypair().public_key(),
self.key_manager.encryption_keypair().public_key().clone(),
// TODO: below only works under assumption that gateway address == gateway id
// (which currently is true)
NodeAddressBytes::try_from_base58_string(self.config.get_gateway_id()).unwrap(),
NodeIdentity::from_base58_string(self.config.get_base().get_gateway_id()).unwrap(),
)
}
// future constantly pumping loop cover traffic at some specified average rate
// the pumped traffic goes to the MixTrafficController
fn start_cover_traffic_stream<T: 'static + NymTopology>(
fn start_cover_traffic_stream(
&self,
topology_accessor: TopologyAccessor<T>,
topology_accessor: TopologyAccessor,
mix_tx: MixMessageSender,
) {
info!("Starting loop cover traffic stream...");
@@ -88,37 +108,53 @@ impl NymClient {
self.runtime
.enter(|| {
LoopCoverTrafficStream::new(
self.key_manager.ack_key(),
self.config.get_base().get_average_ack_delay(),
self.config.get_base().get_average_packet_delay(),
self.config
.get_base()
.get_loop_cover_traffic_average_delay(),
mix_tx,
self.as_mix_recipient(),
topology_accessor,
self.config.get_loop_cover_traffic_average_delay(),
self.config.get_average_packet_delay(),
)
})
.start(self.runtime.handle());
}
fn start_real_traffic_stream<T: 'static + NymTopology>(
fn start_real_traffic_controller(
&self,
topology_accessor: TopologyAccessor<T>,
mix_tx: MixMessageSender,
input_rx: InputMessageReceiver,
topology_accessor: TopologyAccessor,
reply_key_storage: ReplyKeyStorage,
ack_receiver: AcknowledgementReceiver,
input_receiver: InputMessageReceiver,
mix_sender: MixMessageSender,
) {
let controller_config = real_messages_control::Config::new(
self.key_manager.ack_key(),
self.config.get_base().get_ack_wait_multiplier(),
self.config.get_base().get_ack_wait_addition(),
self.config.get_base().get_average_ack_delay(),
self.config.get_base().get_message_sending_average_delay(),
self.config.get_base().get_average_packet_delay(),
self.as_mix_recipient(),
);
info!("Starting real traffic stream...");
// we need to explicitly enter runtime due to "next_delay: time::delay_for(Default::default())"
// set in the constructor which HAS TO be called within context of a tokio runtime
self.runtime
.enter(|| {
real_traffic_stream::OutQueueControl::new(
mix_tx,
input_rx,
self.as_mix_recipient(),
topology_accessor,
self.config.get_average_packet_delay(),
self.config.get_message_sending_average_delay(),
)
})
.start(self.runtime.handle());
// set in the constructor [of OutQueueControl] which HAS TO be called within context of a tokio runtime
// When refactoring this restriction should definitely be removed.
let real_messages_controller = self.runtime.enter(|| {
RealMessagesController::new(
controller_config,
ack_receiver,
input_receiver,
mix_sender,
topology_accessor,
reply_key_storage,
)
});
real_messages_controller.start(self.runtime.handle());
}
// buffer controlling all messages fetched from provider
@@ -126,91 +162,75 @@ impl NymClient {
fn start_received_messages_buffer_controller(
&self,
query_receiver: ReceivedBufferRequestReceiver,
sphinx_receiver: SphinxPacketReceiver,
mixnet_receiver: MixnetMessageReceiver,
reply_key_storage: ReplyKeyStorage,
) {
info!("Starting received messages buffer controller...");
ReceivedMessagesBufferController::new(query_receiver, sphinx_receiver)
.start(self.runtime.handle())
ReceivedMessagesBufferController::new(
self.key_manager.encryption_keypair(),
query_receiver,
mixnet_receiver,
reply_key_storage,
)
.start(self.runtime.handle())
}
fn start_gateway_client(
&mut self,
sphinx_packet_sender: SphinxPacketSender,
gateway_address: url::Url,
mixnet_message_sender: MixnetMessageSender,
ack_sender: AcknowledgementSender,
) -> GatewayClient<'static, url::Url> {
let auth_token = self
.config
.get_gateway_auth_token()
.map(|str_token| AuthToken::try_from_base58_string(str_token).ok())
.unwrap_or(None);
let gateway_id = self.config.get_base().get_gateway_id();
if gateway_id.is_empty() {
panic!("The identity of the gateway is unknown - did you run `nym-client` init?")
}
let gateway_address_str = self.config.get_base().get_gateway_listener();
if gateway_address_str.is_empty() {
panic!("The address of the gateway is unknown - did you run `nym-client` init?")
}
let gateway_identity = identity::PublicKey::from_base58_string(gateway_id)
.expect("provided gateway id is invalid!");
// TODO: since we presumably now get something valid-ish from the `init`, can we just
// ditch url::Url and operate on `String`?
let gateway_address =
url::Url::parse(&gateway_address_str).expect("provided gateway address is invalid!");
let mut gateway_client = GatewayClient::new(
gateway_address,
self.as_mix_recipient().destination(),
auth_token,
sphinx_packet_sender,
self.config.get_gateway_response_timeout(),
self.key_manager.identity_keypair(),
gateway_identity,
Some(self.key_manager.gateway_shared_key()),
mixnet_message_sender,
ack_sender,
self.config.get_base().get_gateway_response_timeout(),
);
let auth_token = self.runtime.block_on(async {
self.runtime.block_on(async {
gateway_client
.establish_connection()
.authenticate_and_start()
.await
.expect("could not establish initial connection with the gateway");
gateway_client
.perform_initial_authentication()
.await
.expect("could not perform initial authentication with the gateway")
.expect("could not authenticate and start up the gateway connection")
});
// TODO: if we didn't have an auth_token initially, save it to config or something?
info!(
"Performed initial authentication. Auth token is {:?}",
auth_token.to_base58_string()
);
gateway_client
}
// TODO: this information should be just put in the config on init...
async fn get_gateway_address<T: NymTopology>(
gateway_id: String,
topology_accessor: TopologyAccessor<T>,
) -> url::Url {
// we already have our gateway written in the config
let gateway_address = topology_accessor
.get_gateway_socket_url(&gateway_id)
.await
.unwrap_or_else(|| {
panic!(
"Could not find gateway with id {:?}.\
It does not seem to be present in the current network topology.\
Are you sure it is still online?\
Perhaps try to run `nym-client init` again to obtain a new gateway",
gateway_id
)
});
url::Url::parse(&gateway_address).expect("provided gateway address is invalid!")
}
// future responsible for periodically polling directory server and updating
// the current global view of topology
fn start_topology_refresher<T: 'static + NymTopology>(
&mut self,
topology_accessor: TopologyAccessor<T>,
) {
fn start_topology_refresher(&mut self, topology_accessor: TopologyAccessor) {
let topology_refresher_config = TopologyRefresherConfig::new(
self.config.get_directory_server(),
self.config.get_topology_refresh_rate(),
self.config.get_base().get_directory_server(),
self.config.get_base().get_topology_refresh_rate(),
);
let mut topology_refresher =
TopologyRefresher::new(topology_refresher_config, topology_accessor);
TopologyRefresher::new_directory_client(topology_refresher_config, topology_accessor);
// before returning, block entire runtime to refresh the current network view so that any
// components depending on topology would see a non-empty view
info!(
"Obtaining initial network topology from {}",
self.config.get_directory_server()
self.config.get_base().get_directory_server()
);
self.runtime.block_on(topology_refresher.refresh());
@@ -242,20 +262,15 @@ impl NymClient {
MixTrafficController::new(mix_rx, gateway_client).start(self.runtime.handle());
}
fn start_websocket_listener<T: 'static + NymTopology>(
fn start_websocket_listener(
&self,
topology_accessor: TopologyAccessor<T>,
buffer_requester: ReceivedBufferRequestSender,
msg_input: InputMessageSender,
) {
info!("Starting websocket listener...");
let websocket_handler = websocket::Handler::new(
msg_input,
buffer_requester,
self.as_mix_recipient(),
topology_accessor,
);
let websocket_handler =
websocket::Handler::new(msg_input, buffer_requester, self.as_mix_recipient());
websocket::Listener::new(self.config.get_listening_port())
.start(self.runtime.handle(), websocket_handler);
@@ -264,20 +279,27 @@ impl NymClient {
/// EXPERIMENTAL DIRECT RUST API
/// It's untested and there are absolutely no guarantees about it (but seems to have worked
/// well enough in local tests)
pub fn send_message(&mut self, recipient: Recipient, message: Vec<u8>) {
let split_message = split_and_prepare_payloads(&message);
debug!(
"Splitting message into {:?} fragments!",
split_message.len()
);
for message_fragment in split_message {
let input_msg = InputMessage::new(recipient.clone(), message_fragment);
self.input_tx
.as_ref()
.expect("start method was not called before!")
.unbounded_send(input_msg)
.unwrap()
}
pub fn send_message(&mut self, recipient: Recipient, message: Vec<u8>, with_reply_surb: bool) {
let input_msg = InputMessage::new_fresh(recipient, message, with_reply_surb);
self.input_tx
.as_ref()
.expect("start method was not called before!")
.unbounded_send(input_msg)
.unwrap();
}
/// EXPERIMENTAL DIRECT RUST API
/// It's untested and there are absolutely no guarantees about it (but seems to have worked
/// well enough in local tests)
pub fn send_reply(&mut self, reply_surb: ReplySURB, message: Vec<u8>) {
let input_msg = InputMessage::new_reply(reply_surb, message);
self.input_tx
.as_ref()
.expect("start method was not called before!")
.unbounded_send(input_msg)
.unwrap();
}
/// EXPERIMENTAL DIRECT RUST API
@@ -285,7 +307,8 @@ impl NymClient {
/// well enough in local tests)
/// Note: it waits for the first occurrence of messages being sent to ourselves. If you expect multiple
/// messages, you might have to call this function repeatedly.
pub async fn wait_for_messages(&mut self) -> Vec<Vec<u8>> {
// TODO: I guess this should really return something that `impl Stream<Item=ReconstructedMessage>`
pub async fn wait_for_messages(&mut self) -> Vec<ReconstructedMessage> {
use futures::StreamExt;
self.receive_tx
@@ -323,9 +346,9 @@ impl NymClient {
// sphinx_message_receiver is the receiver used by MixTrafficController that sends the actual traffic
let (sphinx_message_sender, sphinx_message_receiver) = mpsc::unbounded();
// unwrapped_sphinx_sender is the transmitter of [unwrapped] sphinx messages received from the gateway
// unwrapped_sphinx_sender is the transmitter of mixnet messages received from the gateway
// unwrapped_sphinx_receiver is the receiver for said messages - used by ReceivedMessagesBuffer
let (unwrapped_sphinx_sender, unwrapped_sphinx_receiver) = mpsc::unbounded();
let (mixnet_messages_sender, mixnet_messages_receiver) = mpsc::unbounded();
// used for announcing connection or disconnection of a channel for pushing re-assembled messages to
let (received_buffer_request_sender, received_buffer_request_receiver) = mpsc::unbounded();
@@ -333,39 +356,39 @@ impl NymClient {
// channels responsible for controlling real messages
let (input_sender, input_receiver) = mpsc::unbounded::<InputMessage>();
// TODO: when we switch to our graph topology, we need to remember to change 'presence::Topology' type
let shared_topology_accessor = TopologyAccessor::<presence::Topology>::new();
// channels responsible for controlling ack messages
let (ack_sender, ack_receiver) = mpsc::unbounded();
let shared_topology_accessor = TopologyAccessor::new();
let reply_key_storage =
ReplyKeyStorage::load(self.config.get_base().get_reply_encryption_key_store_path())
.expect("Failed to load reply key storage!");
// the components are started in very specific order. Unless you know what you are doing,
// do not change that.
self.start_topology_refresher(shared_topology_accessor.clone());
self.start_received_messages_buffer_controller(
received_buffer_request_receiver,
unwrapped_sphinx_receiver,
mixnet_messages_receiver,
reply_key_storage.clone(),
);
let gateway_url = self.runtime.block_on(Self::get_gateway_address(
self.config.get_gateway_id(),
shared_topology_accessor.clone(),
));
let gateway_client = self.start_gateway_client(unwrapped_sphinx_sender, gateway_url);
let gateway_client = self.start_gateway_client(mixnet_messages_sender, ack_sender);
self.start_mix_traffic_controller(sphinx_message_receiver, gateway_client);
self.start_cover_traffic_stream(
self.start_real_traffic_controller(
shared_topology_accessor.clone(),
reply_key_storage,
ack_receiver,
input_receiver,
sphinx_message_sender.clone(),
);
self.start_real_traffic_stream(
shared_topology_accessor.clone(),
sphinx_message_sender,
input_receiver,
);
self.start_cover_traffic_stream(shared_topology_accessor, sphinx_message_sender);
match self.config.get_socket_type() {
SocketType::WebSocket => self.start_websocket_listener(
shared_topology_accessor,
received_buffer_request_sender,
input_sender,
),
SocketType::WebSocket => {
self.start_websocket_listener(received_buffer_request_sender, input_sender)
}
SocketType::None => {
// if we did not start the socket, it means we're running (supposedly) in the native mode
// and hence we should announce 'ourselves' to the buffer
@@ -384,5 +407,6 @@ impl NymClient {
}
info!("Client startup finished!");
info!("The address of this client is: {}", self.as_mix_recipient());
}
}
clients/native/src/client/real_traffic_stream.rs
-189
View File
@@ -1,189 +0,0 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::client::inbound_messages::InputMessage;
use crate::client::mix_traffic::MixMessage;
use crate::client::topology_control::TopologyAccessor;
use futures::channel::mpsc;
use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt};
use log::{error, info, trace, warn};
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::utils::{encapsulation, poisson};
use std::pin::Pin;
use std::time::Duration;
use tokio::runtime::Handle;
use tokio::task::JoinHandle;
use tokio::time;
use topology::NymTopology;
pub(crate) struct OutQueueControl<T: NymTopology> {
average_packet_delay: Duration,
average_message_sending_delay: Duration,
next_delay: time::Delay,
mix_tx: mpsc::UnboundedSender<MixMessage>,
input_rx: mpsc::UnboundedReceiver<InputMessage>,
our_full_destination: Recipient,
topology_access: TopologyAccessor<T>,
}
pub(crate) enum StreamMessage {
Cover,
Real(InputMessage),
}
impl<T: NymTopology> Stream for OutQueueControl<T> {
type Item = StreamMessage;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
// it is not yet time to return a message
if Pin::new(&mut self.next_delay).poll(cx).is_pending() {
return Poll::Pending;
};
// we know it's time to send a message, so let's prepare delay for the next one
// Get the `now` by looking at the current `delay` deadline
let now = self.next_delay.deadline();
let next_poisson_delay = poisson::sample(self.average_message_sending_delay);
// The next interval value is `next_poisson_delay` after the one that just
// yielded.
let next = now + next_poisson_delay;
self.next_delay.reset(next);
// decide what kind of message to send
match Pin::new(&mut self.input_rx).poll_next(cx) {
// in the case our real message channel stream was closed, we should also indicate we are closed
// (and whoever is using the stream should panic)
Poll::Ready(None) => Poll::Ready(None),
// if there's an actual message - return it
Poll::Ready(Some(real_message)) => Poll::Ready(Some(StreamMessage::Real(real_message))),
// otherwise construct a dummy one
Poll::Pending => Poll::Ready(Some(StreamMessage::Cover)),
}
}
}
impl<T: 'static + NymTopology> OutQueueControl<T> {
pub(crate) fn new(
mix_tx: mpsc::UnboundedSender<MixMessage>,
input_rx: mpsc::UnboundedReceiver<InputMessage>,
our_full_destination: Recipient,
topology_access: TopologyAccessor<T>,
average_packet_delay: Duration,
average_message_sending_delay: Duration,
) -> Self {
OutQueueControl {
average_packet_delay,
average_message_sending_delay,
next_delay: time::delay_for(Default::default()),
mix_tx,
input_rx,
our_full_destination,
topology_access,
}
}
async fn get_route(&self, other_recipient: Option<&Recipient>) -> Option<Vec<nymsphinx::Node>> {
match other_recipient {
// we are sending to ourselves
None => {
self.topology_access
.random_route_to_gateway(&self.our_full_destination.gateway())
.await
}
Some(other_recipient) => {
self.topology_access
.random_route_to_gateway(&other_recipient.gateway())
.await
}
}
}
async fn on_message(&mut self, next_message: StreamMessage) {
trace!("created new message");
let next_packet = match next_message {
StreamMessage::Cover => {
let route = self.get_route(None).await;
if route.is_none() {
warn!("No valid topology detected - won't send any real or loop message this time");
return;
}
let route = route.unwrap();
encapsulation::loop_cover_message_route(
self.our_full_destination.destination().clone(),
route,
self.average_packet_delay,
)
}
StreamMessage::Real(real_message) => {
let (recipient, data) = real_message.destruct();
let route = self.get_route(Some(&recipient)).await;
if route.is_none() {
warn!("No valid topology detected - won't send any real or loop message this time");
return;
}
let route = route.unwrap();
encapsulation::encapsulate_message_route(
recipient.destination(),
data,
route,
self.average_packet_delay,
)
}
};
let next_packet = match next_packet {
Ok(message) => message,
Err(err) => {
error!(
"Somehow we managed to create an invalid traffic message - {:?}",
err
);
return;
}
};
// if this one fails, there's no retrying because it means that either:
// - we run out of memory
// - the receiver channel is closed
// in either case there's no recovery and we can only panic
self.mix_tx
.unbounded_send(MixMessage::new(next_packet.0, next_packet.1))
.unwrap();
// JS: Not entirely sure why or how it fixes stuff, but without the yield call,
// the UnboundedReceiver [of mix_rx] will not get a chance to read anything
// JS2: Basically it was the case that with high enough rate, the stream had already a next value
// ready and hence was immediately re-scheduled causing other tasks to be starved;
// yield makes it go back the scheduling queue regardless of its value availability
tokio::task::yield_now().await;
}
pub(crate) async fn run_out_queue_control(mut self) {
// we should set initial delay only when we actually start the stream
self.next_delay = time::delay_for(poisson::sample(self.average_message_sending_delay));
info!("Starting out queue controller...");
while let Some(next_message) = self.next().await {
self.on_message(next_message).await;
}
}
pub(crate) fn start(self, handle: &Handle) -> JoinHandle<()> {
handle.spawn(async move { self.run_out_queue_control().await })
}
}
clients/native/src/client/received_buffer.rs
-242
View File
@@ -1,242 +0,0 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use futures::channel::mpsc;
use futures::lock::Mutex;
use futures::StreamExt;
use gateway_client::SphinxPacketReceiver;
use log::*;
use nymsphinx::{
chunking::reconstruction::MessageReconstructor,
utils::encapsulation::LOOP_COVER_MESSAGE_PAYLOAD,
};
use std::sync::Arc;
use tokio::runtime::Handle;
use tokio::task::JoinHandle;
// Buffer Requests to say "hey, send any reconstructed messages to this channel"
// or to say "hey, I'm going offline, don't send anything more to me. Just buffer them instead"
pub(crate) type ReceivedBufferRequestSender = mpsc::UnboundedSender<ReceivedBufferMessage>;
pub(crate) type ReceivedBufferRequestReceiver = mpsc::UnboundedReceiver<ReceivedBufferMessage>;
// The channel set for the above
pub(crate) type ReconstructedMessagesSender = mpsc::UnboundedSender<Vec<Vec<u8>>>;
pub(crate) type ReconstructedMessagesReceiver = mpsc::UnboundedReceiver<Vec<Vec<u8>>>;
struct ReceivedMessagesBufferInner {
messages: Vec<Vec<u8>>,
message_reconstructor: MessageReconstructor,
message_sender: Option<ReconstructedMessagesSender>,
}
#[derive(Debug, Clone)]
// Note: you should NEVER create more than a single instance of this using 'new()'.
// You should always use .clone() to create additional instances
struct ReceivedMessagesBuffer {
inner: Arc<Mutex<ReceivedMessagesBufferInner>>,
}
impl ReceivedMessagesBuffer {
fn new() -> Self {
ReceivedMessagesBuffer {
inner: Arc::new(Mutex::new(ReceivedMessagesBufferInner {
messages: Vec::new(),
message_reconstructor: MessageReconstructor::new(),
message_sender: None,
})),
}
}
async fn disconnect_sender(&mut self) {
let mut guard = self.inner.lock().await;
if guard.message_sender.is_none() {
// in theory we could just ignore it, but that situation should have never happened
// in the first place, so this way we at least know we have an important bug to fix
panic!("trying to disconnect non-existent sender!")
}
guard.message_sender = None;
}
async fn connect_sender(&mut self, sender: ReconstructedMessagesSender) {
let mut guard = self.inner.lock().await;
if guard.message_sender.is_some() {
// in theory we could just ignore it, but that situation should have never happened
// in the first place, so this way we at least know we have an important bug to fix
panic!("trying overwrite an existing sender!")
}
// while we're at it, also empty the buffer if we happened to receive anything while
// no sender was connected
let stored_messages = std::mem::replace(&mut guard.messages, Vec::new());
if !stored_messages.is_empty() {
if let Err(err) = sender.unbounded_send(stored_messages) {
error!(
"The sender channel we just received is already invalidated - {:?}",
err
);
// put the values back to the buffer
// the returned error has two fields: err: SendError and val: T,
// where val is the value that was failed to get sent;
// it's returned by the `into_inner` call
guard.messages = err.into_inner();
return;
}
}
guard.message_sender = Some(sender);
}
async fn add_reconstructed_messages(&mut self, msgs: Vec<Vec<u8>>) {
debug!("Adding {:?} new messages to the buffer!", msgs.len());
trace!("Adding new messages to the buffer! {:?}", msgs);
self.inner.lock().await.messages.extend(msgs)
}
async fn add_new_message_fragments(&mut self, msgs: Vec<Vec<u8>>) {
debug!(
"Adding {:?} new message fragments to the buffer!",
msgs.len()
);
trace!("Adding new message fragments to the buffer! {:?}", msgs);
let mut completed_messages = Vec::new();
let mut inner_guard = self.inner.lock().await;
for msg_fragment in msgs {
if msg_fragment == LOOP_COVER_MESSAGE_PAYLOAD {
trace!("The message was a loop cover message! Skipping it");
continue;
}
if let Some(reconstructed_message) =
inner_guard.message_reconstructor.new_fragment(msg_fragment)
{
completed_messages.push(reconstructed_message);
}
}
if !completed_messages.is_empty() {
if let Some(sender) = &inner_guard.message_sender {
trace!("Sending reconstructed messages to announced sender");
if let Err(err) = sender.unbounded_send(completed_messages) {
warn!("The reconstructed message receiver went offline without explicit notification (relevant error: - {:?})", err);
// make sure to drop the lock to not deadlock
// (it is required by `add_reconstructed_messages`)
inner_guard.message_sender = None;
drop(inner_guard);
self.add_reconstructed_messages(err.into_inner()).await;
}
} else {
// make sure to drop the lock to not deadlock
// (it is required by `add_reconstructed_messages`)
drop(inner_guard);
trace!("No sender available - buffering reconstructed messages");
self.add_reconstructed_messages(completed_messages).await;
}
}
}
}
pub(crate) enum ReceivedBufferMessage {
// Signals a websocket connection (or a native implementation) was established and we should stop buffering messages,
// and instead send them directly to the received channel
ReceiverAnnounce(ReconstructedMessagesSender),
// Explicit signal that Receiver connection will no longer accept messages
ReceiverDisconnect,
}
struct RequestReceiver {
received_buffer: ReceivedMessagesBuffer,
query_receiver: ReceivedBufferRequestReceiver,
}
impl RequestReceiver {
fn new(
received_buffer: ReceivedMessagesBuffer,
query_receiver: ReceivedBufferRequestReceiver,
) -> Self {
RequestReceiver {
received_buffer,
query_receiver,
}
}
fn start(mut self, handle: &Handle) -> JoinHandle<()> {
handle.spawn(async move {
while let Some(request) = self.query_receiver.next().await {
match request {
ReceivedBufferMessage::ReceiverAnnounce(sender) => {
self.received_buffer.connect_sender(sender).await;
}
ReceivedBufferMessage::ReceiverDisconnect => {
self.received_buffer.disconnect_sender().await
}
}
}
})
}
}
struct FragmentedMessageReceiver {
received_buffer: ReceivedMessagesBuffer,
sphinx_packet_receiver: SphinxPacketReceiver,
}
impl FragmentedMessageReceiver {
fn new(
received_buffer: ReceivedMessagesBuffer,
sphinx_packet_receiver: SphinxPacketReceiver,
) -> Self {
FragmentedMessageReceiver {
received_buffer,
sphinx_packet_receiver,
}
}
fn start(mut self, handle: &Handle) -> JoinHandle<()> {
handle.spawn(async move {
while let Some(new_messages) = self.sphinx_packet_receiver.next().await {
self.received_buffer
.add_new_message_fragments(new_messages)
.await;
}
})
}
}
pub(crate) struct ReceivedMessagesBufferController {
fragmented_messsage_receiver: FragmentedMessageReceiver,
request_receiver: RequestReceiver,
}
impl ReceivedMessagesBufferController {
pub(crate) fn new(
query_receiver: ReceivedBufferRequestReceiver,
sphinx_packet_receiver: SphinxPacketReceiver,
) -> Self {
let received_buffer = ReceivedMessagesBuffer::new();
ReceivedMessagesBufferController {
fragmented_messsage_receiver: FragmentedMessageReceiver::new(
received_buffer.clone(),
sphinx_packet_receiver,
),
request_receiver: RequestReceiver::new(received_buffer, query_receiver),
}
}
pub(crate) fn start(self, handle: &Handle) {
// TODO: should we do anything with JoinHandle(s) returned by start methods?
self.fragmented_messsage_receiver.start(handle);
self.request_receiver.start(handle);
}
}
clients/native/src/client/topology_control.rs
-187
View File
@@ -1,187 +0,0 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::built_info;
use futures::lock::Mutex;
use log::*;
use nymsphinx::NodeAddressBytes;
use std::sync::Arc;
use std::time;
use std::time::Duration;
use tokio::runtime::Handle;
use tokio::task::JoinHandle;
use topology::{provider, NymTopology};
struct TopologyAccessorInner<T: NymTopology>(Option<T>);
impl<T: NymTopology> TopologyAccessorInner<T> {
fn new() -> Self {
TopologyAccessorInner(None)
}
fn update(&mut self, new: Option<T>) {
self.0 = new;
}
}
#[derive(Clone, Debug)]
pub(crate) struct TopologyAccessor<T: NymTopology> {
// TODO: this requires some actual benchmarking to determine if obtaining mutex is not going
// to cause some bottlenecking and whether perhaps RwLock would be better
inner: Arc<Mutex<TopologyAccessorInner<T>>>,
}
impl<T: NymTopology> TopologyAccessor<T> {
pub(crate) fn new() -> Self {
TopologyAccessor {
inner: Arc::new(Mutex::new(TopologyAccessorInner::new())),
}
}
async fn update_global_topology(&mut self, new_topology: Option<T>) {
self.inner.lock().await.update(new_topology);
}
pub(crate) async fn get_gateway_socket_url(&self, id: &str) -> Option<String> {
match &self.inner.lock().await.0 {
None => None,
Some(ref topology) => topology
.gateways()
.iter()
.find(|gateway| gateway.pub_key == id)
.map(|gateway| gateway.client_listener.clone()),
}
}
// only used by the client at startup to get a slightly more reasonable error message
// (currently displays as unused because healthchecker is disabled due to required changes)
pub(crate) async fn is_routable(&self) -> bool {
match &self.inner.lock().await.0 {
None => false,
Some(ref topology) => topology.can_construct_path_through(),
}
}
pub(crate) async fn get_all_clients(&self) -> Option<Vec<provider::Client>> {
// TODO: this will need to be modified to instead return pairs (provider, client)
match &self.inner.lock().await.0 {
None => None,
Some(ref topology) => Some(
topology
.providers()
.iter()
.flat_map(|provider| provider.registered_clients.iter())
.cloned()
.collect::<Vec<_>>(),
),
}
}
pub(crate) async fn random_route_to_gateway(
&self,
gateway: &NodeAddressBytes,
) -> Option<Vec<nymsphinx::Node>> {
let b58_address = gateway.to_base58_string();
let guard = self.inner.lock().await;
let topology = guard.0.as_ref()?;
let gateway = topology
.gateways()
.iter()
.cloned()
.find(|gateway| gateway.pub_key == b58_address.clone())?;
topology.random_route_to_gateway(gateway.into()).ok()
}
// // this is a rather temporary solution as each client will have an associated provider
// // currently that is not implemented yet and there only exists one provider in the network
// pub(crate) async fn random_route(&self) -> Option<Vec<nymsphinx::Node>> {
// match &self.inner.lock().await.0 {
// None => None,
// Some(ref topology) => {
// let mut gateways = topology.gateways();
// if gateways.is_empty() {
// return None;
// }
// // unwrap is fine here as we asserted there is at least single provider
// let provider = gateways.pop().unwrap().into();
// topology.random_route_to_gateway(provider).ok()
// }
// }
// }
}
pub(crate) struct TopologyRefresherConfig {
directory_server: String,
refresh_rate: time::Duration,
}
impl TopologyRefresherConfig {
pub(crate) fn new(directory_server: String, refresh_rate: time::Duration) -> Self {
TopologyRefresherConfig {
directory_server,
refresh_rate,
}
}
}
pub(crate) struct TopologyRefresher<T: NymTopology> {
directory_server: String,
topology_accessor: TopologyAccessor<T>,
refresh_rate: Duration,
}
impl<T: 'static + NymTopology> TopologyRefresher<T> {
pub(crate) fn new(
cfg: TopologyRefresherConfig,
topology_accessor: TopologyAccessor<T>,
) -> Self {
TopologyRefresher {
directory_server: cfg.directory_server,
topology_accessor,
refresh_rate: cfg.refresh_rate,
}
}
async fn get_current_compatible_topology(&self) -> T {
// note: this call makes it necessary that `T::new()`does *not* have 'static lifetime
let full_topology = T::new(self.directory_server.clone()).await;
// just filter by version and assume the validators will remove all bad behaving
// nodes with the staking
full_topology.filter_system_version(built_info::PKG_VERSION)
}
pub(crate) async fn refresh(&mut self) {
trace!("Refreshing the topology");
let new_topology = Some(self.get_current_compatible_topology().await);
self.topology_accessor
.update_global_topology(new_topology)
.await;
}
pub(crate) async fn is_topology_routable(&self) -> bool {
self.topology_accessor.is_routable().await
}
pub(crate) fn start(mut self, handle: &Handle) -> JoinHandle<()> {
handle.spawn(async move {
loop {
tokio::time::delay_for(self.refresh_rate).await;
self.refresh().await;
}
})
}
}
clients/native/src/commands/init.rs
+70 -86
View File
@@ -13,20 +13,21 @@
// limitations under the License.
use crate::built_info;
use crate::client::config::Config;
use crate::commands::override_config;
use crate::config::persistence::pathfinder::ClientPathfinder;
use clap::{App, Arg, ArgMatches};
use client_core::client::key_manager::KeyManager;
use client_core::config::persistence::key_pathfinder::ClientKeyPathfinder;
use config::NymConfig;
use crypto::identity::MixIdentityKeyPair;
use directory_client::presence::Topology;
use futures::channel::mpsc;
use crypto::asymmetric::identity;
use directory_client::DirectoryClient;
use gateway_client::GatewayClient;
use gateway_requests::AuthToken;
use nymsphinx::DestinationAddressBytes;
use pemstore::pemstore::PemStore;
use gateway_requests::registration::handshake::SharedKeys;
use rand::rngs::OsRng;
use std::convert::TryInto;
use std::sync::Arc;
use std::time::Duration;
use topology::gateway::Node;
use topology::NymTopology;
use topology::{gateway, NymTopology};
pub fn command_args<'a, 'b>() -> clap::App<'a, 'b> {
App::new("init")
@@ -39,8 +40,9 @@ pub fn command_args<'a, 'b>() -> clap::App<'a, 'b> {
)
.arg(Arg::with_name("gateway")
.long("gateway")
.help("Id of the gateway we have preference to connect to. If left empty, a random gateway will be chosen.")
.help("Id of the gateway we are going to connect to. If unsure what to put here - 'D6YaMzLSY7mANtSQRKXsmMZpqgqiVkeiagKM4V4oFPFr' is a safe choice for the testnet")
.takes_value(true)
.required(true)
)
.arg(Arg::with_name("directory")
.long("directory")
@@ -64,93 +66,80 @@ pub fn command_args<'a, 'b>() -> clap::App<'a, 'b> {
)
}
async fn try_gateway_registration(
gateways: Vec<Node>,
our_address: DestinationAddressBytes,
) -> Option<(String, AuthToken)> {
// TODO: having to do something like this suggests that perhaps GatewayClient's constructor
// could be improved
let (sphinx_tx, _) = mpsc::unbounded();
async fn register_with_gateway(
gateway: &gateway::Node,
our_identity: Arc<identity::KeyPair>,
) -> SharedKeys {
let timeout = Duration::from_millis(1500);
for gateway in gateways {
let mut gateway_client = GatewayClient::new(
url::Url::parse(&gateway.client_listener).unwrap(),
our_address.clone(),
None,
sphinx_tx.clone(),
timeout,
);
if gateway_client.establish_connection().await.is_ok() {
if let Ok(token) = gateway_client.register().await {
return Some((gateway.pub_key, token));
}
}
}
None
let mut gateway_client = GatewayClient::new_init(
url::Url::parse(&gateway.client_listener).unwrap(),
gateway.identity_key,
our_identity.clone(),
timeout,
);
gateway_client
.establish_connection()
.await
.expect("failed to establish connection with the gateway!");
gateway_client
.register()
.await
.expect("failed to register with the gateway!")
}
async fn choose_gateway(
directory_server: String,
our_address: DestinationAddressBytes,
) -> (String, AuthToken) {
// TODO: once we change to graph topology this here will need to be updated!
let topology = Topology::new(directory_server.clone()).await;
let version_filtered_topology = topology.filter_system_version(built_info::PKG_VERSION);
// don't care about health of the networks as mixes can go up and down any time,
// but DO care about gateways
let gateways = version_filtered_topology.gateways();
async fn gateway_details(directory_server: &str, gateway_id: &str) -> gateway::Node {
let directory_client_config = directory_client::Config::new(directory_server.to_string());
let directory_client = directory_client::Client::new(directory_client_config);
let topology = directory_client.get_topology().await.unwrap();
let nym_topology: NymTopology = topology.try_into().expect("Invalid topology data!");
let version_filtered_topology = nym_topology.filter_system_version(built_info::PKG_VERSION);
// try to perform registration so that we wouldn't need to do it at startup
// + at the same time we'll know if we can actually talk with that gateway
let registration_result = try_gateway_registration(gateways, our_address).await;
match registration_result {
None => {
// while technically there's no issue client-side, it will be impossible to execute
// `nym-client run` as no gateway is available so it might be best to not finalize
// the init and rely on users trying to init another time?
panic!(
"Currently there are no valid gateways available on the network ({}). \
Please try to run `init` again at later time or change your directory server",
directory_server
)
}
Some((gateway_id, auth_token)) => (gateway_id, auth_token),
}
version_filtered_topology
.gateways()
.iter()
.find(|gateway| gateway.identity_key.to_base58_string() == gateway_id)
.expect(&*format!("no gateway with id {} exists!", gateway_id))
.clone()
}
pub fn execute(matches: &ArgMatches) {
println!("Initialising client...");
let id = matches.value_of("id").unwrap(); // required for now
let mut config = crate::config::Config::new(id);
let mut config = Config::new(id);
let mut rng = OsRng;
config = override_config(config, matches);
if matches.is_present("fastmode") {
config = config.set_high_default_traffic_volume();
config.get_base_mut().set_high_default_traffic_volume();
}
let mix_identity_keys = MixIdentityKeyPair::new();
// create identity, encryption and ack keys.
let mut key_manager = KeyManager::new(&mut rng);
// if there is no gateway chosen, get a random-ish one from the topology
if config.get_gateway_id().is_empty() {
let our_address = mix_identity_keys.public_key().derive_address();
// TODO: is there perhaps a way to make it work without having to spawn entire runtime?
let mut rt = tokio::runtime::Runtime::new().unwrap();
let (gateway_id, auth_token) =
rt.block_on(choose_gateway(config.get_directory_server(), our_address));
let gateway_id = matches.value_of("gateway").unwrap();
// TODO: this isn't really a gateway, but gateway, yet another change to make
config = config
.with_gateway_id(gateway_id)
.with_gateway_auth_token(auth_token);
}
let registration_fut = async {
let gate_details =
gateway_details(&config.get_base().get_directory_server(), gateway_id).await;
let shared_keys =
register_with_gateway(&gate_details, key_manager.identity_keypair()).await;
(shared_keys, gate_details.client_listener)
};
let pathfinder = ClientPathfinder::new_from_config(&config);
let pem_store = PemStore::new(pathfinder);
pem_store
.write_identity(mix_identity_keys)
.expect("Failed to save identity keys");
println!("Saved mixnet identity keypair");
// TODO: is there perhaps a way to make it work without having to spawn entire runtime?
let mut rt = tokio::runtime::Runtime::new().unwrap();
let (shared_keys, gateway_listener) = rt.block_on(registration_fut);
config
.get_base_mut()
.with_gateway_listener(gateway_listener);
key_manager.insert_gateway_shared_key(shared_keys);
let pathfinder = ClientKeyPathfinder::new_from_config(config.get_base());
key_manager
.store_keys(&pathfinder)
.expect("Failed to generated keys");
println!("Saved all generated keys");
let config_save_location = config.get_config_file_save_location();
config
@@ -160,13 +149,8 @@ pub fn execute(matches: &ArgMatches) {
println!(
"Unless overridden in all `nym-client run` we will be talking to the following gateway: {}...",
config.get_gateway_id(),
config.get_base().get_gateway_id(),
);
if config.get_gateway_auth_token().is_some() {
println!(
"using optional AuthToken: {:?}",
config.get_gateway_auth_token().unwrap()
)
}
println!("Client configuration completed.\n\n\n")
}
clients/native/src/commands/mod.rs
+3 -3
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::config::{Config, SocketType};
use crate::client::config::{Config, SocketType};
use clap::ArgMatches;
pub mod init;
@@ -20,11 +20,11 @@ pub mod run;
pub(crate) fn override_config(mut config: Config, matches: &ArgMatches) -> Config {
if let Some(directory) = matches.value_of("directory") {
config = config.with_custom_directory(directory);
config.get_base_mut().with_custom_directory(directory);
}
if let Some(gateway_id) = matches.value_of("gateway") {
config = config.with_gateway_id(gateway_id);
config.get_base_mut().with_gateway_id(gateway_id);
}
if matches.is_present("disable-socket") {
clients/native/src/commands/run.rs
+3 -16
View File
@@ -12,13 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::client::config::Config;
use crate::client::NymClient;
use crate::commands::override_config;
use crate::config::{persistence::pathfinder::ClientPathfinder, Config};
use clap::{App, Arg, ArgMatches};
use config::NymConfig;
use crypto::identity::MixIdentityKeyPair;
use pemstore::pemstore::PemStore;
pub fn command_args<'a, 'b>() -> clap::App<'a, 'b> {
App::new("run")
@@ -57,17 +55,6 @@ pub fn command_args<'a, 'b>() -> clap::App<'a, 'b> {
)
}
fn load_identity_keys(config_file: &Config) -> MixIdentityKeyPair {
let identity_keypair = PemStore::new(ClientPathfinder::new_from_config(&config_file))
.read_identity()
.expect("Failed to read stored identity key files");
println!(
"Public identity key: {}\n",
identity_keypair.public_key.to_base58_string()
);
identity_keypair
}
pub fn execute(matches: &ArgMatches) {
let id = matches.value_of("id").unwrap();
@@ -76,6 +63,6 @@ pub fn execute(matches: &ArgMatches) {
.expect("Failed to load config file");
config = override_config(config, matches);
let identity_keypair = load_identity_keys(&config);
NymClient::new(config, identity_keypair).run_forever();
NymClient::new(config).run_forever();
}
clients/native/src/config/persistence/pathfinder.rs
-60
View File
@@ -1,60 +0,0 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::config::Config;
use pemstore::pathfinder::PathFinder;
use std::path::PathBuf;
#[derive(Debug)]
pub struct ClientPathfinder {
pub config_dir: PathBuf,
pub private_mix_key: PathBuf,
pub public_mix_key: PathBuf,
}
impl ClientPathfinder {
pub fn new(id: String) -> Self {
let os_config_dir = dirs::config_dir().unwrap(); // grabs the OS default config dir
let config_dir = os_config_dir.join("nym").join("clients").join(id);
let private_mix_key = config_dir.join("private.pem");
let public_mix_key = config_dir.join("public.pem");
ClientPathfinder {
config_dir,
private_mix_key,
public_mix_key,
}
}
pub fn new_from_config(config: &Config) -> Self {
ClientPathfinder {
config_dir: config.get_config_file_save_location(),
private_mix_key: config.get_private_identity_key_file(),
public_mix_key: config.get_public_identity_key_file(),
}
}
}
impl PathFinder for ClientPathfinder {
fn config_dir(&self) -> PathBuf {
self.config_dir.clone()
}
fn private_identity_key(&self) -> PathBuf {
self.private_mix_key.clone()
}
fn public_identity_key(&self) -> PathBuf {
self.public_mix_key.clone()
}
}
clients/native/src/lib.rs
-1
View File
@@ -14,5 +14,4 @@
pub mod built_info;
pub mod client;
pub mod config;
pub mod websocket;
clients/native/src/main.rs
+2 -2
View File
@@ -16,14 +16,14 @@ use clap::{App, ArgMatches};
pub mod built_info;
pub mod client;
mod commands;
pub mod config;
pub mod commands;
pub mod websocket;
fn main() {
dotenv::dotenv().ok();
setup_logging();
println!("{}", banner());
println!("\n\nIf unsure which gateway to choose - 'D6YaMzLSY7mANtSQRKXsmMZpqgqiVkeiagKM4V4oFPFr' is a safe choice for the testnet\n\n");
let arg_matches = App::new("Nym Client")
.version(built_info::PKG_VERSION)
clients/native/src/websocket/handler.rs
+111 -134
View File
@@ -12,27 +12,25 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use super::types::{BinaryClientRequest, ClientRequest, ServerResponse};
use crate::client::{
use client_core::client::{
inbound_messages::{InputMessage, InputMessageSender},
received_buffer::{
ReceivedBufferMessage, ReceivedBufferRequestSender, ReconstructedMessagesReceiver,
},
topology_control::TopologyAccessor,
};
use futures::channel::mpsc;
use futures::{SinkExt, StreamExt};
use log::*;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::chunking::split_and_prepare_payloads;
use std::convert::TryFrom;
use nymsphinx::anonymous_replies::ReplySURB;
use nymsphinx::receiver::ReconstructedMessage;
use tokio::net::TcpStream;
use tokio_tungstenite::{
accept_async,
tungstenite::{protocol::Message, Error as WsError},
tungstenite::{protocol::Message as WsMessage, Error as WsError},
WebSocketStream,
};
use topology::NymTopology;
use websocket_requests::{requests::ClientRequest, responses::ServerResponse};
enum ReceivedResponseType {
Binary,
@@ -45,30 +43,28 @@ impl Default for ReceivedResponseType {
}
}
pub(crate) struct Handler<T: NymTopology> {
pub(crate) struct Handler {
msg_input: InputMessageSender,
buffer_requester: ReceivedBufferRequestSender,
self_full_address: Recipient,
topology_accessor: TopologyAccessor<T>,
socket: Option<WebSocketStream<TcpStream>>,
received_response_type: ReceivedResponseType,
}
// clone is used to use handler on a new connection, which initially is `None`
impl<T: NymTopology> Clone for Handler<T> {
impl Clone for Handler {
fn clone(&self) -> Self {
Handler {
msg_input: self.msg_input.clone(),
buffer_requester: self.buffer_requester.clone(),
self_full_address: self.self_full_address.clone(),
topology_accessor: self.topology_accessor.clone(),
socket: None,
received_response_type: Default::default(),
}
}
}
impl<T: NymTopology> Drop for Handler<T> {
impl Drop for Handler {
fn drop(&mut self) {
self.buffer_requester
.unbounded_send(ReceivedBufferMessage::ReceiverDisconnect)
@@ -76,163 +72,142 @@ impl<T: NymTopology> Drop for Handler<T> {
}
}
impl<T: NymTopology> Handler<T> {
impl Handler {
pub(crate) fn new(
msg_input: InputMessageSender,
buffer_requester: ReceivedBufferRequestSender,
self_full_address: Recipient,
topology_accessor: TopologyAccessor<T>,
) -> Self {
Handler {
msg_input,
buffer_requester,
self_full_address,
topology_accessor,
socket: None,
received_response_type: Default::default(),
}
}
fn handle_text_send(&mut self, msg: String, full_recipient_address: String) -> ServerResponse {
let message_bytes = msg.into_bytes();
fn handle_send(
&mut self,
recipient: Recipient,
message: Vec<u8>,
with_reply_surb: bool,
) -> Option<ServerResponse> {
// the ack control is now responsible for chunking, etc.
let input_msg = InputMessage::new_fresh(recipient, message, with_reply_surb);
self.msg_input.unbounded_send(input_msg).unwrap();
let recipient = match Recipient::try_from_string(full_recipient_address) {
Ok(address) => address,
Err(e) => {
trace!("failed to parse received Recipient: {:?}", e);
return ServerResponse::new_error("malformed recipient address");
}
};
None
}
// in case the message is too long and needs to be split into multiple packets:
let split_message = split_and_prepare_payloads(&message_bytes);
for message_fragment in split_message {
let input_msg = InputMessage::new(recipient.clone(), message_fragment);
self.msg_input.unbounded_send(input_msg).unwrap();
fn handle_reply(&mut self, reply_surb: ReplySURB, message: Vec<u8>) -> Option<ServerResponse> {
if message.len() > ReplySURB::max_msg_len(Default::default()) {
return Some(ServerResponse::new_error(format!("too long message to put inside a reply SURB. Received: {} bytes and maximum is {} bytes", message.len(), ReplySURB::max_msg_len(Default::default()))));
}
let input_msg = InputMessage::new_reply(reply_surb, message);
self.msg_input.unbounded_send(input_msg).unwrap();
None
}
fn handle_self_address(&self) -> ServerResponse {
ServerResponse::SelfAddress(self.self_full_address.clone())
}
fn handle_request(&mut self, request: ClientRequest) -> Option<ServerResponse> {
match request {
ClientRequest::Send {
recipient,
message,
with_reply_surb,
} => self.handle_send(recipient, message, with_reply_surb),
ClientRequest::Reply {
message,
reply_surb,
} => self.handle_reply(reply_surb, message),
ClientRequest::SelfAddress => Some(self.handle_self_address()),
}
}
fn handle_text_message(&mut self, msg: String) -> Option<WsMessage> {
debug!("Handling text message request");
trace!("Content: {:?}", msg);
self.received_response_type = ReceivedResponseType::Text;
let client_request = ClientRequest::try_from_text(msg);
ServerResponse::Send
let response = match client_request {
Err(err) => Some(ServerResponse::Error(err)),
Ok(req) => self.handle_request(req),
};
response.map(|resp| WsMessage::text(resp.into_text()))
}
async fn handle_text_get_clients(&mut self) -> ServerResponse {
match self.topology_accessor.get_all_clients().await {
Some(clients) => {
let client_keys = clients.into_iter().map(|client| client.pub_key).collect();
ServerResponse::GetClients {
clients: client_keys,
}
}
None => ServerResponse::new_error("invalid network topology"),
}
}
fn handle_text_self_address(&self) -> ServerResponse {
ServerResponse::SelfAddress {
address: self.self_full_address.to_string(),
}
}
async fn handle_text_message(&mut self, msg: String) -> Message {
debug!("Handling text message request");
trace!("Content: {:?}", msg.clone());
match ClientRequest::try_from(msg) {
Err(e) => ServerResponse::Error {
message: format!("received invalid request. err: {:?}", e),
}
.into(),
Ok(req) => match req {
ClientRequest::Send { message, recipient } => {
self.handle_text_send(message, recipient)
}
ClientRequest::GetClients => self.handle_text_get_clients().await,
ClientRequest::SelfAddress => self.handle_text_self_address(),
}
.into(),
}
}
async fn handle_binary_send(&mut self, recipient: Recipient, data: Vec<u8>) -> ServerResponse {
// in case the message is too long and needs to be split into multiple packets:
let split_message = split_and_prepare_payloads(&data);
for message_fragment in split_message {
let input_msg = InputMessage::new(recipient.clone(), message_fragment);
self.msg_input.unbounded_send(input_msg).unwrap();
}
self.received_response_type = ReceivedResponseType::Binary;
ServerResponse::Send
}
// if it's binary we assume it's a sphinx packet formatted the same way as we'd have sent
// it to the gateway
async fn handle_binary_message(&mut self, msg: Vec<u8>) -> Message {
fn handle_binary_message(&mut self, msg: Vec<u8>) -> Option<WsMessage> {
debug!("Handling binary message request");
self.received_response_type = ReceivedResponseType::Binary;
// make sure it is correctly formatted
let binary_request = BinaryClientRequest::try_from_bytes(&msg);
if binary_request.is_none() {
return ServerResponse::new_error("invalid binary request").into();
}
match binary_request.unwrap() {
BinaryClientRequest::Send { recipient, data } => {
self.handle_binary_send(recipient, data).await
}
}
.into()
let client_request = ClientRequest::try_from_binary(msg);
let response = match client_request {
Err(err) => Some(ServerResponse::Error(err)),
Ok(req) => self.handle_request(req),
};
response.map(|resp| WsMessage::Binary(resp.into_binary()))
}
async fn handle_request(&mut self, raw_request: Message) -> Option<Message> {
fn handle_ws_request(&mut self, raw_request: WsMessage) -> Option<WsMessage> {
// apparently tungstenite auto-handles ping/pong/close messages so for now let's ignore
// them and let's test that claim. If that's not the case, just copy code from
// old version of this file.
match raw_request {
Message::Text(text_message) => Some(self.handle_text_message(text_message).await),
Message::Binary(binary_message) => {
Some(self.handle_binary_message(binary_message).await)
}
WsMessage::Text(text_message) => self.handle_text_message(text_message),
WsMessage::Binary(binary_message) => self.handle_binary_message(binary_message),
_ => None,
}
}
// I'm still not entirely sure why `send_all` requires `TryStream` rather than `Stream`, but
// let's just play along for now
fn prepare_reconstructed_binary(
&self,
reconstructed_messages: Vec<ReconstructedMessage>,
) -> Vec<Result<WsMessage, WsError>> {
reconstructed_messages
.into_iter()
.map(ServerResponse::Received)
.map(|resp| Ok(WsMessage::Binary(resp.into_binary())))
.collect()
}
// I'm still not entirely sure why `send_all` requires `TryStream` rather than `Stream`, but
// let's just play along for now
fn prepare_reconstructed_text(
&self,
reconstructed_messages: Vec<ReconstructedMessage>,
) -> Vec<Result<WsMessage, WsError>> {
reconstructed_messages
.into_iter()
.map(ServerResponse::Received)
.map(|resp| Ok(WsMessage::Text(resp.into_text())))
.collect()
}
async fn push_websocket_received_plaintexts(
&mut self,
messages_bytes: Vec<Vec<u8>>,
reconstructed_messages: Vec<ReconstructedMessage>,
) -> Result<(), WsError> {
let response_messages: Vec<_> = match self.received_response_type {
ReceivedResponseType::Binary => messages_bytes
.into_iter()
.map(|msg| Ok(Message::Binary(msg)))
.collect(),
ReceivedResponseType::Text => {
let mut decoded_messages = Vec::new();
// either all succeed or all fall back
let mut did_fail = false;
for message in messages_bytes.iter() {
match std::str::from_utf8(message) {
Ok(msg) => decoded_messages.push(msg),
Err(err) => {
did_fail = true;
warn!("Invalid UTF-8 sequence in response message - {:?}", err);
break;
}
}
}
if did_fail {
messages_bytes
.into_iter()
.map(|msg| Ok(Message::Binary(msg)))
.collect()
} else {
decoded_messages
.into_iter()
.map(|msg| Ok(Message::Text(msg.to_string())))
.collect()
}
// TODO: later there might be a flag on the reconstructed message itself to tell us
// if it's text or binary, but for time being we use the naive assumption that if
// client is sending Message::Text it expects text back. Same for Message::Binary
let response_messages = match self.received_response_type {
ReceivedResponseType::Binary => {
self.prepare_reconstructed_binary(reconstructed_messages)
}
ReceivedResponseType::Text => self.prepare_reconstructed_text(reconstructed_messages),
};
let mut send_stream = futures::stream::iter(response_messages);
@@ -243,7 +218,7 @@ impl<T: NymTopology> Handler<T> {
.await
}
async fn send_websocket_response(&mut self, msg: Message) -> Result<(), WsError> {
async fn send_websocket_response(&mut self, msg: WsMessage) -> Result<(), WsError> {
match self.socket {
// TODO: more closely investigate difference between `Sink::send` and `Sink::send_all`
// it got something to do with batching and flushing - it might be important if it
@@ -253,7 +228,7 @@ impl<T: NymTopology> Handler<T> {
}
}
async fn next_websocket_request(&mut self) -> Option<Result<Message, WsError>> {
async fn next_websocket_request(&mut self) -> Option<Result<WsMessage, WsError>> {
match self.socket {
Some(ref mut ws_stream) => ws_stream.next().await,
None => None,
@@ -263,6 +238,7 @@ impl<T: NymTopology> Handler<T> {
async fn listen_for_requests(&mut self, mut msg_receiver: ReconstructedMessagesReceiver) {
loop {
tokio::select! {
// we can either get a client request from the websocket
socket_msg = self.next_websocket_request() => {
if socket_msg.is_none() {
break;
@@ -270,7 +246,7 @@ impl<T: NymTopology> Handler<T> {
let socket_msg = match socket_msg.unwrap() {
Ok(socket_msg) => socket_msg,
Err(err) => {
error!("failed to obtain message from websocket stream! stopping connection handler: {}", err);
warn!("failed to obtain message from websocket stream! stopping connection handler: {}", err);
break;
}
};
@@ -279,7 +255,7 @@ impl<T: NymTopology> Handler<T> {
break;
}
if let Some(response) = self.handle_request(socket_msg).await {
if let Some(response) = self.handle_ws_request(socket_msg) {
if let Err(err) = self.send_websocket_response(response).await {
warn!(
"Failed to send message over websocket: {}. Assuming the connection is dead.",
@@ -289,6 +265,7 @@ impl<T: NymTopology> Handler<T> {
}
}
}
// or a reconstructed mix message that we need to push back to the client
mix_messages = msg_receiver.next() => {
let mix_messages = mix_messages.expect(
"mix messages sender was unexpectedly closed! this shouldn't have ever happened!",
clients/native/src/websocket/listener.rs
+2 -7
View File
@@ -20,7 +20,6 @@ use std::{
};
use tokio::runtime;
use tokio::{sync::Notify, task::JoinHandle};
use topology::NymTopology;
enum State {
Connected,
@@ -50,7 +49,7 @@ impl Listener {
}
}
pub(crate) async fn run<T: NymTopology + 'static>(&mut self, handler: Handler<T>) {
pub(crate) async fn run(&mut self, handler: Handler) {
let mut tcp_listener = tokio::net::TcpListener::bind(self.address)
.await
.expect("Failed to start websocket listener");
@@ -100,11 +99,7 @@ impl Listener {
}
}
pub(crate) fn start<T: NymTopology + 'static>(
mut self,
rt_handle: &runtime::Handle,
handler: Handler<T>,
) -> JoinHandle<()> {
pub(crate) fn start(mut self, rt_handle: &runtime::Handle, handler: Handler) -> JoinHandle<()> {
info!("Running websocket on {:?}", self.address.to_string());
rt_handle.spawn(async move { self.run(handler).await })
clients/native/src/websocket/mod.rs
-3
View File
@@ -14,9 +14,6 @@
pub(crate) mod handler;
pub(crate) mod listener;
pub(crate) mod types;
pub(crate) use handler::Handler;
pub(crate) use listener::Listener;
pub use types::{BinaryClientRequest, ClientRequest, ServerResponse};
clients/native/src/websocket/types.rs
-114
View File
@@ -1,114 +0,0 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use nymsphinx::addressing::clients::Recipient;
use serde::{Deserialize, Serialize};
use std::convert::TryFrom;
use tokio_tungstenite::tungstenite::protocol::Message;
#[derive(Serialize, Deserialize, Debug)]
#[serde(tag = "type", rename_all = "camelCase")]
pub enum ClientRequest {
Send { message: String, recipient: String },
GetClients,
SelfAddress,
}
impl TryFrom<String> for ClientRequest {
type Error = serde_json::Error;
fn try_from(msg: String) -> Result<Self, Self::Error> {
serde_json::from_str(&msg)
}
}
impl Into<Message> for ClientRequest {
fn into(self) -> Message {
let str_req = serde_json::to_string(&self).unwrap();
Message::Text(str_req)
}
}
pub enum BinaryClientRequest {
Send { recipient: Recipient, data: Vec<u8> },
}
impl BinaryClientRequest {
// TODO: perhaps do it the proper way and introduce an error type
pub fn try_from_bytes(req: &[u8]) -> Option<Self> {
if req.len() < Recipient::LEN {
return None;
}
let mut recipient_bytes = [0u8; Recipient::LEN];
recipient_bytes.copy_from_slice(&req[..Recipient::LEN]);
let recipient = Recipient::from_bytes(recipient_bytes);
Some(BinaryClientRequest::Send {
recipient,
data: req[Recipient::LEN..].to_vec(),
})
}
pub fn into_bytes(self) -> Vec<u8> {
match self {
Self::Send { recipient, data } => recipient
.into_bytes()
.iter()
.cloned()
.chain(data.into_iter())
.collect(),
}
}
}
impl Into<Message> for BinaryClientRequest {
fn into(self) -> Message {
Message::Binary(self.into_bytes())
}
}
#[derive(Serialize, Deserialize, Debug)]
#[serde(tag = "type", rename_all = "camelCase")]
pub enum ServerResponse {
Send,
Received { messages: Vec<String> },
GetClients { clients: Vec<String> },
SelfAddress { address: String },
Error { message: String },
}
impl ServerResponse {
pub fn new_error<S: Into<String>>(msg: S) -> Self {
ServerResponse::Error {
message: msg.into(),
}
}
}
impl TryFrom<String> for ServerResponse {
type Error = serde_json::Error;
fn try_from(msg: String) -> Result<Self, <ServerResponse as TryFrom<String>>::Error> {
serde_json::from_str(&msg)
}
}
impl Into<Message> for ServerResponse {
fn into(self) -> Message {
// it should be safe to call `unwrap` here as the message is generated by the server
// so if it fails (and consequently panics) it's a bug that should be resolved
let str_res = serde_json::to_string(&self).unwrap();
Message::Text(str_res)
}
}
clients/native/websocket-requests/Cargo.toml
+13
View File
@@ -0,0 +1,13 @@
[package]
name = "websocket-requests"
version = "0.1.0"
authors = ["Jędrzej Stuczyński <andrew@nymtech.net>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
nymsphinx = { path = "../../../common/nymsphinx" }
clients/native/websocket-requests/src/error.rs
+92
View File
@@ -0,0 +1,92 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use serde::{Deserialize, Serialize};
use std::fmt;
// no need to go fancy here like we've done in other places.
#[derive(PartialEq, Clone, Serialize, Deserialize)]
pub struct Error {
pub kind: ErrorKind,
pub message: String,
}
impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}:{}", self.kind.as_str(), self.message)
}
}
impl fmt::Debug for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
fmt::Display::fmt(self, f)
}
}
impl Error {
pub fn new(kind: ErrorKind, message: String) -> Self {
Error { kind, message }
}
}
#[repr(u8)]
#[derive(PartialEq, Clone, Serialize, Deserialize)]
pub enum ErrorKind {
/// The received request contained no data.
EmptyRequest = 0x01,
/// The received request did not contain enough data to be fully parsed.
TooShortRequest = 0x02,
/// The received request tag is not defined.
UnknownRequest = 0x03,
/// The received request is malformed.
MalformedRequest = 0x04,
// that's an arbitrary division but lets keep 1-127 (hex 0x01 - 0x7F) values request-specific
// and 128-254 (hex 0x80 - 0xFE) for responses
/// The received response contained no data.
EmptyResponse = 0x80,
/// The received response did not contain enough data to be fully parsed.
TooShortResponse = 0x81,
/// The received response tag is not defined.
UnknownResponse = 0x82,
/// The received response is malformed.
MalformedResponse = 0x83,
/// The error is due to something else.
Other = 0xFF,
}
impl ErrorKind {
pub(crate) fn as_str(&self) -> &'static str {
match *self {
ErrorKind::EmptyRequest => "received request contained no data",
ErrorKind::TooShortRequest => "received request did not contain enough data",
ErrorKind::UnknownRequest => "unknown request type",
ErrorKind::MalformedRequest => "malformed request",
ErrorKind::EmptyResponse => "received response contained no data",
ErrorKind::TooShortResponse => "received response did not contain enough data",
ErrorKind::UnknownResponse => "unknown response type",
ErrorKind::MalformedResponse => "malformed response",
ErrorKind::Other => "other",
}
}
}
clients/native/websocket-requests/src/lib.rs
+18
View File
@@ -0,0 +1,18 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub mod error;
pub mod requests;
pub mod responses;
mod text;
clients/native/websocket-requests/src/requests.rs
+366
View File
@@ -0,0 +1,366 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// all variable size data is always prefixed with u64 length
// tags are u8
use crate::error::{self, ErrorKind};
use crate::text::ClientRequestText;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::anonymous_replies::ReplySURB;
use std::convert::{TryFrom, TryInto};
use std::mem::size_of;
/// Value tag representing [`Send`] variant of the [`ClientRequest`]
pub const SEND_REQUEST_TAG: u8 = 0x00;
/// Value tag representing [`Reply`] variant of the [`ClientRequest`]
pub const REPLY_REQUEST_TAG: u8 = 0x01;
/// Value tag representing [`SelfAddress`] variant of the [`ClientRequest`]
pub const SELF_ADDRESS_REQUEST_TAG: u8 = 0x02;
#[allow(non_snake_case)]
#[derive(Debug)]
pub enum ClientRequest {
Send {
recipient: Recipient,
message: Vec<u8>,
// Perhaps we could change it to a number to indicate how many reply_SURBs we want to include?
with_reply_surb: bool,
},
Reply {
message: Vec<u8>,
reply_surb: ReplySURB,
},
SelfAddress,
}
// we could have been parsing it directly TryFrom<WsMessage>, but we want to retain
// information about whether it came from binary or text to send appropriate response back
impl ClientRequest {
// SEND_REQUEST_TAG || with_surb || recipient || data_len || data
fn serialize_send(recipient: Recipient, data: Vec<u8>, with_reply_surb: bool) -> Vec<u8> {
let data_len_bytes = (data.len() as u64).to_be_bytes();
std::iter::once(SEND_REQUEST_TAG)
.chain(std::iter::once(with_reply_surb as u8))
.chain(recipient.to_bytes().iter().cloned()) // will not be length prefixed because the length is constant
.chain(data_len_bytes.iter().cloned())
.chain(data.into_iter())
.collect()
}
// SEND_REQUEST_TAG || with_reply || recipient || data_len || data
fn deserialize_send(b: &[u8]) -> Result<Self, error::Error> {
// we need to have at least 1 (tag) + 1 (reply flag) + Recipient::LEN + sizeof<u64> bytes
if b.len() < 2 + Recipient::LEN + size_of::<u64>() {
return Err(error::Error::new(
ErrorKind::TooShortRequest,
"not enough data provided to recover 'send'".to_string(),
));
}
// this MUST match because it was called by 'deserialize'
debug_assert_eq!(b[0], SEND_REQUEST_TAG);
let with_reply_surb = match b[1] {
0 => false,
1 => true,
n => {
return Err(error::Error::new(
ErrorKind::MalformedRequest,
format!("invalid reply surb flag {}", n),
))
}
};
let mut recipient_bytes = [0u8; Recipient::LEN];
recipient_bytes.copy_from_slice(&b[2..2 + Recipient::LEN]);
let recipient = match Recipient::try_from_bytes(recipient_bytes) {
Ok(recipient) => recipient,
Err(err) => {
return Err(error::Error::new(
ErrorKind::MalformedRequest,
format!("malformed recipient: {:?}", err),
))
}
};
let data_len_bytes = &b[2 + Recipient::LEN..2 + Recipient::LEN + size_of::<u64>()];
let data_len = u64::from_be_bytes(data_len_bytes.try_into().unwrap());
let data = &b[2 + Recipient::LEN + size_of::<u64>()..];
if data.len() as u64 != data_len {
return Err(error::Error::new(
ErrorKind::MalformedRequest,
format!(
"data len has inconsistent length. specified: {} got: {}",
data_len,
data.len()
),
));
}
Ok(ClientRequest::Send {
with_reply_surb,
recipient,
message: data.to_vec(),
})
}
// REPLY_REQUEST_TAG || surb_len || surb || message_len || message
fn serialize_reply(message: Vec<u8>, reply_surb: ReplySURB) -> Vec<u8> {
let reply_surb_bytes = reply_surb.to_bytes();
let surb_len_bytes = (reply_surb_bytes.len() as u64).to_be_bytes();
let message_len_bytes = (message.len() as u64).to_be_bytes();
std::iter::once(REPLY_REQUEST_TAG)
.chain(surb_len_bytes.iter().cloned())
.chain(reply_surb_bytes.into_iter())
.chain(message_len_bytes.iter().cloned())
.chain(message.into_iter())
.collect()
}
// REPLY_REQUEST_TAG || surb_len || surb || message_len || message
fn deserialize_reply(b: &[u8]) -> Result<Self, error::Error> {
// we need to have at the very least 2 * sizeof<u64> bytes (in case, for some peculiar reason
// message and reply surb were 0 len - the request would still be malformed, but would in theory
// be parse'able)
if b.len() < 1 + 2 * size_of::<u64>() {
return Err(error::Error::new(
ErrorKind::TooShortRequest,
"not enough data provided to recover 'reply'".to_string(),
));
}
// this MUST match because it was called by 'deserialize'
debug_assert_eq!(b[0], REPLY_REQUEST_TAG);
let reply_surb_len =
u64::from_be_bytes(b[1..1 + size_of::<u64>()].as_ref().try_into().unwrap());
// make sure we won't go out of bounds here
if reply_surb_len > (b.len() - 1 + 2 * size_of::<u64>()) as u64 {
return Err(error::Error::new(
ErrorKind::MalformedRequest,
format!(
"not enough data to recover reply surb with specified length {}",
reply_surb_len
),
));
}
let surb_bound = 1 + size_of::<u64>() + reply_surb_len as usize;
let reply_surb_bytes = &b[1 + size_of::<u64>()..surb_bound];
let reply_surb = match ReplySURB::from_bytes(reply_surb_bytes) {
Ok(reply_surb) => reply_surb,
Err(err) => {
return Err(error::Error::new(
ErrorKind::MalformedRequest,
format!("malformed reply surb: {:?}", err),
))
}
};
let message_len = u64::from_be_bytes(
b[surb_bound..surb_bound + size_of::<u64>()]
.as_ref()
.try_into()
.unwrap(),
);
let message = &b[surb_bound + size_of::<u64>()..];
if message.len() as u64 != message_len {
return Err(error::Error::new(
ErrorKind::MalformedRequest,
format!(
"message len has inconsistent length. specified: {} got: {}",
message_len,
message.len()
),
));
}
// TODO: should this blow HERE, i.e. during deserialization that the data you're trying
// to send via reply is too long?
Ok(ClientRequest::Reply {
reply_surb,
message: message.to_vec(),
})
}
// SELF_ADDRESS_REQUEST_TAG
fn serialize_self_address() -> Vec<u8> {
std::iter::once(SELF_ADDRESS_REQUEST_TAG).collect()
}
// SELF_ADDRESS_REQUEST_TAG
fn deserialize_self_address(b: &[u8]) -> Result<Self, error::Error> {
// this MUST match because it was called by 'deserialize'
debug_assert_eq!(b[0], SELF_ADDRESS_REQUEST_TAG);
Ok(ClientRequest::SelfAddress)
}
pub fn serialize(self) -> Vec<u8> {
match self {
ClientRequest::Send {
recipient,
message,
with_reply_surb,
} => Self::serialize_send(recipient, message, with_reply_surb),
ClientRequest::Reply {
message,
reply_surb,
} => Self::serialize_reply(message, reply_surb),
ClientRequest::SelfAddress => Self::serialize_self_address(),
}
}
pub fn deserialize(b: &[u8]) -> Result<Self, error::Error> {
if b.is_empty() {
// technically I'm not even sure this can ever be returned, because reading empty
// request would imply closed socket, but let's include it for completion sake
return Err(error::Error::new(
ErrorKind::EmptyRequest,
"no data provided".to_string(),
));
}
if b.len() < size_of::<u8>() {
return Err(error::Error::new(
ErrorKind::TooShortRequest,
format!(
"not enough data provided to recover request tag. Provided only {} bytes",
b.len()
),
));
}
let request_tag = b[0];
// determine what kind of request that is and try to deserialize it
match request_tag {
SEND_REQUEST_TAG => Self::deserialize_send(b),
REPLY_REQUEST_TAG => Self::deserialize_reply(b),
SELF_ADDRESS_REQUEST_TAG => Self::deserialize_self_address(b),
n => Err(error::Error::new(
ErrorKind::UnknownRequest,
format!("type {}", n),
)),
}
}
pub fn try_from_binary(raw_req: Vec<u8>) -> Result<Self, error::Error> {
Self::deserialize(&raw_req)
}
pub fn try_from_text(raw_req: String) -> Result<Self, error::Error> {
// use the intermediate string structure and let serde do bunch of work for us
let text_req = ClientRequestText::try_from(raw_req).map_err(|json_err| {
error::Error::new(ErrorKind::MalformedRequest, json_err.to_string())
})?;
text_req.try_into()
}
}
#[cfg(test)]
mod tests {
use super::*;
// very basic tests to check for obvious errors like off by one
#[test]
fn send_request_serialization_works() {
let recipient = Recipient::try_from_base58_string("CytBseW6yFXUMzz4SGAKdNLGR7q3sJLLYxyBGvutNEQV.4QXYyEVc5fUDjmmi8PrHN9tdUFV4PCvSJE1278cHyvoe@4sBbL1ngf1vtNqykydQKTFh26sQCw888GpUqvPvyNB4f").unwrap();
let recipient_string = recipient.to_string();
let send_request_no_surb = ClientRequest::Send {
recipient: recipient.clone(),
message: b"foomp".to_vec(),
with_reply_surb: false,
};
let bytes = send_request_no_surb.serialize();
let recovered = ClientRequest::deserialize(&bytes).unwrap();
match recovered {
ClientRequest::Send {
recipient,
message,
with_reply_surb,
} => {
assert_eq!(recipient.to_string(), recipient_string);
assert_eq!(message, b"foomp".to_vec());
assert!(!with_reply_surb)
}
_ => unreachable!(),
}
let send_request_surb = ClientRequest::Send {
recipient,
message: b"foomp".to_vec(),
with_reply_surb: true,
};
let bytes = send_request_surb.serialize();
let recovered = ClientRequest::deserialize(&bytes).unwrap();
match recovered {
ClientRequest::Send {
recipient,
message,
with_reply_surb,
} => {
assert_eq!(recipient.to_string(), recipient_string);
assert_eq!(message, b"foomp".to_vec());
assert!(with_reply_surb)
}
_ => unreachable!(),
}
}
#[test]
fn reply_request_serialization_works() {
let reply_surb_string = "CjfVbHbfAjbC3W1BvNHGXmM8KNAnDNYGaHMLqVDxRYeo352csAihstup9bvqXam4dTWgfHak6KYwL9STaxWJ47E8XFZbSEvs7hEsfCkxr6K9WJuSBPK84GDDEvad8ZAuMCoaXsAd5S2Lj9a5eYyzG4SL1jHzhSMni55LyJwumxo1ZTGZNXggxw1RREosvyzNrW9Rsi3owyPqLCwXpiei2tHZty8w8midVvg8vDa7ZEJD842CLv8D4ohynSG7gDpqTrhkRaqYAuz7dzqNbMXLJRM7v823Jn16fA1L7YQxmcaUdUigyRSgTdb4i9ebiLGSyJ1iDe6Acz613PQZh6Ua3bZ2zVKq3dSycpDm9ngarRK4zJrAaUxRkdih8YzW3BY4nL9eqkfKA4N1TWCLaRU7zpSaf8yMEwrAZReU3d5zLV8c5KBfa2w8R5anhQeBojduZEGEad8kkHuKU52Zg93FeWHvH1qgZaEJMHH4nN7gKXz9mvWDhYwyF4vt3Uy2NhCHC3N5pL1gMme27YcoPcTEia1fxKZtnt6rtEozzTrAgCJGswigkFbkafiV5QaJwLKTUxtzhkZ57eEuLPte9UvJHzhhXUQ2CV7R2BUkJjYZy3Zsx6YYvdYWiAFFkWUwNEGA4QpShUHciBfsQVHQ7pN41YcyYUhbywQDFnTVgEmdUZ1XCBi3gyK5U3tDQmFzP1u9m3mWrUA8qB9mRDE7ptNDm5c3c1458L6uXLUth7sdMaa1Was5LCmCdmNDtvNpCDAEt1in6q6mrZFR85aCSU9b1baNGwZoCqPpPvydkVe63gXWoi8ebvdyxARrqACFrSB3ZdY3uJBw8CTMNkKK6MvcefMkSVVsbLd36TQAtYSCqrpiMc5dQuKcEu5QfciwvWYXYx8WFNAgKwP2mv49KCTvfozNDUCbjzDwSx92Zv5zjG8HbFpB13bY9UZGeyTPvv7gGxCzjGjJGbW6FRAheRQaaje5fUgCNM95Tv7wBmAMRHHFgWafeK1sdFH7dtCX9u898HucGTaboSKLsVh8J78gbbkHErwjMh7y9YRkceq5TTYS5da4kHnyNKYWSbxgZrmFg44XGKoeYcqoHB3XTZrdsf7F5fFeNwnihkmADvhAcaxXUmVqq4rQFZH84a1iC3WBWXYcqiZH2L7ujGWV7mMDT4HBEerDYjc8rNY4xGTPfivCrBCJW1i14aqW8xRdsdgTM88eTksvC3WPJLJ7iMzfKXeL7fMW1Ek6QGyQtLBW98vEESpdcDg6DeZ5rMz6VqjTGGqcCaFGfHoqtfxMDaBAEsyQ8h7XDX6dg1wq9wH6j4Tw7Tj1MEv1b8uj5NJkozZdzVdYA2QyE2Dp8vuurQG6uVdTDNww2d88RBQ8sVgjxN8gR45y4woJLhFAaNTAtrY6wDTxyXST13ni6oyqdYxjFVk9Am4v3DzH7Y2K8iRVSHfTk4FRbPULyaeK6wt2anvMJH1XdvVRgc14h67MnBxMgMD1UFk8AErN7CDj26fppe3c5G6KozJe4cSqQUGbBjVzBnrHCruqrfZBn5hNZHTV37bQiomqhRQXohxhuKEnNrGbAe1xNvJr9X";
let reply_surb = ReplySURB::from_base58_string(reply_surb_string).unwrap();
let reply_request = ClientRequest::Reply {
message: b"foomp".to_vec(),
reply_surb,
};
let bytes = reply_request.serialize();
let recovered = ClientRequest::deserialize(&bytes).unwrap();
match recovered {
ClientRequest::Reply {
reply_surb,
message,
} => {
assert_eq!(reply_surb.to_base58_string(), reply_surb_string);
assert_eq!(message, b"foomp".to_vec());
}
_ => unreachable!(),
}
}
#[test]
fn self_address_request_serialization_works() {
let self_address_request = ClientRequest::SelfAddress;
let bytes = self_address_request.serialize();
let recovered = ClientRequest::deserialize(&bytes).unwrap();
match recovered {
ClientRequest::SelfAddress => (),
_ => unreachable!(),
}
}
}
clients/native/websocket-requests/src/responses.rs
+401
View File
@@ -0,0 +1,401 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// all variable size data is always prefixed with u64 length
// tags are u8
use crate::error::{self, ErrorKind};
use crate::text::ServerResponseText;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::anonymous_replies::ReplySURB;
use nymsphinx::receiver::ReconstructedMessage;
use std::convert::TryInto;
use std::mem::size_of;
/// Value tag representing [`Error`] variant of the [`ServerResponse`]
pub const ERROR_RESPONSE_TAG: u8 = 0x00;
/// Value tag representing [`Received`] variant of the [`ServerResponse`]
pub const RECEIVED_RESPONSE_TAG: u8 = 0x01;
/// Value tag representing [`SelfAddress`] variant of the [`ServerResponse`]
pub const SELF_ADDRESS_RESPONSE_TAG: u8 = 0x02;
#[derive(Debug)]
pub enum ServerResponse {
Received(ReconstructedMessage),
SelfAddress(Recipient),
Error(error::Error),
}
impl ServerResponse {
pub fn new_error<S: Into<String>>(message: S) -> Self {
ServerResponse::Error(error::Error {
kind: ErrorKind::Other,
message: message.into(),
})
}
// RECEIVED_RESPONSE_TAG || with_reply || (surb_len || surb) || msg_len || msg
fn serialize_received(reconstructed_message: ReconstructedMessage) -> Vec<u8> {
let message_len_bytes = (reconstructed_message.message.len() as u64).to_be_bytes();
if let Some(reply_surb) = reconstructed_message.reply_surb {
let reply_surb_bytes = reply_surb.to_bytes();
let surb_len_bytes = (reply_surb_bytes.len() as u64).to_be_bytes();
// with_reply || surb_len || surb || msg_len || msg
std::iter::once(RECEIVED_RESPONSE_TAG)
.chain(std::iter::once(true as u8))
.chain(surb_len_bytes.iter().cloned())
.chain(reply_surb_bytes.iter().cloned())
.chain(message_len_bytes.iter().cloned())
.chain(reconstructed_message.message.into_iter())
.collect()
} else {
// without_reply || msg_len || msg
std::iter::once(RECEIVED_RESPONSE_TAG)
.chain(std::iter::once(false as u8))
.chain(message_len_bytes.iter().cloned())
.chain(reconstructed_message.message.into_iter())
.collect()
}
}
// RECEIVED_RESPONSE_TAG || with_reply || (surb_len || surb) || msg_len || msg
fn deserialize_received(b: &[u8]) -> Result<Self, error::Error> {
// this MUST match because it was called by 'deserialize'
debug_assert_eq!(b[0], RECEIVED_RESPONSE_TAG);
// we must be able to read at the very least if it has a reply_surb and length of some field
if b.len() < 2 + size_of::<u64>() {
return Err(error::Error::new(
ErrorKind::TooShortResponse,
"not enough data provided to recover 'received'".to_string(),
));
}
let with_reply_surb = match b[1] {
0 => false,
1 => true,
n => {
return Err(error::Error::new(
ErrorKind::MalformedResponse,
format!("invalid reply flag {}", n),
))
}
};
if with_reply_surb {
let reply_surb_len =
u64::from_be_bytes(b[2..2 + size_of::<u64>()].as_ref().try_into().unwrap());
// make sure we won't go out of bounds here
if reply_surb_len > (b.len() - 2 + 2 * size_of::<u64>()) as u64 {
return Err(error::Error::new(
ErrorKind::MalformedResponse,
"not enough bytes to read reply_surb bytes!".to_string(),
));
}
let surb_bound = 2 + size_of::<u64>() + reply_surb_len as usize;
let reply_surb_bytes = &b[2 + size_of::<u64>()..surb_bound];
let reply_surb = match ReplySURB::from_bytes(reply_surb_bytes) {
Ok(reply_surb) => reply_surb,
Err(err) => {
return Err(error::Error::new(
ErrorKind::MalformedResponse,
format!("malformed reply SURB: {:?}", err),
))
}
};
let message_len = u64::from_be_bytes(
b[surb_bound..surb_bound + size_of::<u64>()]
.as_ref()
.try_into()
.unwrap(),
);
let message = &b[surb_bound + size_of::<u64>()..];
if message.len() as u64 != message_len {
return Err(error::Error::new(
ErrorKind::MalformedResponse,
format!(
"message len has inconsistent length. specified: {} got: {}",
message_len,
message.len()
),
));
}
Ok(ServerResponse::Received(ReconstructedMessage {
message: message.to_vec(),
reply_surb: Some(reply_surb),
}))
} else {
let message_len =
u64::from_be_bytes(b[2..2 + size_of::<u64>()].as_ref().try_into().unwrap());
let message = &b[2 + size_of::<u64>()..];
if message.len() as u64 != message_len {
return Err(error::Error::new(
ErrorKind::MalformedResponse,
format!(
"message len has inconsistent length. specified: {} got: {}",
message_len,
message.len()
),
));
}
Ok(ServerResponse::Received(ReconstructedMessage {
message: message.to_vec(),
reply_surb: None,
}))
}
}
// SELF_ADDRESS_RESPONSE_TAG || self_address
fn serialize_self_address(address: Recipient) -> Vec<u8> {
std::iter::once(SELF_ADDRESS_RESPONSE_TAG)
.chain(address.to_bytes().iter().cloned())
.collect()
}
// SELF_ADDRESS_RESPONSE_TAG || self_address
fn deserialize_self_address(b: &[u8]) -> Result<Self, error::Error> {
// this MUST match because it was called by 'deserialize'
debug_assert_eq!(b[0], SELF_ADDRESS_RESPONSE_TAG);
if b.len() != 1 + Recipient::LEN {
return Err(error::Error::new(
ErrorKind::TooShortResponse,
"not enough data provided to recover 'self_address'".to_string(),
));
}
let mut recipient_bytes = [0u8; Recipient::LEN];
recipient_bytes.copy_from_slice(&b[1..1 + Recipient::LEN]);
let recipient = match Recipient::try_from_bytes(recipient_bytes) {
Ok(recipient) => recipient,
Err(err) => {
return Err(error::Error::new(
ErrorKind::MalformedResponse,
format!("malformed Recipient: {:?}", err),
))
}
};
Ok(ServerResponse::SelfAddress(recipient))
}
// ERROR_RESPONSE_TAG || err_code || msg_len || msg
fn serialize_error(error: error::Error) -> Vec<u8> {
let message_len_bytes = (error.message.len() as u64).to_be_bytes();
std::iter::once(ERROR_RESPONSE_TAG)
.chain(std::iter::once(error.kind as u8))
.chain(message_len_bytes.iter().cloned())
.chain(error.message.into_bytes().into_iter())
.collect()
}
// ERROR_RESPONSE_TAG || err_code || msg_len || msg
fn deserialize_error(b: &[u8]) -> Result<Self, error::Error> {
// this MUST match because it was called by 'deserialize'
debug_assert_eq!(b[0], ERROR_RESPONSE_TAG);
if b.len() < size_of::<u8>() + size_of::<u64>() {
return Err(error::Error::new(
ErrorKind::TooShortResponse,
"not enough data provided to recover 'error'".to_string(),
));
}
let error_kind = match b[1] {
_ if b[1] == (ErrorKind::EmptyRequest as u8) => ErrorKind::EmptyRequest,
_ if b[1] == (ErrorKind::TooShortRequest as u8) => ErrorKind::TooShortRequest,
_ if b[1] == (ErrorKind::UnknownRequest as u8) => ErrorKind::UnknownRequest,
_ if b[1] == (ErrorKind::MalformedRequest as u8) => ErrorKind::MalformedRequest,
_ if b[1] == (ErrorKind::EmptyResponse as u8) => ErrorKind::EmptyResponse,
_ if b[1] == (ErrorKind::TooShortResponse as u8) => ErrorKind::TooShortResponse,
_ if b[1] == (ErrorKind::UnknownResponse as u8) => ErrorKind::UnknownResponse,
_ if b[1] == (ErrorKind::MalformedResponse as u8) => ErrorKind::MalformedResponse,
_ if b[1] == (ErrorKind::Other as u8) => ErrorKind::Other,
n => {
return Err(error::Error::new(
ErrorKind::MalformedResponse,
format!("invalid error code {}", n),
))
}
};
let message_len =
u64::from_be_bytes(b[2..2 + size_of::<u64>()].as_ref().try_into().unwrap());
let message = &b[2 + size_of::<u64>()..];
if message.len() as u64 != message_len {
return Err(error::Error::new(
ErrorKind::MalformedResponse,
format!(
"message len has inconsistent length. specified: {} got: {}",
message_len,
message.len()
),
));
}
let err_message = match String::from_utf8(message.to_vec()) {
Ok(msg) => msg,
Err(err) => {
return Err(error::Error::new(
ErrorKind::MalformedResponse,
format!("malformed error message: {:?}", err),
))
}
};
Ok(ServerResponse::Error(error::Error::new(
error_kind,
err_message,
)))
}
pub fn serialize(self) -> Vec<u8> {
match self {
ServerResponse::Received(reconstructed_message) => {
Self::serialize_received(reconstructed_message)
}
ServerResponse::SelfAddress(address) => Self::serialize_self_address(address),
ServerResponse::Error(err) => Self::serialize_error(err),
}
}
pub fn deserialize(b: &[u8]) -> Result<Self, error::Error> {
if b.is_empty() {
// technically I'm not even sure this can ever be returned, because reading empty
// request would imply closed socket, but let's include it for completion sake
return Err(error::Error::new(
ErrorKind::EmptyResponse,
"no data provided".to_string(),
));
}
if b.len() < size_of::<u8>() {
return Err(error::Error::new(
ErrorKind::TooShortResponse,
format!(
"not enough data provided to recover response tag. Provided only {} bytes",
b.len()
),
));
}
let response_tag = b[0];
// determine what kind of response that is and try to deserialize it
match response_tag {
RECEIVED_RESPONSE_TAG => Self::deserialize_received(b),
SELF_ADDRESS_RESPONSE_TAG => Self::deserialize_self_address(b),
ERROR_RESPONSE_TAG => Self::deserialize_error(b),
n => {
return Err(error::Error::new(
ErrorKind::UnknownResponse,
format!("type {}", n),
))
}
}
}
pub fn into_binary(self) -> Vec<u8> {
self.serialize()
}
pub fn into_text(self) -> String {
// use the intermediate string structure and let serde do bunch of work for us
let text_resp = ServerResponseText::from(self);
text_resp.into()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn received_response_serialization_works() {
let reply_surb_string = "CjfVbHbfAjbC3W1BvNHGXmM8KNAnDNYGaHMLqVDxRYeo352csAihstup9bvqXam4dTWgfHak6KYwL9STaxWJ47E8XFZbSEvs7hEsfCkxr6K9WJuSBPK84GDDEvad8ZAuMCoaXsAd5S2Lj9a5eYyzG4SL1jHzhSMni55LyJwumxo1ZTGZNXggxw1RREosvyzNrW9Rsi3owyPqLCwXpiei2tHZty8w8midVvg8vDa7ZEJD842CLv8D4ohynSG7gDpqTrhkRaqYAuz7dzqNbMXLJRM7v823Jn16fA1L7YQxmcaUdUigyRSgTdb4i9ebiLGSyJ1iDe6Acz613PQZh6Ua3bZ2zVKq3dSycpDm9ngarRK4zJrAaUxRkdih8YzW3BY4nL9eqkfKA4N1TWCLaRU7zpSaf8yMEwrAZReU3d5zLV8c5KBfa2w8R5anhQeBojduZEGEad8kkHuKU52Zg93FeWHvH1qgZaEJMHH4nN7gKXz9mvWDhYwyF4vt3Uy2NhCHC3N5pL1gMme27YcoPcTEia1fxKZtnt6rtEozzTrAgCJGswigkFbkafiV5QaJwLKTUxtzhkZ57eEuLPte9UvJHzhhXUQ2CV7R2BUkJjYZy3Zsx6YYvdYWiAFFkWUwNEGA4QpShUHciBfsQVHQ7pN41YcyYUhbywQDFnTVgEmdUZ1XCBi3gyK5U3tDQmFzP1u9m3mWrUA8qB9mRDE7ptNDm5c3c1458L6uXLUth7sdMaa1Was5LCmCdmNDtvNpCDAEt1in6q6mrZFR85aCSU9b1baNGwZoCqPpPvydkVe63gXWoi8ebvdyxARrqACFrSB3ZdY3uJBw8CTMNkKK6MvcefMkSVVsbLd36TQAtYSCqrpiMc5dQuKcEu5QfciwvWYXYx8WFNAgKwP2mv49KCTvfozNDUCbjzDwSx92Zv5zjG8HbFpB13bY9UZGeyTPvv7gGxCzjGjJGbW6FRAheRQaaje5fUgCNM95Tv7wBmAMRHHFgWafeK1sdFH7dtCX9u898HucGTaboSKLsVh8J78gbbkHErwjMh7y9YRkceq5TTYS5da4kHnyNKYWSbxgZrmFg44XGKoeYcqoHB3XTZrdsf7F5fFeNwnihkmADvhAcaxXUmVqq4rQFZH84a1iC3WBWXYcqiZH2L7ujGWV7mMDT4HBEerDYjc8rNY4xGTPfivCrBCJW1i14aqW8xRdsdgTM88eTksvC3WPJLJ7iMzfKXeL7fMW1Ek6QGyQtLBW98vEESpdcDg6DeZ5rMz6VqjTGGqcCaFGfHoqtfxMDaBAEsyQ8h7XDX6dg1wq9wH6j4Tw7Tj1MEv1b8uj5NJkozZdzVdYA2QyE2Dp8vuurQG6uVdTDNww2d88RBQ8sVgjxN8gR45y4woJLhFAaNTAtrY6wDTxyXST13ni6oyqdYxjFVk9Am4v3DzH7Y2K8iRVSHfTk4FRbPULyaeK6wt2anvMJH1XdvVRgc14h67MnBxMgMD1UFk8AErN7CDj26fppe3c5G6KozJe4cSqQUGbBjVzBnrHCruqrfZBn5hNZHTV37bQiomqhRQXohxhuKEnNrGbAe1xNvJr9X";
let received_with_surb = ServerResponse::Received(ReconstructedMessage {
message: b"foomp".to_vec(),
reply_surb: Some(ReplySURB::from_base58_string(reply_surb_string).unwrap()),
});
let bytes = received_with_surb.serialize();
let recovered = ServerResponse::deserialize(&bytes).unwrap();
match recovered {
ServerResponse::Received(reconstructed) => {
assert_eq!(reconstructed.message, b"foomp".to_vec());
assert_eq!(
reconstructed.reply_surb.unwrap().to_base58_string(),
reply_surb_string
)
}
_ => unreachable!(),
}
let received_without_surb = ServerResponse::Received(ReconstructedMessage {
message: b"foomp".to_vec(),
reply_surb: None,
});
let bytes = received_without_surb.serialize();
let recovered = ServerResponse::deserialize(&bytes).unwrap();
match recovered {
ServerResponse::Received(reconstructed) => {
assert_eq!(reconstructed.message, b"foomp".to_vec());
assert!(reconstructed.reply_surb.is_none())
}
_ => unreachable!(),
}
}
#[test]
fn self_address_response_serialization_works() {
let recipient = Recipient::try_from_base58_string("CytBseW6yFXUMzz4SGAKdNLGR7q3sJLLYxyBGvutNEQV.4QXYyEVc5fUDjmmi8PrHN9tdUFV4PCvSJE1278cHyvoe@4sBbL1ngf1vtNqykydQKTFh26sQCw888GpUqvPvyNB4f").unwrap();
let recipient_string = recipient.to_string();
let self_address_response = ServerResponse::SelfAddress(recipient);
let bytes = self_address_response.serialize();
let recovered = ServerResponse::deserialize(&bytes).unwrap();
match recovered {
ServerResponse::SelfAddress(recipient) => {
assert_eq!(recipient.to_string(), recipient_string)
}
_ => unreachable!(),
}
}
#[test]
fn error_response_serialization_works() {
let dummy_error = error::Error::new(ErrorKind::UnknownRequest, "foomp message".to_string());
let error_response = ServerResponse::Error(dummy_error.clone());
let bytes = error_response.serialize();
let recovered = ServerResponse::deserialize(&bytes).unwrap();
match recovered {
ServerResponse::Error(error) => assert_eq!(error, dummy_error),
_ => unreachable!(),
}
}
}
clients/native/websocket-requests/src/text.rs
+151
View File
@@ -0,0 +1,151 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::error::ErrorKind;
use crate::requests::ClientRequest;
use crate::responses::ServerResponse;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::anonymous_replies::ReplySURB;
use serde::{Deserialize, Serialize};
use std::convert::{TryFrom, TryInto};
// local text equivalent of `ClientRequest` for easier serialization + deserialization with serde
// TODO: figure out if there's an easy way to avoid defining it
#[derive(Serialize, Deserialize, Debug)]
#[serde(tag = "type", rename_all = "camelCase")]
pub(super) enum ClientRequestText {
#[serde(rename_all = "camelCase")]
Send {
message: String,
recipient: String,
with_reply_surb: bool,
},
SelfAddress,
#[serde(rename_all = "camelCase")]
Reply {
message: String,
reply_surb: String,
},
}
impl TryFrom<String> for ClientRequestText {
type Error = serde_json::Error;
fn try_from(msg: String) -> Result<Self, Self::Error> {
serde_json::from_str(&msg)
}
}
impl TryInto<ClientRequest> for ClientRequestText {
type Error = crate::error::Error;
fn try_into(self) -> Result<ClientRequest, Self::Error> {
match self {
ClientRequestText::Send {
message,
recipient,
with_reply_surb,
} => {
let message_bytes = message.into_bytes();
let recipient = Recipient::try_from_base58_string(recipient).map_err(|err| {
Self::Error::new(ErrorKind::MalformedRequest, err.to_string())
})?;
Ok(ClientRequest::Send {
message: message_bytes,
recipient,
with_reply_surb,
})
}
ClientRequestText::SelfAddress => Ok(ClientRequest::SelfAddress),
ClientRequestText::Reply {
message,
reply_surb,
} => {
let message_bytes = message.into_bytes();
let reply_surb = ReplySURB::from_base58_string(reply_surb).map_err(|err| {
Self::Error::new(ErrorKind::MalformedRequest, err.to_string())
})?;
Ok(ClientRequest::Reply {
message: message_bytes,
reply_surb,
})
}
}
}
}
// local text equivalent of `ServerResponse` for easier serialization + deserialization with serde
// TODO: figure out if there's an easy way to avoid defining it
#[derive(Serialize, Deserialize, Debug)]
#[serde(tag = "type", rename_all = "camelCase")]
pub(super) enum ServerResponseText {
#[serde(rename_all = "camelCase")]
Received {
message: String,
reply_surb: Option<String>,
},
SelfAddress {
address: String,
},
Error {
message: String,
},
}
impl TryFrom<String> for ServerResponseText {
type Error = serde_json::Error;
fn try_from(msg: String) -> Result<Self, <ServerResponseText as TryFrom<String>>::Error> {
serde_json::from_str(&msg)
}
}
impl Into<String> for ServerResponseText {
fn into(self) -> String {
// per serde_json docs:
/*
/// Serialization can fail if `T`'s implementation of `Serialize` decides to
/// fail, or if `T` contains a map with non-string keys.
*/
// this is not the case here.
serde_json::to_string(&self).unwrap()
}
}
impl From<ServerResponse> for ServerResponseText {
fn from(resp: ServerResponse) -> Self {
match resp {
ServerResponse::Received(reconstructed) => {
ServerResponseText::Received {
// TODO: ask DH what is more appropriate, lossy utf8 conversion or returning error and then
// pure binary later
message: String::from_utf8_lossy(&reconstructed.message).into_owned(),
reply_surb: reconstructed
.reply_surb
.map(|reply_surb| reply_surb.to_base58_string()),
}
}
ServerResponse::SelfAddress(recipient) => ServerResponseText::SelfAddress {
address: recipient.to_string(),
},
ServerResponse::Error(err) => ServerResponseText::Error {
message: err.to_string(),
},
}
}
}
clients/socks5/Cargo.toml
+43
View File
@@ -0,0 +1,43 @@
[package]
build = "build.rs"
name = "nym-socks5-client"
version = "0.8.0"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
edition = "2018"
[lib]
name = "nym_socks5"
path = "src/lib.rs"
[dependencies]
clap = "2.33.0"
dirs = "3.0" # for determining default store directories in config
dotenv = "0.15.0"
futures = "0.3"
log = "0.4"
pin-project = "0.4"
pretty_env_logger = "0.3"
rand = { version = "0.7.3", features = ["wasm-bindgen"] }
serde = { version = "1.0", features = ["derive"] } # for config serialization/deserialization
snafu = "0.4.1"
tokio = { version = "0.2", features = ["rt-threaded"] }
url = "2.1"
# internal
client-core = { path = "../client-core" }
config = { path = "../../common/config" }
crypto = { path = "../../common/crypto" }
directory-client = { path = "../../common/client-libs/directory-client" }
gateway-client = { path = "../../common/client-libs/gateway-client" }
gateway-requests = { path = "../../gateway/gateway-requests" }
nymsphinx = { path = "../../common/nymsphinx" }
ordered-buffer = {path = "../../common/socks5/ordered-buffer"}
socks5-requests = { path = "../../common/socks5/requests" }
topology = { path = "../../common/topology" }
proxy-helpers = { path = "../../common/socks5/proxy-helpers" }
[build-dependencies]
built = "0.4.3"
[dev-dependencies]
tempfile = "3.1.0"
common/healthcheck/src/config.rs → clients/socks5/build.rs
+3 -13
View File
@@ -12,18 +12,8 @@
// See the License for the specific language governing permissions and
// limitations under the License.
use serde_derive::Deserialize;
use built;
#[derive(Deserialize, Debug)]
pub struct HealthCheck {
#[serde(rename(deserialize = "directory-server"))]
pub directory_server: String,
pub interval: f64, // in seconds
#[serde(rename(deserialize = "resolution-timeout"))]
pub resolution_timeout: f64, // in seconds
#[serde(rename(deserialize = "test-packets-per-node"))]
pub num_test_packets: usize,
fn main() {
built::write_built_file().expect("Failed to acquire build-time information");
}
clients/socks5/src/built_info.rs
+16
View File
@@ -0,0 +1,16 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// The file has been placed there by the build script.
include!(concat!(env!("OUT_DIR"), "/built.rs"));
clients/socks5/src/client/config/mod.rs
+154
View File
@@ -0,0 +1,154 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::client::config::template::config_template;
use client_core::config::Config as BaseConfig;
use config::NymConfig;
use nymsphinx::addressing::clients::Recipient;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
mod template;
const DEFAULT_LISTENING_PORT: u16 = 1080;
#[derive(Debug, Default, Deserialize, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Config {
#[serde(flatten)]
base: BaseConfig<Config>,
socks5: Socks5,
}
impl NymConfig for Config {
fn template() -> &'static str {
config_template()
}
fn config_file_name() -> String {
"config.toml".to_string()
}
fn default_root_directory() -> PathBuf {
dirs::home_dir()
.expect("Failed to evaluate $HOME value")
.join(".nym")
.join("socks5-clients")
}
fn root_directory(&self) -> PathBuf {
self.base.get_nym_root_directory()
}
fn config_directory(&self) -> PathBuf {
self.root_directory()
.join(self.base.get_id())
.join("config")
}
fn data_directory(&self) -> PathBuf {
self.root_directory().join(self.base.get_id()).join("data")
}
}
impl Config {
pub fn new<S: Into<String>>(id: S, provider_mix_address: S) -> Self {
Config {
base: BaseConfig::new(id),
socks5: Socks5::new(provider_mix_address),
}
}
pub fn with_port(mut self, port: u16) -> Self {
self.socks5.listening_port = port;
self
}
pub fn with_provider_mix_address(mut self, address: String) -> Self {
self.socks5.provider_mix_address = address;
self
}
// getters
pub fn get_config_file_save_location(&self) -> PathBuf {
self.config_directory().join(Self::config_file_name())
}
pub fn get_provider_mix_address(&self) -> Recipient {
Recipient::try_from_base58_string(&self.socks5.provider_mix_address)
.expect("malformed provider address")
}
pub fn get_base(&self) -> &BaseConfig<Self> {
&self.base
}
pub fn get_base_mut(&mut self) -> &mut BaseConfig<Self> {
&mut self.base
}
pub fn get_listening_port(&self) -> u16 {
self.socks5.listening_port
}
}
#[derive(Debug, Deserialize, PartialEq, Serialize)]
#[serde(deny_unknown_fields)]
pub struct Socks5 {
/// The port on which the client will be listening for incoming requests
listening_port: u16,
/// The mix address of the provider to which all requests are going to be sent.
provider_mix_address: String,
}
impl Socks5 {
pub fn new<S: Into<String>>(provider_mix_address: S) -> Self {
Socks5 {
listening_port: DEFAULT_LISTENING_PORT,
provider_mix_address: provider_mix_address.into(),
}
}
}
impl Default for Socks5 {
fn default() -> Self {
Socks5 {
listening_port: DEFAULT_LISTENING_PORT,
provider_mix_address: "".into(),
}
}
}
#[cfg(test)]
mod client_config {
use super::*;
#[test]
fn after_saving_default_config_the_loaded_one_is_identical() {
// need to figure out how to do something similar but without touching the disk
// or the file system at all...
let temp_location = tempfile::tempdir().unwrap().path().join("config.toml");
let fake_address = "CytBseW6yFXUMzz4SGAKdNLGR7q3sJLLYxyBGvutNEQV.4QXYyEVc5fUDjmmi8PrHN9tdUFV4PCvSJE1278cHyvoe@FioFa8nMmPpQnYi7JyojoTuwGLeyNS8BF4ChPr29zUML";
let default_config = Config::new("foomp", fake_address);
default_config
.save_to_file(Some(temp_location.clone()))
.unwrap();
let loaded_config = Config::load_from_file(Some(temp_location), None).unwrap();
assert_eq!(default_config, loaded_config);
}
}
clients/socks5/src/client/config/template.rs
+101
View File
@@ -0,0 +1,101 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub(crate) fn config_template() -> &'static str {
// While using normal toml marshalling would have been way simpler with less overhead,
// I think it's useful to have comments attached to the saved config file to explain behaviour of
// particular fields.
// Note: any changes to the template must be reflected in the appropriate structs in mod.rs.
r#"
# This is a TOML config file.
# For more information, see https://github.com/toml-lang/toml
##### main base client config options #####
[client]
# Human readable ID of this particular client.
id = '{{ client.id }}'
# URL to the directory server.
directory_server = '{{ client.directory_server }}'
# Path to file containing private identity key.
private_identity_key_file = '{{ client.private_identity_key_file }}'
# Path to file containing public identity key.
public_identity_key_file = '{{ client.public_identity_key_file }}'
# Path to file containing private encryption key.
private_encryption_key_file = '{{ client.private_encryption_key_file }}'
# Path to file containing public encryption key.
public_encryption_key_file = '{{ client.public_encryption_key_file }}'
# Full path to file containing reply encryption keys of all reply-SURBs we have ever
# sent but not received back.
reply_encryption_key_store_path = '{{ client.reply_encryption_key_store_path }}'
##### additional client config options #####
# ID of the gateway from which the client should be fetching messages.
gateway_id = '{{ client.gateway_id }}'
# Address of the gateway listener to which all client requests should be sent.
gateway_listener = '{{ client.gateway_listener }}'
# A gateway specific, optional, base58 stringified shared key used for
# communication with particular gateway.
gateway_shared_key_file = '{{ client.gateway_shared_key_file }}'
# Path to file containing key used for encrypting and decrypting the content of an
# acknowledgement so that nobody besides the client knows which packet it refers to.
ack_key_file = '{{ client.ack_key_file }}'
##### advanced configuration options #####
# Absolute path to the home Nym Clients directory.
nym_root_directory = '{{ client.nym_root_directory }}'
##### socket config options #####
[socks5]
# The mix address of the provider to which all requests are going to be sent.
provider_mix_address = '{{ socks5.provider_mix_address }}'
# The port on which the client will be listening for incoming requests
listening_port = {{ socks5.listening_port }}
##### logging configuration options #####
[logging]
# TODO
##### debug configuration options #####
# The following options should not be modified unless you know EXACTLY what you are doing
# as if set incorrectly, they may impact your anonymity.
[debug]
average_packet_delay = {{ debug.average_packet_delay }}
average_ack_delay = {{ debug.average_ack_delay }}
loop_cover_traffic_average_delay = {{ debug.loop_cover_traffic_average_delay }}
message_sending_average_delay = {{ debug.message_sending_average_delay }}
"#
}
clients/socks5/src/client/mod.rs
+344
View File
@@ -0,0 +1,344 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::client::config::Config;
use crate::socks::{
authentication::{AuthenticationMethods, Authenticator, User},
server::SphinxSocksServer,
};
use client_core::client::cover_traffic_stream::LoopCoverTrafficStream;
use client_core::client::inbound_messages::{
InputMessage, InputMessageReceiver, InputMessageSender,
};
use client_core::client::key_manager::KeyManager;
use client_core::client::mix_traffic::{
MixMessageReceiver, MixMessageSender, MixTrafficController,
};
use client_core::client::real_messages_control::RealMessagesController;
use client_core::client::received_buffer::{
ReceivedBufferRequestReceiver, ReceivedBufferRequestSender, ReceivedMessagesBufferController,
};
use client_core::client::reply_key_storage::ReplyKeyStorage;
use client_core::client::topology_control::{
TopologyAccessor, TopologyRefresher, TopologyRefresherConfig,
};
use client_core::config::persistence::key_pathfinder::ClientKeyPathfinder;
use crypto::asymmetric::identity;
use futures::channel::mpsc;
use gateway_client::{
AcknowledgementReceiver, AcknowledgementSender, GatewayClient, MixnetMessageReceiver,
MixnetMessageSender,
};
use log::*;
use nymsphinx::addressing::clients::Recipient;
use nymsphinx::addressing::nodes::NodeIdentity;
use tokio::runtime::Runtime;
pub(crate) mod config;
pub struct NymClient {
/// Client configuration options, including, among other things, packet sending rates,
/// key filepaths, etc.
config: Config,
/// Tokio runtime used for futures execution.
// TODO: JS: Personally I think I prefer the implicit way of using it that we've done with the
// gateway.
runtime: Runtime,
/// KeyManager object containing smart pointers to all relevant keys used by the client.
key_manager: KeyManager,
}
impl NymClient {
pub fn new(config: Config) -> Self {
let pathfinder = ClientKeyPathfinder::new_from_config(config.get_base());
let key_manager = KeyManager::load_keys(&pathfinder).expect("failed to load stored keys");
NymClient {
runtime: Runtime::new().unwrap(),
config,
key_manager,
}
}
pub fn as_mix_recipient(&self) -> Recipient {
Recipient::new(
*self.key_manager.identity_keypair().public_key(),
self.key_manager.encryption_keypair().public_key().clone(),
// TODO: below only works under assumption that gateway address == gateway id
// (which currently is true)
NodeIdentity::from_base58_string(self.config.get_base().get_gateway_id()).unwrap(),
)
}
// future constantly pumping loop cover traffic at some specified average rate
// the pumped traffic goes to the MixTrafficController
fn start_cover_traffic_stream(
&self,
topology_accessor: TopologyAccessor,
mix_tx: MixMessageSender,
) {
info!("Starting loop cover traffic stream...");
// we need to explicitly enter runtime due to "next_delay: time::delay_for(Default::default())"
// set in the constructor which HAS TO be called within context of a tokio runtime
self.runtime
.enter(|| {
LoopCoverTrafficStream::new(
self.key_manager.ack_key(),
self.config.get_base().get_average_ack_delay(),
self.config.get_base().get_average_packet_delay(),
self.config
.get_base()
.get_loop_cover_traffic_average_delay(),
mix_tx,
self.as_mix_recipient(),
topology_accessor,
)
})
.start(self.runtime.handle());
}
fn start_real_traffic_controller(
&self,
topology_accessor: TopologyAccessor,
reply_key_storage: ReplyKeyStorage,
ack_receiver: AcknowledgementReceiver,
input_receiver: InputMessageReceiver,
mix_sender: MixMessageSender,
) {
let controller_config = client_core::client::real_messages_control::Config::new(
self.key_manager.ack_key(),
self.config.get_base().get_ack_wait_multiplier(),
self.config.get_base().get_ack_wait_addition(),
self.config.get_base().get_average_ack_delay(),
self.config.get_base().get_message_sending_average_delay(),
self.config.get_base().get_average_packet_delay(),
self.as_mix_recipient(),
);
info!("Starting real traffic stream...");
// we need to explicitly enter runtime due to "next_delay: time::delay_for(Default::default())"
// set in the constructor [of OutQueueControl] which HAS TO be called within context of a tokio runtime
// When refactoring this restriction should definitely be removed.
let real_messages_controller = self.runtime.enter(|| {
RealMessagesController::new(
controller_config,
ack_receiver,
input_receiver,
mix_sender,
topology_accessor,
reply_key_storage,
)
});
real_messages_controller.start(self.runtime.handle());
}
// buffer controlling all messages fetched from provider
// required so that other components would be able to use them (say the websocket)
fn start_received_messages_buffer_controller(
&self,
query_receiver: ReceivedBufferRequestReceiver,
mixnet_receiver: MixnetMessageReceiver,
reply_key_storage: ReplyKeyStorage,
) {
info!("Starting received messages buffer controller...");
ReceivedMessagesBufferController::new(
self.key_manager.encryption_keypair(),
query_receiver,
mixnet_receiver,
reply_key_storage,
)
.start(self.runtime.handle())
}
fn start_gateway_client(
&mut self,
mixnet_message_sender: MixnetMessageSender,
ack_sender: AcknowledgementSender,
) -> GatewayClient<'static, url::Url> {
let gateway_id = self.config.get_base().get_gateway_id();
if gateway_id.is_empty() {
panic!("The identity of the gateway is unknown - did you run `nym-client` init?")
}
let gateway_address_str = self.config.get_base().get_gateway_listener();
if gateway_address_str.is_empty() {
panic!("The address of the gateway is unknown - did you run `nym-client` init?")
}
let gateway_identity = identity::PublicKey::from_base58_string(gateway_id)
.expect("provided gateway id is invalid!");
// TODO: since we presumably now get something valid-ish from the `init`, can we just
// ditch url::Url and operate on `String`?
let gateway_address =
url::Url::parse(&gateway_address_str).expect("provided gateway address is invalid!");
let mut gateway_client = GatewayClient::new(
gateway_address,
self.key_manager.identity_keypair(),
gateway_identity,
Some(self.key_manager.gateway_shared_key()),
mixnet_message_sender,
ack_sender,
self.config.get_base().get_gateway_response_timeout(),
);
self.runtime.block_on(async {
gateway_client
.authenticate_and_start()
.await
.expect("could not authenticate and start up the gateway connection")
});
gateway_client
}
// future responsible for periodically polling directory server and updating
// the current global view of topology
fn start_topology_refresher(&mut self, topology_accessor: TopologyAccessor) {
let topology_refresher_config = TopologyRefresherConfig::new(
self.config.get_base().get_directory_server(),
self.config.get_base().get_topology_refresh_rate(),
);
let mut topology_refresher =
TopologyRefresher::new_directory_client(topology_refresher_config, topology_accessor);
// before returning, block entire runtime to refresh the current network view so that any
// components depending on topology would see a non-empty view
info!(
"Obtaining initial network topology from {}",
self.config.get_base().get_directory_server()
);
self.runtime.block_on(topology_refresher.refresh());
// TODO: a slightly more graceful termination here
if !self
.runtime
.block_on(topology_refresher.is_topology_routable())
{
panic!(
"The current network topology seem to be insufficient to route any packets through\
- check if enough nodes and a gateway are online"
);
}
info!("Starting topology refresher...");
topology_refresher.start(self.runtime.handle());
}
// controller for sending sphinx packets to mixnet (either real traffic or cover traffic)
// TODO: if we want to send control messages to gateway_client, this CAN'T take the ownership
// over it. Perhaps GatewayClient needs to be thread-shareable or have some channel for
// requests?
fn start_mix_traffic_controller(
&mut self,
mix_rx: MixMessageReceiver,
gateway_client: GatewayClient<'static, url::Url>,
) {
info!("Starting mix traffic controller...");
MixTrafficController::new(mix_rx, gateway_client).start(self.runtime.handle());
}
fn start_socks5_listener(
&self,
buffer_requester: ReceivedBufferRequestSender,
msg_input: InputMessageSender,
) {
info!("Starting socks5 listener...");
let mut auth_methods: Vec<u8> = Vec::new();
auth_methods.push(AuthenticationMethods::NoAuth as u8);
let allowed_users: Vec<User> = Vec::new();
let authenticator = Authenticator::new(auth_methods, allowed_users);
let mut sphinx_socks = SphinxSocksServer::new(
self.config.get_listening_port(),
authenticator,
self.config.get_provider_mix_address(),
self.as_mix_recipient(),
);
self.runtime
.spawn(async move { sphinx_socks.serve(msg_input, buffer_requester).await });
}
/// blocking version of `start` method. Will run forever (or until SIGINT is sent)
pub fn run_forever(&mut self) {
self.start();
if let Err(e) = self.runtime.block_on(tokio::signal::ctrl_c()) {
error!(
"There was an error while capturing SIGINT - {:?}. We will terminate regardless",
e
);
}
println!(
"Received SIGINT - the client will terminate now (threads are not YET nicely stopped)"
);
}
pub fn start(&mut self) {
info!("Starting nym client");
// channels for inter-component communication
// TODO: make the channels be internally created by the relevant components
// rather than creating them here, so say for example the buffer controller would create the request channels
// and would allow anyone to clone the sender channel
// sphinx_message_sender is the transmitter for any component generating sphinx packets that are to be sent to the mixnet
// they are used by cover traffic stream and real traffic stream
// sphinx_message_receiver is the receiver used by MixTrafficController that sends the actual traffic
let (sphinx_message_sender, sphinx_message_receiver) = mpsc::unbounded();
// unwrapped_sphinx_sender is the transmitter of mixnet messages received from the gateway
// unwrapped_sphinx_receiver is the receiver for said messages - used by ReceivedMessagesBuffer
let (mixnet_messages_sender, mixnet_messages_receiver) = mpsc::unbounded();
// used for announcing connection or disconnection of a channel for pushing re-assembled messages to
let (received_buffer_request_sender, received_buffer_request_receiver) = mpsc::unbounded();
// channels responsible for controlling real messages
let (input_sender, input_receiver) = mpsc::unbounded::<InputMessage>();
// channels responsible for controlling ack messages
let (ack_sender, ack_receiver) = mpsc::unbounded();
let shared_topology_accessor = TopologyAccessor::new();
let reply_key_storage =
ReplyKeyStorage::load(self.config.get_base().get_reply_encryption_key_store_path())
.expect("Failed to load reply key storage!");
// the components are started in very specific order. Unless you know what you are doing,
// do not change that.
self.start_topology_refresher(shared_topology_accessor.clone());
self.start_received_messages_buffer_controller(
received_buffer_request_receiver,
mixnet_messages_receiver,
reply_key_storage.clone(),
);
let gateway_client = self.start_gateway_client(mixnet_messages_sender, ack_sender);
self.start_mix_traffic_controller(sphinx_message_receiver, gateway_client);
self.start_real_traffic_controller(
shared_topology_accessor.clone(),
reply_key_storage,
ack_receiver,
input_receiver,
sphinx_message_sender.clone(),
);
self.start_cover_traffic_stream(shared_topology_accessor, sphinx_message_sender);
self.start_socks5_listener(received_buffer_request_sender, input_sender);
info!("Client startup finished!");
info!("The address of this client is: {}", self.as_mix_recipient());
}
}
clients/socks5/src/commands/init.rs
+160
View File
@@ -0,0 +1,160 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::built_info;
use crate::client::config::Config;
use crate::commands::override_config;
use clap::{App, Arg, ArgMatches};
use client_core::client::key_manager::KeyManager;
use client_core::config::persistence::key_pathfinder::ClientKeyPathfinder;
use config::NymConfig;
use crypto::asymmetric::identity;
use directory_client::DirectoryClient;
use gateway_client::GatewayClient;
use gateway_requests::registration::handshake::SharedKeys;
use rand::rngs::OsRng;
use std::convert::TryInto;
use std::sync::Arc;
use std::time::Duration;
use topology::{gateway, NymTopology};
pub fn command_args<'a, 'b>() -> clap::App<'a, 'b> {
App::new("init")
.about("Initialise a Nym client. Do this first!")
.arg(Arg::with_name("id")
.long("id")
.help("Id of the nym-mixnet-client we want to create config for.")
.takes_value(true)
.required(true)
)
.arg(Arg::with_name("provider")
.long("provider")
.help("Address of the socks5 provider to send messages to.")
.takes_value(true)
.required(true)
)
.arg(Arg::with_name("gateway")
.long("gateway")
.help("Id of the gateway we are going to connect to. If unsure what to put here - 'D6YaMzLSY7mANtSQRKXsmMZpqgqiVkeiagKM4V4oFPFr' is a safe choice for the testnet")
.takes_value(true)
.required(true)
)
.arg(Arg::with_name("directory")
.long("directory")
.help("Address of the directory server the client is getting topology from")
.takes_value(true),
)
.arg(Arg::with_name("port")
.short("p")
.long("port")
.help("Port for the socket to listen on in all subsequent runs")
.takes_value(true)
)
.arg(Arg::with_name("fastmode")
.long("fastmode")
.hidden(true) // this will prevent this flag from being displayed in `--help`
.help("Mostly debug-related option to increase default traffic rate so that you would not need to modify config post init")
)
}
async fn register_with_gateway(
gateway: &gateway::Node,
our_identity: Arc<identity::KeyPair>,
) -> SharedKeys {
let timeout = Duration::from_millis(1500);
let mut gateway_client = GatewayClient::new_init(
url::Url::parse(&gateway.client_listener).unwrap(),
gateway.identity_key,
our_identity.clone(),
timeout,
);
gateway_client
.establish_connection()
.await
.expect("failed to establish connection with the gateway!");
gateway_client
.register()
.await
.expect("failed to register with the gateway!")
}
async fn gateway_details(directory_server: &str, gateway_id: &str) -> gateway::Node {
let directory_client_config = directory_client::Config::new(directory_server.to_string());
let directory_client = directory_client::Client::new(directory_client_config);
let topology = directory_client.get_topology().await.unwrap();
let nym_topology: NymTopology = topology.try_into().expect("Invalid topology data!");
let version_filtered_topology = nym_topology.filter_system_version(built_info::PKG_VERSION);
version_filtered_topology
.gateways()
.iter()
.find(|gateway| gateway.identity_key.to_base58_string() == gateway_id)
.expect(&*format!("no gateway with id {} exists!", gateway_id))
.clone()
}
pub fn execute(matches: &ArgMatches) {
println!("Initialising client...");
let id = matches.value_of("id").unwrap(); // required for now
let provider_address = matches.value_of("provider").unwrap();
let mut config = Config::new(id, provider_address);
let mut rng = OsRng;
config = override_config(config, matches);
if matches.is_present("fastmode") {
config.get_base_mut().set_high_default_traffic_volume();
}
// create identity, encryption and ack keys.
let mut key_manager = KeyManager::new(&mut rng);
let gateway_id = matches.value_of("gateway").unwrap();
let registration_fut = async {
let gate_details =
gateway_details(&config.get_base().get_directory_server(), gateway_id).await;
let shared_keys =
register_with_gateway(&gate_details, key_manager.identity_keypair()).await;
(shared_keys, gate_details.client_listener)
};
// TODO: is there perhaps a way to make it work without having to spawn entire runtime?
let mut rt = tokio::runtime::Runtime::new().unwrap();
let (shared_keys, gateway_listener) = rt.block_on(registration_fut);
config
.get_base_mut()
.with_gateway_listener(gateway_listener);
key_manager.insert_gateway_shared_key(shared_keys);
let pathfinder = ClientKeyPathfinder::new_from_config(config.get_base());
key_manager
.store_keys(&pathfinder)
.expect("Failed to generated keys");
println!("Saved all generated keys");
let config_save_location = config.get_config_file_save_location();
config
.save_to_file(None)
.expect("Failed to save the config file");
println!("Saved configuration file to {:?}", config_save_location);
println!(
"Unless overridden in all `nym-socks5-client run` we will be talking to the following gateway: {}...",
config.get_base().get_gateway_id(),
);
println!("Client configuration completed.\n\n\n")
}
clients/socks5/src/commands/mod.rs
+39
View File
@@ -0,0 +1,39 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::client::config::Config;
use clap::ArgMatches;
pub mod init;
pub mod run;
pub(crate) fn override_config(mut config: Config, matches: &ArgMatches) -> Config {
if let Some(directory) = matches.value_of("directory") {
config.get_base_mut().with_custom_directory(directory);
}
if let Some(gateway_id) = matches.value_of("gateway") {
config.get_base_mut().with_gateway_id(gateway_id);
}
if let Some(port) = matches.value_of("port").map(|port| port.parse::<u16>()) {
if let Err(err) = port {
// if port was overridden, it must be parsable
panic!("Invalid port value provided - {:?}", err);
}
config = config.with_port(port.unwrap());
}
config
}
clients/socks5/src/commands/run.rs
+69
View File
@@ -0,0 +1,69 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use crate::client::config::Config;
use crate::client::NymClient;
use crate::commands::override_config;
use clap::{App, Arg, ArgMatches};
use config::NymConfig;
pub fn command_args<'a, 'b>() -> clap::App<'a, 'b> {
App::new("run")
.about("Run the Nym client with provided configuration client optionally overriding set parameters")
.arg(Arg::with_name("id")
.long("id")
.help("Id of the nym-mixnet-client we want to run.")
.takes_value(true)
.required(true)
)
// the rest of arguments are optional, they are used to override settings in config file
.arg(Arg::with_name("config")
.long("config")
.help("Custom path to the nym-mixnet-client configuration file")
.takes_value(true)
)
.arg(Arg::with_name("provider")
.long("provider")
.help("Address of the socks5 provider to send messages to.")
.takes_value(true)
)
.arg(Arg::with_name("directory")
.long("directory")
.help("Address of the directory server the client is getting topology from")
.takes_value(true),
)
.arg(Arg::with_name("gateway")
.long("gateway")
.help("Id of the gateway we want to connect to. If overridden, it is user's responsibility to ensure prior registration happened")
.takes_value(true)
)
.arg(Arg::with_name("port")
.short("p")
.long("port")
.help("Port for the socket to listen on")
.takes_value(true)
)
}
pub fn execute(matches: &ArgMatches) {
let id = matches.value_of("id").unwrap();
let mut config =
Config::load_from_file(matches.value_of("config").map(|path| path.into()), Some(id))
.expect("Failed to load config file");
config = override_config(config, matches);
NymClient::new(config).run_forever();
}
clients/socks5/src/lib.rs
+16
View File
@@ -0,0 +1,16 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
pub mod built_info;
pub mod client;
pub mod socks;
clients/socks5/src/main.rs
+86
View File
@@ -0,0 +1,86 @@
// Copyright 2020 Nym Technologies SA
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use clap::{App, ArgMatches};
pub mod built_info;
pub mod client;
mod commands;
pub mod socks;
fn main() {
dotenv::dotenv().ok();
setup_logging();
println!("{}", banner());
println!("\n\nIf unsure which gateway to choose - 'D6YaMzLSY7mANtSQRKXsmMZpqgqiVkeiagKM4V4oFPFr' is a safe choice for the testnet\n\n");
let arg_matches = App::new("Nym Socks5 Proxy")
.version(built_info::PKG_VERSION)
.author("Nymtech")
.about("A Socks5 localhost proxy that converts incoming messages to Sphinx and sends them to a Nym address")
.subcommand(commands::init::command_args())
.subcommand(commands::run::command_args())
.get_matches();
execute(arg_matches);
}
fn execute(matches: ArgMatches) {
match matches.subcommand() {
("init", Some(m)) => commands::init::execute(m),
("run", Some(m)) => commands::run::execute(m),
_ => println!("{}", usage()),
}
}
fn usage() -> &'static str {
"usage: --help to see available options.\n\n"
}
fn banner() -> String {
format!(
r#"
_ __ _ _ _ __ ___
| '_ \| | | | '_ \ _ \
| | | | |_| | | | | | |
|_| |_|\__, |_| |_| |_|
|___/
(socks5 proxy - version {:})
"#,
built_info::PKG_VERSION
)
}
fn setup_logging() {
let mut log_builder = pretty_env_logger::formatted_timed_builder();
if let Ok(s) = ::std::env::var("RUST_LOG") {
log_builder.parse_filters(&s);
} else {
// default to 'Info'
log_builder.filter(None, log::LevelFilter::Info);
}
log_builder
.filter_module("hyper", log::LevelFilter::Warn)
.filter_module("tokio_reactor", log::LevelFilter::Warn)
.filter_module("reqwest", log::LevelFilter::Warn)
.filter_module("mio", log::LevelFilter::Warn)
.filter_module("want", log::LevelFilter::Warn)
.filter_module("tungstenite", log::LevelFilter::Warn)
.filter_module("tokio_tungstenite", log::LevelFilter::Warn)
.init();
}
clients/socks5/src/socks/authentication.rs
+133
View File
@@ -0,0 +1,133 @@
/// Client Authentication Methods
pub(crate) enum AuthenticationMethods {
/// No Authentication
NoAuth = 0x00,
// GssApi = 0x01, // question to DH: why is this commented?
/// Authenticate with a username / password
UserPass = 0x02,
/// Cannot authenticate
NoMethods = 0xFF,
}
#[derive(Clone, Debug, PartialEq)]
/// A socks5 user with a matching password.
pub struct User {
pub username: String,
pub password: String,
}
#[derive(Clone, Debug)]
/// Allows configuration of access methods (no auth required, username/pass, reject all)
/// and keeps a list of users who have access if that method is enabled.
pub(crate) struct Authenticator {
allowed_users: Vec<User>,
pub(crate) auth_methods: Vec<u8>,
}
impl Authenticator {
pub(crate) fn new(auth_methods: Vec<u8>, allowed_users: Vec<User>) -> Authenticator {
Authenticator {
allowed_users,
auth_methods,
}
}
/// Check if username + password pair are valid
pub fn is_allowed(&self, user: &User) -> bool {
if self
.auth_methods
.contains(&(AuthenticationMethods::UserPass as u8))
{
self.allowed_users.contains(user)
} else {
false
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn constructor_works() {
let mut auth_methods: Vec<u8> = Vec::new();
auth_methods.push(AuthenticationMethods::NoAuth as u8);
auth_methods.push(AuthenticationMethods::UserPass as u8);
let admin = User {
username: "foo".to_string(),
password: "bar".to_string(),
};
let allowed_users = vec![admin.clone()];
let authenticator = Authenticator::new(auth_methods, allowed_users);
assert!(authenticator.allowed_users.contains(&admin));
assert!(authenticator
.auth_methods
.contains(&(AuthenticationMethods::NoAuth as u8)));
assert!(authenticator
.auth_methods
.contains(&(AuthenticationMethods::UserPass as u8)));
}
mod without_user_and_password_auth_enabled {
use super::*;
#[test]
fn user_pass_authentication_fails() {
let auth_methods: Vec<u8> = Vec::new(); // it's empty
let admin = User {
username: "foo".to_string(),
password: "bar".to_string(),
};
let allowed_users = vec![admin.clone()];
let authenticator = Authenticator::new(auth_methods, allowed_users);
assert!(!authenticator.is_allowed(&admin));
}
}
#[cfg(test)]
mod with_user_and_password_auth_enabled {
use super::*;
#[test]
fn allowed_user_passes_authentication_check() {
let mut auth_methods: Vec<u8> = Vec::new();
auth_methods.push(AuthenticationMethods::UserPass as u8);
let admin = User {
username: "foo".to_string(),
password: "bar".to_string(),
};
let allowed_users = vec![admin.clone()];
let authenticator = Authenticator::new(auth_methods, allowed_users);
assert!(authenticator.is_allowed(&admin));
}
#[test]
fn disallowed_user_fails_authentication_check() {
let mut auth_methods: Vec<u8> = Vec::new();
auth_methods.push(AuthenticationMethods::UserPass as u8);
let bad_user = User {
username: "ashy".to_string(),
password: "slashy".to_string(),
};
let allowed_users = Vec::new();
let authenticator = Authenticator::new(auth_methods, allowed_users);
assert!(!authenticator.is_allowed(&bad_user));
}
}
}
clients/socks5/src/socks/client.rs
+457
View File
@@ -0,0 +1,457 @@
#![forbid(unsafe_code)]
use super::authentication::{AuthenticationMethods, Authenticator, User};
use super::request::{SocksCommand, SocksRequest};
use super::types::{ResponseCode, SocksProxyError};
use super::{RESERVED, SOCKS_VERSION};
use client_core::client::inbound_messages::InputMessage;
use client_core::client::inbound_messages::InputMessageSender;
use futures::channel::mpsc;
use futures::core_reexport::pin::Pin;
use futures::task::{Context, Poll};
use log::*;
use nymsphinx::addressing::clients::Recipient;
use ordered_buffer::{OrderedMessageBuffer, OrderedMessageSender};
use pin_project::pin_project;
use proxy_helpers::available_reader::AvailableReader;
use proxy_helpers::connection_controller::{
ConnectionReceiver, ControllerCommand, ControllerSender,
};
use proxy_helpers::proxy_runner::ProxyRunner;
use rand::RngCore;
use socks5_requests::{ConnectionId, Request};
use std::net::{Shutdown, SocketAddr};
use tokio::prelude::*;
use tokio::{self, net::TcpStream};
#[pin_project(project = StateProject)]
enum StreamState {
Available(TcpStream),
RunningProxy,
}
impl StreamState {
fn finish_proxy(&mut self, stream: TcpStream) {
match self {
StreamState::RunningProxy => *self = StreamState::Available(stream),
StreamState::Available(_) => panic!("invalid state!"),
}
}
fn run_proxy(&mut self) -> TcpStream {
// It's not the nicest way to do it, but it works
#[allow(unused_assignments)]
let mut stream = None;
*self = match std::mem::replace(self, StreamState::RunningProxy) {
StreamState::Available(inner_stream) => {
stream = Some(inner_stream);
StreamState::RunningProxy
}
StreamState::RunningProxy => panic!("invalid state"),
};
stream.unwrap()
}
/// Returns the remote address that this stream is connected to.
fn peer_addr(&self) -> io::Result<SocketAddr> {
match self {
StreamState::RunningProxy => Err(io::Error::new(
io::ErrorKind::NotFound,
"stream is being used to run the proxy",
)),
StreamState::Available(ref stream) => stream.peer_addr(),
}
}
fn shutdown(&self, how: Shutdown) -> io::Result<()> {
// shutdown should only be called if proxy is not being run. If it is, there's some bug
// somewhere
match self {
StreamState::RunningProxy => panic!("Tried to shutdown stream while proxy is running"),
StreamState::Available(ref stream) => TcpStream::shutdown(stream, how),
}
}
}
// convenience implementations
impl AsyncRead for StreamState {
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut [u8],
) -> Poll<io::Result<usize>> {
match self.project() {
StateProject::RunningProxy => Poll::Ready(Err(io::Error::new(
io::ErrorKind::NotFound,
"stream is being used to run the proxy",
))),
StateProject::Available(ref mut stream) => Pin::new(stream).poll_read(cx, buf),
}
}
}
impl AsyncWrite for StreamState {
fn poll_write(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &[u8],
) -> Poll<io::Result<usize>> {
match self.project() {
StateProject::RunningProxy => Poll::Ready(Err(io::Error::new(
io::ErrorKind::NotFound,
"stream is being used to run the proxy",
))),
StateProject::Available(ref mut stream) => Pin::new(stream).poll_write(cx, buf),
}
}
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
match self.project() {
StateProject::RunningProxy => Poll::Ready(Err(io::Error::new(
io::ErrorKind::NotFound,
"stream is being used to run the proxy",
))),
StateProject::Available(ref mut stream) => Pin::new(stream).poll_flush(cx),
}
}
fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
match self.project() {
StateProject::RunningProxy => Poll::Ready(Err(io::Error::new(
io::ErrorKind::NotFound,
"stream is being used to run the proxy",
))),
StateProject::Available(ref mut stream) => Pin::new(stream).poll_shutdown(cx),
}
}
}
/// A client connecting to the Socks proxy server, because
/// it wants to make a Nym-protected outbound request. Typically, this is
/// something like e.g. a wallet app running on your laptop connecting to
/// SphinxSocksServer.
pub(crate) struct SocksClient {
controller_sender: ControllerSender,
stream: StreamState,
auth_nmethods: u8,
authenticator: Authenticator,
socks_version: u8,
input_sender: InputMessageSender,
connection_id: ConnectionId,
service_provider: Recipient,
self_address: Recipient,
}
impl Drop for SocksClient {
fn drop(&mut self) {
// TODO: decrease to debug/trace
info!("socksclient is going out of scope - the stream is getting dropped!");
info!("Connection {} is getting closed", self.connection_id);
self.controller_sender
.unbounded_send(ControllerCommand::Remove(self.connection_id))
.unwrap();
}
}
impl SocksClient {
/// Create a new SOCKClient
pub fn new(
stream: TcpStream,
authenticator: Authenticator,
input_sender: InputMessageSender,
service_provider: Recipient,
controller_sender: ControllerSender,
self_address: Recipient,
) -> Self {
let connection_id = Self::generate_random();
SocksClient {
controller_sender,
connection_id,
stream: StreamState::Available(stream),
auth_nmethods: 0,
socks_version: 0,
authenticator,
input_sender,
service_provider,
self_address,
}
}
fn generate_random() -> ConnectionId {
let mut rng = rand::rngs::OsRng;
rng.next_u64()
}
// Send an error back to the client
pub async fn error(&mut self, r: ResponseCode) -> Result<(), SocksProxyError> {
self.stream.write_all(&[5, r as u8]).await?;
Ok(())
}
/// Shutdown the TcpStream to the client and end the session
pub fn shutdown(&mut self) -> Result<(), SocksProxyError> {
info!("client is shutting down its TCP stream");
self.stream.shutdown(Shutdown::Both)?;
Ok(())
}
/// Initializes the new client, checking that the correct Socks version (5)
/// is in use and that the client is authenticated, then runs the request.
pub async fn run(&mut self) -> Result<(), SocksProxyError> {
debug!("New connection from: {}", self.stream.peer_addr()?.ip());
let mut header = [0u8; 2];
// Read a byte from the stream and determine the version being requested
self.stream.read_exact(&mut header).await?;
self.socks_version = header[0];
self.auth_nmethods = header[1];
// Handle SOCKS4 requests
if header[0] != SOCKS_VERSION {
warn!("Init: Unsupported version: SOCKS{}", self.socks_version);
self.shutdown()?;
}
// Valid SOCKS5
else {
// Authenticate w/ client
self.authenticate().await?;
// Handle requests
self.handle_request().await?;
}
Ok(())
}
async fn send_request_to_mixnet(&mut self, request: Request) {
self.send_to_mixnet(request.into_bytes()).await;
}
async fn run_proxy(
&mut self,
conn_receiver: ConnectionReceiver,
message_sender: OrderedMessageSender,
) {
let stream = self.stream.run_proxy();
let connection_id = self.connection_id;
let input_sender = self.input_sender.clone();
let recipient = self.service_provider.clone();
let (stream, _) = ProxyRunner::new(
stream,
conn_receiver,
input_sender,
connection_id,
message_sender,
)
.run(move |conn_id, read_data, socket_closed| {
let provider_request = Request::new_send(conn_id, read_data, socket_closed);
InputMessage::new_fresh(recipient, provider_request.into_bytes(), false)
})
.await
.into_inner();
// recover stream from the proxy
self.stream.finish_proxy(stream)
}
/// Handles a client request.
async fn handle_request(&mut self) -> Result<(), SocksProxyError> {
debug!("Handling CONNECT Command");
let request = SocksRequest::from_stream(&mut self.stream).await?;
let remote_address = request.to_string();
// setup for receiving from the mixnet
let (mix_sender, mix_receiver) = mpsc::unbounded();
let ordered_buffer = OrderedMessageBuffer::new();
self.controller_sender
.unbounded_send(ControllerCommand::Insert(
self.connection_id,
mix_sender,
ordered_buffer,
))
.unwrap();
match request.command {
// Use the Proxy to connect to the specified addr/port
SocksCommand::Connect => {
trace!("Connecting to: {:?}", remote_address.clone());
self.acknowledge_socks5().await;
let mut message_sender = OrderedMessageSender::new();
// 'connect' needs to be handled manually due to different structure,
// but still needs to have correct sequence number on it!
// read whatever we can
let available_reader = AvailableReader::new(&mut self.stream);
let (request_data_bytes, _) = available_reader.await?;
let ordered_message = message_sender.wrap_message(request_data_bytes.to_vec());
let socks_provider_request = Request::new_connect(
self.connection_id,
remote_address.clone(),
ordered_message,
self.self_address.clone(),
);
self.send_request_to_mixnet(socks_provider_request).await;
info!("Starting proxy for {}", remote_address.clone());
self.run_proxy(mix_receiver, message_sender).await;
info!("Proxy for {} is finished", remote_address);
}
SocksCommand::Bind => unimplemented!(), // not handled
SocksCommand::UdpAssociate => unimplemented!(), // not handled
};
Ok(())
}
/// Send serialized Socks5 request bytes to the mixnet. The request stream
/// will be chunked up into a series of one or more Sphinx packets and
/// reassembled at the destination service provider at the other end, then
/// sent onwards anonymously.
async fn send_to_mixnet(&self, request_bytes: Vec<u8>) {
let input_message =
InputMessage::new_fresh(self.service_provider.clone(), request_bytes, false);
self.input_sender.unbounded_send(input_message).unwrap();
}
/// Writes a Socks5 header back to the requesting client's TCP stream,
/// basically saying "I acknowledge your request and am dealing with it".
async fn acknowledge_socks5(&mut self) {
self.stream
.write_all(&[
SOCKS_VERSION,
ResponseCode::Success as u8,
RESERVED,
1,
127,
0,
0,
1,
0,
0,
])
.await
.unwrap();
}
/// Authenticate the incoming request. Each request is checked for its
/// authentication method. A user/password request will extract the
/// username and password from the stream, then check with the Authenticator
/// to see if the resulting user is allowed.
///
/// A lot of this could probably be put into the the `SocksRequest::from_stream()`
/// constructor, and/or cleaned up with tokio::codec. It's mostly just
/// read-a-byte-or-two. The bytes being extracted look like this:
///
/// +----+------+----------+------+------------+
/// |ver | ulen | uname | plen | password |
/// +----+------+----------+------+------------+
/// | 1 | 1 | 1 to 255 | 1 | 1 to 255 |
/// +----+------+----------+------+------------+
///
/// Pulling out the stream code into its own home, and moving the if/else logic
/// into the Authenticator (where it'll be more easily testable)
/// would be a good next step.
async fn authenticate(&mut self) -> Result<(), SocksProxyError> {
debug!("Authenticating w/ {}", self.stream.peer_addr()?.ip());
// Get valid auth methods
let methods = self.get_available_methods().await?;
trace!("methods: {:?}", methods);
let mut response = [0u8; 2];
// Set the version in the response
response[0] = SOCKS_VERSION;
if methods.contains(&(AuthenticationMethods::UserPass as u8)) {
// Set the default auth method (NO AUTH)
response[1] = AuthenticationMethods::UserPass as u8;
debug!("Sending USER/PASS packet");
self.stream.write_all(&response).await?;
let mut header = [0u8; 2];
// Read a byte from the stream and determine the version being requested
self.stream.read_exact(&mut header).await?;
// debug!("Auth Header: [{}, {}]", header[0], header[1]);
// Username parsing
let ulen = header[1];
let mut username = Vec::with_capacity(ulen as usize);
// For some reason the vector needs to actually be full
for _ in 0..ulen {
username.push(0);
}
self.stream.read_exact(&mut username).await?;
// Password Parsing
let mut plen = [0u8; 1];
self.stream.read_exact(&mut plen).await?;
let mut password = Vec::with_capacity(plen[0] as usize);
// For some reason the vector needs to actually be full
for _ in 0..plen[0] {
password.push(0);
}
self.stream.read_exact(&mut password).await?;
let username_str = String::from_utf8(username)?;
let password_str = String::from_utf8(password)?;
let user = User {
username: username_str,
password: password_str,
};
// Authenticate passwords
if self.authenticator.is_allowed(&user) {
debug!("Access Granted. User: {}", user.username);
let response = [1, ResponseCode::Success as u8];
self.stream.write_all(&response).await?;
} else {
debug!("Access Denied. User: {}", user.username);
let response = [1, ResponseCode::Failure as u8];
self.stream.write_all(&response).await?;
// Shutdown
self.shutdown()?;
}
Ok(())
} else if methods.contains(&(AuthenticationMethods::NoAuth as u8)) {
// set the default auth method (no auth)
response[1] = AuthenticationMethods::NoAuth as u8;
debug!("Sending NOAUTH packet");
self.stream.write_all(&response).await?;
Ok(())
} else {
warn!("Client has no suitable authentication methods!");
response[1] = AuthenticationMethods::NoMethods as u8;
self.stream.write_all(&response).await?;
self.shutdown()?;
Err(ResponseCode::Failure.into())
}
}
/// Return the available methods based on `self.auth_nmethods`
async fn get_available_methods(&mut self) -> Result<Vec<u8>, SocksProxyError> {
let mut methods: Vec<u8> = Vec::with_capacity(self.auth_nmethods as usize);
for _ in 0..self.auth_nmethods {
let mut method = [0u8; 1];
self.stream.read_exact(&mut method).await?;
if self.authenticator.auth_methods.contains(&method[0]) {
methods.append(&mut method.to_vec());
}
}
Ok(methods)
}
}
clients/socks5/src/socks/mixnet_responses.rs
+72
View File
@@ -0,0 +1,72 @@
use client_core::client::received_buffer::ReconstructedMessagesReceiver;
use client_core::client::received_buffer::{ReceivedBufferMessage, ReceivedBufferRequestSender};
use futures::channel::mpsc;
use futures::StreamExt;
use log::*;
use nymsphinx::receiver::ReconstructedMessage;
use proxy_helpers::connection_controller::{ControllerCommand, ControllerSender};
use socks5_requests::Response;
pub(crate) struct MixnetResponseListener {
buffer_requester: ReceivedBufferRequestSender,
mix_response_receiver: ReconstructedMessagesReceiver,
controller_sender: ControllerSender,
}
impl Drop for MixnetResponseListener {
fn drop(&mut self) {
self.buffer_requester
.unbounded_send(ReceivedBufferMessage::ReceiverDisconnect)
.expect("the buffer request failed!")
}
}
impl MixnetResponseListener {
pub(crate) fn new(
buffer_requester: ReceivedBufferRequestSender,
controller_sender: ControllerSender,
) -> Self {
let (mix_response_sender, mix_response_receiver) = mpsc::unbounded();
buffer_requester
.unbounded_send(ReceivedBufferMessage::ReceiverAnnounce(mix_response_sender))
.unwrap();
MixnetResponseListener {
controller_sender,
buffer_requester,
mix_response_receiver,
}
}
async fn on_message(&self, reconstructed_message: ReconstructedMessage) {
let raw_message = reconstructed_message.message;
if reconstructed_message.reply_surb.is_some() {
warn!("this message had a surb - we didn't do anything with it");
}
let response = match Response::try_from_bytes(&raw_message) {
Err(err) => {
warn!("failed to parse received response - {:?}", err);
return;
}
Ok(data) => data,
};
self.controller_sender
.unbounded_send(ControllerCommand::Send(
response.connection_id,
response.data,
response.is_closed,
))
.unwrap();
}
pub(crate) async fn run(&mut self) {
while let Some(received_responses) = self.mix_response_receiver.next().await {
for reconstructed_message in received_responses {
self.on_message(reconstructed_message).await;
}
}
error!("We should never see this message");
}
}
clients/socks5/src/socks/mod.rs
+14
View File
@@ -0,0 +1,14 @@
#![forbid(unsafe_code)]
pub mod authentication;
mod client;
pub(crate) mod mixnet_responses;
mod request;
pub mod server;
pub mod types;
pub mod utils;
/// Version of socks
const SOCKS_VERSION: u8 = 0x05;
const RESERVED: u8 = 0x00;
clients/socks5/src/socks/request.rs
+125
View File
@@ -0,0 +1,125 @@
use super::types::{AddrType, ResponseCode, SocksProxyError};
use super::{utils as socks_utils, SOCKS_VERSION};
use log::*;
use tokio::prelude::*;
/// A Socks5 request hitting the proxy.
pub(crate) struct SocksRequest {
pub version: u8,
pub command: SocksCommand,
pub addr_type: AddrType,
pub addr: Vec<u8>,
pub port: u16,
}
impl SocksRequest {
/// Parse a SOCKS5 request from a TcpStream
pub async fn from_stream<R>(stream: &mut R) -> Result<Self, SocksProxyError>
where
R: AsyncRead + Unpin,
{
let mut packet = [0u8; 4];
// Read a byte from the stream and determine the version being requested
stream.read_exact(&mut packet).await?;
if packet[0] != SOCKS_VERSION {
warn!("from_stream Unsupported version: SOCKS{}", packet[0]);
return Err(SocksProxyError::UnsupportedProxyVersion(packet[0]));
}
// Get command
let mut command: SocksCommand = SocksCommand::Connect;
match SocksCommand::from(packet[1] as usize) {
Some(com) => {
command = com;
Ok(())
}
None => {
warn!("Invalid Command");
Err(ResponseCode::CommandNotSupported)
}
}?;
// DST.address
let mut addr_type: AddrType = AddrType::V6;
match AddrType::from(packet[3] as usize) {
Some(addr) => {
addr_type = addr;
Ok(())
}
None => {
error!("No Addr");
Err(ResponseCode::AddrTypeNotSupported)
}
}?;
trace!("Getting Addr");
// Get Addr from addr_type and stream
let addr: Result<Vec<u8>, SocksProxyError> = match addr_type {
AddrType::Domain => {
let mut domain_length = [0u8; 1];
stream.read_exact(&mut domain_length).await?;
let mut domain = vec![0u8; domain_length[0] as usize];
stream.read_exact(&mut domain).await?;
Ok(domain)
}
AddrType::V4 => {
let mut addr = [0u8; 4];
stream.read_exact(&mut addr).await?;
Ok(addr.to_vec())
}
AddrType::V6 => {
let mut addr = [0u8; 16];
stream.read_exact(&mut addr).await?;
Ok(addr.to_vec())
}
};
let addr = addr?;
// read DST.port
let mut port = [0u8; 2];
stream.read_exact(&mut port).await?;
// Merge two u8s into u16
let port = (u16::from(port[0]) << 8) | u16::from(port[1]);
// Return parsed request
Ok(SocksRequest {
version: packet[0],
command,
addr_type,
addr,
port,
})
}
/// Print out the address and port to a String.
/// This might return domain:port, ipv6:port, or ipv4:port.
pub(crate) fn to_string(&self) -> String {
let address = socks_utils::pretty_print_addr(&self.addr_type, &self.addr);
format!("{}:{}", address, self.port)
}
}
/// SOCK5 CMD type
#[derive(Debug)]
pub(crate) enum SocksCommand {
Connect = 0x01,
Bind = 0x02,
UdpAssociate = 0x3,
}
impl SocksCommand {
/// Parse bytes to SocksCommand
fn from(n: usize) -> Option<SocksCommand> {
match n {
1 => Some(SocksCommand::Connect),
2 => Some(SocksCommand::Bind),
3 => Some(SocksCommand::UdpAssociate),
_ => None,
}
}
}
clients/socks5/src/socks/server.rs
+114
View File
@@ -0,0 +1,114 @@
use super::authentication::Authenticator;
use super::client::SocksClient;
use super::{
mixnet_responses::MixnetResponseListener,
types::{ResponseCode, SocksProxyError},
};
use client_core::client::{
inbound_messages::InputMessageSender, received_buffer::ReceivedBufferRequestSender,
};
use log::*;
use nymsphinx::addressing::clients::Recipient;
use proxy_helpers::connection_controller::Controller;
use std::net::SocketAddr;
use tokio::net::TcpListener;
/// A Socks5 server that listens for connections.
pub struct SphinxSocksServer {
authenticator: Authenticator,
listening_address: SocketAddr,
service_provider: Recipient,
self_address: Recipient,
}
impl SphinxSocksServer {
/// Create a new SphinxSocks instance
pub(crate) fn new(
port: u16,
authenticator: Authenticator,
service_provider: Recipient,
self_address: Recipient,
) -> Self {
// hardcode ip as we (presumably) ONLY want to listen locally. If we change it, we can
// just modify the config
let ip = "127.0.0.1";
info!("Listening on {}:{}", ip, port);
SphinxSocksServer {
authenticator,
listening_address: format!("{}:{}", ip, port).parse().unwrap(),
service_provider,
self_address,
}
}
/// Set up the listener and initiate connection handling when something
/// connects to the server.
pub(crate) async fn serve(
&mut self,
input_sender: InputMessageSender,
buffer_requester: ReceivedBufferRequestSender,
) -> Result<(), SocksProxyError> {
let mut listener = TcpListener::bind(self.listening_address).await.unwrap();
info!("Serving Connections...");
// controller for managing all active connections
let (mut active_streams_controller, controller_sender) = Controller::new();
tokio::spawn(async move {
active_streams_controller.run().await;
});
// listener for mix messages
let mut mixnet_response_listener =
MixnetResponseListener::new(buffer_requester, controller_sender.clone());
tokio::spawn(async move {
mixnet_response_listener.run().await;
});
loop {
if let Ok((stream, _remote)) = listener.accept().await {
// TODO Optimize this
let mut client = SocksClient::new(
stream,
self.authenticator.clone(),
input_sender.clone(),
self.service_provider.clone(),
controller_sender.clone(),
self.self_address.clone(),
);
tokio::spawn(async move {
{
match client.run().await {
Ok(_) => {}
Err(error) => {
error!("Error! {}", error);
let error_text = format!("{}", error);
let response: ResponseCode;
if error_text.contains("Host") {
response = ResponseCode::HostUnreachable;
} else if error_text.contains("Network") {
response = ResponseCode::NetworkUnreachable;
} else if error_text.contains("ttl") {
response = ResponseCode::TtlExpired
} else {
response = ResponseCode::Failure
}
if client.error(response).await.is_err() {
warn!("Failed to send error code");
};
if client.shutdown().is_err() {
warn!("Failed to shutdown TcpStream");
};
}
};
// client gets dropped here
}
});
}
}
}
}
clients/socks5/src/socks/types.rs
+68
View File
@@ -0,0 +1,68 @@
use snafu::Snafu;
#[derive(Debug, Snafu)]
/// Possible SOCKS5 Response Codes
pub(crate) enum ResponseCode {
Success = 0x00,
#[snafu(display("SOCKS5 Server Failure"))]
Failure = 0x01,
#[snafu(display("SOCKS5 Rule failure"))]
RuleFailure = 0x02,
#[snafu(display("network unreachable"))]
NetworkUnreachable = 0x03,
#[snafu(display("host unreachable"))]
HostUnreachable = 0x04,
#[snafu(display("connection refused"))]
ConnectionRefused = 0x05,
#[snafu(display("TTL expired"))]
TtlExpired = 0x06,
#[snafu(display("Command not supported"))]
CommandNotSupported = 0x07,
#[snafu(display("Addr Type not supported"))]
AddrTypeNotSupported = 0x08,
}
#[derive(Debug)]
pub enum SocksProxyError {
GenericError(Box<dyn std::error::Error + Send + Sync>),
UnsupportedProxyVersion(u8),
}
impl std::fmt::Display for SocksProxyError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
SocksProxyError::GenericError(err) => write!(f, "GenericError - {}", err),
SocksProxyError::UnsupportedProxyVersion(version) => {
write!(f, "Unsupported proxy version {}", version)
}
}
}
}
impl<E> From<E> for SocksProxyError
where
E: std::error::Error + Send + Sync + 'static,
{
fn from(err: E) -> Self {
SocksProxyError::GenericError(Box::new(err))
}
}
/// DST.addr variant types
#[derive(PartialEq)]
pub(crate) enum AddrType {
V4 = 0x01,
Domain = 0x03,
V6 = 0x04,
}
impl AddrType {
/// Parse Byte to Command
pub(crate) fn from(n: usize) -> Option<AddrType> {
match n {
1 => Some(AddrType::V4),
3 => Some(AddrType::Domain),
4 => Some(AddrType::V6),
_ => None,
}
}
}
clients/socks5/src/socks/utils.rs
+24
View File
@@ -0,0 +1,24 @@
use super::types::AddrType;
/// Convert an AddrType and address to String
pub(crate) fn pretty_print_addr(addr_type: &AddrType, addr: &[u8]) -> String {
match addr_type {
AddrType::Domain => String::from_utf8_lossy(addr).to_string(),
AddrType::V4 => addr
.iter()
.map(std::string::ToString::to_string)
.collect::<Vec<String>>()
.join("."),
AddrType::V6 => {
let addr_16 = (0..8)
.map(|x| (u16::from(addr[(x * 2)]) << 8) | u16::from(addr[(x * 2) + 1]))
.collect::<Vec<u16>>();
addr_16
.iter()
.map(|x| format!("{:x}", x))
.collect::<Vec<String>>()
.join(":")
}
}
}
clients/webassembly/Cargo.toml
+4 -2
View File
@@ -6,25 +6,27 @@ edition = "2018"
keywords = ["nym", "sphinx", "wasm", "webassembly", "privacy", "client"]
license = "Apache-2.0"
repository = "https://github.com/nymtech/nym"
description = "A webassembly client which can be used to interact with the the Nym privacy platform. Wasm is used for Sphinx packet generation."
[lib]
crate-type = ["cdylib", "rlib"]
[features]
default = ["console_error_panic_hook"]
offline-test = []
[dependencies]
log = "0.4"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
slice_as_array = "1.1.0"
wasm-bindgen = "0.2"
rand = "0.7.2"
rand = { version = "0.7.3", features = ["wasm-bindgen"] }
# internal
crypto = { path = "../../common/crypto" }
nymsphinx = { path = "../../common/nymsphinx" }
topology = { path = "../../common/topology" }
directory-client-models = { path = "../../common/client-libs/directory-client/models" }
# The `console_error_panic_hook` crate provides better debugging of panics by
# logging them with `console.error`. This is great for development, but requires
clients/webassembly/js-example/index.html
+3 -2
View File
@@ -8,7 +8,8 @@
</head>
<body>
<p>
<h1> The WASM code is currently under work. The example should be restored in 0.8.1! </h1>
<!-- <p>
<label>Sender: </label><input disabled="true" size="85" type="text" id="sender" value="">
</p>
@@ -30,7 +31,7 @@
<p>
<span id="output"></div>
</p>
<script src="./bootstrap.js"></script>
<script src="./bootstrap.js"></script> -->
</body>
clients/webassembly/js-example/index.js
+13 -13
View File
@@ -19,21 +19,21 @@ import {
async function main() {
// Set up identity and client
let directory = "https://directory.nymtech.net";
let identity = new Identity();
let nymClient = new Client(directory, identity, null);
// let directory = "https://directory.nymtech.net";
// let identity = new Identity();
// let nymClient = new Client(directory, identity, null);
// Wire up events callbacks
nymClient.onConnect = (_) => displaySenderAddress(nymClient);
nymClient.onText = displayReceived;
nymClient.onErrorResponse = (event) => alert("Received invalid gateway response", event.data);
const sendButton = document.querySelector('#send-button');
sendButton.onclick = function () {
sendMessageTo(nymClient);
}
// // Wire up events callbacks
// nymClient.onConnect = (_) => displaySenderAddress(nymClient);
// nymClient.onText = displayReceived;
// nymClient.onErrorResponse = (event) => alert("Received invalid gateway response", event.data);
// const sendButton = document.querySelector('#send-button');
// sendButton.onclick = function () {
// sendMessageTo(nymClient);
// }
// Start the Nym client. Connects to a Nym gateway via websocket.
await nymClient.start();
// // Start the Nym client. Connects to a Nym gateway via websocket.
// await nymClient.start();
}
/**
clients/webassembly/js-example/package-lock.json
Generated
+156 -223
View File
@@ -5,9 +5,19 @@
"requires": true,
"dependencies": {
"@nymproject/nym-client-wasm": {
"version": "0.7.3",
"resolved": "https://registry.npmjs.org/@nymproject/nym-client-wasm/-/nym-client-wasm-0.7.3.tgz",
"integrity": "sha512-sCLX4BGK+BjWh1lP+URexWsJoZQtyXSy2AmCaDUgCJm2rVFWhgeS2odkg2S+tdTeTeYO9Fu2YPvD8P3CFIelLg=="
"version": "0.7.5",
"resolved": "https://registry.npmjs.org/@nymproject/nym-client-wasm/-/nym-client-wasm-0.7.5.tgz",
"integrity": "sha512-r5RW4aSqyzWUvnO01EwqWdA0UGcULGrNSTe9eTDo+EoprCp1ab3vFpJs4pcg64DhxkbNbTgFVbi6TZVxObKhPQ==",
"requires": {
"semver": "^7.3.2"
},
"dependencies": {
"semver": {
"version": "7.3.2",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.3.2.tgz",
"integrity": "sha512-OrOb32TeeambH6UrhtShmF7CRDqhL6/5XpPNp2DuRH6+9QLw/orhp72j87v8Qa1ScDkvrrBNpZcDejAirJmfXQ=="
}
}
},
"@types/events": {
"version": "3.0.0",
@@ -355,20 +365,21 @@
"dev": true
},
"asn1.js": {
"version": "4.10.1",
"resolved": "https://registry.npmjs.org/asn1.js/-/asn1.js-4.10.1.tgz",
"integrity": "sha512-p32cOF5q0Zqs9uBiONKYLm6BClCoBCM5O9JfeUSlnQLBTxYdTK+pW+nXflm8UkKd2UYlEbYz5qEi0JuZR9ckSw==",
"version": "5.4.1",
"resolved": "https://registry.npmjs.org/asn1.js/-/asn1.js-5.4.1.tgz",
"integrity": "sha512-+I//4cYPccV8LdmBLiX8CYvf9Sp3vQsrqu2QNXRcrbiWvcx/UdlFiqUJJzxRQxgsZmvhXhn4cSKeSmoFjVdupA==",
"dev": true,
"requires": {
"bn.js": "^4.0.0",
"inherits": "^2.0.1",
"minimalistic-assert": "^1.0.0"
"minimalistic-assert": "^1.0.0",
"safer-buffer": "^2.1.0"
},
"dependencies": {
"bn.js": {
"version": "4.11.8",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.8.tgz",
"integrity": "sha512-ItfYfPLkWHUjckQCk8xC+LwxgK8NYcXywGigJgSwOP8Y2iyWT4f2vsZnoOXTTbo+o5yXmIUJ4gn5538SO5S3gA==",
"version": "4.11.9",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.9.tgz",
"integrity": "sha512-E6QoYqCKZfgatHTdHzs1RRKP7ip4vvm+EyRUeE2RF0NblwVvb0p6jSVeNTOFxPn26QXN2o6SMfNxKp6kU8zQaw==",
"dev": true
}
}
@@ -513,9 +524,9 @@
"dev": true
},
"binary-extensions": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.0.0.tgz",
"integrity": "sha512-Phlt0plgpIIBOGTT/ehfFnbNlfsDEiqmzE2KRXoX1bLIlir4X/MR+zSyBEkL05ffWgnRSf/DXv+WrUAVr93/ow==",
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.1.0.tgz",
"integrity": "sha512-1Yj8h9Q+QDF5FzhMs/c9+6UntbD5MkRfRwac8DoEm9ZfUBZ7tZ55YcGVAzEe4bXsdQHEk+s9S5wsOKVdZrw0tQ==",
"dev": true,
"optional": true
},
@@ -526,9 +537,9 @@
"dev": true
},
"bn.js": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-5.1.1.tgz",
"integrity": "sha512-IUTD/REb78Z2eodka1QZyyEk66pciRcP6Sroka0aI3tG/iwIdYLrBD62RsubR7vqdt3WyX8p4jxeatzmRSphtA==",
"version": "5.1.3",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-5.1.3.tgz",
"integrity": "sha512-GkTiFpjFtUzU9CbMeJ5iazkCzGL3jrhzerzZIuqLABjbwRaFt33I9tUdSNryIptM+RxDet6OKm2WnLXzW51KsQ==",
"dev": true
},
"body-parser": {
@@ -618,7 +629,7 @@
},
"browserify-aes": {
"version": "1.2.0",
"resolved": "http://registry.npmjs.org/browserify-aes/-/browserify-aes-1.2.0.tgz",
"resolved": "https://registry.npmjs.org/browserify-aes/-/browserify-aes-1.2.0.tgz",
"integrity": "sha512-+7CHXqGuspUn/Sl5aO7Ea0xWGAtETPXNSAjHo48JfLdPWcMng33Xe4znFvQweqc/uzk5zSOI3H52CYnjCfb5hA==",
"dev": true,
"requires": {
@@ -655,7 +666,7 @@
},
"browserify-rsa": {
"version": "4.0.1",
"resolved": "http://registry.npmjs.org/browserify-rsa/-/browserify-rsa-4.0.1.tgz",
"resolved": "https://registry.npmjs.org/browserify-rsa/-/browserify-rsa-4.0.1.tgz",
"integrity": "sha1-IeCr+vbyApzy+vsTNWenAdQTVSQ=",
"dev": true,
"requires": {
@@ -664,27 +675,28 @@
},
"dependencies": {
"bn.js": {
"version": "4.11.8",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.8.tgz",
"integrity": "sha512-ItfYfPLkWHUjckQCk8xC+LwxgK8NYcXywGigJgSwOP8Y2iyWT4f2vsZnoOXTTbo+o5yXmIUJ4gn5538SO5S3gA==",
"version": "4.11.9",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.9.tgz",
"integrity": "sha512-E6QoYqCKZfgatHTdHzs1RRKP7ip4vvm+EyRUeE2RF0NblwVvb0p6jSVeNTOFxPn26QXN2o6SMfNxKp6kU8zQaw==",
"dev": true
}
}
},
"browserify-sign": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/browserify-sign/-/browserify-sign-4.1.0.tgz",
"integrity": "sha512-VYxo7cDCeYUoBZ0ZCy4UyEUCP3smyBd4DRQM5nrFS1jJjPJjX7rP3oLRpPoWfkhQfyJ0I9ZbHbKafrFD/SGlrg==",
"version": "4.2.1",
"resolved": "https://registry.npmjs.org/browserify-sign/-/browserify-sign-4.2.1.tgz",
"integrity": "sha512-/vrA5fguVAKKAVTNJjgSm1tRQDHUU6DbwO9IROu/0WAzC8PKhucDSh18J0RMvVeHAn5puMd+QHC2erPRNf8lmg==",
"dev": true,
"requires": {
"bn.js": "^5.1.1",
"browserify-rsa": "^4.0.1",
"create-hash": "^1.2.0",
"create-hmac": "^1.1.7",
"elliptic": "^6.5.2",
"elliptic": "^6.5.3",
"inherits": "^2.0.4",
"parse-asn1": "^5.1.5",
"readable-stream": "^3.6.0"
"readable-stream": "^3.6.0",
"safe-buffer": "^5.2.0"
},
"dependencies": {
"readable-stream": {
@@ -697,6 +709,12 @@
"string_decoder": "^1.1.1",
"util-deprecate": "^1.0.1"
}
},
"safe-buffer": {
"version": "5.2.1",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
"integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
"dev": true
}
}
},
@@ -711,7 +729,7 @@
},
"buffer": {
"version": "4.9.2",
"resolved": "http://registry.npmjs.org/buffer/-/buffer-4.9.2.tgz",
"resolved": "https://registry.npmjs.org/buffer/-/buffer-4.9.2.tgz",
"integrity": "sha512-xq+q3SRMOxGivLhBNaUdC64hDTQwejJ+H0T/NB1XMtTVEwNTrfFF3gAxiyW0Bu/xWEGhjVKgUcMhCrUy2+uCWg==",
"dev": true,
"requires": {
@@ -819,9 +837,9 @@
}
},
"chokidar": {
"version": "3.4.0",
"resolved": "https://registry.npmjs.org/chokidar/-/chokidar-3.4.0.tgz",
"integrity": "sha512-aXAaho2VJtisB/1fg1+3nlLJqGOuewTzQpd/Tz0yTg2R0e4IGtshYvtjowyEumcBv2z+y4+kc75Mz7j5xJskcQ==",
"version": "3.4.2",
"resolved": "https://registry.npmjs.org/chokidar/-/chokidar-3.4.2.tgz",
"integrity": "sha512-IZHaDeBeI+sZJRX7lGcXsdzgvZqKv6sECqsbErJA4mHWfpRrD8B97kSFN4cQz6nGBGiuFia1MKR4d6c1o8Cv7A==",
"dev": true,
"optional": true,
"requires": {
@@ -1094,9 +1112,9 @@
"dev": true
},
"copy-webpack-plugin": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/copy-webpack-plugin/-/copy-webpack-plugin-5.1.1.tgz",
"integrity": "sha512-P15M5ZC8dyCjQHWwd4Ia/dm0SgVvZJMYeykVIVYXbGyqO4dWB5oyPHp9i7wjwo5LhtlhKbiBCdS2NvM07Wlybg==",
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/copy-webpack-plugin/-/copy-webpack-plugin-5.1.2.tgz",
"integrity": "sha512-Uh7crJAco3AjBvgAy9Z75CjK8IG+gxaErro71THQ+vv/bl4HaQcpkexAY8KVW/T6D2W2IRr+couF/knIRkZMIQ==",
"dev": true,
"requires": {
"cacache": "^12.0.3",
@@ -1109,7 +1127,7 @@
"normalize-path": "^3.0.0",
"p-limit": "^2.2.1",
"schema-utils": "^1.0.0",
"serialize-javascript": "^2.1.2",
"serialize-javascript": "^4.0.0",
"webpack-log": "^2.0.0"
}
},
@@ -1120,26 +1138,26 @@
"dev": true
},
"create-ecdh": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/create-ecdh/-/create-ecdh-4.0.3.tgz",
"integrity": "sha512-GbEHQPMOswGpKXM9kCWVrremUcBmjteUaQ01T9rkKCPDXfUHX0IoP9LpHYo2NPFampa4e+/pFDc3jQdxrxQLaw==",
"version": "4.0.4",
"resolved": "https://registry.npmjs.org/create-ecdh/-/create-ecdh-4.0.4.tgz",
"integrity": "sha512-mf+TCx8wWc9VpuxfP2ht0iSISLZnt0JgWlrOKZiNqyUZWnjIaCIVNQArMHnCZKfEYRg6IM7A+NeJoN8gf/Ws0A==",
"dev": true,
"requires": {
"bn.js": "^4.1.0",
"elliptic": "^6.0.0"
"elliptic": "^6.5.3"
},
"dependencies": {
"bn.js": {
"version": "4.11.8",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.8.tgz",
"integrity": "sha512-ItfYfPLkWHUjckQCk8xC+LwxgK8NYcXywGigJgSwOP8Y2iyWT4f2vsZnoOXTTbo+o5yXmIUJ4gn5538SO5S3gA==",
"version": "4.11.9",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.9.tgz",
"integrity": "sha512-E6QoYqCKZfgatHTdHzs1RRKP7ip4vvm+EyRUeE2RF0NblwVvb0p6jSVeNTOFxPn26QXN2o6SMfNxKp6kU8zQaw==",
"dev": true
}
}
},
"create-hash": {
"version": "1.2.0",
"resolved": "http://registry.npmjs.org/create-hash/-/create-hash-1.2.0.tgz",
"resolved": "https://registry.npmjs.org/create-hash/-/create-hash-1.2.0.tgz",
"integrity": "sha512-z00bCGNHDG8mHAkP7CtT1qVu+bFQUPjYq/4Iv3C3kWjTFV10zIjfSoeqXo9Asws8gwSHDGj/hl2u4OGIjapeCg==",
"dev": true,
"requires": {
@@ -1152,7 +1170,7 @@
},
"create-hmac": {
"version": "1.1.7",
"resolved": "http://registry.npmjs.org/create-hmac/-/create-hmac-1.1.7.tgz",
"resolved": "https://registry.npmjs.org/create-hmac/-/create-hmac-1.1.7.tgz",
"integrity": "sha512-MJG9liiZ+ogc4TzUwuvbER1JRdgvUFSB5+VR/g5h82fGaIRWMWddtKBHi7/sVhfjQZ6SehlyhvQYrcYkaUIpLg==",
"dev": true,
"requires": {
@@ -1371,7 +1389,7 @@
},
"diffie-hellman": {
"version": "5.0.3",
"resolved": "http://registry.npmjs.org/diffie-hellman/-/diffie-hellman-5.0.3.tgz",
"resolved": "https://registry.npmjs.org/diffie-hellman/-/diffie-hellman-5.0.3.tgz",
"integrity": "sha512-kqag/Nl+f3GwyK25fhUMYj81BUOrZ9IuJsjIcDE5icNM9FJHAVm3VcUDxdLPoQtTuUylWm6ZIknYJwwaPxsUzg==",
"dev": true,
"requires": {
@@ -1381,9 +1399,9 @@
},
"dependencies": {
"bn.js": {
"version": "4.11.8",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.8.tgz",
"integrity": "sha512-ItfYfPLkWHUjckQCk8xC+LwxgK8NYcXywGigJgSwOP8Y2iyWT4f2vsZnoOXTTbo+o5yXmIUJ4gn5538SO5S3gA==",
"version": "4.11.9",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.9.tgz",
"integrity": "sha512-E6QoYqCKZfgatHTdHzs1RRKP7ip4vvm+EyRUeE2RF0NblwVvb0p6jSVeNTOFxPn26QXN2o6SMfNxKp6kU8zQaw==",
"dev": true
}
}
@@ -1447,9 +1465,9 @@
"dev": true
},
"elliptic": {
"version": "6.5.2",
"resolved": "https://registry.npmjs.org/elliptic/-/elliptic-6.5.2.tgz",
"integrity": "sha512-f4x70okzZbIQl/NSRLkI/+tteV/9WqL98zx+SQ69KbXxmVrmjwsNUPn/gYJJ0sHvEak24cZgHIPegRePAtA/xw==",
"version": "6.5.3",
"resolved": "https://registry.npmjs.org/elliptic/-/elliptic-6.5.3.tgz",
"integrity": "sha512-IMqzv5wNQf+E6aHeIqATs0tOLeOTwj1QKbRcS3jBbYkl5oLAserA8yJTT7/VyHUYG91PRmPyeQDObKLPpeS4dw==",
"dev": true,
"requires": {
"bn.js": "^4.4.0",
@@ -1462,9 +1480,9 @@
},
"dependencies": {
"bn.js": {
"version": "4.11.8",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.8.tgz",
"integrity": "sha512-ItfYfPLkWHUjckQCk8xC+LwxgK8NYcXywGigJgSwOP8Y2iyWT4f2vsZnoOXTTbo+o5yXmIUJ4gn5538SO5S3gA==",
"version": "4.11.9",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.9.tgz",
"integrity": "sha512-E6QoYqCKZfgatHTdHzs1RRKP7ip4vvm+EyRUeE2RF0NblwVvb0p6jSVeNTOFxPn26QXN2o6SMfNxKp6kU8zQaw==",
"dev": true
}
}
@@ -1497,9 +1515,9 @@
}
},
"enhanced-resolve": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-4.1.1.tgz",
"integrity": "sha512-98p2zE+rL7/g/DzMHMTF4zZlCgeVdJ7yr6xzEpJRYwFYrGi9ANdn5DnJURg6RpBkyk60XYDnWIv51VfIhfNGuA==",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-4.3.0.tgz",
"integrity": "sha512-3e87LvavsdxyoCfGusJnrZ5G8SLPOFeHSNpZI/ATL9a5leXo2k0w6MKnbqhdBad9qTobSfB20Ld7UmgoNbAZkQ==",
"dev": true,
"requires": {
"graceful-fs": "^4.1.2",
@@ -1581,12 +1599,20 @@
}
},
"esrecurse": {
"version": "4.2.1",
"resolved": "https://registry.npmjs.org/esrecurse/-/esrecurse-4.2.1.tgz",
"integrity": "sha512-64RBB++fIOAXPw3P9cy89qfMlvZEXZkqqJkjqqXIvzP5ezRZjW+lPWjw35UX/3EhUPFYbg5ER4JYgDw4007/DQ==",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/esrecurse/-/esrecurse-4.3.0.tgz",
"integrity": "sha512-KmfKL3b6G+RXvP8N1vr3Tq1kL/oCFgn2NYXEtqP8/L3pKapUA4G8cFVaoF3SU323CD4XypR/ffioHmkti6/Tag==",
"dev": true,
"requires": {
"estraverse": "^4.1.0"
"estraverse": "^5.2.0"
},
"dependencies": {
"estraverse": {
"version": "5.2.0",
"resolved": "https://registry.npmjs.org/estraverse/-/estraverse-5.2.0.tgz",
"integrity": "sha512-BxbNGGNm0RyRYvUdHpIwv9IWzeM9XClbOxwoATuFdOE7ZE6wHL+HQ5T8hoPM+zHvmKzzsEqhgy0GrQ5X13afiQ==",
"dev": true
}
}
},
"estraverse": {
@@ -1608,9 +1634,9 @@
"dev": true
},
"events": {
"version": "3.1.0",
"resolved": "http://registry.npmjs.org/events/-/events-3.1.0.tgz",
"integrity": "sha512-Rv+u8MLHNOdMjTAFeT3nCjHn2aGlx435FP/sDHNaRhDEMwyI/aB22Kj2qIN8R0cw3z28psEQLYwxVKLsKrMgWg==",
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/events/-/events-3.2.0.tgz",
"integrity": "sha512-/46HWwbfCX2xTawVfkKLGxMifJYQBWMwY1mjywRtb4c9x8l5NP3KoJtnIOiL1hfdRkIuYhETxQlo62IF8tcnlg==",
"dev": true
},
"eventsource": {
@@ -2424,15 +2450,9 @@
}
},
"interpret": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/interpret/-/interpret-1.2.0.tgz",
"integrity": "sha512-mT34yGKMNceBQUoVn7iCDKDntA7SC6gycMAWzGx1z/CMCTV7b2AAtXlo3nRyHZ1FelRkQbQjprHSYGwzLtkVbw==",
"dev": true
},
"invert-kv": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/invert-kv/-/invert-kv-2.0.0.tgz",
"integrity": "sha512-wPVv/y/QQ/Uiirj/vh3oP+1Ww+AWehmi1g5fFWGPF6IpCBCDVrhgHRMvrLfdYcwDh3QJbGXDW4JAuzxElLSqKA==",
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/interpret/-/interpret-1.4.0.tgz",
"integrity": "sha512-agE4QfB2Lkp9uICn7BAqoscw4SZP9kTE2hxiFI3jBPmXJfdqiahTbUuKGsMoN2GtqL9AxhYioAcVvgsb1HvRbA==",
"dev": true
},
"ip": {
@@ -2725,15 +2745,6 @@
"integrity": "sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==",
"dev": true
},
"lcid": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/lcid/-/lcid-2.0.0.tgz",
"integrity": "sha512-avPEb8P8EGnwXKClwsNUgryVjllcRqtMYa49NTsbQagYuT1DcXnl1915oxWjoyGrXR6zH/Y0Zc96xWsPcoDKeA==",
"dev": true,
"requires": {
"invert-kv": "^2.0.0"
}
},
"loader-runner": {
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/loader-runner/-/loader-runner-2.4.0.tgz",
@@ -2792,15 +2803,6 @@
"semver": "^5.6.0"
}
},
"map-age-cleaner": {
"version": "0.1.3",
"resolved": "https://registry.npmjs.org/map-age-cleaner/-/map-age-cleaner-0.1.3.tgz",
"integrity": "sha512-bJzx6nMoP6PDLPBFmg7+xRKeFZvFboMrGlxmNj9ClvX53KrmvM5bXFXEWjbz4cz1AFn+jWJ9z/DJSz7hrs0w3w==",
"dev": true,
"requires": {
"p-defer": "^1.0.0"
}
},
"map-cache": {
"version": "0.2.2",
"resolved": "https://registry.npmjs.org/map-cache/-/map-cache-0.2.2.tgz",
@@ -2833,17 +2835,6 @@
"integrity": "sha1-hxDXrwqmJvj/+hzgAWhUUmMlV0g=",
"dev": true
},
"mem": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/mem/-/mem-4.3.0.tgz",
"integrity": "sha512-qX2bG48pTqYRVmDB37rn/6PT7LcR8T7oAX3bf99u1Tt1nzxYfxkgqDwUwolPlXweM0XzBOBFzSx4kfp7KP1s/w==",
"dev": true,
"requires": {
"map-age-cleaner": "^0.1.1",
"mimic-fn": "^2.0.0",
"p-is-promise": "^2.0.0"
}
},
"memory-fs": {
"version": "0.4.1",
"resolved": "https://registry.npmjs.org/memory-fs/-/memory-fs-0.4.1.tgz",
@@ -2898,9 +2889,9 @@
},
"dependencies": {
"bn.js": {
"version": "4.11.8",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.8.tgz",
"integrity": "sha512-ItfYfPLkWHUjckQCk8xC+LwxgK8NYcXywGigJgSwOP8Y2iyWT4f2vsZnoOXTTbo+o5yXmIUJ4gn5538SO5S3gA==",
"version": "4.11.9",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.9.tgz",
"integrity": "sha512-E6QoYqCKZfgatHTdHzs1RRKP7ip4vvm+EyRUeE2RF0NblwVvb0p6jSVeNTOFxPn26QXN2o6SMfNxKp6kU8zQaw==",
"dev": true
}
}
@@ -2926,12 +2917,6 @@
"mime-db": "1.44.0"
}
},
"mimic-fn": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/mimic-fn/-/mimic-fn-2.1.0.tgz",
"integrity": "sha512-OqbOk5oEQeAZ8WXWydlu9HJjz9WVdEIvamMCcXmuqUYjTknH/sqsWvhQ3vgwKFRR1HpjvNBKQ37nbJgYzGqGcg==",
"dev": true
},
"minimalistic-assert": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/minimalistic-assert/-/minimalistic-assert-1.0.1.tgz",
@@ -3069,9 +3054,9 @@
"dev": true
},
"neo-async": {
"version": "2.6.1",
"resolved": "https://registry.npmjs.org/neo-async/-/neo-async-2.6.1.tgz",
"integrity": "sha512-iyam8fBuCUpWeKPGpaNMetEocMt364qkCsfL9JuhjXX6dRnguRVOfk2GZaDpPjcOKiiXCPINZC1GczQ7iTq3Zw==",
"version": "2.6.2",
"resolved": "https://registry.npmjs.org/neo-async/-/neo-async-2.6.2.tgz",
"integrity": "sha512-Yd3UES5mWCSqR+qNT93S3UoYUkqAZ9lLg8a7g9rimsWmYGK8cVToA4/sF3RrshdyV3sAGMXVUmpMYOw+dLpOuw==",
"dev": true
},
"nice-try": {
@@ -3283,35 +3268,12 @@
"integrity": "sha1-hUNzx/XCMVkU/Jv8a9gjj92h7Cc=",
"dev": true
},
"os-locale": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/os-locale/-/os-locale-3.1.0.tgz",
"integrity": "sha512-Z8l3R4wYWM40/52Z+S265okfFj8Kt2cC2MKY+xNi3kFs+XGI7WXu/I309QQQYbRW4ijiZ+yxs9pqEhJh0DqW3Q==",
"dev": true,
"requires": {
"execa": "^1.0.0",
"lcid": "^2.0.0",
"mem": "^4.0.0"
}
},
"p-defer": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/p-defer/-/p-defer-1.0.0.tgz",
"integrity": "sha1-n26xgvbJqozXQwBKfU+WsZaw+ww=",
"dev": true
},
"p-finally": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/p-finally/-/p-finally-1.0.0.tgz",
"integrity": "sha1-P7z7FbiZpEEjs0ttzBi3JDNqLK4=",
"dev": true
},
"p-is-promise": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/p-is-promise/-/p-is-promise-2.1.0.tgz",
"integrity": "sha512-Y3W0wlRPK8ZMRbNq97l4M5otioeA5lm1z7bkNkxCka8HSPjR0xRWmpCmc9utiaLP9Jb1eD8BgeIxTW4AIF45Pg==",
"dev": true
},
"p-limit": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/p-limit/-/p-limit-2.3.0.tgz",
@@ -3369,14 +3331,13 @@
}
},
"parse-asn1": {
"version": "5.1.5",
"resolved": "http://registry.npmjs.org/parse-asn1/-/parse-asn1-5.1.5.tgz",
"integrity": "sha512-jkMYn1dcJqF6d5CpU689bq7w/b5ALS9ROVSpQDPrZsqqesUJii9qutvoT5ltGedNXMO2e16YUWIghG9KxaViTQ==",
"version": "5.1.6",
"resolved": "https://registry.npmjs.org/parse-asn1/-/parse-asn1-5.1.6.tgz",
"integrity": "sha512-RnZRo1EPU6JBnra2vGHj0yhp6ebyjBZpmUCLHWiFhxlzvBCCpAuZ7elsBp1PVAbQN0/04VD/19rfzlBSwLstMw==",
"dev": true,
"requires": {
"asn1.js": "^4.0.0",
"asn1.js": "^5.2.0",
"browserify-aes": "^1.0.0",
"create-hash": "^1.1.0",
"evp_bytestokey": "^1.0.0",
"pbkdf2": "^3.0.3",
"safe-buffer": "^5.1.1"
@@ -3460,9 +3421,9 @@
}
},
"pbkdf2": {
"version": "3.0.17",
"resolved": "https://registry.npmjs.org/pbkdf2/-/pbkdf2-3.0.17.tgz",
"integrity": "sha512-U/il5MsrZp7mGg3mSQfn742na2T+1/vHDCG5/iTI3X9MKUuYUZVLQhyRsg06mCgDBTd57TxzgZt7P+fYfjRLtA==",
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/pbkdf2/-/pbkdf2-3.1.1.tgz",
"integrity": "sha512-4Ejy1OPxi9f2tt1rRV7Go7zmfDQ+ZectEQz3VGUQhgq62HtIRPDyG/JtnwIxs6x3uNMwo2V7q1fMvKjb+Tnpqg==",
"dev": true,
"requires": {
"create-hash": "^1.1.2",
@@ -3592,9 +3553,9 @@
},
"dependencies": {
"bn.js": {
"version": "4.11.8",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.8.tgz",
"integrity": "sha512-ItfYfPLkWHUjckQCk8xC+LwxgK8NYcXywGigJgSwOP8Y2iyWT4f2vsZnoOXTTbo+o5yXmIUJ4gn5538SO5S3gA==",
"version": "4.11.9",
"resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.9.tgz",
"integrity": "sha512-E6QoYqCKZfgatHTdHzs1RRKP7ip4vvm+EyRUeE2RF0NblwVvb0p6jSVeNTOFxPn26QXN2o6SMfNxKp6kU8zQaw==",
"dev": true
}
}
@@ -3955,10 +3916,13 @@
}
},
"serialize-javascript": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/serialize-javascript/-/serialize-javascript-2.1.2.tgz",
"integrity": "sha512-rs9OggEUF0V4jUSecXazOYsLfu7OGK2qIn3c7IPBiffz32XniEp/TX9Xmc9LQfK2nQ2QKHvZ2oygKUGU0lG4jQ==",
"dev": true
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/serialize-javascript/-/serialize-javascript-4.0.0.tgz",
"integrity": "sha512-GaNA54380uFefWghODBWEGisLZFj00nS5ACs6yHa9nLqlLpVLO8ChDGeKRjZnV4Nh4n0Qi7nhYZD/9fCPzEqkw==",
"dev": true,
"requires": {
"randombytes": "^2.1.0"
}
},
"serve-index": {
"version": "1.9.1",
@@ -4056,7 +4020,7 @@
},
"sha.js": {
"version": "2.4.11",
"resolved": "http://registry.npmjs.org/sha.js/-/sha.js-2.4.11.tgz",
"resolved": "https://registry.npmjs.org/sha.js/-/sha.js-2.4.11.tgz",
"integrity": "sha512-QMEp5B7cftE7APOjk5Y6xgrbWu+WkLVQwk8JNjZ8nKRciZaByEW6MubieAiToS7+dwvrjGhH8jRXz3MVd0AYqQ==",
"dev": true,
"requires": {
@@ -4547,9 +4511,9 @@
"dev": true
},
"terser": {
"version": "4.6.13",
"resolved": "https://registry.npmjs.org/terser/-/terser-4.6.13.tgz",
"integrity": "sha512-wMvqukYgVpQlymbnNbabVZbtM6PN63AzqexpwJL8tbh/mRT9LE5o+ruVduAGL7D6Fpjl+Q+06U5I9Ul82odAhw==",
"version": "4.8.0",
"resolved": "https://registry.npmjs.org/terser/-/terser-4.8.0.tgz",
"integrity": "sha512-EAPipTNeWsb/3wLPeup1tVPaXfIaU68xMnVdPafIL1TV05OhASArYyIfFvnvJCNrR2NIOvDVNNTFRa+Re2MWyw==",
"dev": true,
"requires": {
"commander": "^2.20.0",
@@ -4566,16 +4530,16 @@
}
},
"terser-webpack-plugin": {
"version": "1.4.3",
"resolved": "https://registry.npmjs.org/terser-webpack-plugin/-/terser-webpack-plugin-1.4.3.tgz",
"integrity": "sha512-QMxecFz/gHQwteWwSo5nTc6UaICqN1bMedC5sMtUc7y3Ha3Q8y6ZO0iCR8pq4RJC8Hjf0FEPEHZqcMB/+DFCrA==",
"version": "1.4.5",
"resolved": "https://registry.npmjs.org/terser-webpack-plugin/-/terser-webpack-plugin-1.4.5.tgz",
"integrity": "sha512-04Rfe496lN8EYruwi6oPQkG0vo8C+HT49X687FZnpPF0qMAIHONI6HEXYPKDOE8e5HjXTyKfqRd/agHtH0kOtw==",
"dev": true,
"requires": {
"cacache": "^12.0.2",
"find-cache-dir": "^2.1.0",
"is-wsl": "^1.1.0",
"schema-utils": "^1.0.0",
"serialize-javascript": "^2.1.2",
"serialize-javascript": "^4.0.0",
"source-map": "^0.6.1",
"terser": "^4.1.2",
"webpack-sources": "^1.4.0",
@@ -4864,9 +4828,9 @@
"dev": true
},
"v8-compile-cache": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/v8-compile-cache/-/v8-compile-cache-2.0.3.tgz",
"integrity": "sha512-CNmdbwQMBjwr9Gsmohvm0pbL954tJrNzf6gWL3K+QMQf00PF7ERGrEiLgjuU3mKreLC2MeGhUsNV9ybTbLgd3w==",
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/v8-compile-cache/-/v8-compile-cache-2.1.1.tgz",
"integrity": "sha512-8OQ9CL+VWyt3JStj7HX7/ciTL2V3Rl1Wf5OL+SNTm0yK1KvtReVulksyeRnCANHHuUxHlQig+JJDlUhBt1NQDQ==",
"dev": true
},
"vary": {
@@ -4882,12 +4846,12 @@
"dev": true
},
"watchpack": {
"version": "1.7.2",
"resolved": "https://registry.npmjs.org/watchpack/-/watchpack-1.7.2.tgz",
"integrity": "sha512-ymVbbQP40MFTp+cNMvpyBpBtygHnPzPkHqoIwRRj/0B8KhqQwV8LaKjtbaxF2lK4vl8zN9wCxS46IFCU5K4W0g==",
"version": "1.7.4",
"resolved": "https://registry.npmjs.org/watchpack/-/watchpack-1.7.4.tgz",
"integrity": "sha512-aWAgTW4MoSJzZPAicljkO1hsi1oKj/RRq/OJQh2PKI2UKL04c2Bs+MBOB+BBABHTXJpf9mCwHN7ANCvYsvY2sg==",
"dev": true,
"requires": {
"chokidar": "^3.4.0",
"chokidar": "^3.4.1",
"graceful-fs": "^4.1.2",
"neo-async": "^2.5.0",
"watchpack-chokidar2": "^2.0.0"
@@ -4995,9 +4959,9 @@
}
},
"webpack": {
"version": "4.43.0",
"resolved": "https://registry.npmjs.org/webpack/-/webpack-4.43.0.tgz",
"integrity": "sha512-GW1LjnPipFW2Y78OOab8NJlCflB7EFskMih2AHdvjbpKMeDJqEgSx24cXXXiPS65+WSwVyxtDsJH6jGX2czy+g==",
"version": "4.44.1",
"resolved": "https://registry.npmjs.org/webpack/-/webpack-4.44.1.tgz",
"integrity": "sha512-4UOGAohv/VGUNQJstzEywwNxqX417FnjZgZJpJQegddzPmTvph37eBIRbRTfdySXzVtJXLJfbMN3mMYhM6GdmQ==",
"dev": true,
"requires": {
"@webassemblyjs/ast": "1.9.0",
@@ -5008,7 +4972,7 @@
"ajv": "^6.10.2",
"ajv-keywords": "^3.4.1",
"chrome-trace-event": "^1.0.2",
"enhanced-resolve": "^4.1.0",
"enhanced-resolve": "^4.3.0",
"eslint-scope": "^4.0.3",
"json-parse-better-errors": "^1.0.2",
"loader-runner": "^2.4.0",
@@ -5021,57 +4985,27 @@
"schema-utils": "^1.0.0",
"tapable": "^1.1.3",
"terser-webpack-plugin": "^1.4.3",
"watchpack": "^1.6.1",
"watchpack": "^1.7.4",
"webpack-sources": "^1.4.1"
}
},
"webpack-cli": {
"version": "3.3.11",
"resolved": "https://registry.npmjs.org/webpack-cli/-/webpack-cli-3.3.11.tgz",
"integrity": "sha512-dXlfuml7xvAFwYUPsrtQAA9e4DOe58gnzSxhgrO/ZM/gyXTBowrsYeubyN4mqGhYdpXMFNyQ6emjJS9M7OBd4g==",
"version": "3.3.12",
"resolved": "https://registry.npmjs.org/webpack-cli/-/webpack-cli-3.3.12.tgz",
"integrity": "sha512-NVWBaz9k839ZH/sinurM+HcDvJOTXwSjYp1ku+5XKeOC03z8v5QitnK/x+lAxGXFyhdayoIf/GOpv85z3/xPag==",
"dev": true,
"requires": {
"chalk": "2.4.2",
"cross-spawn": "6.0.5",
"enhanced-resolve": "4.1.0",
"findup-sync": "3.0.0",
"global-modules": "2.0.0",
"import-local": "2.0.0",
"interpret": "1.2.0",
"loader-utils": "1.2.3",
"supports-color": "6.1.0",
"v8-compile-cache": "2.0.3",
"yargs": "13.2.4"
},
"dependencies": {
"emojis-list": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/emojis-list/-/emojis-list-2.1.0.tgz",
"integrity": "sha1-TapNnbAPmBmIDHn6RXrlsJof04k=",
"dev": true
},
"enhanced-resolve": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-4.1.0.tgz",
"integrity": "sha512-F/7vkyTtyc/llOIn8oWclcB25KdRaiPBpZYDgJHgh/UHtpgT2p2eldQgtQnLtUvfMKPKxbRaQM/hHkvLHt1Vng==",
"dev": true,
"requires": {
"graceful-fs": "^4.1.2",
"memory-fs": "^0.4.0",
"tapable": "^1.0.0"
}
},
"loader-utils": {
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/loader-utils/-/loader-utils-1.2.3.tgz",
"integrity": "sha512-fkpz8ejdnEMG3s37wGL07iSBDg99O9D5yflE9RGNH3hRdx9SOwYfnGYdZOUIZitN8E+E2vkq3MUMYMvPYl5ZZA==",
"dev": true,
"requires": {
"big.js": "^5.2.2",
"emojis-list": "^2.0.0",
"json5": "^1.0.1"
}
}
"chalk": "^2.4.2",
"cross-spawn": "^6.0.5",
"enhanced-resolve": "^4.1.1",
"findup-sync": "^3.0.0",
"global-modules": "^2.0.0",
"import-local": "^2.0.0",
"interpret": "^1.4.0",
"loader-utils": "^1.4.0",
"supports-color": "^6.1.0",
"v8-compile-cache": "^2.1.1",
"yargs": "^13.3.2"
}
},
"webpack-dev-middleware": {
@@ -5304,9 +5238,9 @@
}
},
"websocket-extensions": {
"version": "0.1.3",
"resolved": "https://registry.npmjs.org/websocket-extensions/-/websocket-extensions-0.1.3.tgz",
"integrity": "sha512-nqHUnMXmBzT0w570r2JpJxfiSD1IzoI+HGVdd3aZ0yNi3ngvQ4jv1dtHt5VGxfI2yj5yqImPhOK4vmIh2xMbGg==",
"version": "0.1.4",
"resolved": "https://registry.npmjs.org/websocket-extensions/-/websocket-extensions-0.1.4.tgz",
"integrity": "sha512-OqedPIGOfsDlo31UNwYbCFMSaO9m9G/0faIHj5/dZFDMFqPTcx6UwqyOy3COEaEOg/9VsGIpdqn62W5KhoKSpg==",
"dev": true
},
"which": {
@@ -5378,22 +5312,21 @@
"dev": true
},
"yargs": {
"version": "13.2.4",
"resolved": "https://registry.npmjs.org/yargs/-/yargs-13.2.4.tgz",
"integrity": "sha512-HG/DWAJa1PAnHT9JAhNa8AbAv3FPaiLzioSjCcmuXXhP8MlpHO5vwls4g4j6n30Z74GVQj8Xa62dWVx1QCGklg==",
"version": "13.3.2",
"resolved": "https://registry.npmjs.org/yargs/-/yargs-13.3.2.tgz",
"integrity": "sha512-AX3Zw5iPruN5ie6xGRIDgqkT+ZhnRlZMLMHAs8tg7nRruy2Nb+i5o9bwghAogtM08q1dpr2LVoS8KSTMYpWXUw==",
"dev": true,
"requires": {
"cliui": "^5.0.0",
"find-up": "^3.0.0",
"get-caller-file": "^2.0.1",
"os-locale": "^3.1.0",
"require-directory": "^2.1.1",
"require-main-filename": "^2.0.0",
"set-blocking": "^2.0.0",
"string-width": "^3.0.0",
"which-module": "^2.0.0",
"y18n": "^4.0.0",
"yargs-parser": "^13.1.0"
"yargs-parser": "^13.1.2"
}
},
"yargs-parser": {
clients/webassembly/js-example/package.json
+4 -4
View File
@@ -27,13 +27,13 @@
},
"homepage": "https://nymtech.net/docs",
"devDependencies": {
"copy-webpack-plugin": "^5.0.0",
"copy-webpack-plugin": "^5.1.2",
"hello-wasm-pack": "^0.1.0",
"webpack": "^4.29.3",
"webpack-cli": "^3.1.0",
"webpack": "^4.44.1",
"webpack-cli": "^3.3.12",
"webpack-dev-server": "^3.11.0"
},
"dependencies": {
"@nymproject/nym-client-wasm": "^0.7.5"
}
}
}
clients/webassembly/js-example/yarn.lock
-3905
View File
File diff suppressed because it is too large Load Diff
clients/webassembly/react-example/.gitignore
+23
View File
@@ -0,0 +1,23 @@
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
# dependencies
/node_modules
/.pnp
.pnp.js
# testing
/coverage
# production
/build
# misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local
npm-debug.log*
yarn-debug.log*
yarn-error.log*
clients/webassembly/react-example/README.md
+20
View File
@@ -0,0 +1,20 @@
This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
# Nym Sphinx Wasm Demo
This example application demonstrates how to use WebAssembly to create Sphinx packets, in the browser, and forward them to a Nym gateway.
## 🚴 Usage
```
npm install # set up dependencies
npm run start # starts a web server at http://localhost:3000
```
Check your dev console for output.
### Rebuild after Rust source changes
Install `wasm-pack`. Instruction are at the [Rust WASM tutorial](https://rustwasm.github.io/docs/book/game-of-life/hello-world.html).
`wasm-pack build` in the `clients/webassembly` directory (one up) will rebuild the wasm package if you make changes to the Rust source. That will be automatically picked up (and reloaded, if need be) by the npm dev server.
clients/webassembly/react-example/config-overrides.js
Vendored
+25
View File
@@ -0,0 +1,25 @@
const path = require('path');
module.exports = function override(config, env) {
const wasmExtensionRegExp = /\.wasm$/;
config.resolve.extensions.push('.wasm');
config.module.rules.forEach(rule => {
(rule.oneOf || []).forEach(oneOf => {
if (oneOf.loader && oneOf.loader.indexOf('file-loader') >= 0) {
// make file-loader ignore WASM files
oneOf.exclude.push(wasmExtensionRegExp);
}
});
});
// add a dedicated loader for WASM
config.module.rules.push({
test: wasmExtensionRegExp,
include: path.resolve(__dirname, 'src'),
use: [{ loader: require.resolve('wasm-loader'), options: {} }]
});
return config;
};
clients/webassembly/react-example/package-lock.json
Generated
+14640
View File
File diff suppressed because it is too large Load Diff
clients/webassembly/react-example/package.json
+47
View File
@@ -0,0 +1,47 @@
{
"name": "react-example",
"version": "0.1.0",
"private": true,
"dependencies": {
"@nymproject/nym-client-wasm": "^0.7.5",
"@testing-library/jest-dom": "^4.2.4",
"@testing-library/react": "^9.5.0",
"@testing-library/user-event": "^7.2.1",
"bootstrap": "^4.5.0",
"node-sass": "^4.14.1",
"react": "^16.13.1",
"react-dom": "^16.13.1",
"react-scripts": "3.4.1",
"reactstrap": "^8.5.1",
"request": "^2.79.0"
},
"scripts": {
"start": "react-app-rewired start",
"build": "react-app-rewired build",
"test": "react-app-rewired test",
"eject": "react-app-rewired eject"
},
"eslintConfig": {
"extends": "react-app"
},
"browserslist": {
"production": [
">0.2%",
"not dead",
"not op_mini all"
],
"development": [
"last 1 chrome version",
"last 1 firefox version",
"last 1 safari version"
]
},
"devDependencies": {
"copy-webpack-plugin": "^5.0.0",
"hello-wasm-pack": "^0.1.0",
"react-app-rewired": "^2.1.6",
"wasm-loader": "^1.3.0",
"webpack": "^4.29.3",
"webpack-cli": "^3.1.0"
}
}
clients/webassembly/react-example/public/favicon.ico
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.1 KiB

Some files were not shown because too many files have changed in this diff Show More