GRIN/nym
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'release/v0.4.0-rc.1'

Browse Source
This commit is contained in:
Dave Hrycyszyn
2020-01-28 12:06:31 +00:00
parent bec0223899 1bc68ddfa1
commit c9cefa9b66
89 changed files with 2296 additions and 1488 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1
View File
@@ -5,3 +5,4 @@
.idea
target
.env
/.vscode/settings.json
CHANGELOG.md
+121
View File
@@ -0,0 +1,121 @@
# Changelog
## [Unreleased](https://github.com/nymtech/nym/tree/HEAD)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.3...HEAD)
**Closed issues:**
- Check Sphinx packet length in client [\#98](https://github.com/nymtech/nym/issues/98)
- workflow test [\#97](https://github.com/nymtech/nym/issues/97)
- Client SemVer [\#85](https://github.com/nymtech/nym/issues/85)
- Move PemStore [\#74](https://github.com/nymtech/nym/issues/74)
- Periodic client refresh [\#70](https://github.com/nymtech/nym/issues/70)
- Logging [\#68](https://github.com/nymtech/nym/issues/68)
- Nym-client refactor [\#67](https://github.com/nymtech/nym/issues/67)
- Stop panicking! [\#66](https://github.com/nymtech/nym/issues/66)
- Fix Mixnode Panic on Sphinx packet replay [\#65](https://github.com/nymtech/nym/issues/65)
- Convert older code to start using logging [\#35](https://github.com/nymtech/nym/issues/35)
- Convert to non-url-safe base64 \(everywhere\) [\#28](https://github.com/nymtech/nym/issues/28)
- If a thread blows at startup, panic the entire application [\#15](https://github.com/nymtech/nym/issues/15)
- Split provider/mod.rs [\#10](https://github.com/nymtech/nym/issues/10)
**Merged pull requests:**
- Feature/health checker with existing keys [\#105](https://github.com/nymtech/nym/pull/105) ([jstuczyn](https://github.com/jstuczyn))
- Feature/remove topology equality check [\#104](https://github.com/nymtech/nym/pull/104) ([futurechimp](https://github.com/futurechimp))
- Feature/base58 [\#102](https://github.com/nymtech/nym/pull/102) ([futurechimp](https://github.com/futurechimp))
- Feature/panic improvements [\#101](https://github.com/nymtech/nym/pull/101) ([jstuczyn](https://github.com/jstuczyn))
- Feature/fix sphinx unwraps [\#100](https://github.com/nymtech/nym/pull/100) ([futurechimp](https://github.com/futurechimp))
- Feature/check packet length [\#99](https://github.com/nymtech/nym/pull/99) ([futurechimp](https://github.com/futurechimp))
- Feature/version filtering improvements [\#96](https://github.com/nymtech/nym/pull/96) ([futurechimp](https://github.com/futurechimp))
- Feature/refreshing topology [\#94](https://github.com/nymtech/nym/pull/94) ([jstuczyn](https://github.com/jstuczyn))
- Feature/consistent logging [\#93](https://github.com/nymtech/nym/pull/93) ([futurechimp](https://github.com/futurechimp))
- Feature/semver client [\#92](https://github.com/nymtech/nym/pull/92) ([futurechimp](https://github.com/futurechimp))
- Feature/client refactor [\#91](https://github.com/nymtech/nym/pull/91) ([jstuczyn](https://github.com/jstuczyn))
- Release builds should no longer silently fail - everything will be im… [\#89](https://github.com/nymtech/nym/pull/89) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.3](https://github.com/nymtech/nym/tree/v0.3.3) (2020-01-20)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.2...v0.3.3)
**Fixed bugs:**
- Nym client crashing and disconnecting local websocket with complaint about binary data [\#56](https://github.com/nymtech/nym/issues/56)
**Closed issues:**
- Websocket text fix [\#64](https://github.com/nymtech/nym/issues/64)
- Restore nym-client lib in crate [\#63](https://github.com/nymtech/nym/issues/63)
- Make websocket not crash on ping or pong messages [\#62](https://github.com/nymtech/nym/issues/62)
- Messages returned by fetch are base64 encoded [\#55](https://github.com/nymtech/nym/issues/55)
- Check layer 1 connectivity at client start [\#38](https://github.com/nymtech/nym/issues/38)
- Check required sfw-provider args [\#27](https://github.com/nymtech/nym/issues/27)
- Make Electron app work with new Rust mixnet client [\#16](https://github.com/nymtech/nym/issues/16)
- Take version numbers into account when picking routes [\#14](https://github.com/nymtech/nym/issues/14)
**Merged pull requests:**
- Feature/websocket improvements [\#88](https://github.com/nymtech/nym/pull/88) ([jstuczyn](https://github.com/jstuczyn))
- Using println rather than log for startup banner, it's not an error [\#87](https://github.com/nymtech/nym/pull/87) ([futurechimp](https://github.com/futurechimp))
- Feature/nym client lib [\#61](https://github.com/nymtech/nym/pull/61) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.2](https://github.com/nymtech/nym/tree/v0.3.2) (2020-01-17)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.1...v0.3.2)
**Merged pull requests:**
- Feature/separate presence address [\#59](https://github.com/nymtech/nym/pull/59) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.1](https://github.com/nymtech/nym/tree/v0.3.1) (2020-01-16)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.3.0...v0.3.1)
**Merged pull requests:**
- Bugfix/presence client crash [\#58](https://github.com/nymtech/nym/pull/58) ([jstuczyn](https://github.com/jstuczyn))
## [v0.3.0](https://github.com/nymtech/nym/tree/v0.3.0) (2020-01-14)
[Full Changelog](https://github.com/nymtech/nym/compare/v0.2.0...v0.3.0)
**Closed issues:**
- Error referring to mismatched types caused by topology [\#46](https://github.com/nymtech/nym/issues/46)
- Provider needs to announce two of its addresses \(+ issue of hardcoded port\) [\#32](https://github.com/nymtech/nym/issues/32)
- Port run\_network.sh from old Go mixnet [\#30](https://github.com/nymtech/nym/issues/30)
- Health Checker inside Validator [\#29](https://github.com/nymtech/nym/issues/29)
- Combine Rust repositories into a single master repo containing multiple crates [\#24](https://github.com/nymtech/nym/issues/24)
- Fix the version numbers on provider and mixnode [\#23](https://github.com/nymtech/nym/issues/23)
- Travis releases for Nym [\#22](https://github.com/nymtech/nym/issues/22)
- Add version number to presence [\#4](https://github.com/nymtech/nym/issues/4)
- Add version number to presence [\#1](https://github.com/nymtech/nym/issues/1)
**Merged pull requests:**
- Feature/client topology filtering [\#54](https://github.com/nymtech/nym/pull/54) ([jstuczyn](https://github.com/jstuczyn))
- print public key for nym client tools [\#53](https://github.com/nymtech/nym/pull/53) ([mileschet](https://github.com/mileschet))
- Showing binding warning on binding to localhost, 0.0.0.0 or 127.0.0.1 [\#52](https://github.com/nymtech/nym/pull/52) ([jstuczyn](https://github.com/jstuczyn))
- validator: moving sample config files into sample configs directory [\#51](https://github.com/nymtech/nym/pull/51) ([futurechimp](https://github.com/futurechimp))
- Feature/validator health checker [\#50](https://github.com/nymtech/nym/pull/50) ([jstuczyn](https://github.com/jstuczyn))
- Improving websocket connection errors [\#49](https://github.com/nymtech/nym/pull/49) ([futurechimp](https://github.com/futurechimp))
- Feature/crypto crate [\#48](https://github.com/nymtech/nym/pull/48) ([jstuczyn](https://github.com/jstuczyn))
- Feature/chill log messages [\#43](https://github.com/nymtech/nym/pull/43) ([futurechimp](https://github.com/futurechimp))
- persistence: improving PEM file reading and parsing failure messages [\#42](https://github.com/nymtech/nym/pull/42) ([futurechimp](https://github.com/futurechimp))
- Providing a nicer error than "failed on unwrap\(\)" when topology retri… [\#41](https://github.com/nymtech/nym/pull/41) ([futurechimp](https://github.com/futurechimp))
- Prettying up sfw-provider start sequence a bit. [\#40](https://github.com/nymtech/nym/pull/40) ([futurechimp](https://github.com/futurechimp))
- Removing the run command from code and documentation [\#39](https://github.com/nymtech/nym/pull/39) ([futurechimp](https://github.com/futurechimp))
- Feature/script/localnet [\#37](https://github.com/nymtech/nym/pull/37) ([futurechimp](https://github.com/futurechimp))
- Feature/arguments fix [\#36](https://github.com/nymtech/nym/pull/36) ([jstuczyn](https://github.com/jstuczyn))
- Feature/crates cleanup [\#34](https://github.com/nymtech/nym/pull/34) ([jstuczyn](https://github.com/jstuczyn))
- Features/version number to presence [\#25](https://github.com/nymtech/nym/pull/25) ([aniampio](https://github.com/aniampio))
- Feature/minor client changes [\#7](https://github.com/nymtech/nym/pull/7) ([jstuczyn](https://github.com/jstuczyn))
## [v0.2.0](https://github.com/nymtech/nym/tree/v0.2.0) (2020-01-07)
[Full Changelog](https://github.com/nymtech/nym/compare/3c64a2facd753f4f2f431e7f888e54842e2bc64e...v0.2.0)
\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*
Cargo.lock
Generated
+141 -120
View File
@@ -3,6 +3,10 @@
[[package]]
name = "addressing"
version = "0.1.0"
dependencies = [
"log",
"pretty_env_logger",
]
[[package]]
name = "adler32"
@@ -70,9 +74,9 @@ checksum = "d7b8a9123b8027467bce0099fe556c628a53c8d83df0507084c31e9ba2e39aff"
[[package]]
name = "arrayref"
version = "0.3.5"
version = "0.3.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d382e583f07208808f6b1249e60848879ba3543f57c32277bf52d69c2f0f0ee"
checksum = "a4c527152e37cf757a3f78aae5a06fbeefdb07ccc535c980a3208ee3060dd544"
[[package]]
name = "arrayvec"
@@ -115,9 +119,9 @@ checksum = "f8aac770f1885fd7e387acedd76065302551364496e46b3dd00860b2f8359b9d"
[[package]]
name = "backtrace"
version = "0.3.40"
version = "0.3.42"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "924c76597f0d9ca25d762c25a4d369d51267536465dc5064bdf0eb073ed477ea"
checksum = "b4b1549d804b6c73f4817df2ba073709e96e426f12987127c48e6745568c350b"
dependencies = [
"backtrace-sys",
"cfg-if",
@@ -209,6 +213,12 @@ dependencies = [
"byte-tools",
]
[[package]]
name = "bs58"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b170cd256a3f9fa6b9edae3e44a7dfdfc77e8124dbc3e2612d75f9c3e2396dae"
[[package]]
name = "built"
version = "0.3.2"
@@ -246,9 +256,9 @@ dependencies = [
[[package]]
name = "bytes"
version = "0.5.3"
version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10004c15deb332055f7a4a208190aed362cf9a7c2f6ab70a305fba50e1105f38"
checksum = "130aac562c0dd69c56b3b1cc8ffd2e17be31d0b6c25b61c96b76231aa23e39e1"
[[package]]
name = "c2-chacha"
@@ -330,9 +340,9 @@ dependencies = [
[[package]]
name = "colored"
version = "1.9.1"
version = "1.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f930f8b286023ed451756fe2527d73484d667adf9e905e9932e81d52996a343a"
checksum = "8815e2ab78f3a59928fc32e141fbeece88320a240e43f47b2fd64ea3a88a5b3d"
dependencies = [
"atty",
"lazy_static",
@@ -341,9 +351,9 @@ dependencies = [
[[package]]
name = "constant_time_eq"
version = "0.1.4"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "995a44c877f9212528ccc74b21a232f66ad69001e40ede5bcee2ac9ef2657120"
checksum = "245097e9a4535ee1e3e3931fcfcd55a796a44c643e8596ff6566d68f09b87bbc"
[[package]]
name = "cookie"
@@ -456,10 +466,13 @@ dependencies = [
name = "crypto"
version = "0.1.0"
dependencies = [
"base64 0.11.0",
"bs58",
"curve25519-dalek",
"rand 0.7.2",
"log",
"pretty_env_logger",
"rand 0.7.3",
"rand_os",
"sphinx",
]
[[package]]
@@ -514,7 +527,9 @@ dependencies = [
name = "directory-client"
version = "0.1.0"
dependencies = [
"log",
"mockito",
"pretty_env_logger",
"reqwest",
"serde",
"topology",
@@ -582,19 +597,6 @@ dependencies = [
"termcolor",
]
[[package]]
name = "env_logger"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44533bbbb3bb3c1fa17d9f2e4e38bbbaf8396ba82193c4cb1b6445d711445d36"
dependencies = [
"atty",
"humantime",
"log",
"regex",
"termcolor",
]
[[package]]
name = "error-chain"
version = "0.12.1"
@@ -856,12 +858,15 @@ dependencies = [
"itertools",
"log",
"mix-client",
"pretty_env_logger",
"provider-client",
"rand 0.7.3",
"rand_os",
"serde",
"serde_derive",
"sfw-provider-requests",
"sphinx",
"tokio 0.2.9",
"tokio 0.2.10",
"topology",
]
@@ -874,12 +879,6 @@ dependencies = [
"libc",
]
[[package]]
name = "hex"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "023b39be39e3a2da62a94feb433e91e8bcd37676fbc8bea371daf52b7a769a3e"
[[package]]
name = "hkdf"
version = "0.8.0"
@@ -1005,11 +1004,11 @@ dependencies = [
[[package]]
name = "indexmap"
version = "1.3.0"
version = "1.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712d7b3ea5827fcb9d4fda14bf4da5f136f0db2ae9c8f4bd4e2d1c6fde4e6db2"
checksum = "0b54058f0a6ff80b6803da8faf8997cde53872b38f4023728f6830b06cd3c0dc"
dependencies = [
"autocfg 0.1.7",
"autocfg 1.0.0",
]
[[package]]
@@ -1041,15 +1040,15 @@ dependencies = [
[[package]]
name = "itoa"
version = "0.4.4"
version = "0.4.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "501266b7edd0174f8530248f87f99c88fbe60ca4ef3dd486835b8d8d53136f7f"
checksum = "b8b7a7c0c47db5545ed3fef7468ee7bb5b74691498139e4b3f6a20685dc6dd8e"
[[package]]
name = "jobserver"
version = "0.1.18"
version = "0.1.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "230ae9adf468173aecd4176c7233bddc84a15871a586c5971ace9a55f881c075"
checksum = "67b06c1b455f1cf4269a8cfc320ab930a810e2375a42af5075eb8a8b36405ce0"
dependencies = [
"libc",
]
@@ -1150,9 +1149,9 @@ checksum = "60302e4db3a61da70c0cb7991976248362f30319e88850c487b9b95bbf059e00"
[[package]]
name = "memchr"
version = "2.2.1"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88579771288728879b57485cc7d6b07d648c9f0141eb955f8ab7f9d45394468e"
checksum = "3197e20c7edb283f87c071ddfc7a2cca8f8e0b888c242959846a6fce03c72223"
[[package]]
name = "memoffset"
@@ -1256,9 +1255,14 @@ dependencies = [
name = "mix-client"
version = "0.1.0"
dependencies = [
"addressing",
"log",
"pretty_env_logger",
"rand 0.7.3",
"rand_distr",
"sphinx",
"tokio 0.2.9",
"tokio 0.2.10",
"topology",
]
[[package]]
@@ -1274,7 +1278,7 @@ dependencies = [
"lazy_static",
"log",
"percent-encoding 1.0.1",
"rand 0.7.2",
"rand 0.7.3",
"regex",
"serde_json",
]
@@ -1329,9 +1333,9 @@ dependencies = [
[[package]]
name = "num_cpus"
version = "1.11.1"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76dac5ed2a876980778b8b85f75a71b6cbf0db0b1232ee12f826bccb00d09d72"
checksum = "46203554f085ff89c235cd12f7075f3233af9b11ed7c9e16dfe2560d03313ce6"
dependencies = [
"hermit-abi",
"libc",
@@ -1339,32 +1343,31 @@ dependencies = [
[[package]]
name = "nym-client"
version = "0.3.3"
version = "0.4.0-rc.1"
dependencies = [
"addressing",
"base64 0.11.0",
"bs58",
"built",
"clap",
"crypto",
"curve25519-dalek",
"directory-client",
"dirs",
"env_logger 0.7.1",
"dotenv",
"futures 0.3.1",
"healthcheck",
"hex",
"log",
"mix-client",
"pem",
"pemstore",
"pretty_env_logger",
"provider-client",
"rand 0.7.2",
"rand_distr",
"reqwest",
"serde",
"serde_json",
"sfw-provider-requests",
"sphinx",
"tokio 0.2.9",
"tokio 0.2.10",
"tokio-tungstenite",
"topology",
"tungstenite",
@@ -1372,49 +1375,53 @@ dependencies = [
[[package]]
name = "nym-mixnode"
version = "0.3.3"
version = "0.4.0-rc.1"
dependencies = [
"addressing",
"base64 0.11.0",
"bs58",
"built",
"clap",
"curve25519-dalek",
"directory-client",
"dotenv",
"futures 0.3.1",
"log",
"pretty_env_logger",
"sphinx",
"tokio 0.2.9",
"tokio 0.2.10",
]
[[package]]
name = "nym-sfw-provider"
version = "0.3.3"
version = "0.4.0-rc.1"
dependencies = [
"base64 0.11.0",
"bs58",
"built",
"clap",
"crypto",
"curve25519-dalek",
"directory-client",
"dotenv",
"futures 0.3.1",
"hex",
"hmac",
"log",
"rand 0.7.2",
"pretty_env_logger",
"rand 0.7.3",
"serde",
"serde_json",
"sfw-provider-requests",
"sha2",
"sphinx",
"tokio 0.2.9",
"tokio 0.2.10",
]
[[package]]
name = "nym-validator"
version = "0.1.0"
version = "0.4.0-rc.1"
dependencies = [
"built",
"clap",
"crypto",
"dotenv",
"futures 0.3.1",
"healthcheck",
@@ -1422,7 +1429,7 @@ dependencies = [
"pretty_env_logger",
"serde",
"serde_derive",
"tokio 0.2.9",
"tokio 0.2.10",
"toml",
]
@@ -1502,6 +1509,16 @@ dependencies = [
"regex",
]
[[package]]
name = "pemstore"
version = "0.1.0"
dependencies = [
"crypto",
"log",
"pem",
"pretty_env_logger",
]
[[package]]
name = "percent-encoding"
version = "1.0.1"
@@ -1516,18 +1533,18 @@ checksum = "d4fd5641d01c8f18a23da7b6fe29298ff4b55afcccdf78973b24cf3175fee32e"
[[package]]
name = "pin-project"
version = "0.4.6"
version = "0.4.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94b90146c7216e4cb534069fb91366de4ea0ea353105ee45ed297e2d1619e469"
checksum = "75fca1c4ff21f60ca2d37b80d72b63dab823a9d19d3cda3a81d18bc03f0ba8c5"
dependencies = [
"pin-project-internal",
]
[[package]]
name = "pin-project-internal"
version = "0.4.6"
version = "0.4.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44ca92f893f0656d3cba8158dd0f2b99b94de256a4a54e870bd6922fcc6c8355"
checksum = "6544cd4e4ecace61075a6ec78074beeef98d58aa9a3d07d053d993b2946a90d6"
dependencies = [
"proc-macro2",
"quote",
@@ -1536,9 +1553,9 @@ dependencies = [
[[package]]
name = "pin-project-lite"
version = "0.1.2"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8822eb8bb72452f038ebf6048efa02c3fe22bf83f76519c9583e47fc194a422"
checksum = "237844750cfbb86f67afe27eee600dfbbcb6188d734139b534cbfbf4f96792ae"
[[package]]
name = "pin-utils"
@@ -1565,7 +1582,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "717ee476b1690853d222af4634056d830b5197ffd747726a9a1eee6da9f49074"
dependencies = [
"chrono",
"env_logger 0.6.2",
"env_logger",
"log",
]
@@ -1588,9 +1605,9 @@ checksum = "369a6ed065f249a159e06c45752c780bda2fb53c995718f9e484d08daa9eb42e"
[[package]]
name = "proc-macro2"
version = "1.0.7"
version = "1.0.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0319972dcae462681daf4da1adeeaa066e3ebd29c69be96c6abb1259d2ee2bcc"
checksum = "3acb317c6ff86a4e579dfa00fc5e6cca91ecbb4e7eb2df0468805b674eb88548"
dependencies = [
"unicode-xid",
]
@@ -1601,9 +1618,10 @@ version = "0.1.0"
dependencies = [
"futures 0.3.1",
"log",
"pretty_env_logger",
"sfw-provider-requests",
"sphinx",
"tokio 0.2.9",
"tokio 0.2.10",
]
[[package]]
@@ -1655,9 +1673,9 @@ dependencies = [
[[package]]
name = "rand"
version = "0.7.2"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3ae1b169243eaf61759b8475a998f0a385e42042370f3a7dbaf35246eacc8412"
checksum = "6a6b1679d49b24bbfe0c803429aa1874472f50d9b363131f0e89fc356b544d03"
dependencies = [
"getrandom",
"libc",
@@ -1716,7 +1734,7 @@ version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96977acbdd3a6576fb1d27391900035bf3863d4a16422973a409b488cf29ffb2"
dependencies = [
"rand 0.7.2",
"rand 0.7.3",
]
[[package]]
@@ -1807,12 +1825,11 @@ checksum = "2439c63f3f6139d1b57529d16bc3b8bb855230c8efcc5d3a896c8bea7c3b1e84"
[[package]]
name = "redox_users"
version = "0.3.1"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ecedbca3bf205f8d8f5c2b44d83cd0690e39ee84b951ed649e9f1841132b66d"
checksum = "09b23093265f8d200fa7b4c2c76297f47e681c655f6f1285a8780d6a022f7431"
dependencies = [
"failure",
"rand_os",
"getrandom",
"redox_syscall",
"rust-argon2",
]
@@ -1880,13 +1897,14 @@ dependencies = [
[[package]]
name = "rust-argon2"
version = "0.5.1"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ca4eaef519b494d1f2848fc602d18816fed808a981aedf4f1f00ceb7c9d32cf"
checksum = "2bc8af4bda8e1ff4932523b94d3dd20ee30a87232323eda55903ffd71d2fb017"
dependencies = [
"base64 0.10.1",
"base64 0.11.0",
"blake2b_simd",
"crossbeam-utils 0.6.6",
"constant_time_eq",
"crossbeam-utils 0.7.0",
]
[[package]]
@@ -1984,9 +2002,9 @@ dependencies = [
[[package]]
name = "serde_json"
version = "1.0.44"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "48c575e0cc52bdd09b47f330f646cf59afc586e9c4e3ccd6fc1f625b8ea1dad7"
checksum = "eab8f15f15d6c41a154c1b128a22f2dfabe350ef53c40953d84e36155c91192b"
dependencies = [
"itoa",
"ryu",
@@ -2082,19 +2100,20 @@ dependencies = [
[[package]]
name = "sphinx"
version = "0.1.0"
source = "git+https://github.com/nymtech/sphinx?rev=1d8cefcb6a0cb8e87d00d89eb1ccf2839e92aa1f#1d8cefcb6a0cb8e87d00d89eb1ccf2839e92aa1f"
source = "git+https://github.com/nymtech/sphinx?rev=8424f4b0933b4a7f64ae828b2edefc5ba43a9e79#8424f4b0933b4a7f64ae828b2edefc5ba43a9e79"
dependencies = [
"aes-ctr",
"arrayref",
"base64 0.11.0",
"blake2",
"bs58",
"byteorder",
"chacha",
"curve25519-dalek",
"hkdf",
"hmac",
"lioness",
"rand 0.7.2",
"log",
"rand 0.7.3",
"rand_distr",
"rand_os",
"sha2",
@@ -2138,9 +2157,9 @@ checksum = "7c65d530b10ccaeac294f349038a597e435b18fb456aadd0840a623f83b9e941"
[[package]]
name = "syn"
version = "1.0.13"
version = "1.0.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e4ff033220a41d1a57d8125eab57bf5263783dfdcc18688b1dacc6ce9651ef8"
checksum = "af6f3550d8dff9ef7dc34d384ac6f107e5d31c8f57d9f28e0081503f547ac8f5"
dependencies = [
"proc-macro2",
"quote",
@@ -2167,7 +2186,7 @@ checksum = "7a6e24d9338a0a5be79593e2fa15a648add6138caa803e2d5bc782c371732ca9"
dependencies = [
"cfg-if",
"libc",
"rand 0.7.2",
"rand 0.7.3",
"redox_syscall",
"remove_dir_all",
"winapi 0.3.8",
@@ -2175,11 +2194,11 @@ dependencies = [
[[package]]
name = "termcolor"
version = "1.0.5"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96d6098003bde162e4277c70665bd87c326f5a0c3f3fbfb285787fa482d54e6e"
checksum = "bb6bfa289a4d7c5766392812c0a1f4c1ba45afa1ad47803c11e1f407d846d75f"
dependencies = [
"wincolor",
"winapi-util",
]
[[package]]
@@ -2193,9 +2212,9 @@ dependencies = [
[[package]]
name = "thread_local"
version = "1.0.0"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88ddf1ad580c7e3d1efff877d972bcc93f995556b9087a5a259630985c88ceab"
checksum = "d40c6d1b69745a6ec6fb1ca717914848da4b44ae29d9b3080cbee91d72a69b14"
dependencies = [
"lazy_static",
]
@@ -2232,11 +2251,11 @@ dependencies = [
[[package]]
name = "tokio"
version = "0.2.9"
version = "0.2.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ffa2fdcfa937b20cb3c822a635ceecd5fc1a27a6a474527e5516aa24b8c8820a"
checksum = "c1fc73332507b971a5010664991a441b5ee0de92017f5a0e8b00fd684573045b"
dependencies = [
"bytes 0.5.3",
"bytes 0.5.4",
"fnv",
"futures-core",
"iovec",
@@ -2386,15 +2405,15 @@ dependencies = [
"futures 0.3.1",
"log",
"pin-project",
"tokio 0.2.9",
"tokio 0.2.10",
"tungstenite",
]
[[package]]
name = "toml"
version = "0.5.5"
version = "0.5.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "01d1404644c8b12b16bfcffa4322403a91a451584daaaa7c28d3152e6cbc98cf"
checksum = "ffc92d160b1eef40665be3a05630d003936a3bc7da7421277846c2613e92c71a"
dependencies = [
"serde",
]
@@ -2404,13 +2423,14 @@ name = "topology"
version = "0.1.0"
dependencies = [
"addressing",
"base64 0.11.0",
"curve25519-dalek",
"bs58",
"itertools",
"log",
"rand 0.7.2",
"pretty_env_logger",
"rand 0.7.3",
"serde",
"sphinx",
"version-checker",
]
[[package]]
@@ -2442,7 +2462,7 @@ dependencies = [
"input_buffer",
"log",
"native-tls",
"rand 0.7.2",
"rand 0.7.3",
"sha-1",
"url 2.1.1",
"utf-8",
@@ -2474,9 +2494,9 @@ dependencies = [
[[package]]
name = "unicode-normalization"
version = "0.1.11"
version = "0.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b561e267b2326bb4cebfc0ef9e68355c7abe6c6f522aeac2f5bf95d56c59bdcf"
checksum = "5479532badd04e128284890390c1e876ef7a993d0570b3597ae43dfa1d59afa4"
dependencies = [
"smallvec 1.1.0",
]
@@ -2533,6 +2553,10 @@ dependencies = [
[[package]]
name = "validator-client"
version = "0.1.0"
dependencies = [
"log",
"pretty_env_logger",
]
[[package]]
name = "vcpkg"
@@ -2546,6 +2570,13 @@ version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05c78687fb1a80548ae3250346c3db86a80a7cdd77bda190189f2d0a0987c81a"
[[package]]
name = "version-checker"
version = "0.1.0"
dependencies = [
"semver",
]
[[package]]
name = "version_check"
version = "0.1.5"
@@ -2605,9 +2636,9 @@ checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[[package]]
name = "winapi-util"
version = "0.1.2"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7168bab6e1daee33b4557efd0e95d5ca70a03706d39fa5f3fe7a236f584b03c9"
checksum = "4ccfbf554c6ad11084fb7517daca16cfdcaccbdadba4fc336f032a8b12c2ad80"
dependencies = [
"winapi 0.3.8",
]
@@ -2618,16 +2649,6 @@ version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "wincolor"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96f5016b18804d24db43cebf3c77269e7569b8954a8464501c216cc5e070eaa9"
dependencies = [
"winapi 0.3.8",
"winapi-util",
]
[[package]]
name = "winreg"
version = "0.6.2"
Cargo.toml
+4
View File
@@ -1,3 +1,6 @@
[profile.release]
panic = "abort"
[workspace]
members = [
@@ -8,6 +11,7 @@ members = [
"common/addressing",
"common/crypto",
"common/healthcheck",
"common/pemstore",
"common/topology",
"mixnode",
"nym-client",
README.md
+2 -2
View File
@@ -4,8 +4,8 @@ This repository contains the full Nym platform, written in Rust.
The platform is composed of multiple Rust crates. Top-level executable binary crates include:
* client - an executable which you can build into your own applications. Use it for interacting with Nym nodes.
* mixnode - the mixnode crate.
* mixnode - shuffles [Sphinx](https://github.com/nymtech/sphinx) packets together to provide privacy against network-level attackers.
* nym-client - an executable which you can build into your own applications. Use it for interacting with Nym nodes.
* sfw-provider - a store-and-forward service provider. The provider acts sort of like a mailbox for mixnet messages.
* validator - currently just starting development. Handles consensus ordering of transactions, mixmining, and coconut credential generation and validation.
common/addressing/Cargo.toml
+2
View File
@@ -7,3 +7,5 @@ edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
log = "0.4"
pretty_env_logger = "0.3"
common/addressing/src/lib.rs
+3 -3
View File
@@ -62,8 +62,8 @@ pub fn encoded_bytes_from_socket_address(address: SocketAddr) -> [u8; 32] {
address_bytes
}
pub fn socket_address_from_encoded_bytes(b: [u8; 32]) -> SocketAddr {
let address_type: AddressType = b[0].try_into().unwrap();
pub fn socket_address_from_encoded_bytes(b: [u8; 32]) -> Result<SocketAddr, AddressTypeError> {
let address_type: AddressType = b[0].try_into()?;
let port: u16 = u16::from_be_bytes([b[1], b[2]]);
@@ -76,5 +76,5 @@ pub fn socket_address_from_encoded_bytes(b: [u8; 32]) -> SocketAddr {
}
};
SocketAddr::new(ip, port)
Ok(SocketAddr::new(ip, port))
}
common/clients/directory-client/Cargo.toml
+2
View File
@@ -7,6 +7,8 @@ edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
log = "0.4"
pretty_env_logger = "0.3"
reqwest = "0.9.22"
serde = { version = "1.0.104", features = ["derive"] }
common/clients/directory-client/src/presence.rs
-244
View File
@@ -1,244 +0,0 @@
use crate::requests::presence_topology_get::PresenceTopologyGetRequester;
use crate::{Client, Config, DirectoryClient};
use serde::{Deserialize, Serialize};
use std::convert::TryInto;
use std::io;
use std::net::ToSocketAddrs;
use topology::{CocoNode, MixNode, MixProviderNode, NymTopology};
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct CocoPresence {
pub host: String,
pub pub_key: String,
pub last_seen: u64,
pub version: String,
}
impl Into<topology::CocoNode> for CocoPresence {
fn into(self) -> topology::CocoNode {
topology::CocoNode {
host: self.host,
pub_key: self.pub_key,
last_seen: self.last_seen,
version: self.version,
}
}
}
impl From<topology::CocoNode> for CocoPresence {
fn from(cn: CocoNode) -> Self {
CocoPresence {
host: cn.host,
pub_key: cn.pub_key,
last_seen: cn.last_seen,
version: cn.version,
}
}
}
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct MixNodePresence {
pub host: String,
pub pub_key: String,
pub layer: u64,
pub last_seen: u64,
pub version: String,
}
impl TryInto<topology::MixNode> for MixNodePresence {
type Error = io::Error;
fn try_into(self) -> Result<MixNode, Self::Error> {
let resolved_hostname = self.host.to_socket_addrs()?.next();
if resolved_hostname.is_none() {
return Err(io::Error::new(
io::ErrorKind::Other,
"no valid socket address",
));
}
Ok(topology::MixNode {
host: resolved_hostname.unwrap(),
pub_key: self.pub_key,
layer: self.layer,
last_seen: self.last_seen,
version: self.version,
})
}
}
impl From<topology::MixNode> for MixNodePresence {
fn from(mn: MixNode) -> Self {
MixNodePresence {
host: mn.host.to_string(),
pub_key: mn.pub_key,
layer: mn.layer,
last_seen: mn.last_seen,
version: mn.version,
}
}
}
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct MixProviderPresence {
pub client_listener: String,
pub mixnet_listener: String,
pub pub_key: String,
pub registered_clients: Vec<MixProviderClient>,
pub last_seen: u64,
pub version: String,
}
impl Into<topology::MixProviderNode> for MixProviderPresence {
fn into(self) -> topology::MixProviderNode {
topology::MixProviderNode {
client_listener: self.client_listener.parse().unwrap(),
mixnet_listener: self.mixnet_listener.parse().unwrap(),
pub_key: self.pub_key,
registered_clients: self
.registered_clients
.into_iter()
.map(|c| c.into())
.collect(),
last_seen: self.last_seen,
version: self.version,
}
}
}
impl From<topology::MixProviderNode> for MixProviderPresence {
fn from(mpn: MixProviderNode) -> Self {
MixProviderPresence {
client_listener: mpn.client_listener.to_string(),
mixnet_listener: mpn.mixnet_listener.to_string(),
pub_key: mpn.pub_key,
registered_clients: mpn
.registered_clients
.into_iter()
.map(|c| c.into())
.collect(),
last_seen: mpn.last_seen,
version: mpn.version,
}
}
}
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct MixProviderClient {
pub pub_key: String,
}
impl Into<topology::MixProviderClient> for MixProviderClient {
fn into(self) -> topology::MixProviderClient {
topology::MixProviderClient {
pub_key: self.pub_key,
}
}
}
impl From<topology::MixProviderClient> for MixProviderClient {
fn from(mpc: topology::MixProviderClient) -> Self {
MixProviderClient {
pub_key: mpc.pub_key,
}
}
}
// Topology shows us the current state of the overall Nym network
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct Topology {
pub coco_nodes: Vec<CocoPresence>,
pub mix_nodes: Vec<MixNodePresence>,
pub mix_provider_nodes: Vec<MixProviderPresence>,
}
impl NymTopology for Topology {
fn new(directory_server: String) -> Self {
println!("Using directory server: {:?}", directory_server);
let directory_config = Config {
base_url: directory_server,
};
let directory = Client::new(directory_config);
let topology = directory
.presence_topology
.get()
.expect("Failed to retrieve network topology.");
topology
}
fn new_from_nodes(
mix_nodes: Vec<MixNode>,
mix_provider_nodes: Vec<MixProviderNode>,
coco_nodes: Vec<CocoNode>,
) -> Self {
Topology {
coco_nodes: coco_nodes.into_iter().map(|node| node.into()).collect(),
mix_nodes: mix_nodes.into_iter().map(|node| node.into()).collect(),
mix_provider_nodes: mix_provider_nodes
.into_iter()
.map(|node| node.into())
.collect(),
}
}
fn get_mix_nodes(&self) -> Vec<topology::MixNode> {
self.mix_nodes
.iter()
.filter_map(|x| x.clone().try_into().ok())
.collect()
}
fn get_mix_provider_nodes(&self) -> Vec<topology::MixProviderNode> {
self.mix_provider_nodes
.iter()
.map(|x| x.clone().into())
.collect()
}
fn get_coco_nodes(&self) -> Vec<topology::CocoNode> {
self.coco_nodes.iter().map(|x| x.clone().into()).collect()
}
}
#[cfg(test)]
mod converting_mixnode_presence_into_topology_mixnode {
use super::*;
#[test]
fn it_returns_error_on_unresolvable_hostname() {
let unresolvable_hostname = "foomp.foomp.foomp:1234";
let mix_presence = MixNodePresence {
host: unresolvable_hostname.to_string(),
pub_key: "".to_string(),
layer: 0,
last_seen: 0,
version: "".to_string(),
};
let result: Result<topology::MixNode, io::Error> = mix_presence.try_into();
assert!(result.is_err())
}
#[test]
fn it_returns_resolved_ip_on_resolvable_hostname() {
let resolvable_hostname = "nymtech.net:1234";
let mix_presence = MixNodePresence {
host: resolvable_hostname.to_string(),
pub_key: "".to_string(),
layer: 0,
last_seen: 0,
version: "".to_string(),
};
let result: Result<topology::MixNode, io::Error> = mix_presence.try_into();
assert!(result.is_ok())
}
}
common/clients/directory-client/src/presence/coconodes.rs
+33
View File
@@ -0,0 +1,33 @@
use serde::{Deserialize, Serialize};
use topology::coco;
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct CocoPresence {
pub host: String,
pub pub_key: String,
pub last_seen: u64,
pub version: String,
}
impl Into<topology::coco::Node> for CocoPresence {
fn into(self) -> topology::coco::Node {
topology::coco::Node {
host: self.host,
pub_key: self.pub_key,
last_seen: self.last_seen,
version: self.version,
}
}
}
impl From<topology::coco::Node> for CocoPresence {
fn from(cn: coco::Node) -> Self {
CocoPresence {
host: cn.host,
pub_key: cn.pub_key,
last_seen: cn.last_seen,
version: cn.version,
}
}
}
common/clients/directory-client/src/presence/mixnodes.rs
+49
View File
@@ -0,0 +1,49 @@
use serde::{Deserialize, Serialize};
use std::convert::TryInto;
use std::io;
use std::net::ToSocketAddrs;
use topology::mix;
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct MixNodePresence {
pub host: String,
pub pub_key: String,
pub layer: u64,
pub last_seen: u64,
pub version: String,
}
impl TryInto<topology::mix::Node> for MixNodePresence {
type Error = io::Error;
fn try_into(self) -> Result<topology::mix::Node, Self::Error> {
let resolved_hostname = self.host.to_socket_addrs()?.next();
if resolved_hostname.is_none() {
return Err(io::Error::new(
io::ErrorKind::Other,
"no valid socket address",
));
}
Ok(topology::mix::Node {
host: resolved_hostname.unwrap(),
pub_key: self.pub_key,
layer: self.layer,
last_seen: self.last_seen,
version: self.version,
})
}
}
impl From<topology::mix::Node> for MixNodePresence {
fn from(mn: mix::Node) -> Self {
MixNodePresence {
host: mn.host.to_string(),
pub_key: mn.pub_key,
layer: mn.layer,
last_seen: mn.last_seen,
version: mn.version,
}
}
}
common/clients/directory-client/src/presence/mod.rs
+108
View File
@@ -0,0 +1,108 @@
use crate::requests::presence_topology_get::PresenceTopologyGetRequester;
use crate::{Client, Config, DirectoryClient};
use log::*;
use serde::{Deserialize, Serialize};
use std::convert::TryInto;
use topology::coco;
use topology::mix;
use topology::provider;
use topology::NymTopology;
pub mod coconodes;
pub mod mixnodes;
pub mod providers;
// Topology shows us the current state of the overall Nym network
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct Topology {
pub coco_nodes: Vec<coconodes::CocoPresence>,
pub mix_nodes: Vec<mixnodes::MixNodePresence>,
pub mix_provider_nodes: Vec<providers::MixProviderPresence>,
}
impl NymTopology for Topology {
fn new(directory_server: String) -> Self {
debug!("Using directory server: {:?}", directory_server);
let directory_config = Config {
base_url: directory_server,
};
let directory = Client::new(directory_config);
let topology = directory
.presence_topology
.get()
.expect("Failed to retrieve network topology.");
topology
}
fn new_from_nodes(
mix_nodes: Vec<mix::Node>,
mix_provider_nodes: Vec<provider::Node>,
coco_nodes: Vec<coco::Node>,
) -> Self {
Topology {
coco_nodes: coco_nodes.into_iter().map(|node| node.into()).collect(),
mix_nodes: mix_nodes.into_iter().map(|node| node.into()).collect(),
mix_provider_nodes: mix_provider_nodes
.into_iter()
.map(|node| node.into())
.collect(),
}
}
fn mix_nodes(&self) -> Vec<mix::Node> {
self.mix_nodes
.iter()
.filter_map(|x| x.clone().try_into().ok())
.collect()
}
fn providers(&self) -> Vec<provider::Node> {
self.mix_provider_nodes
.iter()
.map(|x| x.clone().into())
.collect()
}
fn coco_nodes(&self) -> Vec<topology::coco::Node> {
self.coco_nodes.iter().map(|x| x.clone().into()).collect()
}
}
#[cfg(test)]
mod converting_mixnode_presence_into_topology_mixnode {
use super::*;
#[test]
fn it_returns_error_on_unresolvable_hostname() {
let unresolvable_hostname = "foomp.foomp.foomp:1234";
let mix_presence = mixnodes::MixNodePresence {
host: unresolvable_hostname.to_string(),
pub_key: "".to_string(),
layer: 0,
last_seen: 0,
version: "".to_string(),
};
let result: Result<mix::Node, std::io::Error> = mix_presence.try_into();
assert!(result.is_err())
}
#[test]
fn it_returns_resolved_ip_on_resolvable_hostname() {
let resolvable_hostname = "nymtech.net:1234";
let mix_presence = mixnodes::MixNodePresence {
host: resolvable_hostname.to_string(),
pub_key: "".to_string(),
layer: 0,
last_seen: 0,
version: "".to_string(),
};
let result: Result<topology::mix::Node, std::io::Error> = mix_presence.try_into();
assert!(result.is_ok())
}
}
common/clients/directory-client/src/presence/providers.rs
+69
View File
@@ -0,0 +1,69 @@
use serde::{Deserialize, Serialize};
use topology::provider;
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct MixProviderPresence {
pub client_listener: String,
pub mixnet_listener: String,
pub pub_key: String,
pub registered_clients: Vec<MixProviderClient>,
pub last_seen: u64,
pub version: String,
}
impl Into<topology::provider::Node> for MixProviderPresence {
fn into(self) -> topology::provider::Node {
topology::provider::Node {
client_listener: self.client_listener.parse().unwrap(),
mixnet_listener: self.mixnet_listener.parse().unwrap(),
pub_key: self.pub_key,
registered_clients: self
.registered_clients
.into_iter()
.map(|c| c.into())
.collect(),
last_seen: self.last_seen,
version: self.version,
}
}
}
impl From<topology::provider::Node> for MixProviderPresence {
fn from(mpn: provider::Node) -> Self {
MixProviderPresence {
client_listener: mpn.client_listener.to_string(),
mixnet_listener: mpn.mixnet_listener.to_string(),
pub_key: mpn.pub_key,
registered_clients: mpn
.registered_clients
.into_iter()
.map(|c| c.into())
.collect(),
last_seen: mpn.last_seen,
version: mpn.version,
}
}
}
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct MixProviderClient {
pub pub_key: String,
}
impl Into<topology::provider::Client> for MixProviderClient {
fn into(self) -> topology::provider::Client {
topology::provider::Client {
pub_key: self.pub_key,
}
}
}
impl From<topology::provider::Client> for MixProviderClient {
fn from(mpc: topology::provider::Client) -> Self {
MixProviderClient {
pub_key: mpc.pub_key,
}
}
}
common/clients/directory-client/src/requests/presence_coconodes_post.rs
+2 -2
View File
@@ -1,4 +1,4 @@
use crate::presence::CocoPresence;
use crate::presence::coconodes::CocoPresence;
use reqwest::Response;
pub struct Request {
@@ -73,7 +73,7 @@ mod metrics_get_request {
#[cfg(test)]
mod fixtures {
use crate::presence::CocoPresence;
use crate::presence::coconodes::CocoPresence;
pub fn new_presence() -> CocoPresence {
CocoPresence {
common/clients/directory-client/src/requests/presence_mixnodes_post.rs
+2 -2
View File
@@ -1,4 +1,4 @@
use crate::presence::MixNodePresence;
use crate::presence::mixnodes::MixNodePresence;
use reqwest::Response;
pub struct Request {
@@ -73,7 +73,7 @@ mod metrics_get_request {
#[cfg(test)]
mod fixtures {
use crate::presence::MixNodePresence;
use crate::presence::mixnodes::MixNodePresence;
pub fn new_presence() -> MixNodePresence {
MixNodePresence {
common/clients/directory-client/src/requests/presence_providers_post.rs
+2 -2
View File
@@ -1,4 +1,4 @@
use crate::presence::MixProviderPresence;
use crate::presence::providers::MixProviderPresence;
use reqwest::Response;
pub struct Request {
@@ -72,7 +72,7 @@ mod metrics_get_request {
}
#[cfg(test)]
mod fixtures {
use crate::presence::MixProviderPresence;
use crate::presence::providers::MixProviderPresence;
pub fn new_presence() -> MixProviderPresence {
MixProviderPresence {
common/clients/mix-client/Cargo.toml
+10 -1
View File
@@ -8,7 +8,16 @@ edition = "2018"
[dependencies]
log = "0.4.8"
pretty_env_logger = "0.3"
rand = "0.7.2"
rand_distr = "0.2.2"
tokio = { version = "0.2", features = ["full"] }
## internal
addressing = {path = "../../addressing"}
topology = {path = "../../topology"}
## will be moved to proper dependencies once released
sphinx = { git = "https://github.com/nymtech/sphinx", rev="1d8cefcb6a0cb8e87d00d89eb1ccf2839e92aa1f" }
sphinx = { git = "https://github.com/nymtech/sphinx", rev="8424f4b0933b4a7f64ae828b2edefc5ba43a9e79" }
# sphinx = { path = "../../../../sphinx"}
common/clients/mix-client/src/lib.rs
+4 -2
View File
@@ -3,6 +3,9 @@ use sphinx::SphinxPacket;
use std::net::SocketAddr;
use tokio::prelude::*;
pub mod packet;
pub mod poisson;
pub struct MixClient {}
impl MixClient {
@@ -17,8 +20,7 @@ impl MixClient {
mix_addr: SocketAddr,
) -> Result<(), Box<dyn std::error::Error>> {
let bytes = packet.to_bytes();
info!("socket addr: {:?}", mix_addr);
debug!("Sending to the following address: {:?}", mix_addr);
let mut stream = tokio::net::TcpStream::connect(mix_addr).await?;
stream.write_all(&bytes[..]).await?;
common/clients/mix-client/src/packet.rs
+82
View File
@@ -0,0 +1,82 @@
use addressing;
use addressing::AddressTypeError;
use sphinx::route::{Destination, DestinationAddressBytes, SURBIdentifier};
use sphinx::SphinxPacket;
use std::net::SocketAddr;
use topology::{NymTopology, NymTopologyError};
pub const LOOP_COVER_MESSAGE_PAYLOAD: &[u8] = b"The cake is a lie!";
pub const LOOP_COVER_MESSAGE_AVERAGE_DELAY: f64 = 2.0;
#[derive(Debug)]
pub enum SphinxPacketEncapsulationError {
NoValidProvidersError,
InvalidTopologyError,
SphinxEncapsulationError(sphinx::header::SphinxUnwrapError),
InvalidFirstMixAddress,
}
impl From<topology::NymTopologyError> for SphinxPacketEncapsulationError {
fn from(_: NymTopologyError) -> Self {
use SphinxPacketEncapsulationError::*;
InvalidTopologyError
}
}
// it is correct error we're converting from, it just has an unfortunate name
// related issue: https://github.com/nymtech/sphinx/issues/40
impl From<sphinx::header::SphinxUnwrapError> for SphinxPacketEncapsulationError {
fn from(err: sphinx::header::SphinxUnwrapError) -> Self {
use SphinxPacketEncapsulationError::*;
SphinxEncapsulationError(err)
}
}
impl From<AddressTypeError> for SphinxPacketEncapsulationError {
fn from(_: AddressTypeError) -> Self {
use SphinxPacketEncapsulationError::*;
InvalidFirstMixAddress
}
}
pub fn loop_cover_message<T: NymTopology>(
our_address: DestinationAddressBytes,
surb_id: SURBIdentifier,
topology: &T,
) -> Result<(SocketAddr, SphinxPacket), SphinxPacketEncapsulationError> {
let destination = Destination::new(our_address, surb_id);
encapsulate_message(
destination,
LOOP_COVER_MESSAGE_PAYLOAD.to_vec(),
topology,
LOOP_COVER_MESSAGE_AVERAGE_DELAY,
)
}
pub fn encapsulate_message<T: NymTopology>(
recipient: Destination,
message: Vec<u8>,
topology: &T,
average_delay: f64,
) -> Result<(SocketAddr, SphinxPacket), SphinxPacketEncapsulationError> {
let mut providers = topology.providers();
if providers.len() == 0 {
return Err(SphinxPacketEncapsulationError::NoValidProvidersError);
}
// unwrap is fine here as we asserted there is at least single provider
let provider = providers.pop().unwrap().into();
let route = topology.route_to(provider)?;
let delays = sphinx::header::delays::generate(route.len(), average_delay);
// build the packet
let packet = sphinx::SphinxPacket::new(message, &route[..], &recipient, &delays)?;
// we know the mix route must be valid otherwise we would have already returned an error
let first_node_address =
addressing::socket_address_from_encoded_bytes(route.first().unwrap().address.to_bytes())?;
Ok((first_node_address, packet))
}
common/clients/mix-client/src/poisson.rs
+9
View File
@@ -0,0 +1,9 @@
use rand_distr::{Distribution, Exp};
pub fn sample(average_delay: f64) -> f64 {
// this is our internal code used by our traffic streams
// the error is only thrown if average delay is less than 0, which will never happen
// so call to unwrap is perfectly safe here
let exp = Exp::new(1.0 / average_delay).unwrap();
exp.sample(&mut rand::thread_rng())
}
common/clients/provider-client/Cargo.toml
+2 -1
View File
@@ -9,10 +9,11 @@ edition = "2018"
[dependencies]
futures = "0.3.1"
log = "0.4.8"
pretty_env_logger = "0.3"
tokio = { version = "0.2", features = ["full"] }
## internal
sfw-provider-requests = { path = "../../../sfw-provider/sfw-provider-requests" }
## will be moved to proper dependencies once released
sphinx = { git = "https://github.com/nymtech/sphinx", rev="1d8cefcb6a0cb8e87d00d89eb1ccf2839e92aa1f" }
sphinx = { git = "https://github.com/nymtech/sphinx", rev="8424f4b0933b4a7f64ae828b2edefc5ba43a9e79" }
common/clients/provider-client/src/lib.rs
+18 -14
View File
@@ -1,5 +1,5 @@
use futures::io::Error;
use log::info;
use log::*;
use sfw_provider_requests::requests::{ProviderRequest, PullRequest, RegisterRequest};
use sfw_provider_requests::responses::{
ProviderResponse, ProviderResponseError, PullResponse, RegisterResponse,
@@ -65,34 +65,34 @@ impl ProviderClient {
pub async fn send_request(&self, bytes: Vec<u8>) -> Result<Vec<u8>, ProviderClientError> {
let mut socket = tokio::net::TcpStream::connect(self.provider_network_address).await?;
info!("keep alive: {:?}", socket.keepalive());
socket.set_keepalive(Some(Duration::from_secs(2))).unwrap();
socket.set_keepalive(Some(Duration::from_secs(2)))?;
socket.write_all(&bytes[..]).await?;
if let Err(_e) = socket.shutdown(Shutdown::Write) {
// TODO: make it a silent log once we have a proper logging library
// eprintln!("failed to close write part of the socket; err = {:?}", e)
if let Err(e) = socket.shutdown(Shutdown::Write) {
warn!("failed to close write part of the socket; err = {:?}", e)
}
let mut response = Vec::new();
socket.read_to_end(&mut response).await?;
if let Err(_e) = socket.shutdown(Shutdown::Read) {
// TODO: make it a silent log once we have a proper logging library
// eprintln!("failed to close read part of the socket; err = {:?}", e)
if let Err(e) = socket.shutdown(Shutdown::Read) {
debug!("failed to close read part of the socket; err = {:?}. It was probably already closed by the provider", e)
}
Ok(response)
}
pub async fn retrieve_messages(&self) -> Result<Vec<Vec<u8>>, ProviderClientError> {
if self.auth_token.is_none() {
return Err(ProviderClientError::EmptyAuthTokenError);
}
let auth_token = match self.auth_token {
Some(token) => token,
None => {
return Err(ProviderClientError::EmptyAuthTokenError);
}
};
let pull_request = PullRequest::new(self.our_address, self.auth_token.unwrap());
let pull_request = PullRequest::new(self.our_address, auth_token);
let bytes = pull_request.to_bytes();
let response = self.send_request(bytes).await?;
info!("Received the following response: {:?}", response);
let parsed_response = PullResponse::from_bytes(&response)?;
Ok(parsed_response.messages)
@@ -111,4 +111,8 @@ impl ProviderClient {
Ok(parsed_response.auth_token)
}
pub fn is_registered(&self) -> bool {
self.auth_token.is_some()
}
}
common/clients/validator-client/Cargo.toml
+2
View File
@@ -7,3 +7,5 @@ edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
log = "0.4"
pretty_env_logger = "0.3"
common/crypto/Cargo.toml
+6 -1
View File
@@ -7,7 +7,12 @@ edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
base64 = "0.11.0"
bs58 = "0.3.0"
curve25519-dalek = "1.2.3"
log = "0.4"
pretty_env_logger = "0.3"
rand = "0.7.2"
rand_os = "0.1"
## will be moved to proper dependencies once released
sphinx = { git = "https://github.com/nymtech/sphinx", rev="8424f4b0933b4a7f64ae828b2edefc5ba43a9e79" }
common/crypto/src/encryption/x25519.rs
+1
View File
@@ -56,6 +56,7 @@ impl MixnetEncryptionPrivateKey for PrivateKey {
fn from_bytes(b: &[u8]) -> Self {
let mut bytes = [0; 32];
bytes.copy_from_slice(&b[..]);
// due to trait restriction we have no choice but to panic if this fails
let key = Scalar::from_canonical_bytes(bytes).unwrap();
Self(key)
}
common/crypto/src/identity/mod.rs
+22 -8
View File
@@ -2,9 +2,11 @@ use crate::encryption::{
MixnetEncryptionKeyPair, MixnetEncryptionPrivateKey, MixnetEncryptionPublicKey,
};
use crate::{encryption, PemStorable};
use bs58;
use curve25519_dalek::scalar::Scalar;
use sphinx::route::DestinationAddressBytes;
pub trait MixnetIdentityKeyPair<Priv, Pub>
pub trait MixnetIdentityKeyPair<Priv, Pub>: Clone
where
Priv: MixnetIdentityPrivateKey,
Pub: MixnetIdentityPublicKey,
@@ -18,16 +20,20 @@ where
}
pub trait MixnetIdentityPublicKey:
Sized + PemStorable + for<'a> From<&'a <Self as MixnetIdentityPublicKey>::PrivateKeyMaterial>
Sized
+ PemStorable
+ Clone
+ for<'a> From<&'a <Self as MixnetIdentityPublicKey>::PrivateKeyMaterial>
{
// we need to couple public and private keys together
type PrivateKeyMaterial: MixnetIdentityPrivateKey<PublicKeyMaterial = Self>;
fn derive_address(&self) -> DestinationAddressBytes;
fn to_bytes(&self) -> Vec<u8>;
fn from_bytes(b: &[u8]) -> Self;
}
pub trait MixnetIdentityPrivateKey: Sized + PemStorable {
pub trait MixnetIdentityPrivateKey: Sized + PemStorable + Clone {
// we need to couple public and private keys together
type PublicKeyMaterial: MixnetIdentityPublicKey<PrivateKeyMaterial = Self>;
@@ -44,7 +50,7 @@ pub trait MixnetIdentityPrivateKey: Sized + PemStorable {
// for time being define a dummy identity using x25519 encryption keys (as we've done so far)
// and replace it with proper keys, like ed25519 later on
#[derive(Clone)]
pub struct DummyMixIdentityKeyPair {
pub private_key: DummyMixIdentityPrivateKey,
pub public_key: DummyMixIdentityPublicKey,
@@ -83,6 +89,14 @@ pub struct DummyMixIdentityPublicKey(encryption::x25519::PublicKey);
impl MixnetIdentityPublicKey for DummyMixIdentityPublicKey {
type PrivateKeyMaterial = DummyMixIdentityPrivateKey;
fn derive_address(&self) -> DestinationAddressBytes {
let mut temporary_address = [0u8; 32];
let public_key_bytes = self.to_bytes();
temporary_address.copy_from_slice(&public_key_bytes[..]);
temporary_address
}
fn to_bytes(&self) -> Vec<u8> {
self.0.to_bytes()
}
@@ -99,13 +113,13 @@ impl PemStorable for DummyMixIdentityPublicKey {
}
impl DummyMixIdentityPublicKey {
pub fn to_b64_string(&self) -> String {
base64::encode_config(&self.to_bytes(), base64::URL_SAFE)
pub fn to_base58_string(&self) -> String {
bs58::encode(&self.to_bytes()).into_string()
}
#[allow(dead_code)]
fn from_b64_string(val: String) -> Self {
Self::from_bytes(&base64::decode_config(&val, base64::URL_SAFE).unwrap())
fn from_base58_string(val: String) -> Self {
Self::from_bytes(&bs58::decode(&val).into_vec().unwrap())
}
}
common/healthcheck/Cargo.toml
+5 -1
View File
@@ -10,6 +10,9 @@ edition = "2018"
futures = "0.3.1"
itertools = "0.8.2"
log = "0.4.8"
pretty_env_logger = "0.3"
rand_os = "0.1"
rand = "0.7.2"
serde = "1.0.104"
serde_derive = "1.0.104"
tokio = { version = "0.2", features = ["full"] }
@@ -24,6 +27,7 @@ sfw-provider-requests = { path = "../../sfw-provider/sfw-provider-requests" }
topology = {path = "../topology" }
## will be moved to proper dependencies once released
sphinx = { git = "https://github.com/nymtech/sphinx", rev="1d8cefcb6a0cb8e87d00d89eb1ccf2839e92aa1f" }
sphinx = { git = "https://github.com/nymtech/sphinx", rev="8424f4b0933b4a7f64ae828b2edefc5ba43a9e79" }
# sphinx = { path = "../../../sphinx"}
[dev-dependencies]
common/healthcheck/src/lib.rs
+25 -3
View File
@@ -1,8 +1,10 @@
use crate::result::HealthCheckResult;
use crypto::identity::{MixnetIdentityKeyPair, MixnetIdentityPrivateKey, MixnetIdentityPublicKey};
use directory_client::requests::presence_topology_get::PresenceTopologyGetRequester;
use directory_client::DirectoryClient;
use log::{debug, error, info, trace};
use std::fmt::{Error, Formatter};
use std::marker::PhantomData;
use std::time::Duration;
use topology::NymTopologyError;
@@ -34,15 +36,29 @@ impl From<topology::NymTopologyError> for HealthCheckerError {
}
}
pub struct HealthChecker {
pub struct HealthChecker<IDPair, Priv, Pub>
where
IDPair: MixnetIdentityKeyPair<Priv, Pub>,
Priv: MixnetIdentityPrivateKey,
Pub: MixnetIdentityPublicKey,
{
directory_client: directory_client::Client,
interval: Duration,
num_test_packets: usize,
resolution_timeout: Duration,
identity_keypair: IDPair,
_phantom_private: PhantomData<Priv>,
_phantom_public: PhantomData<Pub>,
}
impl HealthChecker {
pub fn new(config: config::HealthCheck) -> Self {
impl<IDPair, Priv, Pub> HealthChecker<IDPair, Priv, Pub>
where
IDPair: crypto::identity::MixnetIdentityKeyPair<Priv, Pub>,
Priv: crypto::identity::MixnetIdentityPrivateKey,
Pub: crypto::identity::MixnetIdentityPublicKey,
{
pub fn new(config: config::HealthCheck, identity_keypair: IDPair) -> Self {
debug!(
"healthcheck will be using the following directory server: {:?}",
config.directory_server
@@ -53,11 +69,16 @@ impl HealthChecker {
interval: Duration::from_secs_f64(config.interval),
resolution_timeout: Duration::from_secs_f64(config.resolution_timeout),
num_test_packets: config.num_test_packets,
identity_keypair,
_phantom_private: PhantomData,
_phantom_public: PhantomData,
}
}
pub async fn do_check(&self) -> Result<HealthCheckResult, HealthCheckerError> {
trace!("going to perform a healthcheck!");
let current_topology = match self.directory_client.presence_topology.get() {
Ok(topology) => topology,
Err(err) => {
@@ -71,6 +92,7 @@ impl HealthChecker {
current_topology,
self.num_test_packets,
self.resolution_timeout,
&self.identity_keypair,
)
.await;
healthcheck_result.sort_scores();
common/healthcheck/src/path_check.rs
+61 -27
View File
@@ -1,12 +1,12 @@
use crypto::identity::{DummyMixIdentityKeyPair, MixnetIdentityKeyPair, MixnetIdentityPublicKey};
use crypto::identity::{MixnetIdentityKeyPair, MixnetIdentityPrivateKey, MixnetIdentityPublicKey};
use itertools::Itertools;
use log::{debug, error, trace, warn};
use log::{debug, error, info, trace, warn};
use mix_client::MixClient;
use provider_client::ProviderClient;
use provider_client::{ProviderClient, ProviderClientError};
use sphinx::header::delays::Delay;
use sphinx::route::{Destination, Node as SphinxNode};
use std::collections::HashMap;
use topology::MixProviderNode;
use topology::provider;
#[derive(Debug, PartialEq, Clone)]
pub enum PathStatus {
@@ -23,28 +23,36 @@ pub(crate) struct PathChecker {
layer_one_clients: HashMap<[u8; 32], Option<MixClient>>,
paths_status: HashMap<Vec<u8>, PathStatus>,
our_destination: Destination,
check_id: [u8; 16],
}
impl PathChecker {
pub(crate) async fn new(
providers: Vec<MixProviderNode>,
ephemeral_keys: DummyMixIdentityKeyPair,
) -> Self {
pub(crate) async fn new<IDPair, Priv, Pub>(
providers: Vec<provider::Node>,
identity_keys: &IDPair,
check_id: [u8; 16],
) -> Self
where
IDPair: MixnetIdentityKeyPair<Priv, Pub>,
Priv: MixnetIdentityPrivateKey,
Pub: MixnetIdentityPublicKey,
{
let mut provider_clients = HashMap::new();
let mut temporary_address = [0u8; 32];
let public_key_bytes = ephemeral_keys.public_key().to_bytes();
temporary_address.copy_from_slice(&public_key_bytes[..]);
let address = identity_keys.public_key().derive_address();
for provider in providers {
let mut provider_client =
ProviderClient::new(provider.client_listener, temporary_address, None);
let mut provider_client = ProviderClient::new(provider.client_listener, address, None);
let insertion_result = match provider_client.register().await {
Ok(token) => {
debug!("registered at provider {}", provider.pub_key);
provider_client.update_token(token);
provider_clients.insert(provider.get_pub_key_bytes(), Some(provider_client))
}
Err(ProviderClientError::ClientAlreadyRegisteredError) => {
info!("We were already registered");
provider_clients.insert(provider.get_pub_key_bytes(), Some(provider_client))
}
Err(err) => {
warn!(
"failed to register at provider {} - {:?}",
@@ -62,15 +70,19 @@ impl PathChecker {
PathChecker {
provider_clients,
layer_one_clients: HashMap::new(),
our_destination: Destination::new(temporary_address, Default::default()),
our_destination: Destination::new(address, Default::default()),
paths_status: HashMap::new(),
check_id,
}
}
// iteration is used to distinguish packets sent through the same path (as the healthcheck
// may try to send say 10 packets through given path)
fn unique_path_key(path: &Vec<SphinxNode>, iteration: u8) -> Vec<u8> {
std::iter::once(iteration)
fn unique_path_key(path: &Vec<SphinxNode>, check_id: [u8; 16], iteration: u8) -> Vec<u8> {
check_id
.iter()
.cloned()
.chain(std::iter::once(iteration))
.chain(
path.iter()
.map(|node| node.pub_key.to_bytes().to_vec())
@@ -80,10 +92,10 @@ impl PathChecker {
}
pub(crate) fn path_key_to_node_keys(path_key: Vec<u8>) -> Vec<[u8; 32]> {
assert_eq!(path_key.len() % 32, 1);
assert_eq!(path_key.len() % 32, 17);
path_key
.into_iter()
.skip(1) // remove first byte as it represents the iteration number which we do not care about now
.skip(16 + 1) // remove 16 + 1 bytes as it represents check_id and the iteration number which we do not care about now
.chunks(32)
.into_iter()
.map(|key_chunk| {
@@ -135,6 +147,8 @@ impl PathChecker {
if msg == sfw_provider_requests::DUMMY_MESSAGE_CONTENT {
// finish iterating the loop as the messages might not be ordered
should_stop = true;
} else if msg[..16] != self.check_id {
warn!("received response from previous healthcheck")
} else {
provider_messages.push(msg);
}
@@ -153,25 +167,38 @@ impl PathChecker {
let mut provider_messages = Vec::new();
for provider_client in self.provider_clients.values() {
// if it was none all associated paths were already marked as unhealthy
if provider_client.is_some() {
let pc = provider_client.as_ref().unwrap();
provider_messages.extend(self.resolve_pending_provider_checks(pc).await);
}
let pc = match provider_client {
Some(pc) => pc,
None => continue,
};
provider_messages.extend(self.resolve_pending_provider_checks(pc).await);
}
self.update_path_statuses(provider_messages);
}
pub(crate) async fn send_test_packet(&mut self, path: &Vec<SphinxNode>, iteration: u8) {
if path.len() == 0 {
warn!("trying to send test packet through an empty path!");
return;
}
debug!("Checking path: {:?} ({})", path, iteration);
let path_identifier = PathChecker::unique_path_key(path, iteration);
let path_identifier = PathChecker::unique_path_key(path, self.check_id, iteration);
// check if there is even any point in sending the packet
// does provider exist?
let provider_client = self
.provider_clients
.get(&path.last().unwrap().pub_key.to_bytes())
.get(
&path
.last()
.expect("We checked the path to contain at least one entry")
.pub_key
.to_bytes(),
)
.unwrap();
if provider_client.is_none() {
@@ -186,10 +213,15 @@ impl PathChecker {
return;
}
let layer_one_mix = path.first().unwrap();
let layer_one_mix = path
.first()
.expect("We checked the path to contain at least one entry");
let first_node_key = layer_one_mix.pub_key.to_bytes();
// we generated the bytes data so unwrap is fine
let first_node_address =
addressing::socket_address_from_encoded_bytes(layer_one_mix.address.to_bytes());
addressing::socket_address_from_encoded_bytes(layer_one_mix.address.to_bytes())
.unwrap();
let first_node_client = self
.layer_one_clients
@@ -208,10 +240,12 @@ impl PathChecker {
return;
}
// we already checked for 'None' case
let first_node_client = first_node_client.as_ref().unwrap();
let delays: Vec<_> = path.iter().map(|_| Delay::new(0)).collect();
// all of the data used to create the packet was created by us
let packet = sphinx::SphinxPacket::new(
path_identifier.clone(),
&path[..],
@@ -223,7 +257,7 @@ impl PathChecker {
debug!("sending test packet to {}", first_node_address);
match first_node_client.send(packet, first_node_address).await {
Err(err) => {
warn!("failed to send packet to {} - {}", first_node_address, err);
info!("failed to send packet to {} - {}", first_node_address, err);
if self
.paths_status
.insert(path_identifier, PathStatus::Unhealthy)
common/healthcheck/src/result.rs
+36 -23
View File
@@ -1,7 +1,8 @@
use crate::path_check::{PathChecker, PathStatus};
use crate::score::NodeScore;
use crypto::identity::{DummyMixIdentityKeyPair, MixnetIdentityKeyPair};
use crypto::identity::{MixnetIdentityKeyPair, MixnetIdentityPrivateKey, MixnetIdentityPublicKey};
use log::{debug, error, info, warn};
use rand_os::rand_core::RngCore;
use sphinx::route::NodeAddressBytes;
use std::collections::HashMap;
use std::fmt::{Error, Formatter};
@@ -14,9 +15,9 @@ pub struct HealthCheckResult(Vec<NodeScore>);
impl std::fmt::Display for HealthCheckResult {
fn fmt(&self, f: &mut Formatter) -> Result<(), Error> {
write!(f, "NETWORK HEALTH\n==============\n")?;
self.0
.iter()
.for_each(|score| write!(f, "{}\n", score).unwrap());
for score in self.0.iter() {
write!(f, "{}\n", score)?
}
Ok(())
}
}
@@ -28,8 +29,8 @@ impl HealthCheckResult {
fn zero_score<T: NymTopology>(topology: T) -> Self {
warn!("The network is unhealthy, could not send any packets - returning zero score!");
let mixes = topology.get_mix_nodes();
let providers = topology.get_mix_provider_nodes();
let mixes = topology.mix_nodes();
let providers = topology.providers();
let health = mixes
.into_iter()
@@ -58,10 +59,10 @@ impl HealthCheckResult {
score_threshold: f64,
) -> T {
let filtered_mix_nodes = topology
.get_mix_nodes()
.mix_nodes()
.into_iter()
.filter(|node| {
match self.node_score(NodeAddressBytes::from_b64_string(node.pub_key.clone())) {
match self.node_score(NodeAddressBytes::from_base58_string(node.pub_key.clone())) {
None => {
error!("Unknown node in topology - {:?}", node);
false
@@ -72,10 +73,10 @@ impl HealthCheckResult {
.collect();
let filtered_provider_nodes = topology
.get_mix_provider_nodes()
.providers()
.into_iter()
.filter(|node| {
match self.node_score(NodeAddressBytes::from_b64_string(node.pub_key.clone())) {
match self.node_score(NodeAddressBytes::from_base58_string(node.pub_key.clone())) {
None => {
error!("Unknown node in topology - {:?}", node);
false
@@ -85,7 +86,7 @@ impl HealthCheckResult {
})
.collect();
// coco nodes remain unchanged as no healthcheck is being run on them or time being
let filtered_coco_nodes = topology.get_coco_nodes();
let filtered_coco_nodes = topology.coco_nodes();
T::new_from_nodes(
filtered_mix_nodes,
@@ -94,15 +95,31 @@ impl HealthCheckResult {
)
}
pub async fn calculate<T: NymTopology>(
fn generate_check_id() -> [u8; 16] {
let mut id = [0u8; 16];
let mut rng = rand_os::OsRng::new().unwrap();
rng.fill_bytes(&mut id);
id
}
pub async fn calculate<T, IDPair, Priv, Pub>(
topology: T,
iterations: usize,
resolution_timeout: Duration,
) -> Self {
identity_keys: &IDPair,
) -> Self
where
T: NymTopology,
IDPair: MixnetIdentityKeyPair<Priv, Pub>,
Priv: MixnetIdentityPrivateKey,
Pub: MixnetIdentityPublicKey,
{
// currently healthchecker supports only up to 255 iterations - if we somehow
// find we need more, it's relatively easy change
assert!(iterations <= 255);
let check_id = Self::generate_check_id();
let all_paths = match topology.all_paths() {
Ok(paths) => paths,
Err(_) => return Self::zero_score(topology),
@@ -110,21 +127,17 @@ impl HealthCheckResult {
// create entries for all nodes
let mut score_map = HashMap::new();
topology.get_mix_nodes().into_iter().for_each(|node| {
topology.mix_nodes().into_iter().for_each(|node| {
score_map.insert(node.get_pub_key_bytes(), NodeScore::from_mixnode(node));
});
topology
.get_mix_provider_nodes()
.into_iter()
.for_each(|node| {
score_map.insert(node.get_pub_key_bytes(), NodeScore::from_provider(node));
});
topology.providers().into_iter().for_each(|node| {
score_map.insert(node.get_pub_key_bytes(), NodeScore::from_provider(node));
});
let ephemeral_keys = DummyMixIdentityKeyPair::new();
let providers = topology.get_mix_provider_nodes();
let providers = topology.providers();
let mut path_checker = PathChecker::new(providers, ephemeral_keys).await;
let mut path_checker = PathChecker::new(providers, identity_keys, check_id).await;
for i in 0..iterations {
debug!("running healthcheck iteration {} / {}", i + 1, iterations);
for path in &all_paths {
common/healthcheck/src/score.rs
+7 -6
View File
@@ -4,7 +4,8 @@ use std::cmp::Ordering;
use std::fmt::Error;
use std::fmt::Formatter;
use std::net::SocketAddr;
use topology::{MixNode, MixProviderNode};
use topology::mix;
use topology::provider;
// TODO: should 'nodetype' really be part of healthcheck::score
@@ -91,10 +92,10 @@ impl PartialEq for NodeScore {
}
impl NodeScore {
pub(crate) fn from_mixnode(node: MixNode) -> Self {
pub(crate) fn from_mixnode(node: mix::Node) -> Self {
NodeScore {
typ: NodeType::Mix,
pub_key: NodeAddressBytes::from_b64_string(node.pub_key),
pub_key: NodeAddressBytes::from_base58_string(node.pub_key),
addresses: vec![node.host],
version: node.version,
layer: format!("layer {}", node.layer),
@@ -103,10 +104,10 @@ impl NodeScore {
}
}
pub(crate) fn from_provider(node: MixProviderNode) -> Self {
pub(crate) fn from_provider(node: provider::Node) -> Self {
NodeScore {
typ: NodeType::MixProvider,
pub_key: NodeAddressBytes::from_b64_string(node.pub_key),
pub_key: NodeAddressBytes::from_base58_string(node.pub_key),
addresses: vec![node.mixnet_listener, node.client_listener],
version: node.version,
layer: format!("provider"),
@@ -148,7 +149,7 @@ impl std::fmt::Display for NodeScore {
return Err(std::fmt::Error);
}
};
let stringified_key = self.pub_key.to_b64_string();
let stringified_key = self.pub_key.to_base58_string();
write!(
f,
"({})\t{}/{}\t({}%)\t|| {}\tv{} <{}> - {}",
common/pemstore/Cargo.toml
+15
View File
@@ -0,0 +1,15 @@
[package]
name = "pemstore"
version = "0.1.0"
authors = ["Jedrzej Stuczynski <andrew@nymtech.net>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
log = "0.4"
pem = "0.7.0"
pretty_env_logger = "0.3"
## internal
crypto = {path = "../crypto"}
nym-client/src/persistence/mod.rs → common/pemstore/src/lib.rs
View File
common/pemstore/src/pathfinder.rs
+15
View File
@@ -0,0 +1,15 @@
use std::path::PathBuf;
pub trait PathFinder {
fn config_dir(&self) -> PathBuf;
fn private_identity_key(&self) -> PathBuf;
fn public_identity_key(&self) -> PathBuf;
// Optional:
fn private_encryption_key(&self) -> Option<PathBuf> {
None
}
fn public_encryption_key(&self) -> Option<PathBuf> {
None
}
}
nym-client/src/persistence/pemstore.rs → common/pemstore/src/pemstore.rs
+37 -35
View File
@@ -1,18 +1,10 @@
use crate::persistence::pathfinder::Pathfinder;
use crate::pathfinder::PathFinder;
use pem::{encode, parse, Pem};
use std::fs::File;
use std::io;
use std::io::prelude::*;
use std::path::PathBuf;
pub fn read_mix_identity_keypair_from_disk(
id: String,
) -> crypto::identity::DummyMixIdentityKeyPair {
let pathfinder = Pathfinder::new(id);
let pem_store = PemStore::new(pathfinder);
let keypair = pem_store.read_identity();
keypair
}
#[allow(dead_code)]
pub fn read_mix_encryption_keypair_from_disk(_id: String) -> crypto::encryption::x25519::KeyPair {
unimplemented!()
@@ -25,51 +17,58 @@ pub struct PemStore {
}
impl PemStore {
pub fn new(pathfinder: Pathfinder) -> PemStore {
pub fn new<P: PathFinder>(pathfinder: P) -> PemStore {
PemStore {
config_dir: pathfinder.config_dir,
private_mix_key: pathfinder.private_mix_key,
public_mix_key: pathfinder.public_mix_key,
config_dir: pathfinder.config_dir(),
private_mix_key: pathfinder.private_identity_key(),
public_mix_key: pathfinder.public_identity_key(),
}
}
pub fn read_identity<IDPair, Priv, Pub>(&self) -> IDPair
pub fn read_identity<IDPair, Priv, Pub>(&self) -> io::Result<IDPair>
where
IDPair: crypto::identity::MixnetIdentityKeyPair<Priv, Pub>,
Priv: crypto::identity::MixnetIdentityPrivateKey,
Pub: crypto::identity::MixnetIdentityPublicKey,
{
let private_pem = self.read_pem_file(self.private_mix_key.clone());
let public_pem = self.read_pem_file(self.public_mix_key.clone());
let private_pem = self.read_pem_file(self.private_mix_key.clone())?;
let public_pem = self.read_pem_file(self.public_mix_key.clone())?;
let key_pair = IDPair::from_bytes(&private_pem.contents, &public_pem.contents);
assert_eq!(key_pair.private_key().pem_type(), private_pem.tag);
assert_eq!(key_pair.public_key().pem_type(), public_pem.tag);
if key_pair.private_key().pem_type() != private_pem.tag {
return Err(io::Error::new(
io::ErrorKind::Other,
"unexpected private key pem tag",
));
}
key_pair
if key_pair.public_key().pem_type() != public_pem.tag {
return Err(io::Error::new(
io::ErrorKind::Other,
"unexpected public key pem tag",
));
}
Ok(key_pair)
}
fn read_pem_file(&self, filepath: PathBuf) -> Pem {
let mut pem_bytes = File::open(filepath).expect("Could not open stored keys from disk.");
fn read_pem_file(&self, filepath: PathBuf) -> io::Result<Pem> {
let mut pem_bytes = File::open(filepath)?;
let mut buf = Vec::new();
pem_bytes
.read_to_end(&mut buf)
.expect("PEM bytes reading failed.");
let pem = parse(&buf).expect("PEM parsing failed while reading keys");
pem
pem_bytes.read_to_end(&mut buf)?;
parse(&buf).map_err(|e| io::Error::new(io::ErrorKind::Other, e))
}
// This should be refactored and made more generic for when we have other kinds of
// KeyPairs that we want to persist (e.g. validator keypairs, or keys for
// signing vs encryption). However, for the moment, it does the job.
pub fn write_identity<IDPair, Priv, Pub>(&self, key_pair: IDPair)
pub fn write_identity<IDPair, Priv, Pub>(&self, key_pair: IDPair) -> io::Result<()>
where
IDPair: crypto::identity::MixnetIdentityKeyPair<Priv, Pub>,
Priv: crypto::identity::MixnetIdentityPrivateKey,
Pub: crypto::identity::MixnetIdentityPublicKey,
{
std::fs::create_dir_all(self.config_dir.clone()).unwrap();
std::fs::create_dir_all(self.config_dir.clone())?;
let private_key = key_pair.private_key();
let public_key = key_pair.public_key();
@@ -77,22 +76,25 @@ impl PemStore {
self.private_mix_key.clone(),
private_key.to_bytes(),
private_key.pem_type(),
);
)?;
self.write_pem_file(
self.public_mix_key.clone(),
public_key.to_bytes(),
public_key.pem_type(),
);
)?;
Ok(())
}
fn write_pem_file(&self, filepath: PathBuf, data: Vec<u8>, tag: String) {
fn write_pem_file(&self, filepath: PathBuf, data: Vec<u8>, tag: String) -> io::Result<()> {
let pem = Pem {
tag,
contents: data,
};
let key = encode(&pem);
let mut file = File::create(filepath).unwrap();
file.write_all(key.as_bytes()).unwrap();
let mut file = File::create(filepath)?;
file.write_all(key.as_bytes())?;
Ok(())
}
}
common/topology/Cargo.toml
+6 -4
View File
@@ -7,15 +7,17 @@ edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
base64 = "0.11.0"
curve25519-dalek = "1.2.3"
bs58 = "0.3.0"
itertools = "0.8.2"
log = "0.4.8"
log = "0.4"
pretty_env_logger = "0.3"
rand = "0.7.2"
serde = { version = "1.0.104", features = ["derive"] }
## internal
addressing = {path = "../addressing"}
version-checker = {path = "../version-checker" }
## will be moved to proper dependencies once released
sphinx = { git = "https://github.com/nymtech/sphinx", rev="1d8cefcb6a0cb8e87d00d89eb1ccf2839e92aa1f" }
sphinx = { git = "https://github.com/nymtech/sphinx", rev="8424f4b0933b4a7f64ae828b2edefc5ba43a9e79" }
# sphinx = { path = "../../../sphinx"}
common/topology/src/coco.rs
+15
View File
@@ -0,0 +1,15 @@
use crate::filter;
#[derive(Debug, Clone)]
pub struct Node {
pub host: String,
pub pub_key: String,
pub last_seen: u64,
pub version: String,
}
impl filter::Versioned for Node {
fn version(&self) -> String {
self.version.clone()
}
}
common/topology/src/filter.rs
+18
View File
@@ -0,0 +1,18 @@
pub trait Versioned: Clone {
fn version(&self) -> String;
}
pub trait VersionFilterable<T> {
fn filter_by_version(&self, expected_version: &str) -> Self;
}
impl<T: Versioned> VersionFilterable<T> for Vec<T> {
fn filter_by_version(&self, expected_version: &str) -> Self {
self.iter()
.filter(|node| {
version_checker::is_minor_version_compatible(&node.version(), expected_version)
})
.cloned()
.collect()
}
}
common/topology/src/lib.rs
+36 -114
View File
@@ -1,102 +1,29 @@
use addressing;
use curve25519_dalek::montgomery::MontgomeryPoint;
use crate::filter::VersionFilterable;
use itertools::Itertools;
use rand::seq::IteratorRandom;
use sphinx::route::{Node as SphinxNode, NodeAddressBytes};
use sphinx::route::Node as SphinxNode;
use std::cmp::max;
use std::collections::HashMap;
use std::net::SocketAddr;
#[derive(Debug, Clone)]
pub struct MixNode {
pub host: SocketAddr,
pub pub_key: String,
pub layer: u64,
pub last_seen: u64,
pub version: String,
}
pub mod coco;
mod filter;
pub mod mix;
pub mod provider;
impl Into<SphinxNode> for MixNode {
fn into(self) -> SphinxNode {
let address_bytes = addressing::encoded_bytes_from_socket_address(self.host);
let key_bytes = self.get_pub_key_bytes();
let key = MontgomeryPoint(key_bytes);
SphinxNode::new(NodeAddressBytes::from_bytes(address_bytes), key)
}
}
impl MixNode {
pub fn get_pub_key_bytes(&self) -> [u8; 32] {
let decoded_key_bytes = base64::decode_config(&self.pub_key, base64::URL_SAFE).unwrap();
let mut key_bytes = [0; 32];
key_bytes.copy_from_slice(&decoded_key_bytes[..]);
key_bytes
}
}
#[derive(Debug, Clone)]
pub struct MixProviderClient {
pub pub_key: String,
}
#[derive(Debug, Clone)]
pub struct MixProviderNode {
pub client_listener: SocketAddr,
pub mixnet_listener: SocketAddr,
pub pub_key: String,
pub registered_clients: Vec<MixProviderClient>,
pub last_seen: u64,
pub version: String,
}
impl Into<SphinxNode> for MixProviderNode {
fn into(self) -> SphinxNode {
let address_bytes = addressing::encoded_bytes_from_socket_address(self.mixnet_listener);
let key_bytes = self.get_pub_key_bytes();
let key = MontgomeryPoint(key_bytes);
SphinxNode::new(NodeAddressBytes::from_bytes(address_bytes), key)
}
}
impl MixProviderNode {
pub fn get_pub_key_bytes(&self) -> [u8; 32] {
let decoded_key_bytes = base64::decode_config(&self.pub_key, base64::URL_SAFE).unwrap();
let mut key_bytes = [0; 32];
key_bytes.copy_from_slice(&decoded_key_bytes[..]);
key_bytes
}
}
#[derive(Debug, Clone)]
pub struct CocoNode {
pub host: String,
pub pub_key: String,
pub last_seen: u64,
pub version: String,
}
#[derive(Debug)]
pub enum NymTopologyError {
InvalidMixLayerError,
MissingLayerError(Vec<u64>),
}
pub trait NymTopology: Sized {
pub trait NymTopology: Sized + std::fmt::Debug + Send + Sync {
fn new(directory_server: String) -> Self;
fn new_from_nodes(
mix_nodes: Vec<MixNode>,
mix_provider_nodes: Vec<MixProviderNode>,
coco_nodes: Vec<CocoNode>,
mix_nodes: Vec<mix::Node>,
mix_provider_nodes: Vec<provider::Node>,
coco_nodes: Vec<coco::Node>,
) -> Self;
fn get_mix_nodes(&self) -> Vec<MixNode>;
fn get_mix_provider_nodes(&self) -> Vec<MixProviderNode>;
fn get_coco_nodes(&self) -> Vec<CocoNode>;
fn make_layered_topology(&self) -> Result<HashMap<u64, Vec<MixNode>>, NymTopologyError> {
let mut layered_topology: HashMap<u64, Vec<MixNode>> = HashMap::new();
fn mix_nodes(&self) -> Vec<mix::Node>;
fn providers(&self) -> Vec<provider::Node>;
fn coco_nodes(&self) -> Vec<coco::Node>;
fn make_layered_topology(&self) -> Result<HashMap<u64, Vec<mix::Node>>, NymTopologyError> {
let mut layered_topology: HashMap<u64, Vec<mix::Node>> = HashMap::new();
let mut highest_layer = 0;
for mix in self.get_mix_nodes() {
for mix in self.mix_nodes() {
// we need to have extra space for provider
if mix.layer > sphinx::constants::MAX_PATH_LENGTH as u64 {
return Err(NymTopologyError::InvalidMixLayerError);
@@ -124,10 +51,14 @@ pub trait NymTopology: Sized {
Ok(layered_topology)
}
// Tries to get a route through the mix network
fn mix_route(&self) -> Result<Vec<SphinxNode>, NymTopologyError> {
let mut layered_topology = self.make_layered_topology()?;
let num_layers = layered_topology.len();
let route = (1..=num_layers as u64)
// unwrap is safe for 'remove' as it it failed, it implied the entry never existed
// in the map in the first place which would contradict what we've just done
.map(|layer| layered_topology.remove(&layer).unwrap()) // for each layer
.map(|nodes| nodes.into_iter().choose(&mut rand::thread_rng()).unwrap()) // choose random node
.map(|random_node| random_node.into()) // and convert it into sphinx specific node format
@@ -136,7 +67,7 @@ pub trait NymTopology: Sized {
Ok(route)
}
// sets a route to specific provider
// Sets up a route to a specific provider
fn route_to(&self, provider_node: SphinxNode) -> Result<Vec<SphinxNode>, NymTopologyError> {
Ok(self
.mix_route()?
@@ -147,7 +78,7 @@ pub trait NymTopology: Sized {
fn all_paths(&self) -> Result<Vec<Vec<SphinxNode>>, NymTopologyError> {
let mut layered_topology = self.make_layered_topology()?;
let providers = self.get_mix_provider_nodes();
let providers = self.providers();
let sorted_layers: Vec<Vec<SphinxNode>> = (1..=layered_topology.len() as u64)
.map(|layer| layered_topology.remove(&layer).unwrap()) // get all nodes per layer
@@ -167,30 +98,17 @@ pub trait NymTopology: Sized {
fn filter_node_versions(
&self,
mix_version: &str,
provider_version: &str,
coco_version: &str,
expected_mix_version: &str,
expected_provider_version: &str,
expected_coco_version: &str,
) -> Self {
let filtered_mixes = self
.get_mix_nodes()
.iter()
.cloned()
.filter(|mix_node| mix_node.version == mix_version)
.collect();
let filtered_providers = self
.get_mix_provider_nodes()
.iter()
.cloned()
.filter(|provider_node| provider_node.version == provider_version)
.collect();
let filtered_coco_nodes = self
.get_coco_nodes()
.iter()
.cloned()
.filter(|coco_node| coco_node.version == coco_version)
.collect();
let mixes = self.mix_nodes().filter_by_version(expected_mix_version);
let providers = self
.providers()
.filter_by_version(expected_provider_version);
let cocos = self.coco_nodes().filter_by_version(expected_coco_version);
Self::new_from_nodes(filtered_mixes, filtered_providers, filtered_coco_nodes)
Self::new_from_nodes(mixes, providers, cocos)
}
fn can_construct_path_through(&self) -> bool {
@@ -201,4 +119,8 @@ pub trait NymTopology: Sized {
}
}
// TODO: tests...
#[derive(Debug)]
pub enum NymTopologyError {
InvalidMixLayerError,
MissingLayerError(Vec<u64>),
}
common/topology/src/mix.rs
+37
View File
@@ -0,0 +1,37 @@
use crate::filter;
use sphinx::route::Node as SphinxNode;
use sphinx::route::NodeAddressBytes;
use std::net::SocketAddr;
#[derive(Debug, Clone)]
pub struct Node {
pub host: SocketAddr,
pub pub_key: String,
pub layer: u64,
pub last_seen: u64,
pub version: String,
}
impl Node {
pub fn get_pub_key_bytes(&self) -> [u8; 32] {
let mut key_bytes = [0; 32];
bs58::decode(&self.pub_key).into(&mut key_bytes).unwrap();
key_bytes
}
}
impl filter::Versioned for Node {
fn version(&self) -> String {
self.version.clone()
}
}
impl Into<SphinxNode> for Node {
fn into(self) -> SphinxNode {
let address_bytes = addressing::encoded_bytes_from_socket_address(self.host);
let key_bytes = self.get_pub_key_bytes();
let key = sphinx::key::new(key_bytes);
SphinxNode::new(NodeAddressBytes::from_bytes(address_bytes), key)
}
}
common/topology/src/provider.rs
+43
View File
@@ -0,0 +1,43 @@
use crate::filter;
use sphinx::route::Node as SphinxNode;
use sphinx::route::NodeAddressBytes;
use std::net::SocketAddr;
#[derive(Debug, Clone)]
pub struct Client {
pub pub_key: String,
}
#[derive(Debug, Clone)]
pub struct Node {
pub client_listener: SocketAddr,
pub mixnet_listener: SocketAddr,
pub pub_key: String,
pub registered_clients: Vec<Client>,
pub last_seen: u64,
pub version: String,
}
impl Node {
pub fn get_pub_key_bytes(&self) -> [u8; 32] {
let mut key_bytes = [0; 32];
bs58::decode(&self.pub_key).into(&mut key_bytes).unwrap();
key_bytes
}
}
impl filter::Versioned for Node {
fn version(&self) -> String {
self.version.clone()
}
}
impl Into<SphinxNode> for Node {
fn into(self) -> SphinxNode {
let address_bytes = addressing::encoded_bytes_from_socket_address(self.mixnet_listener);
let key_bytes = self.get_pub_key_bytes();
let key = sphinx::key::new(key_bytes);
SphinxNode::new(NodeAddressBytes::from_bytes(address_bytes), key)
}
}
common/version-checker/.gitignore
+3
View File
@@ -0,0 +1,3 @@
/target
**/*.rs.bk
Cargo.lock
common/version-checker/Cargo.toml
+10
View File
@@ -0,0 +1,10 @@
[package]
name = "version-checker"
version = "0.1.0"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
semver = "0.9.0"
common/version-checker/src/lib.rs
+67
View File
@@ -0,0 +1,67 @@
use semver::Version;
use semver::VersionReq;
/// Checks whether given `version` is compatible with a given semantic version requirement `req`
/// according to major-minor semver rules. The semantic version requirement can be passed as a full,
/// concrete version number, because that's what we'll have in our Cargo.toml files (e.g. 0.3.2).
/// The patch number in the requirement gets dropped and replaced with a wildcard (0.3.*) as all
/// minor versions should be compatible with each other.
pub fn is_minor_version_compatible(version: &str, req: &str) -> bool {
let version = match Version::parse(version) {
Ok(v) => v,
Err(_) => return false,
};
let tmp = match Version::parse(req) {
Ok(v) => v,
Err(_) => return false,
};
let wildcard = format!("{}.{}.*", tmp.major, tmp.minor).to_string();
let semver_requirement = VersionReq::parse(&wildcard).expect("panicked on semver requirement parsing. This should never happen as inputs should already have been sanitized.");
semver_requirement.matches(&version)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn version_0_3_0_is_compatible_with_requirement_0_3_x() {
assert!(is_minor_version_compatible("0.3.0", "0.3.2"));
}
#[test]
fn version_0_3_1_is_compatible_with_minimum_requirement_0_3_x() {
assert!(is_minor_version_compatible("0.3.1", "0.3.2"));
}
#[test]
fn version_0_3_2_is_compatible_with_minimum_requirement_0_3_x() {
assert!(is_minor_version_compatible("0.3.2", "0.3.0"));
}
#[test]
fn version_0_2_0_is_not_compatible_with_requirement_0_3_x() {
assert!(!is_minor_version_compatible("0.2.0", "0.3.2"));
}
#[test]
fn version_0_4_0_is_not_compatible_with_requirement_0_3_x() {
assert!(!is_minor_version_compatible("0.4.0", "0.3.2"));
}
#[test]
fn version_1_3_2_is_not_compatible_with_requirement_0_3_x() {
assert!(!is_minor_version_compatible("1.3.2", "0.3.2"));
}
#[test]
fn returns_false_on_foo_version() {
assert!(!is_minor_version_compatible("foo", "0.3.2"));
}
#[test]
fn returns_false_on_bar_version() {
assert!(!is_minor_version_compatible("0.3.2", "bar"));
}
}
mixnode/CHANGELOG.md
-39
View File
@@ -1,39 +0,0 @@
# nym-mixnode Changelog
## 0.3.3
* Version increase for consistency with `nym-client`
## 0.3.2
* added separate announce address
* allows announcing dns hostname instead of an ip address
## 0.3.1
* Fixed crash when directory server goes down
## 0.3.0
* cleaned up a lot of internal dependencies
* reporting version to the directory server
* printing warning on trying to bind to "localhost", "127.0.0.1" or "0.0.0.0"
* more informative error messages
* generalised identity keys
* generalised Topology handling
* started slow transition to `log` crate by `nym-client`
* start of 'MixMining'
* start of validator node
## 0.2.0
* removed the `--local` flag
* introduced `--directory` argument to support arbitrary directory servers. Leaving it out will point the node at the "https://directory.nymtech.net" alpha testnet server
* the `host` argument is now required
* IPv6 support
* mixnode version number is now shown at node start
* directory server location is now shown at node start
## 0.1.0 - Initial Release
* The bare minimum set of features required by a Nym Mixnode
mixnode/Cargo.toml
+6 -4
View File
@@ -1,18 +1,20 @@
[package]
build = "build.rs"
name = "nym-mixnode"
version = "0.3.3"
version = "0.4.0-rc.1"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
base64 = "0.11.0"
bs58 = "0.3.0"
clap = "2.33.0"
curve25519-dalek = "1.2.3"
dotenv = "0.15.0"
futures = "0.3.1"
log = "0.4.8"
log = "0.4"
pretty_env_logger = "0.3"
tokio = { version = "0.2", features = ["full"] }
## internal
@@ -20,7 +22,7 @@ addressing = {path = "../common/addressing" }
directory-client = { path = "../common/clients/directory-client" }
## will be moved to proper dependencies once released
sphinx = { git = "https://github.com/nymtech/sphinx", rev="1d8cefcb6a0cb8e87d00d89eb1ccf2839e92aa1f" }
sphinx = { git = "https://github.com/nymtech/sphinx", rev="8424f4b0933b4a7f64ae828b2edefc5ba43a9e79" }
[build-dependencies]
built = "0.3.2"
mixnode/src/main.rs
+5 -1
View File
@@ -1,10 +1,14 @@
use clap::{App, Arg, ArgMatches, SubCommand};
use log::*;
use std::process;
mod mix_peer;
mod node;
fn main() {
dotenv::dotenv().ok();
pretty_env_logger::init();
let arg_matches = App::new("Nym Mixnode")
.version(built_info::PKG_VERSION)
.author("Nymtech")
@@ -54,7 +58,7 @@ fn main() {
.get_matches();
if let Err(e) = execute(arg_matches) {
println!("{}", e);
error!("{}", e);
process::exit(1);
}
}
mixnode/src/mix_peer.rs
+18 -4
View File
@@ -1,4 +1,5 @@
use addressing;
use addressing::AddressTypeError;
use sphinx::route::NodeAddressBytes;
use std::error::Error;
use std::net::SocketAddr;
@@ -9,14 +10,27 @@ pub struct MixPeer {
connection: SocketAddr,
}
#[derive(Debug)]
pub enum MixPeerError {
InvalidAddressError,
}
impl From<addressing::AddressTypeError> for MixPeerError {
fn from(_: AddressTypeError) -> Self {
use MixPeerError::*;
InvalidAddressError
}
}
impl MixPeer {
// note that very soon `next_hop_address` will be changed to `next_hop_metadata`
pub fn new(next_hop_address: NodeAddressBytes) -> MixPeer {
pub fn new(next_hop_address: NodeAddressBytes) -> Result<MixPeer, MixPeerError> {
let next_hop_socket_address =
addressing::socket_address_from_encoded_bytes(next_hop_address.to_bytes());
MixPeer {
addressing::socket_address_from_encoded_bytes(next_hop_address.to_bytes())?;
Ok(MixPeer {
connection: next_hop_socket_address,
}
})
}
pub async fn send(&self, bytes: Vec<u8>) -> Result<(), Box<dyn Error>> {
mixnode/src/node/mod.rs
+42 -18
View File
@@ -6,6 +6,7 @@ use curve25519_dalek::scalar::Scalar;
use futures::channel::mpsc;
use futures::lock::Mutex;
use futures::SinkExt;
use log::*;
use sphinx::header::delays::Delay as SphinxDelay;
use sphinx::{ProcessedPacket, SphinxPacket};
use std::net::SocketAddr;
@@ -30,7 +31,7 @@ pub struct Config {
impl Config {
pub fn public_key_string(&self) -> String {
let key_bytes = self.public_key.to_bytes().to_vec();
base64::encode_config(&key_bytes, base64::URL_SAFE)
bs58::encode(&key_bytes).into_string()
}
}
@@ -38,6 +39,8 @@ impl Config {
pub enum MixProcessingError {
SphinxRecoveryError,
ReceivedFinalHopError,
SphinxProcessingError,
InvalidHopAddress,
}
impl From<sphinx::ProcessingError> for MixProcessingError {
@@ -107,20 +110,32 @@ impl PacketProcessor {
) -> Result<ForwardingData, MixProcessingError> {
// we received something resembling a sphinx packet, report it!
let processing_data = processing_data.lock().await;
let mut received_sender = processing_data.received_metrics_tx.clone();
let mut received_metrics_tx = processing_data.received_metrics_tx.clone();
received_sender.send(()).await.unwrap();
// if unwrap failed it means our metrics reporter died, so we should exit application and
// force restart
if received_metrics_tx.send(()).await.is_err() {
error!("failed to send metrics data to the controller - the underlying thread probably died!");
std::process::exit(1);
}
let packet = SphinxPacket::from_bytes(packet_data.to_vec())?;
let (next_packet, next_hop_address, delay) =
match packet.process(processing_data.secret_key) {
ProcessedPacket::ProcessedPacketForwardHop(packet, address, delay) => {
Ok(ProcessedPacket::ProcessedPacketForwardHop(packet, address, delay)) => {
(packet, address, delay)
}
_ => return Err(MixProcessingError::ReceivedFinalHopError),
Ok(_) => return Err(MixProcessingError::ReceivedFinalHopError),
Err(e) => {
warn!("Failed to unwrap Sphinx packet: {:?}", e);
return Err(MixProcessingError::SphinxProcessingError);
}
};
let next_mix = MixPeer::new(next_hop_address);
let next_mix = match MixPeer::new(next_hop_address) {
Ok(next_mix) => next_mix,
Err(_) => return Err(MixProcessingError::InvalidHopAddress),
};
let fwd_data = ForwardingData::new(
next_packet,
@@ -134,13 +149,18 @@ impl PacketProcessor {
async fn wait_and_forward(mut forwarding_data: ForwardingData) {
let delay_duration = Duration::from_nanos(forwarding_data.delay.get_value());
tokio::time::delay_for(delay_duration).await;
forwarding_data
if forwarding_data
.sent_metrics_tx
.send(forwarding_data.recipient.to_string())
.await
.unwrap();
.is_err()
{
error!("failed to send metrics data to the controller - the underlying thread probably died!");
std::process::exit(1);
}
println!("RECIPIENT: {:?}", forwarding_data.recipient);
trace!("RECIPIENT: {:?}", forwarding_data.recipient);
match forwarding_data
.recipient
.send(forwarding_data.packet.to_bytes())
@@ -148,7 +168,7 @@ impl PacketProcessor {
{
Ok(()) => (),
Err(e) => {
println!(
warn!(
"failed to write bytes to next mix peer. err = {:?}",
e.to_string()
);
@@ -182,7 +202,6 @@ impl MixNode {
mut socket: tokio::net::TcpStream,
processing_data: Arc<Mutex<ProcessingData>>,
) {
// NOTE: processing_data is copied here!!
let mut buf = [0u8; sphinx::PACKET_SIZE];
// In a loop, read data from the socket and write the data back.
@@ -190,27 +209,33 @@ impl MixNode {
match socket.read(&mut buf).await {
// socket closed
Ok(n) if n == 0 => {
println!("Remote connection closed.");
trace!("Remote connection closed.");
return;
}
Ok(_) => {
let fwd_data = PacketProcessor::process_sphinx_data_packet(
let fwd_data = match PacketProcessor::process_sphinx_data_packet(
buf.as_ref(),
processing_data.clone(),
)
.await
.unwrap();
{
Ok(fwd_data) => fwd_data,
Err(e) => {
warn!("failed to process sphinx packet: {:?}", e);
return;
}
};
PacketProcessor::wait_and_forward(fwd_data).await;
}
Err(e) => {
println!("failed to read from socket; err = {:?}", e);
warn!("failed to read from socket; err = {:?}", e);
return;
}
};
// Write the some data back
if let Err(e) = socket.write_all(b"foomp").await {
println!("failed to write reply to socket; err = {:?}", e);
warn!("failed to write reply to socket; err = {:?}", e);
return;
}
}
@@ -226,8 +251,7 @@ impl MixNode {
let directory_cfg = directory_client::Config {
base_url: self.directory_server.clone(),
};
let pub_key_str =
base64::encode_config(&self.public_key.to_bytes().to_vec(), base64::URL_SAFE);
let pub_key_str = bs58::encode(&self.public_key.to_bytes().to_vec()).into_string();
rt.spawn({
let presence_notifier = presence::Notifier::new(&config);
mixnode/src/node/presence.rs
+1 -1
View File
@@ -1,5 +1,5 @@
use crate::node;
use directory_client::presence::MixNodePresence;
use directory_client::presence::mixnodes::MixNodePresence;
use directory_client::requests::presence_mixnodes_post::PresenceMixNodesPoster;
use directory_client::DirectoryClient;
use log::{debug, error};
nym-client/CHANGELOG.md
-41
View File
@@ -1,41 +0,0 @@
# nym-client Changelog
## 0.3.3
* websocket handling of 'ping', 'pong' and 'close' messages
* websocket not crashing on binary messages
* websocket returning text rather than base64
* restored `nym-client` lib functionality
## 0.3.2
* allows receiving topology with dns hostname instead of an ip address
## 0.3.1
* Version increase for consistency with `nym-mixnode` and `nym-sfw-provider`
## 0.3.0
* cleaned up a lot of internal dependencies
* reporting version to the directory server
* printing warning on trying to bind to "localhost", "127.0.0.1" or "0.0.0.0"
* more informative error messages
* generalised identity keys
* generalised Topology handling
* started slow transition to `log` crate by `nym-client`
* start of 'MixMining'
* start of validator node
## 0.2.0
* removed the `--local` flag
* introduced `--directory` argument to support arbitrary directory servers. Leaving it out will point the node at the "https://directory.nymtech.net" alpha testnet server
* IPv6 support
* client version number is now shown at node start
* directory server location is now shown at node start
* decrease default delays
## 0.1.0 - Initial Release
* The bare minimum set of features required by a Nym Client
nym-client/Cargo.toml
+8 -8
View File
@@ -1,7 +1,7 @@
[package]
build = "build.rs"
name = "nym-client"
version = "0.3.3"
version = "0.4.0-rc.1"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
edition = "2018"
@@ -12,17 +12,15 @@ name = "nym_client"
path = "src/lib.rs"
[dependencies]
base64 = "0.11.0"
bs58 = "0.3.0"
clap = "2.33.0"
curve25519-dalek = "1.2.3"
dirs = "2.0.2"
env_logger = "0.7.1"
dotenv = "0.15.0"
futures = "0.3.1"
hex = "0.4.0"
log = "0.4.8"
log = "0.4"
pem = "0.7.0"
rand = "0.7.2"
rand_distr = "0.2.2"
pretty_env_logger = "0.3"
reqwest = "0.9.22"
serde = { version = "1.0.104", features = ["derive"] }
serde_json = "1.0.44"
@@ -35,12 +33,14 @@ crypto = {path = "../common/crypto"}
directory-client = { path = "../common/clients/directory-client" }
healthcheck = { path = "../common/healthcheck" }
mix-client = { path = "../common/clients/mix-client" }
pemstore = {path = "../common/pemstore"}
provider-client = { path = "../common/clients/provider-client" }
sfw-provider-requests = { path = "../sfw-provider/sfw-provider-requests" }
topology = {path = "../common/topology" }
## will be moved to proper dependencies once released
sphinx = { git = "https://github.com/nymtech/sphinx", rev="1d8cefcb6a0cb8e87d00d89eb1ccf2839e92aa1f" }
sphinx = { git = "https://github.com/nymtech/sphinx", rev="8424f4b0933b4a7f64ae828b2edefc5ba43a9e79" }
# sphinx = { path = "../../sphinx"}
# putting this explicitly below everything and most likely, the next time we look into it, it will already have a proper release
tokio-tungstenite = { git = "https://github.com/snapview/tokio-tungstenite", rev="308d9680c0e59dd1e8651659a775c05df937934e" }
nym-client/src/client/cover_traffic_stream.rs
+53
View File
@@ -0,0 +1,53 @@
use crate::client::mix_traffic::MixMessage;
use crate::client::topology_control::TopologyInnerRef;
use crate::client::LOOP_COVER_AVERAGE_DELAY;
use futures::channel::mpsc;
use log::{error, info, trace, warn};
use sphinx::route::Destination;
use std::time::Duration;
use topology::NymTopology;
pub(crate) async fn start_loop_cover_traffic_stream<T: NymTopology>(
tx: mpsc::UnboundedSender<MixMessage>,
our_info: Destination,
topology_ctrl_ref: TopologyInnerRef<T>,
) {
info!("Starting loop cover traffic stream");
loop {
trace!("next cover message!");
let delay = mix_client::poisson::sample(LOOP_COVER_AVERAGE_DELAY);
let delay_duration = Duration::from_secs_f64(delay);
tokio::time::delay_for(delay_duration).await;
let read_lock = topology_ctrl_ref.read().await;
let topology = match read_lock.topology.as_ref() {
None => {
warn!("No valid topology detected - won't send any loop cover message this time");
continue;
}
Some(topology) => topology,
};
let cover_message = match mix_client::packet::loop_cover_message(
our_info.address,
our_info.identifier,
topology,
) {
Ok(message) => message,
Err(err) => {
error!(
"Somehow we managed to create an invalid cover message - {:?}",
err
);
continue;
}
};
// if this one fails, there's no retrying because it means that either:
// - we run out of memory
// - the receiver channel is closed
// in either case there's no recovery and we can only panic
tx.unbounded_send(MixMessage::new(cover_message.0, cover_message.1))
.unwrap();
}
}
nym-client/src/client/mix_traffic.rs
+37
View File
@@ -0,0 +1,37 @@
use futures::channel::mpsc;
use futures::StreamExt;
use log::{debug, error, info, trace};
use sphinx::SphinxPacket;
use std::net::SocketAddr;
pub(crate) struct MixMessage(SocketAddr, SphinxPacket);
impl MixMessage {
pub(crate) fn new(address: SocketAddr, packet: SphinxPacket) -> Self {
MixMessage(address, packet)
}
}
pub(crate) struct MixTrafficController;
impl MixTrafficController {
pub(crate) async fn run(mut rx: mpsc::UnboundedReceiver<MixMessage>) {
info!("Mix Traffic Controller started!");
let mix_client = mix_client::MixClient::new();
while let Some(mix_message) = rx.next().await {
debug!("Got a mix_message for {:?}", mix_message.0);
let send_res = mix_client.send(mix_message.1, mix_message.0).await;
match send_res {
Ok(_) => {
trace!("sent a mix message");
}
// TODO: should there be some kind of threshold of failed messages
// that if reached, the application blows?
Err(e) => error!(
"We failed to send the message to {} :( - {:?}",
mix_message.0, e
),
};
}
}
}
nym-client/src/client/mod.rs
+249
View File
@@ -0,0 +1,249 @@
use crate::client::mix_traffic::MixTrafficController;
use crate::client::received_buffer::ReceivedMessagesBuffer;
use crate::client::topology_control::TopologyInnerRef;
use crate::sockets::tcp;
use crate::sockets::ws;
use crypto::identity::{MixnetIdentityKeyPair, MixnetIdentityPrivateKey, MixnetIdentityPublicKey};
use directory_client::presence::Topology;
use futures::channel::mpsc;
use futures::join;
use log::*;
use sfw_provider_requests::AuthToken;
use sphinx::route::Destination;
use std::marker::PhantomData;
use std::net::SocketAddr;
use tokio::runtime::Runtime;
use topology::NymTopology;
mod cover_traffic_stream;
mod mix_traffic;
mod provider_poller;
mod real_traffic_stream;
pub mod received_buffer;
pub mod topology_control;
// TODO: all of those constants should probably be moved to config file
const LOOP_COVER_AVERAGE_DELAY: f64 = 0.5;
// seconds
const MESSAGE_SENDING_AVERAGE_DELAY: f64 = 0.5;
// seconds;
const FETCH_MESSAGES_DELAY: f64 = 1.0; // seconds;
const TOPOLOGY_REFRESH_RATE: f64 = 10.0; // seconds
pub enum SocketType {
TCP,
WebSocket,
None,
}
pub struct NymClient<IDPair, Priv, Pub>
where
IDPair: MixnetIdentityKeyPair<Priv, Pub> + Send + Sync,
Priv: MixnetIdentityPrivateKey + Send + Sync,
Pub: MixnetIdentityPublicKey + Send + Sync,
{
keypair: IDPair,
// to be used by "send" function or socket, etc
pub input_tx: mpsc::UnboundedSender<InputMessage>,
input_rx: mpsc::UnboundedReceiver<InputMessage>,
socket_listening_address: SocketAddr,
directory: String,
auth_token: Option<AuthToken>,
socket_type: SocketType,
_phantom_private: PhantomData<Priv>,
_phantom_public: PhantomData<Pub>,
}
#[derive(Debug)]
pub struct InputMessage(pub Destination, pub Vec<u8>);
impl<IDPair: 'static, Priv: 'static, Pub: 'static> NymClient<IDPair, Priv, Pub>
where
IDPair: MixnetIdentityKeyPair<Priv, Pub> + Send + Sync,
Priv: MixnetIdentityPrivateKey + Send + Sync,
Pub: MixnetIdentityPublicKey + Send + Sync,
{
pub fn new(
keypair: IDPair,
socket_listening_address: SocketAddr,
directory: String,
auth_token: Option<AuthToken>,
socket_type: SocketType,
) -> Self {
let (input_tx, input_rx) = mpsc::unbounded::<InputMessage>();
NymClient {
keypair,
input_tx,
input_rx,
socket_listening_address,
directory,
auth_token,
socket_type,
_phantom_private: PhantomData,
_phantom_public: PhantomData,
}
}
async fn get_provider_socket_address<T: NymTopology>(
&self,
topology_ctrl_ref: TopologyInnerRef<T>,
) -> SocketAddr {
// this is temporary and assumes there exists only a single provider.
topology_ctrl_ref.read().await.topology.as_ref().unwrap()
.providers()
.first()
.expect("Could not get a provider from the initial network topology, are you using the right directory server?")
.client_listener
}
pub fn start(self) -> Result<(), Box<dyn std::error::Error>> {
info!("Starting nym client");
let mut rt = Runtime::new()?;
// channels for inter-component communication
// mix_tx is the transmitter for any component generating sphinx packets that are to be sent to the mixnet
// they are used by cover traffic stream and real traffic stream
// mix_rx is the receiver used by MixTrafficController that sends the actual traffic
let (mix_tx, mix_rx) = mpsc::unbounded();
// poller_input_tx is the transmitter of messages fetched from the provider - used by ProviderPoller
// poller_input_rx is the receiver for said messages - used by ReceivedMessagesBuffer
let (poller_input_tx, poller_input_rx) = mpsc::unbounded();
// received_messages_buffer_output_tx is the transmitter for *REQUESTS* for messages contained in ReceivedMessagesBuffer - used by sockets
// the requests contain a oneshot channel to send a reply on
// received_messages_buffer_output_rx is the received for the said requests - used by ReceivedMessagesBuffer
let (received_messages_buffer_output_tx, received_messages_buffer_output_rx) =
mpsc::unbounded();
let self_address = self.keypair.public_key().derive_address();
// generate same type of keys we have as our identity
let healthcheck_keys = IDPair::new();
// TODO: when we switch to our graph topology, we need to remember to change 'Topology' type
let topology_controller =
rt.block_on(topology_control::TopologyControl::<Topology, _, _, _>::new(
self.directory.clone(),
TOPOLOGY_REFRESH_RATE,
healthcheck_keys,
));
let provider_client_listener_address =
rt.block_on(self.get_provider_socket_address(topology_controller.get_inner_ref()));
let mut provider_poller = provider_poller::ProviderPoller::new(
poller_input_tx,
provider_client_listener_address,
self_address,
self.auth_token,
);
// registration
if let Err(err) = rt.block_on(provider_poller.perform_initial_registration()) {
panic!("Failed to perform initial registration: {:?}", err);
};
// setup all of futures for the components running on the client
// buffer controlling all messages fetched from provider
// required so that other components would be able to use them (say the websocket)
let received_messages_buffer_controllers_future = rt.spawn(
ReceivedMessagesBuffer::new()
.start_controllers(poller_input_rx, received_messages_buffer_output_rx),
);
// controller for sending sphinx packets to mixnet (either real traffic or cover traffic)
let mix_traffic_future = rt.spawn(MixTrafficController::run(mix_rx));
// future constantly pumping loop cover traffic at some specified average rate
// the pumped traffic goes to the MixTrafficController
let loop_cover_traffic_future =
rt.spawn(cover_traffic_stream::start_loop_cover_traffic_stream(
mix_tx.clone(),
Destination::new(self_address, Default::default()),
topology_controller.get_inner_ref(),
));
// cloning arguments required by OutQueueControl; required due to move
let input_rx = self.input_rx;
let topology_ref = topology_controller.get_inner_ref();
// future constantly pumping traffic at some specified average rate
// if a real message is available on 'input_rx' that might have been received from say
// the websocket, the real message is used, otherwise a loop cover message is generated
// the pumped traffic goes to the MixTrafficController
let out_queue_control_future = rt.spawn(async move {
real_traffic_stream::OutQueueControl::new(
mix_tx,
input_rx,
Destination::new(self_address, Default::default()),
topology_ref,
)
.run_out_queue_control()
.await
});
// future constantly trying to fetch any received messages from the provider
// the received messages are sent to ReceivedMessagesBuffer to be available to rest of the system
let provider_polling_future = rt.spawn(provider_poller.start_provider_polling());
// a temporary workaround for starting socket listener of specified type
// in the future the actual socket handler should start THIS client instead
match self.socket_type {
SocketType::WebSocket => {
rt.spawn(ws::start_websocket(
self.socket_listening_address,
self.input_tx,
received_messages_buffer_output_tx,
self_address,
topology_controller.get_inner_ref(),
));
}
SocketType::TCP => {
rt.spawn(tcp::start_tcpsocket(
self.socket_listening_address,
self.input_tx,
received_messages_buffer_output_tx,
self_address,
topology_controller.get_inner_ref(),
));
}
SocketType::None => (),
}
// future responsible for periodically polling directory server and updating
// the current global view of topology
let topology_refresher_future = rt.spawn(topology_controller.run_refresher());
rt.block_on(async {
let future_results = join!(
received_messages_buffer_controllers_future,
mix_traffic_future,
loop_cover_traffic_future,
out_queue_control_future,
provider_polling_future,
topology_refresher_future,
);
assert!(
future_results.0.is_ok()
&& future_results.1.is_ok()
&& future_results.2.is_ok()
&& future_results.3.is_ok()
&& future_results.4.is_ok()
&& future_results.5.is_ok()
);
});
// this line in theory should never be reached as the runtime should be permanently blocked on traffic senders
error!("The client went kaput...");
Ok(())
}
}
nym-client/src/client/provider_poller.rs
+88
View File
@@ -0,0 +1,88 @@
use crate::client::FETCH_MESSAGES_DELAY;
use futures::channel::mpsc;
use log::{debug, error, info, trace, warn};
use provider_client::ProviderClientError;
use sfw_provider_requests::AuthToken;
use sphinx::route::DestinationAddressBytes;
use std::net::SocketAddr;
use std::time::Duration;
pub(crate) struct ProviderPoller {
provider_client: provider_client::ProviderClient,
poller_tx: mpsc::UnboundedSender<Vec<Vec<u8>>>,
}
impl ProviderPoller {
pub(crate) fn new(
poller_tx: mpsc::UnboundedSender<Vec<Vec<u8>>>,
provider_client_listener_address: SocketAddr,
client_address: DestinationAddressBytes,
auth_token: Option<AuthToken>,
) -> Self {
ProviderPoller {
provider_client: provider_client::ProviderClient::new(
provider_client_listener_address,
client_address,
auth_token,
),
poller_tx,
}
}
// This method is only temporary until registration is moved to `client init`
pub(crate) async fn perform_initial_registration(&mut self) -> Result<(), ProviderClientError> {
debug!("performing initial provider registration");
if !self.provider_client.is_registered() {
let auth_token = match self.provider_client.register().await {
// in this particular case we can ignore this error
Err(ProviderClientError::ClientAlreadyRegisteredError) => return Ok(()),
Err(err) => return Err(err),
Ok(token) => token,
};
self.provider_client.update_token(auth_token)
} else {
warn!("did not perform registration - we were already registered")
}
Ok(())
}
pub(crate) async fn start_provider_polling(self) {
info!("Starting provider poller");
let loop_message = &mix_client::packet::LOOP_COVER_MESSAGE_PAYLOAD.to_vec();
let dummy_message = &sfw_provider_requests::DUMMY_MESSAGE_CONTENT.to_vec();
let delay_duration = Duration::from_secs_f64(FETCH_MESSAGES_DELAY);
let extended_delay_duration = Duration::from_secs_f64(FETCH_MESSAGES_DELAY * 10.0);
loop {
debug!("Polling provider...");
let messages = match self.provider_client.retrieve_messages().await {
Err(err) => {
error!("Failed to query the provider for messages: {:?}, ... Going to wait {:?} before retrying", err, extended_delay_duration);
tokio::time::delay_for(extended_delay_duration).await;
continue;
}
Ok(messages) => messages,
};
let good_messages = messages
.into_iter()
.filter(|message| message != loop_message && message != dummy_message)
.collect();
trace!("Obtained the following messages: {:?}", good_messages);
// if this one fails, there's no retrying because it means that either:
// - we run out of memory
// - the receiver channel is closed
// in either case there's no recovery and we can only panic
self.poller_tx.unbounded_send(good_messages).unwrap();
tokio::time::delay_for(delay_duration).await;
}
}
}
nym-client/src/client/real_traffic_stream.rs
+132
View File
@@ -0,0 +1,132 @@
use crate::client::mix_traffic::MixMessage;
use crate::client::topology_control::TopologyInnerRef;
use crate::client::{InputMessage, MESSAGE_SENDING_AVERAGE_DELAY};
use futures::channel::mpsc;
use futures::task::{Context, Poll};
use futures::{Future, Stream, StreamExt};
use log::{error, info, trace, warn};
use sphinx::route::Destination;
use std::pin::Pin;
use std::time::Duration;
use tokio::time;
use topology::NymTopology;
// have a rather low value for test sake
const AVERAGE_PACKET_DELAY: f64 = 0.1;
pub(crate) struct OutQueueControl<T: NymTopology> {
delay: time::Delay,
mix_tx: mpsc::UnboundedSender<MixMessage>,
input_rx: mpsc::UnboundedReceiver<InputMessage>,
our_info: Destination,
topology_ctrl_ref: TopologyInnerRef<T>,
}
pub(crate) enum StreamMessage {
Cover,
Real(InputMessage),
}
impl<T: NymTopology> Stream for OutQueueControl<T> {
type Item = StreamMessage;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
// it is not yet time to return a message
if Pin::new(&mut self.delay).poll(cx).is_pending() {
return Poll::Pending;
};
// we know it's time to send a message, so let's prepare delay for the next one
// Get the `now` by looking at the current `delay` deadline
let now = self.delay.deadline();
let next_poisson_delay =
Duration::from_secs_f64(mix_client::poisson::sample(MESSAGE_SENDING_AVERAGE_DELAY));
// The next interval value is `next_poisson_delay` after the one that just
// yielded.
let next = now + next_poisson_delay;
self.delay.reset(next);
// decide what kind of message to send
match Pin::new(&mut self.input_rx).poll_next(cx) {
// in the case our real message channel stream was closed, we should also indicate we are closed
// (and whoever is using the stream should panic)
Poll::Ready(None) => Poll::Ready(None),
// if there's an actual message - return it
Poll::Ready(Some(real_message)) => Poll::Ready(Some(StreamMessage::Real(real_message))),
// otherwise construct a dummy one
Poll::Pending => Poll::Ready(Some(StreamMessage::Cover)),
}
}
}
impl<T: NymTopology> OutQueueControl<T> {
pub(crate) fn new(
mix_tx: mpsc::UnboundedSender<MixMessage>,
input_rx: mpsc::UnboundedReceiver<InputMessage>,
our_info: Destination,
topology: TopologyInnerRef<T>,
) -> Self {
let initial_delay = time::delay_for(Duration::from_secs_f64(MESSAGE_SENDING_AVERAGE_DELAY));
OutQueueControl {
delay: initial_delay,
mix_tx,
input_rx,
our_info,
topology_ctrl_ref: topology,
}
}
pub(crate) async fn run_out_queue_control(mut self) {
info!("starting out queue controller");
while let Some(next_message) = self.next().await {
trace!("created new message");
let read_lock = self.topology_ctrl_ref.read().await;
let topology = match read_lock.topology.as_ref() {
None => {
warn!(
"No valid topology detected - won't send any loop cover message this time"
);
continue;
}
Some(topology) => topology,
};
let next_packet = match next_message {
StreamMessage::Cover => mix_client::packet::loop_cover_message(
self.our_info.address,
self.our_info.identifier,
topology,
),
StreamMessage::Real(real_message) => mix_client::packet::encapsulate_message(
real_message.0,
real_message.1,
topology,
AVERAGE_PACKET_DELAY,
),
};
let next_packet = match next_packet {
Ok(message) => message,
Err(err) => {
error!(
"Somehow we managed to create an invalid traffic message - {:?}",
err
);
continue;
}
};
// if this one fails, there's no retrying because it means that either:
// - we run out of memory
// - the receiver channel is closed
// in either case there's no recovery and we can only panic
self.mix_tx
.unbounded_send(MixMessage::new(next_packet.0, next_packet.1))
.unwrap();
}
}
}
nym-client/src/client/received_buffer.rs
+89
View File
@@ -0,0 +1,89 @@
use futures::channel::{mpsc, oneshot};
use futures::lock::Mutex as FMutex;
use futures::StreamExt;
use log::{error, info, trace};
use std::sync::Arc;
pub type BufferResponse = oneshot::Sender<Vec<Vec<u8>>>;
pub(crate) struct ReceivedMessagesBuffer {
inner: Arc<FMutex<Inner>>,
}
impl ReceivedMessagesBuffer {
pub(crate) fn new() -> Self {
ReceivedMessagesBuffer {
inner: Arc::new(FMutex::new(Inner::new())),
}
}
pub(crate) async fn start_controllers(
self,
poller_rx: mpsc::UnboundedReceiver<Vec<Vec<u8>>>, // to receive new messages
query_receiver: mpsc::UnboundedReceiver<BufferResponse>, // to receive requests to acquire all stored messages
) {
let input_controller_future = tokio::spawn(Self::run_poller_input_controller(
self.inner.clone(),
poller_rx,
));
let output_controller_future = tokio::spawn(Self::run_query_output_controller(
self.inner,
query_receiver,
));
futures::future::select(input_controller_future, output_controller_future).await;
panic!("One of the received buffer controllers failed!")
}
pub(crate) async fn run_poller_input_controller(
buf: Arc<FMutex<Inner>>,
mut poller_rx: mpsc::UnboundedReceiver<Vec<Vec<u8>>>,
) {
info!("Started Received Messages Buffer Input Controller");
while let Some(new_messages) = poller_rx.next().await {
Inner::add_new_messages(&*buf, new_messages).await;
}
}
pub(crate) async fn run_query_output_controller(
buf: Arc<FMutex<Inner>>,
mut query_receiver: mpsc::UnboundedReceiver<BufferResponse>,
) {
info!("Started Received Messages Buffer Output Controller");
while let Some(request) = query_receiver.next().await {
let messages = Inner::acquire_and_empty(&*buf).await;
if let Err(failed_messages) = request.send(messages) {
error!(
"Failed to send the messages to the requester. Adding them back to the buffer"
);
Inner::add_new_messages(&*buf, failed_messages).await;
}
}
}
}
pub(crate) struct Inner {
messages: Vec<Vec<u8>>,
}
impl Inner {
fn new() -> Self {
Inner {
messages: Vec::new(),
}
}
async fn add_new_messages(buf: &FMutex<Self>, msgs: Vec<Vec<u8>>) {
trace!("Adding new messages to the buffer! {:?}", msgs);
let mut unlocked = buf.lock().await;
unlocked.messages.extend(msgs);
}
async fn acquire_and_empty(buf: &FMutex<Self>) -> Vec<Vec<u8>> {
trace!("Emptying the buffer and returning all messages");
let mut unlocked = buf.lock().await;
std::mem::replace(&mut unlocked.messages, Vec::new())
}
}
nym-client/src/client/topology_control.rs
+148
View File
@@ -0,0 +1,148 @@
use crate::built_info;
use crypto::identity::{MixnetIdentityKeyPair, MixnetIdentityPrivateKey, MixnetIdentityPublicKey};
use healthcheck::HealthChecker;
use log::{error, info, trace, warn};
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::RwLock as FRwLock;
use topology::NymTopology;
const NODE_HEALTH_THRESHOLD: f64 = 0.0;
// auxiliary type for ease of use
pub type TopologyInnerRef<T> = Arc<FRwLock<Inner<T>>>;
pub(crate) struct TopologyControl<T, IDPair, Priv, Pub>
where
T: NymTopology,
IDPair: MixnetIdentityKeyPair<Priv, Pub>,
Priv: MixnetIdentityPrivateKey,
Pub: MixnetIdentityPublicKey,
{
directory_server: String,
inner: Arc<FRwLock<Inner<T>>>,
health_checker: HealthChecker<IDPair, Priv, Pub>,
refresh_rate: f64,
}
#[derive(Debug)]
enum TopologyError {
HealthCheckError,
NoValidPathsError,
}
impl<T, IDPair, Priv, Pub> TopologyControl<T, IDPair, Priv, Pub>
where
T: NymTopology,
IDPair: MixnetIdentityKeyPair<Priv, Pub>,
Priv: MixnetIdentityPrivateKey,
Pub: MixnetIdentityPublicKey,
{
pub(crate) async fn new(
directory_server: String,
refresh_rate: f64,
identity_keypair: IDPair,
) -> Self {
// topology control run a healthcheck to determine healthy-ish nodes:
// this is a temporary solution as the healthcheck will eventually be moved to validators
let healthcheck_config = healthcheck::config::HealthCheck {
directory_server: directory_server.clone(),
// those are literally irrelevant when running single check
interval: 100000.0,
resolution_timeout: 5.0,
num_test_packets: 2,
};
let health_checker = healthcheck::HealthChecker::new(healthcheck_config, identity_keypair);
let mut topology_control = TopologyControl {
directory_server,
refresh_rate,
inner: Arc::new(FRwLock::new(Inner::new(None))),
health_checker,
};
// best effort approach to try to get a valid topology after call to 'new'
let initial_topology = match topology_control.get_current_compatible_topology().await {
Ok(topology) => Some(topology),
Err(err) => {
error!("Initial topology is invalid - {:?}. Right now it will be impossible to send any packets through the mixnet!", err);
None
}
};
topology_control
.update_global_topology(initial_topology)
.await;
topology_control
}
async fn get_current_compatible_topology(&self) -> Result<T, TopologyError> {
let full_topology = T::new(self.directory_server.clone());
let healthcheck_result = self.health_checker.do_check().await;
let healthcheck_scores = match healthcheck_result {
Err(err) => {
error!("Error while performing the healthcheck: {:?}", err);
return Err(TopologyError::HealthCheckError);
}
Ok(scores) => scores,
};
let healthy_topology =
healthcheck_scores.filter_topology_by_score(&full_topology, NODE_HEALTH_THRESHOLD);
let versioned_healthy_topology = healthy_topology.filter_node_versions(
built_info::PKG_VERSION,
built_info::PKG_VERSION,
built_info::PKG_VERSION,
);
// make sure you can still send a packet through the network:
if !versioned_healthy_topology.can_construct_path_through() {
return Err(TopologyError::NoValidPathsError);
}
Ok(versioned_healthy_topology)
}
pub(crate) fn get_inner_ref(&self) -> Arc<FRwLock<Inner<T>>> {
self.inner.clone()
}
async fn update_global_topology(&mut self, new_topology: Option<T>) {
// acquire write lock
let mut write_lock = self.inner.write().await;
write_lock.topology = new_topology;
}
pub(crate) async fn run_refresher(mut self) {
info!("Starting topology refresher");
let delay_duration = Duration::from_secs_f64(self.refresh_rate);
loop {
trace!("Refreshing the topology");
let new_topology_res = self.get_current_compatible_topology().await;
let new_topology = match new_topology_res {
Ok(topology) => Some(topology),
Err(err) => {
warn!("the obtained topology seems to be invalid - {:?}, it will be impossible to send packets through", err);
None
}
};
self.update_global_topology(new_topology).await;
tokio::time::delay_for(delay_duration).await;
}
}
}
pub struct Inner<T: NymTopology> {
pub topology: Option<T>,
}
impl<T: NymTopology> Inner<T> {
fn new(topology: Option<T>) -> Self {
Inner { topology }
}
}
nym-client/src/clients/mod.rs
-387
View File
@@ -1,387 +0,0 @@
use crate::built_info;
use crate::sockets::tcp;
use crate::sockets::ws;
use crate::utils;
use directory_client::presence::Topology;
use futures::channel::{mpsc, oneshot};
use futures::join;
use futures::lock::Mutex as FMutex;
use futures::select;
use futures::{SinkExt, StreamExt};
use log::*;
use sfw_provider_requests::AuthToken;
use sphinx::route::{Destination, DestinationAddressBytes};
use sphinx::SphinxPacket;
use std::io;
use std::io::prelude::*;
use std::net::SocketAddr;
use std::sync::Arc;
use std::time::Duration;
use tokio::runtime::Runtime;
use topology::NymTopology;
const LOOP_COVER_AVERAGE_DELAY: f64 = 0.5;
// seconds
const MESSAGE_SENDING_AVERAGE_DELAY: f64 = 0.5;
// seconds;
const FETCH_MESSAGES_DELAY: f64 = 1.0; // seconds;
// provider-poller sends polls service provider; receives messages
// provider-poller sends (TX) to ReceivedBufferController (RX)
// ReceivedBufferController sends (TX) to ... ??Client??
// outQueueController sends (TX) to TrafficStreamController (RX)
// TrafficStreamController sends messages to mixnet
// ... ??Client?? sends (TX) to outQueueController (RX)
// Loop cover traffic stream just sends messages to mixnet without any channel communication
struct MixMessage(SocketAddr, SphinxPacket);
struct MixTrafficController;
impl MixTrafficController {
// this was way more difficult to implement than what this code may suggest...
async fn run(mut rx: mpsc::UnboundedReceiver<MixMessage>) {
let mix_client = mix_client::MixClient::new();
while let Some(mix_message) = rx.next().await {
info!(
"[MIX TRAFFIC CONTROL] - got a mix_message for {:?}",
mix_message.0
);
let send_res = mix_client.send(mix_message.1, mix_message.0).await;
match send_res {
Ok(_) => {
print!(".");
io::stdout().flush().ok().expect("Could not flush stdout");
}
Err(e) => error!("We failed to send the message :( - {:?}", e),
};
}
}
}
pub type BufferResponse = oneshot::Sender<Vec<Vec<u8>>>;
struct ReceivedMessagesBuffer {
messages: Vec<Vec<u8>>,
}
impl ReceivedMessagesBuffer {
fn add_arc_futures_mutex(self) -> Arc<FMutex<Self>> {
Arc::new(FMutex::new(self))
}
fn new() -> Self {
ReceivedMessagesBuffer {
messages: Vec::new(),
}
}
async fn add_new_messages(buf: Arc<FMutex<Self>>, msgs: Vec<Vec<u8>>) {
info!("Adding new messages to the buffer! {:?}", msgs);
let mut unlocked = buf.lock().await;
unlocked.messages.extend(msgs);
}
async fn run_poller_input_controller(
buf: Arc<FMutex<Self>>,
mut poller_rx: mpsc::UnboundedReceiver<Vec<Vec<u8>>>,
) {
while let Some(new_messages) = poller_rx.next().await {
ReceivedMessagesBuffer::add_new_messages(buf.clone(), new_messages).await;
}
}
async fn acquire_and_empty(buf: Arc<FMutex<Self>>) -> Vec<Vec<u8>> {
let mut unlocked = buf.lock().await;
std::mem::replace(&mut unlocked.messages, Vec::new())
}
async fn run_query_output_controller(
buf: Arc<FMutex<Self>>,
mut query_receiver: mpsc::UnboundedReceiver<BufferResponse>,
) {
while let Some(request) = query_receiver.next().await {
let messages = ReceivedMessagesBuffer::acquire_and_empty(buf.clone()).await;
// if this fails, the whole application needs to blow
// because currently only this thread would fail
request.send(messages).unwrap();
}
}
}
pub enum SocketType {
TCP,
WebSocket,
None,
}
pub struct NymClient {
// to be replaced by something else I guess
address: DestinationAddressBytes,
pub input_tx: mpsc::UnboundedSender<InputMessage>,
// to be used by "send" function or socket, etc
input_rx: mpsc::UnboundedReceiver<InputMessage>,
socket_listening_address: SocketAddr,
directory: String,
auth_token: Option<AuthToken>,
socket_type: SocketType,
}
#[derive(Debug)]
pub struct InputMessage(pub Destination, pub Vec<u8>);
impl NymClient {
pub fn new(
address: DestinationAddressBytes,
socket_listening_address: SocketAddr,
directory: String,
auth_token: Option<AuthToken>,
socket_type: SocketType,
) -> Self {
let (input_tx, input_rx) = mpsc::unbounded::<InputMessage>();
NymClient {
address,
input_tx,
input_rx,
socket_listening_address,
directory,
auth_token,
socket_type,
}
}
async fn start_loop_cover_traffic_stream(
mut tx: mpsc::UnboundedSender<MixMessage>,
our_info: Destination,
topology: Topology,
) {
loop {
info!("[LOOP COVER TRAFFIC STREAM] - next cover message!");
let delay = utils::poisson::sample(LOOP_COVER_AVERAGE_DELAY);
let delay_duration = Duration::from_secs_f64(delay);
tokio::time::delay_for(delay_duration).await;
let cover_message =
utils::sphinx::loop_cover_message(our_info.address, our_info.identifier, &topology);
tx.send(MixMessage(cover_message.0, cover_message.1))
.await
.unwrap();
}
}
async fn control_out_queue(
mut mix_tx: mpsc::UnboundedSender<MixMessage>,
mut input_rx: mpsc::UnboundedReceiver<InputMessage>,
our_info: Destination,
topology: Topology,
) {
loop {
info!("[OUT QUEUE] here I will be sending real traffic (or loop cover if nothing is available)");
// TODO: consider replacing select macro with our own proper future definition with polling
let traffic_message = select! {
real_message = input_rx.next() => {
info!("[OUT QUEUE] - we got a real message!");
if real_message.is_none() {
error!("Unexpected 'None' real message!");
std::process::exit(1);
}
let real_message = real_message.unwrap();
println!("real: {:?}", real_message);
utils::sphinx::encapsulate_message(real_message.0, real_message.1, &topology)
},
default => {
info!("[OUT QUEUE] - no real message - going to send extra loop cover");
utils::sphinx::loop_cover_message(our_info.address, our_info.identifier, &topology)
}
};
mix_tx
.send(MixMessage(traffic_message.0, traffic_message.1))
.await
.unwrap();
let delay_duration = Duration::from_secs_f64(MESSAGE_SENDING_AVERAGE_DELAY);
tokio::time::delay_for(delay_duration).await;
}
}
async fn start_provider_polling(
provider_client: provider_client::ProviderClient,
mut poller_tx: mpsc::UnboundedSender<Vec<Vec<u8>>>,
) {
let loop_message = &utils::sphinx::LOOP_COVER_MESSAGE_PAYLOAD.to_vec();
let dummy_message = &sfw_provider_requests::DUMMY_MESSAGE_CONTENT.to_vec();
loop {
let delay_duration = Duration::from_secs_f64(FETCH_MESSAGES_DELAY);
tokio::time::delay_for(delay_duration).await;
info!("[FETCH MSG] - Polling provider...");
let messages = provider_client.retrieve_messages().await.unwrap();
let good_messages = messages
.into_iter()
.filter(|message| message != loop_message && message != dummy_message)
.collect();
// if any of those fails, whole application should blow...
poller_tx.send(good_messages).await.unwrap();
}
}
pub fn start(self) -> Result<(), Box<dyn std::error::Error>> {
let score_threshold = 0.0;
println!("Starting nym client");
let mut rt = Runtime::new()?;
println!("Trying to obtain valid, healthy, topology");
let full_topology = Topology::new(self.directory.clone());
// run a healthcheck to determine healthy-ish nodes:
// this is a temporary solution as the healthcheck will eventually be moved to validators
let healthcheck_config = healthcheck::config::HealthCheck {
directory_server: self.directory.clone(),
// those are literally unrelevant when running single check
interval: 100000.0,
resolution_timeout: 5.0,
num_test_packets: 2,
};
let healthcheck = healthcheck::HealthChecker::new(healthcheck_config);
let healthcheck_result = rt.block_on(healthcheck.do_check());
let healthcheck_scores = match healthcheck_result {
Err(err) => {
error!(
"failed to perform healthcheck to determine healthy topology - {:?}",
err
);
return Err(Box::new(err));
}
Ok(scores) => scores,
};
let healthy_topology =
healthcheck_scores.filter_topology_by_score(&full_topology, score_threshold);
// for time being assume same versioning, i.e. if client is running X.Y.Z,
// we're expecting mixes, providers and coconodes to also be running X.Y.Z
let versioned_healthy_topology = healthy_topology.filter_node_versions(
built_info::PKG_VERSION,
built_info::PKG_VERSION,
built_info::PKG_VERSION,
);
// make sure you can still send a packet through the network:
if !versioned_healthy_topology.can_construct_path_through() {
error!("No valid path exists in the topology");
// TODO: replace panic with proper return type
panic!("No valid path exists in the topology");
}
// this is temporary and assumes there exists only a single provider.
let provider_client_listener_address: SocketAddr = versioned_healthy_topology
.get_mix_provider_nodes()
.first()
.expect("Could not get a provider from the supplied network topology, are you using the right directory server?")
.client_listener;
let mut provider_client = provider_client::ProviderClient::new(
provider_client_listener_address,
self.address,
self.auth_token,
);
// registration
rt.block_on(async {
match self.auth_token {
None => {
let auth_token = provider_client.register().await.unwrap();
provider_client.update_token(auth_token);
info!("Obtained new token! - {:?}", auth_token);
}
Some(token) => println!("Already got the token! - {:?}", token),
}
});
// channels for intercomponent communication
let (mix_tx, mix_rx) = mpsc::unbounded();
let (poller_input_tx, poller_input_rx) = mpsc::unbounded();
let (received_messages_buffer_output_tx, received_messages_buffer_output_rx) =
mpsc::unbounded();
let received_messages_buffer = ReceivedMessagesBuffer::new().add_arc_futures_mutex();
let received_messages_buffer_input_controller_future =
rt.spawn(ReceivedMessagesBuffer::run_poller_input_controller(
received_messages_buffer.clone(),
poller_input_rx,
));
let received_messages_buffer_output_controller_future =
rt.spawn(ReceivedMessagesBuffer::run_query_output_controller(
received_messages_buffer,
received_messages_buffer_output_rx,
));
let mix_traffic_future = rt.spawn(MixTrafficController::run(mix_rx));
let loop_cover_traffic_future = rt.spawn(NymClient::start_loop_cover_traffic_stream(
mix_tx.clone(),
Destination::new(self.address, Default::default()),
versioned_healthy_topology.clone(),
));
let out_queue_control_future = rt.spawn(NymClient::control_out_queue(
mix_tx,
self.input_rx,
Destination::new(self.address, Default::default()),
versioned_healthy_topology.clone(),
));
let provider_polling_future = rt.spawn(NymClient::start_provider_polling(
provider_client,
poller_input_tx,
));
match self.socket_type {
SocketType::WebSocket => {
rt.spawn(ws::start_websocket(
self.socket_listening_address,
self.input_tx,
received_messages_buffer_output_tx,
self.address,
versioned_healthy_topology,
));
}
SocketType::TCP => {
rt.spawn(tcp::start_tcpsocket(
self.socket_listening_address,
self.input_tx,
received_messages_buffer_output_tx,
self.address,
versioned_healthy_topology,
));
}
SocketType::None => (),
}
rt.block_on(async {
let future_results = join!(
received_messages_buffer_input_controller_future,
received_messages_buffer_output_controller_future,
mix_traffic_future,
loop_cover_traffic_future,
out_queue_control_future,
provider_polling_future,
);
assert!(
future_results.0.is_ok()
&& future_results.1.is_ok()
&& future_results.2.is_ok()
&& future_results.3.is_ok()
&& future_results.4.is_ok()
&& future_results.5.is_ok()
);
});
// this line in theory should never be reached as the runtime should be permanently blocked on traffic senders
eprintln!("The client went kaput...");
Ok(())
}
}
nym-client/src/commands/init.rs
+4 -4
View File
@@ -1,18 +1,18 @@
use crate::persistence::pathfinder::Pathfinder;
use crate::persistence::pemstore::PemStore;
use crate::config::persistance::pathfinder::ClientPathfinder;
use clap::ArgMatches;
use crypto::identity::MixnetIdentityKeyPair;
use pemstore::pemstore::PemStore;
pub fn execute(matches: &ArgMatches) {
println!("Initialising client...");
let id = matches.value_of("id").unwrap().to_string(); // required for now
let pathfinder = Pathfinder::new(id);
let pathfinder = ClientPathfinder::new(id);
println!("Writing keypairs to {:?}...", pathfinder.config_dir);
let mix_keys = crypto::identity::DummyMixIdentityKeyPair::new();
let pem_store = PemStore::new(pathfinder);
pem_store.write_identity(mix_keys);
pem_store.write_identity(mix_keys).unwrap();
println!("Client configuration completed.\n\n\n")
}
nym-client/src/commands/tcpsocket.rs
+10 -10
View File
@@ -1,8 +1,8 @@
use crate::clients::{NymClient, SocketType};
use crate::persistence::pemstore;
use crate::client::{NymClient, SocketType};
use crate::config::persistance::pathfinder::ClientPathfinder;
use clap::ArgMatches;
use crypto::identity::{MixnetIdentityKeyPair, MixnetIdentityPublicKey};
use crypto::identity::DummyMixIdentityKeyPair;
use pemstore::pemstore::PemStore;
use std::net::ToSocketAddrs;
pub fn execute(matches: &ArgMatches) {
@@ -26,17 +26,17 @@ pub fn execute(matches: &ArgMatches) {
.next()
.expect("Failed to extract the socket address from the iterator");
let keypair = pemstore::read_mix_identity_keypair_from_disk(id);
// TODO: currently we know we are reading the 'DummyMixIdentityKeyPair', but how to properly assert the type?
let keypair: DummyMixIdentityKeyPair = PemStore::new(ClientPathfinder::new(id))
.read_identity()
.unwrap();
// TODO: reading auth_token from disk (if exists);
println!("Public key: {}", keypair.public_key.to_b64_string());
println!("Public key: {}", keypair.public_key.to_base58_string());
let mut temporary_address = [0u8; 32];
let public_key_bytes = keypair.public_key().to_bytes();
temporary_address.copy_from_slice(&public_key_bytes[..]);
let auth_token = None;
let client = NymClient::new(
temporary_address,
keypair,
socket_address.clone(),
directory_server,
auth_token,
nym-client/src/commands/websocket.rs
+11 -10
View File
@@ -1,8 +1,8 @@
use crate::clients::{NymClient, SocketType};
use crate::persistence::pemstore;
use crate::client::{NymClient, SocketType};
use crate::config::persistance::pathfinder::ClientPathfinder;
use clap::ArgMatches;
use crypto::identity::{MixnetIdentityKeyPair, MixnetIdentityPublicKey};
use crypto::identity::DummyMixIdentityKeyPair;
use pemstore::pemstore::PemStore;
use std::net::ToSocketAddrs;
pub fn execute(matches: &ArgMatches) {
@@ -26,17 +26,18 @@ pub fn execute(matches: &ArgMatches) {
.next()
.expect("Failed to extract the socket address from the iterator");
let keypair = pemstore::read_mix_identity_keypair_from_disk(id);
// TODO: currently we know we are reading the 'DummyMixIdentityKeyPair', but how to properly assert the type?
let keypair: DummyMixIdentityKeyPair = PemStore::new(ClientPathfinder::new(id))
.read_identity()
.unwrap();
// TODO: reading auth_token from disk (if exists);
println!("Public key: {}", keypair.public_key.to_b64_string());
println!("Public key: {}", keypair.public_key.to_base58_string());
let mut temporary_address = [0u8; 32];
let public_key_bytes = keypair.public_key().to_bytes();
temporary_address.copy_from_slice(&public_key_bytes[..]);
let auth_token = None;
let client = NymClient::new(
temporary_address,
keypair,
socket_address,
directory_server,
auth_token,
nym-client/src/config/mod.rs
+1
View File
@@ -0,0 +1 @@
pub mod persistance;
nym-client/src/config/persistance/mod.rs
+1
View File
@@ -0,0 +1 @@
pub mod pathfinder;
nym-client/src/persistence/pathfinder.rs → nym-client/src/config/persistance/pathfinder.rs
+19 -4
View File
@@ -1,21 +1,36 @@
use pemstore::pathfinder::PathFinder;
use std::path::PathBuf;
pub struct Pathfinder {
pub struct ClientPathfinder {
pub config_dir: PathBuf,
pub private_mix_key: PathBuf,
pub public_mix_key: PathBuf,
}
impl Pathfinder {
pub fn new(id: String) -> Pathfinder {
impl ClientPathfinder {
pub fn new(id: String) -> Self {
let os_config_dir = dirs::config_dir().unwrap(); // grabs the OS default config dir
let config_dir = os_config_dir.join("nym").join("clients").join(id);
let private_mix_key = config_dir.join("private.pem");
let public_mix_key = config_dir.join("public.pem");
Pathfinder {
ClientPathfinder {
config_dir,
private_mix_key,
public_mix_key,
}
}
}
impl PathFinder for ClientPathfinder {
fn config_dir(&self) -> PathBuf {
self.config_dir.clone()
}
fn private_identity_key(&self) -> PathBuf {
self.private_mix_key.clone()
}
fn public_identity_key(&self) -> PathBuf {
self.public_mix_key.clone()
}
}
nym-client/src/lib.rs
+3 -7
View File
@@ -1,8 +1,4 @@
#![recursion_limit = "256"]
pub mod built_info;
pub mod clients;
pub mod commands;
pub mod persistence;
pub mod sockets;
pub mod utils;
pub mod client;
pub mod config;
mod sockets;
nym-client/src/main.rs
+4 -6
View File
@@ -1,16 +1,14 @@
#![recursion_limit = "256"]
use clap::{App, Arg, ArgMatches, SubCommand};
pub mod built_info;
pub mod clients;
pub mod client;
mod commands;
mod persistence;
pub mod config;
mod sockets;
pub mod utils;
fn main() {
env_logger::init();
dotenv::dotenv().ok();
pretty_env_logger::init();
let arg_matches = App::new("Nym Client")
.version(built_info::PKG_VERSION)
nym-client/src/persistence/toml.rs
-1
View File
@@ -1 +0,0 @@
// TODO: we can put all the TOML config templating code in here once we get to that.
nym-client/src/sockets/tcp.rs
+62 -53
View File
@@ -1,17 +1,17 @@
use crate::clients::BufferResponse;
use crate::clients::InputMessage;
use directory_client::presence::Topology;
use crate::client::received_buffer::BufferResponse;
use crate::client::topology_control::TopologyInnerRef;
use crate::client::InputMessage;
use futures::channel::{mpsc, oneshot};
use futures::future::FutureExt;
use futures::io::Error;
use futures::SinkExt;
use log::*;
use sphinx::route::{Destination, DestinationAddressBytes};
use std::borrow::Borrow;
use std::convert::TryFrom;
use std::io;
use std::net::SocketAddr;
use std::sync::Arc;
use tokio::prelude::*;
use topology::NymTopology;
const SEND_REQUEST_PREFIX: u8 = 1;
const FETCH_REQUEST_PREFIX: u8 = 2;
@@ -94,54 +94,65 @@ impl ClientRequest {
recipient_address: DestinationAddressBytes,
mut input_tx: mpsc::UnboundedSender<InputMessage>,
) -> ServerResponse {
println!(
"send handle. sending to: {:?}, msg: {:?}",
recipient_address, msg
);
trace!("sending to: {:?}, msg: {:?}", recipient_address, msg);
if msg.len() > sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH {
return ServerResponse::Error {
message: format!(
"too long message. Sent {} bytes while the maximum is {}",
msg.len(),
sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH
)
.to_string(),
};
}
let dummy_surb = [0; 16];
let input_msg = InputMessage(Destination::new(recipient_address, dummy_surb), msg);
println!("ALMOST ABOUT TO SOMEDAY SEND {:?}", input_msg);
input_tx.send(input_msg).await.unwrap();
ServerResponse::Send
}
async fn handle_fetch(mut msg_query: mpsc::UnboundedSender<BufferResponse>) -> ServerResponse {
println!("fetch handle");
trace!("handle_fetch called");
let (res_tx, res_rx) = oneshot::channel();
if msg_query.send(res_tx).await.is_err() {
return ServerResponse::Error {
message: "Server failed to receive messages".to_string(),
};
let e = "Nym-client TCP socket failed to receive messages".to_string();
error!("{}", e);
return ServerResponse::Error { message: e };
}
let messages = res_rx.map(|msg| msg).await;
let messages = match res_rx.map(|msg| msg).await {
Ok(messages) => messages,
Err(e) => {
warn!("Failed to fetch client messages - {:?}", e);
return ServerResponse::Error {
message: format!("Server failed to receive messages").to_string(),
};
}
};
if messages.is_err() {
return ServerResponse::Error {
message: "Server failed to receive messages".to_string(),
};
}
let messages = messages.unwrap();
println!("fetched {} messages", messages.len());
trace!("fetched {} messages", messages.len());
ServerResponse::Fetch { messages }
}
async fn handle_get_clients(topology: &Topology) -> ServerResponse {
println!("get clients handle");
let clients = topology
.mix_provider_nodes
.iter()
.flat_map(|provider| provider.registered_clients.iter())
.map(|client| base64::decode_config(&client.pub_key, base64::URL_SAFE).unwrap()) // TODO: this can potentially throw an error
.collect();
ServerResponse::GetClients { clients }
async fn handle_get_clients<T: NymTopology>(topology: &TopologyInnerRef<T>) -> ServerResponse {
let topology_data = &topology.read().await.topology;
match topology_data {
Some(topology) => {
let clients = topology
.providers()
.iter()
.flat_map(|provider| provider.registered_clients.iter())
.filter_map(|client| bs58::decode(&client.pub_key).into_vec().ok())
.collect();
ServerResponse::GetClients { clients }
}
None => ServerResponse::Error {
message: "Invalid network topology".to_string(),
},
}
}
async fn handle_own_details(self_address_bytes: DestinationAddressBytes) -> ServerResponse {
println!("own details handle");
ServerResponse::OwnDetails {
address: self_address_bytes.to_vec(),
}
@@ -189,8 +200,8 @@ fn encode_fetched_messages(messages: Vec<Vec<u8>>) -> Vec<u8> {
}
fn encode_list_of_clients(clients: Vec<Vec<u8>>) -> Vec<u8> {
println!("clients: {:?}", clients);
// we can just concat all clients since all of them got to be 32 bytes long
debug!("client: {:?}", clients);
// we can just concat all client since all of them got to be 32 bytes long
// (if not, then we have bigger problem somewhere up the line)
// converts [[1,2,3],[4,5,6],...] into [1,2,3,4,5,6,...]
@@ -203,9 +214,9 @@ impl ServerResponse {
}
}
async fn handle_connection(
async fn handle_connection<T: NymTopology>(
data: &[u8],
request_handling_data: RequestHandlingData,
request_handling_data: RequestHandlingData<T>,
) -> Result<ServerResponse, TCPSocketError> {
let request = ClientRequest::try_from(data)?;
let response = match request {
@@ -218,7 +229,7 @@ async fn handle_connection(
}
ClientRequest::Fetch => ClientRequest::handle_fetch(request_handling_data.msg_query).await,
ClientRequest::GetClients => {
ClientRequest::handle_get_clients(request_handling_data.topology.borrow()).await
ClientRequest::handle_get_clients(&request_handling_data.topology).await
}
ClientRequest::OwnDetails => {
ClientRequest::handle_own_details(request_handling_data.self_address).await
@@ -228,26 +239,24 @@ async fn handle_connection(
Ok(response)
}
struct RequestHandlingData {
struct RequestHandlingData<T: NymTopology> {
msg_input: mpsc::UnboundedSender<InputMessage>,
msg_query: mpsc::UnboundedSender<BufferResponse>,
self_address: DestinationAddressBytes,
topology: Arc<Topology>,
topology: TopologyInnerRef<T>,
}
async fn accept_connection(
async fn accept_connection<T: 'static + NymTopology>(
mut socket: tokio::net::TcpStream,
msg_input: mpsc::UnboundedSender<InputMessage>,
msg_query: mpsc::UnboundedSender<BufferResponse>,
self_address: DestinationAddressBytes,
topology: Topology,
topology: TopologyInnerRef<T>,
) {
let address = socket
.peer_addr()
.expect("connected streams should have a peer address");
println!("Peer address: {}", address);
let topology = Arc::new(topology);
debug!("Peer address: {}", address);
let mut buf = [0u8; 2048];
@@ -258,7 +267,7 @@ async fn accept_connection(
let response = match socket.read(&mut buf).await {
// socket closed
Ok(n) if n == 0 => {
println!("Remote connection closed.");
trace!("Remote connection closed.");
return;
}
Ok(n) => {
@@ -274,25 +283,25 @@ async fn accept_connection(
}
}
Err(e) => {
eprintln!("failed to read from socket; err = {:?}", e);
warn!("failed to read from socket; err = {:?}", e);
return;
}
};
let response_vec: Vec<u8> = response.into();
if let Err(e) = socket.write_all(&response_vec).await {
eprintln!("failed to write reply to socket; err = {:?}", e);
warn!("failed to write reply to socket; err = {:?}", e);
return;
}
}
}
pub async fn start_tcpsocket(
pub async fn start_tcpsocket<T: 'static + NymTopology>(
address: SocketAddr,
message_tx: mpsc::UnboundedSender<InputMessage>,
received_messages_query_tx: mpsc::UnboundedSender<BufferResponse>,
self_address: DestinationAddressBytes,
topology: Topology,
topology: TopologyInnerRef<T>,
) -> Result<(), TCPSocketError> {
let mut listener = tokio::net::TcpListener::bind(address).await?;
@@ -308,6 +317,6 @@ pub async fn start_tcpsocket(
));
}
eprintln!("The tcpsocket went kaput...");
error!("The tcpsocket went kaput...");
Ok(())
}
nym-client/src/sockets/ws.rs
+47 -46
View File
@@ -1,6 +1,6 @@
use crate::clients::BufferResponse;
use crate::clients::InputMessage;
use directory_client::presence::Topology;
use crate::client::received_buffer::BufferResponse;
use crate::client::topology_control::TopologyInnerRef;
use crate::client::InputMessage;
use futures::channel::mpsc::{UnboundedReceiver, UnboundedSender};
use futures::channel::{mpsc, oneshot};
use futures::future::FutureExt;
@@ -12,20 +12,21 @@ use sphinx::route::{Destination, DestinationAddressBytes};
use std::convert::TryFrom;
use std::io;
use std::net::SocketAddr;
use topology::NymTopology;
use tungstenite::protocol::frame::coding::CloseCode;
use tungstenite::protocol::{CloseFrame, Message};
struct Connection {
struct Connection<T: NymTopology> {
address: SocketAddr,
msg_input: mpsc::UnboundedSender<InputMessage>,
msg_query: mpsc::UnboundedSender<BufferResponse>,
rx: UnboundedReceiver<Message>,
self_address: DestinationAddressBytes,
topology: Topology,
topology: TopologyInnerRef<T>,
tx: UnboundedSender<Message>,
}
impl Connection {
impl<T: NymTopology> Connection<T> {
async fn handle_text_message(&self, msg: String) -> ServerResponse {
debug!("Handling text message request");
trace!("Content: {:?}", msg.clone());
@@ -48,9 +49,7 @@ impl Connection {
ClientRequest::handle_send(message, recipient_address, self.msg_input.clone()).await
}
ClientRequest::Fetch => ClientRequest::handle_fetch(self.msg_query.clone()).await,
ClientRequest::GetClients => {
ClientRequest::handle_get_clients(self.topology.clone()).await
}
ClientRequest::GetClients => ClientRequest::handle_get_clients(&self.topology).await,
ClientRequest::OwnDetails => ClientRequest::handle_own_details(self.self_address).await,
}
}
@@ -189,32 +188,24 @@ impl ClientRequest {
mut input_tx: mpsc::UnboundedSender<InputMessage>,
) -> ServerResponse {
let message_bytes = msg.into_bytes();
// TODO: wait until 0.4.0 release to replace those constants with newly exposed
// sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH
// we can't do it now for compatibility reasons as most recent sphinx revision
// has breaking changes due to packet format changes
let maximum_plaintext_length = sphinx::constants::PAYLOAD_SIZE
- sphinx::constants::SECURITY_PARAMETER
- sphinx::constants::DESTINATION_ADDRESS_LENGTH
- 1;
if message_bytes.len() > maximum_plaintext_length {
if message_bytes.len() > sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH {
return ServerResponse::Error {
message: format!(
"too long message. Sent {} bytes while the maximum is {}",
"message too long. Sent {} bytes, but the maximum is {}",
message_bytes.len(),
maximum_plaintext_length
sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH
)
.to_string(),
};
}
let address_vec = match base64::decode_config(&recipient_address, base64::URL_SAFE) {
let address_vec = match bs58::decode(&recipient_address).into_vec() {
Err(e) => {
return ServerResponse::Error {
message: e.to_string(),
}
};
}
Ok(hex) => hex,
Ok(bytes) => bytes,
};
if address_vec.len() != 32 {
@@ -243,16 +234,17 @@ impl ClientRequest {
};
}
let messages = res_rx.map(|msg| msg).await;
if messages.is_err() {
warn!("Failed to handle_fetch. messages is an error");
return ServerResponse::Error {
message: "Server failed to receive messages".to_string(),
};
}
let messages = match res_rx.map(|msg| msg).await {
Ok(messages) => messages,
Err(e) => {
warn!("Failed to fetch client messages - {:?}", e);
return ServerResponse::Error {
message: format!("Server failed to receive messages").to_string(),
};
}
};
let messages = messages
.unwrap()
.into_iter()
.map(|message| {
match std::str::from_utf8(&message) {
@@ -269,18 +261,26 @@ impl ClientRequest {
ServerResponse::Fetch { messages }
}
async fn handle_get_clients(topology: Topology) -> ServerResponse {
let clients = topology
.mix_provider_nodes
.into_iter()
.flat_map(|provider| provider.registered_clients.into_iter())
.map(|client| client.pub_key)
.collect();
ServerResponse::GetClients { clients }
async fn handle_get_clients<T: NymTopology>(topology: &TopologyInnerRef<T>) -> ServerResponse {
let topology_data = &topology.read().await.topology;
match topology_data {
Some(topology) => {
let clients = topology
.providers()
.iter()
.flat_map(|provider| provider.registered_clients.iter())
.map(|client| client.pub_key.clone())
.collect();
ServerResponse::GetClients { clients }
}
None => ServerResponse::Error {
message: "Invalid network topology".to_string(),
},
}
}
async fn handle_own_details(self_address_bytes: DestinationAddressBytes) -> ServerResponse {
let self_address = base64::encode_config(&self_address_bytes, base64::URL_SAFE);
let self_address = bs58::encode(&self_address_bytes).into_string();
ServerResponse::OwnDetails {
address: self_address,
}
@@ -306,14 +306,13 @@ impl Into<Message> for ServerResponse {
}
}
async fn accept_connection(
async fn accept_connection<T: 'static + NymTopology>(
stream: tokio::net::TcpStream,
msg_input: mpsc::UnboundedSender<InputMessage>,
msg_query: mpsc::UnboundedSender<BufferResponse>,
self_address: DestinationAddressBytes,
topology: Topology,
topology: TopologyInnerRef<T>,
) {
warn!("accept_connection");
let address = stream
.peer_addr()
.expect("connected streams should have a peer address");
@@ -337,6 +336,8 @@ async fn accept_connection(
msg_query,
self_address,
};
// TODO: make sure this actually doesn't leak memory...
tokio::spawn(conn.handle());
while let Some(message) = ws_stream.next().await {
@@ -385,12 +386,12 @@ async fn accept_connection(
}
}
pub async fn start_websocket(
pub async fn start_websocket<T: 'static + NymTopology>(
address: SocketAddr,
message_tx: mpsc::UnboundedSender<InputMessage>,
received_messages_query_tx: mpsc::UnboundedSender<BufferResponse>,
self_address: DestinationAddressBytes,
topology: Topology,
topology: TopologyInnerRef<T>,
) -> Result<(), WebSocketError> {
let mut listener = tokio::net::TcpListener::bind(address).await?;
@@ -406,6 +407,6 @@ pub async fn start_websocket(
));
}
eprintln!("The websocket went kaput...");
error!("The websocket went kaput...");
Ok(())
}
nym-client/src/utils/bytes.rs
-63
View File
@@ -1,63 +0,0 @@
pub fn zero_pad_to_32(mut bytes: Vec<u8>) -> [u8; 32] {
assert!(bytes.len() <= 32);
if bytes.len() != 32 {
bytes.resize(32, 0);
}
let mut padded_bytes = [0; 32];
padded_bytes.copy_from_slice(&bytes[..]);
padded_bytes
}
#[cfg(test)]
mod zero_padding_to_32_bytes {
use super::*;
#[cfg(test)]
mod with_empty_input {
use super::*;
#[test]
fn it_returns_32_zeros() {
let input = vec![];
let result = zero_pad_to_32(input);
assert_eq!([0u8; 32], result);
}
}
#[cfg(test)]
mod with_all_bytes_set_to_1 {
use super::*;
#[test]
fn it_returns_32_ones() {
let input = vec![1u8; 32];
let result = zero_pad_to_32(input);
assert_eq!([1u8; 32], result);
}
}
#[cfg(test)]
mod with_3_bytes_set {
use super::*;
#[test]
fn it_returns_input_zero_padded_to_32_bytes() {
let input = vec![1u8; 3];
let result = zero_pad_to_32(input);
let expected_content = vec![
1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0,
];
assert_eq!(expected_content, result.to_vec());
}
}
#[cfg(test)]
mod with_oversized_input {
use super::*;
#[test]
#[should_panic]
fn it_panics() {
let input = vec![1u8; 33];
zero_pad_to_32(input);
}
}
}
nym-client/src/utils/mod.rs
-3
View File
@@ -1,3 +0,0 @@
pub mod bytes;
pub mod poisson;
pub mod sphinx;
nym-client/src/utils/poisson.rs
-6
View File
@@ -1,6 +0,0 @@
use rand_distr::{Distribution, Exp};
pub fn sample(average_delay: f64) -> f64 {
let exp = Exp::new(1.0 / average_delay).unwrap();
exp.sample(&mut rand::thread_rng())
}
nym-client/src/utils/sphinx.rs
-40
View File
@@ -1,40 +0,0 @@
use addressing;
use sphinx::route::{Destination, DestinationAddressBytes, SURBIdentifier};
use sphinx::SphinxPacket;
use std::net::SocketAddr;
use topology::NymTopology;
pub const LOOP_COVER_MESSAGE_PAYLOAD: &[u8] = b"The cake is a lie!";
pub fn loop_cover_message<T: NymTopology>(
our_address: DestinationAddressBytes,
surb_id: SURBIdentifier,
topology: &T,
) -> (SocketAddr, SphinxPacket) {
let destination = Destination::new(our_address, surb_id);
encapsulate_message(destination, LOOP_COVER_MESSAGE_PAYLOAD.to_vec(), topology)
}
pub fn encapsulate_message<T: NymTopology>(
recipient: Destination,
message: Vec<u8>,
topology: &T,
) -> (SocketAddr, SphinxPacket) {
let mut providers = topology.get_mix_provider_nodes();
let provider = providers.pop().unwrap().into();
let route = topology.route_to(provider).unwrap();
// Set average packet delay to an arbitrary but at least not super-slow value for testing.
let average_delay = 0.1;
let delays = sphinx::header::delays::generate(route.len(), average_delay);
// build the packet
let packet = sphinx::SphinxPacket::new(message, &route[..], &recipient, &delays).unwrap();
let first_node_address =
addressing::socket_address_from_encoded_bytes(route.first().unwrap().address.to_bytes());
(first_node_address, packet)
}
sfw-provider/CHANGELOG.md
-38
View File
@@ -1,38 +0,0 @@
# nym-sfw-provider Changelog
## 0.3.3
* Version increase for consistency with `nym-client`
## 0.3.2
* Version increase for consistency with `nym-mixnode` and `nym-client`
## 0.3.1
* Fixed crash when directory server goes down
## 0.3.0
* cleaned up a lot of internal dependencies
* reporting version to the directory server
* printing warning on trying to bind to "localhost", "127.0.0.1" or "0.0.0.0"
* more informative error messages
* generalised identity keys
* generalised Topology handling
* started slow transition to `log` crate by `nym-client`
* start of 'MixMining'
* start of validator node
## 0.2.0
* removed the `--local` flag
* introduced `--directory` argument to support arbitrary directory servers. Leaving it out will point the node at the "https://directory.nymtech.net" alpha testnet server
* IPv6 support
* provider version number is now shown at node start
* directory server location is now shown at node start
## 0.1.0 - Initial Release
* The bare minimum set of features required by a Nym Store and Forward Provider
sfw-provider/Cargo.toml
+6 -5
View File
@@ -1,19 +1,20 @@
[package]
build = "build.rs"
name = "nym-sfw-provider"
version = "0.3.3"
version = "0.4.0-rc.1"
authors = ["Dave Hrycyszyn <futurechimp@users.noreply.github.com>", "Jędrzej Stuczyński <andrew@nymtech.net>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
base64 = "0.11.0"
bs58 = "0.3.0"
clap = "2.33.0"
curve25519-dalek = "1.2.3"
hex = "0.4.0"
dotenv = "0.15.0"
futures = "0.3.1"
log = "0.4.8"
log = "0.4"
pretty_env_logger = "0.3"
rand = "0.7.2"
tokio = { version = "0.2.4", features = ["full"] }
sha2 = "0.8.0"
@@ -27,7 +28,7 @@ directory-client = { path = "../common/clients/directory-client" }
sfw-provider-requests = { path = "./sfw-provider-requests" }
## will be moved to proper dependencies once released
sphinx = { git = "https://github.com/nymtech/sphinx", rev="1d8cefcb6a0cb8e87d00d89eb1ccf2839e92aa1f" }
sphinx = { git = "https://github.com/nymtech/sphinx", rev="8424f4b0933b4a7f64ae828b2edefc5ba43a9e79" }
[build-dependencies]
built = "0.3.2"
sfw-provider/sfw-provider-requests/Cargo.toml
+1 -1
View File
@@ -7,4 +7,4 @@ edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
sphinx = { git = "https://github.com/nymtech/sphinx", rev="1d8cefcb6a0cb8e87d00d89eb1ccf2839e92aa1f" }
sphinx = { git = "https://github.com/nymtech/sphinx", rev="8424f4b0933b4a7f64ae828b2edefc5ba43a9e79" }
sfw-provider/sfw-provider-requests/src/responses.rs
+2 -2
View File
@@ -45,8 +45,8 @@ fn read_be_u16(input: &mut &[u8]) -> u16 {
u16::from_be_bytes(int_bytes.try_into().unwrap())
}
// TODO: currently this allows for maximum 64kB payload - if we go over that in sphinx,
// we need to update this code.
// TODO: currently this allows for maximum 64kB payload -
// if we go over that in sphinx we need to update this code.
impl ProviderResponse for PullResponse {
// num_msgs || len1 || len2 || ... || msg1 || msg2 || ...
fn to_bytes(&self) -> Vec<u8> {
sfw-provider/src/main.rs
+5 -1
View File
@@ -1,6 +1,7 @@
use crate::provider::ServiceProvider;
use clap::{App, Arg, ArgMatches, SubCommand};
use crypto::identity::MixnetIdentityKeyPair;
use log::error;
use std::net::ToSocketAddrs;
use std::path::PathBuf;
use std::process;
@@ -8,6 +9,9 @@ use std::process;
pub mod provider;
fn main() {
dotenv::dotenv().ok();
pretty_env_logger::init();
let arg_matches = App::new("Nym Service Provider")
.version(built_info::PKG_VERSION)
.author("Nymtech")
@@ -65,7 +69,7 @@ fn main() {
.get_matches();
if let Err(e) = execute(arg_matches) {
println!("{}", e);
error!("{}", e);
process::exit(1);
}
}
sfw-provider/src/provider/client_handling/mod.rs
+10 -6
View File
@@ -3,6 +3,7 @@ use crate::provider::ClientLedger;
use crypto::identity::{DummyMixIdentityPrivateKey, MixnetIdentityPrivateKey};
use futures::lock::Mutex as FMutex;
use hmac::{Hmac, Mac};
use log::*;
use sfw_provider_requests::requests::{
ProviderRequestError, ProviderRequests, PullRequest, RegisterRequest,
};
@@ -82,7 +83,7 @@ impl ClientRequestProcessor {
processing_data: Arc<ClientProcessingData>,
) -> Result<Vec<u8>, ClientProcessingError> {
let client_request = ProviderRequests::from_bytes(&data)?;
println!("Received the following request: {:?}", client_request);
trace!("Received the following request: {:?}", client_request);
match client_request {
ProviderRequests::Register(req) => Ok(ClientRequestProcessor::register_new_client(
req,
@@ -106,7 +107,6 @@ impl ClientRequestProcessor {
// Wait for https://github.com/nymtech/nym-sfw-provider/issues/19 to resolve.
let unlocked_ledger = processing_data.registered_clients_ledger.lock().await;
println!("Processing pull!");
if unlocked_ledger.has_token(req.auth_token) {
// drop the mutex so that we could do IO without blocking others wanting to get the lock
drop(unlocked_ledger);
@@ -124,7 +124,10 @@ impl ClientRequestProcessor {
req: RegisterRequest,
processing_data: Arc<ClientProcessingData>,
) -> Result<RegisterResponse, ClientProcessingError> {
println!("Processing register new client request!");
debug!(
"Processing register new client request: {:?}",
req.destination_address
);
let mut unlocked_ledger = processing_data.registered_clients_ledger.lock().await;
let auth_token = ClientRequestProcessor::generate_new_auth_token(
@@ -145,14 +148,15 @@ impl ClientRequestProcessor {
client_address: sphinx::route::DestinationAddressBytes,
store_dir: &Path,
) -> io::Result<()> {
let client_dir_name = hex::encode(client_address);
let client_dir_name = bs58::encode(client_address).into_string();
let full_store_dir = store_dir.join(client_dir_name);
std::fs::create_dir_all(full_store_dir)
}
fn generate_new_auth_token(data: Vec<u8>, key: DummyMixIdentityPrivateKey) -> AuthToken {
let mut auth_token_raw =
HmacSha256::new_varkey(&key.to_bytes()).expect("HMAC can take key of any size");
// also note that `new_varkey` doesn't even have an execution branch returning an error
let mut auth_token_raw = HmacSha256::new_varkey(&key.to_bytes())
.expect("HMAC should be able take key of any size");
auth_token_raw.input(&data);
let mut auth_token = [0u8; 32];
auth_token.copy_from_slice(&auth_token_raw.result().code().to_vec());
sfw-provider/src/provider/mix_handling/mod.rs
+18 -6
View File
@@ -1,5 +1,6 @@
use crate::provider::storage::StoreData;
use crypto::identity::DummyMixIdentityPrivateKey;
use log::{error, warn};
use sphinx::{ProcessedPacket, SphinxPacket};
use std::path::PathBuf;
use std::sync::{Arc, RwLock};
@@ -8,11 +9,12 @@ use std::sync::{Arc, RwLock};
// DUPLICATE WITH MIXNODE CODE!!!
#[derive(Debug)]
pub enum MixProcessingError {
SphinxRecoveryError,
ReceivedForwardHopError,
FileIOFailure,
InvalidPayload,
NonMatchingRecipient,
FileIOFailure,
ReceivedForwardHopError,
SphinxRecoveryError,
SphinxProcessingError,
}
impl From<sphinx::ProcessingError> for MixProcessingError {
@@ -60,13 +62,23 @@ impl MixPacketProcessor {
processing_data: &RwLock<MixProcessingData>,
) -> Result<StoreData, MixProcessingError> {
let packet = SphinxPacket::from_bytes(packet_data.to_vec())?;
let read_processing_data = processing_data.read().unwrap();
let read_processing_data = match processing_data.read() {
Ok(guard) => guard,
Err(e) => {
error!("processing data lock was poisoned! - {:?}", e);
std::process::exit(1)
}
};
let (client_address, client_surb_id, payload) =
match packet.process(read_processing_data.secret_key.as_scalar()) {
ProcessedPacket::ProcessedPacketFinalHop(client_address, surb_id, payload) => {
Ok(ProcessedPacket::ProcessedPacketFinalHop(client_address, surb_id, payload)) => {
(client_address, surb_id, payload)
}
_ => return Err(MixProcessingError::ReceivedForwardHopError),
Ok(_) => return Err(MixProcessingError::ReceivedForwardHopError),
Err(e) => {
warn!("Error unwrapping Sphinx packet: {:?}", e);
return Err(MixProcessingError::SphinxProcessingError);
}
};
// TODO: should provider try to be recovering plaintext? this would potentially make client retrieve messages of non-constant length,
sfw-provider/src/provider/mod.rs
+23 -17
View File
@@ -2,9 +2,10 @@ use crate::provider::client_handling::{ClientProcessingData, ClientRequestProces
use crate::provider::mix_handling::{MixPacketProcessor, MixProcessingData};
use crate::provider::storage::ClientStorage;
use crypto::identity::{DummyMixIdentityPrivateKey, DummyMixIdentityPublicKey};
use directory_client::presence::MixProviderClient;
use directory_client::presence::providers::MixProviderClient;
use futures::io::Error;
use futures::lock::Mutex as FMutex;
use log::*;
use sfw_provider_requests::AuthToken;
use sphinx::route::DestinationAddressBytes;
use std::collections::HashMap;
@@ -86,7 +87,7 @@ impl ClientLedger {
fn current_clients(&self) -> Vec<MixProviderClient> {
self.0
.iter()
.map(|(_, v)| base64::encode_config(v, base64::URL_SAFE))
.map(|(_, v)| bs58::encode(v).into_string())
.map(|pub_key| MixProviderClient { pub_key })
.collect()
}
@@ -132,7 +133,7 @@ impl ServiceProvider {
match socket.read(&mut buf).await {
// socket closed
Ok(n) if n == 0 => {
println!("Remote connection closed.");
trace!("Remote connection closed.");
return;
}
Ok(_) => {
@@ -142,28 +143,35 @@ impl ServiceProvider {
) {
Ok(sd) => sd,
Err(e) => {
eprintln!("failed to process sphinx packet; err = {:?}", e);
warn!("failed to process sphinx packet; err = {:?}", e);
return;
}
};
let processing_data_lock = match processing_data.read() {
Ok(guard) => guard,
Err(e) => {
error!("processing data lock was poisoned! - {:?}", e);
std::process::exit(1)
}
};
ClientStorage::store_processed_data(
store_data,
processing_data.read().unwrap().store_dir.as_path(),
processing_data_lock.store_dir.as_path(),
)
.unwrap_or_else(|e| {
eprintln!("failed to store processed sphinx message; err = {:?}", e);
error!("failed to store processed sphinx message; err = {:?}", e);
return;
});
}
Err(e) => {
eprintln!("failed to read from socket; err = {:?}", e);
warn!("failed to read from socket; err = {:?}", e);
return;
}
};
// Write the some data back
if let Err(e) = socket.write_all(b"foomp").await {
eprintln!("failed to write reply to socket; err = {:?}", e);
warn!("failed to write reply to socket; err = {:?}", e);
return;
}
}
@@ -171,10 +179,10 @@ impl ServiceProvider {
async fn send_response(mut socket: tokio::net::TcpStream, data: &[u8]) {
if let Err(e) = socket.write_all(data).await {
eprintln!("failed to write reply to socket; err = {:?}", e)
warn!("failed to write reply to socket; err = {:?}", e)
}
if let Err(e) = socket.shutdown(Shutdown::Write) {
eprintln!("failed to close write part of the socket; err = {:?}", e)
warn!("failed to close write part of the socket; err = {:?}", e)
}
}
@@ -189,7 +197,7 @@ impl ServiceProvider {
let response = match socket.read(&mut buf).await {
// socket closed
Ok(n) if n == 0 => {
println!("Remote connection closed.");
trace!("Remote connection closed.");
Err(())
}
Ok(n) => {
@@ -200,29 +208,27 @@ impl ServiceProvider {
.await
{
Err(e) => {
eprintln!("failed to process client request; err = {:?}", e);
warn!("failed to process client request; err = {:?}", e);
Err(())
}
Ok(res) => Ok(res),
}
}
Err(e) => {
eprintln!("failed to read from socket; err = {:?}", e);
warn!("failed to read from socket; err = {:?}", e);
Err(())
}
};
if let Err(e) = socket.shutdown(Shutdown::Read) {
eprintln!("failed to close read part of the socket; err = {:?}", e)
warn!("failed to close read part of the socket; err = {:?}", e)
}
match response {
Ok(res) => {
println!("should send this response! {:?}", res);
ServiceProvider::send_response(socket, &res).await;
}
_ => {
println!("we failed...");
ServiceProvider::send_response(socket, b"bad foomp").await;
}
}
@@ -308,7 +314,7 @@ impl ServiceProvider {
});
// this line in theory should never be reached as the runtime should be permanently blocked on listeners
eprintln!("The server went kaput...");
error!("The server went kaput...");
Ok(())
}
}
sfw-provider/src/provider/presence.rs
+2 -2
View File
@@ -1,6 +1,6 @@
use crate::provider::ClientLedger;
use crypto::identity::DummyMixIdentityPublicKey;
use directory_client::presence::MixProviderPresence;
use directory_client::presence::providers::MixProviderPresence;
use directory_client::requests::presence_providers_post::PresenceMixProviderPoster;
use directory_client::DirectoryClient;
use futures::lock::Mutex as FMutex;
@@ -34,7 +34,7 @@ impl Notifier {
net_client,
client_listener: client_listener.to_string(),
mixnet_listener: mixnet_listener.to_string(),
pub_key: pub_key.to_b64_string(),
pub_key: pub_key.to_base58_string(),
client_ledger,
}
}
sfw-provider/src/provider/storage/mod.rs
+33 -18
View File
@@ -1,4 +1,5 @@
use crate::provider::{MESSAGE_RETRIEVAL_LIMIT, STORED_MESSAGE_FILENAME_LENGTH};
use log::*;
use rand::Rng;
use sfw_provider_requests::DUMMY_MESSAGE_CONTENT;
use sphinx::route::{DestinationAddressBytes, SURBIdentifier};
@@ -59,10 +60,10 @@ impl ClientStorage {
}
pub fn store_processed_data(store_data: StoreData, store_dir: &Path) -> io::Result<()> {
let client_dir_name = hex::encode(store_data.client_address);
let client_dir_name = bs58::encode(store_data.client_address).into_string();
let full_store_dir = store_dir.join(client_dir_name);
let full_store_path = full_store_dir.join(ClientStorage::generate_random_file_name());
println!(
debug!(
"going to store: {:?} in file: {:?}",
store_data.message, full_store_path
);
@@ -80,27 +81,19 @@ impl ClientStorage {
client_address: DestinationAddressBytes,
store_dir: &Path,
) -> Result<Vec<Vec<u8>>, StoreError> {
let client_dir_name = hex::encode(client_address);
let client_dir_name = bs58::encode(client_address).into_string();
let full_store_dir = store_dir.join(client_dir_name);
println!("going to lookup: {:?}!", full_store_dir);
trace!("going to lookup: {:?}!", full_store_dir);
if !full_store_dir.exists() {
return Err(StoreError::ClientDoesntExistError);
}
let msgs: Vec<_> = std::fs::read_dir(full_store_dir)?
.map(|entry| entry.unwrap())
.filter(|entry| {
let is_file = entry.metadata().unwrap().is_file();
if !is_file {
eprintln!(
"potentially corrupted client inbox! - found a non-file - {:?}",
entry.path()
);
}
is_file
})
.filter_map(|entry| entry.ok())
.filter(|entry| ClientStorage::is_valid_file(entry))
.map(|entry| {
// Not yet sure how to exactly get rid of those unwraps
let content = std::fs::read(entry.path()).unwrap();
ClientStorage::delete_file(entry.path()).unwrap();
content
@@ -109,16 +102,38 @@ impl ClientStorage {
.take(MESSAGE_RETRIEVAL_LIMIT)
.collect();
println!("retrieved the following data: {:?}", msgs);
Ok(msgs)
}
fn is_valid_file(entry: &std::fs::DirEntry) -> bool {
let metadata = match entry.metadata() {
Ok(meta) => meta,
Err(e) => {
error!(
"potentially corrupted client inbox! ({:?} - failed to read its metadata - {:?}",
entry.path(),
e,
);
return false;
}
};
let is_file = metadata.is_file();
if !is_file {
error!(
"potentially corrupted client inbox! - found a non-file - {:?}",
entry.path()
);
}
is_file
}
// TODO: THIS NEEDS A LOCKING MECHANISM!!! (or a db layer on top - basically 'ClientStorage' on steroids)
// TODO 2: This should only be called AFTER we sent the reply. Because if client's connection failed after sending request
// the messages would be deleted but he wouldn't have received them
fn delete_file(path: PathBuf) -> io::Result<()> {
println!("Here {:?} will be deleted!", path);
trace!("Here {:?} will be deleted!", path);
std::fs::remove_file(path) // another argument for db layer -> remove_file is NOT guaranteed to immediately get rid of the file
}
}
validator/CHANGELOG.md
-5
View File
@@ -1,5 +0,0 @@
# nym-validator Changelog
## 0.1.0 - Initial Release
* Initial take on the health checker that is going to be run by the Nym Validator
validator/Cargo.toml
+4 -3
View File
@@ -1,7 +1,7 @@
[package]
build = "build.rs"
name = "nym-validator"
version = "0.1.0"
version = "0.4.0-rc.1"
authors = ["Jedrzej Stuczynski <andrew@nymtech.net>"]
edition = "2018"
@@ -12,14 +12,15 @@ clap = "2.33.0"
# Read notes https://crates.io/crates/dotenv - tl;dr: don't use in production, set environmental variables properly.
dotenv = "0.15.0"
futures = "0.3.1"
log = "0.4.8"
pretty_env_logger = "0.3.1"
log = "0.4"
pretty_env_logger = "0.3"
serde = "1.0.104"
serde_derive = "1.0.104"
tokio = { version = "0.2", features = ["full"] }
toml = "0.5.5"
## internal
crypto = {path = "../common/crypto"}
healthcheck = {path = "../common/healthcheck" }
[build-dependencies]
validator/src/main.rs
+1 -9
View File
@@ -1,7 +1,6 @@
use crate::validator::config::Config;
use crate::validator::Validator;
use clap::{App, Arg, ArgMatches, SubCommand};
use dotenv;
use log::{error, trace};
use std::process;
use toml;
@@ -9,14 +8,7 @@ use toml;
mod validator;
fn main() {
// load environment variables from .env file
if dotenv::dotenv().is_err() {
eprint!("failed to read .env file - the logging is unlikely to work correctly")
}
// if we want to log to file or use different logger, we'd need to replace it here.
// a better alternative, but way more complex would be `slog` crate - we should
// perhaps research it at some point.
dotenv::dotenv().ok();
pretty_env_logger::init();
let arg_matches = App::new("Nym Validator")
validator/src/validator/mod.rs
+20 -4
View File
@@ -1,20 +1,36 @@
use crate::validator::config::Config;
use crypto::identity::{
DummyMixIdentityKeyPair, DummyMixIdentityPrivateKey, DummyMixIdentityPublicKey,
MixnetIdentityKeyPair, MixnetIdentityPrivateKey, MixnetIdentityPublicKey,
};
use healthcheck::HealthChecker;
use log::debug;
use tokio::runtime::Runtime;
pub mod config;
pub struct Validator {
heath_check: HealthChecker,
// allow for a generic validator
pub struct Validator<IDPair, Priv, Pub>
where
IDPair: MixnetIdentityKeyPair<Priv, Pub>,
Priv: MixnetIdentityPrivateKey,
Pub: MixnetIdentityPublicKey,
{
#[allow(dead_code)]
identity_keypair: IDPair,
heath_check: HealthChecker<IDPair, Priv, Pub>,
}
impl Validator {
// but for time being, since it's a dummy one, have it use dummy keys
impl Validator<DummyMixIdentityKeyPair, DummyMixIdentityPrivateKey, DummyMixIdentityPublicKey> {
pub fn new(config: Config) -> Self {
debug!("validator new");
let dummy_keypair = DummyMixIdentityKeyPair::new();
Validator {
heath_check: HealthChecker::new(config.health_check),
identity_keypair: dummy_keypair.clone(),
heath_check: HealthChecker::new(config.health_check, dummy_keypair),
}
}