diff --git a/CHANGELOG.md b/CHANGELOG.md index 8a65d4c2ef..b74e697337 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,8 +1,83 @@ # Changelog -## [Unreleased](https://github.com/nymtech/nym/tree/HEAD) +## [v0.6.0](https://github.com/nymtech/nym/tree/HEAD) -[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.1...HEAD) +[Full Changelog](https://github.com/nymtech/nym/compare/v0.5.0...HEAD) + +This release fixes bugs in v0.5.0. All testnet node operators are advised to upgrade from v0.5.0. + +* fixed premature EOFs on socket connections by using the new multi-TCP client +* fixed a bug causing client and mixnode connection hangs for misconfigured nodes +* by default 'Debug' section of saved configs is now empty and default values are used unless explicitly overridden +* introduced packet chunking allowing clients to send messages of arbitrary length. Note that packet retransmission is not implemented yet, so for longer messages, you might not get anything +* mixnodes now periodically log stats regarding number of packets mixed +* fixed possible client hang ups when sending high rates of traffic +* preventing mixes from starting with same announce-host as an existing node +* fixed overflow multiplication if connection backoff was set to a high value + + +**Closed issues:** + +- Periodic activity summary [\#172](https://github.com/nymtech/nym/issues/172) +- Move contents of 'common/addressing' into 'common/nymsphinx' [\#161](https://github.com/nymtech/nym/issues/161) +- Make builds simpler for node operators [\#114](https://github.com/nymtech/nym/issues/114) +- Chunking in `nym-client` \(receive\) [\#83](https://github.com/nymtech/nym/issues/83) +- Chunking in `nym-client` \(send\) [\#82](https://github.com/nymtech/nym/issues/82) + +**Merged pull requests:** + +- Feature/tcp client connection timeout [\#176](https://github.com/nymtech/nym/pull/176) ([jstuczyn](https://github.com/jstuczyn)) +- Feature/mixing stats logging [\#175](https://github.com/nymtech/nym/pull/175) ([jstuczyn](https://github.com/jstuczyn)) +- Preventing multiplication overflow for reconnection backoff [\#174](https://github.com/nymtech/nym/pull/174) ([jstuczyn](https://github.com/jstuczyn)) +- Feature/non mandatory debug config [\#173](https://github.com/nymtech/nym/pull/173) ([jstuczyn](https://github.com/jstuczyn)) +- Feature/addressing move [\#169](https://github.com/nymtech/nym/pull/169) ([jstuczyn](https://github.com/jstuczyn)) +- Checking if any other node is already announcing the same host [\#168](https://github.com/nymtech/nym/pull/168) ([jstuczyn](https://github.com/jstuczyn)) +- Bugfix/closing tcp client connections on drop [\#167](https://github.com/nymtech/nym/pull/167) ([jstuczyn](https://github.com/jstuczyn)) +- Yielding tokio task upon creating loop/real traffic message [\#166](https://github.com/nymtech/nym/pull/166) ([jstuczyn](https://github.com/jstuczyn)) +- Feature/minor healthchecker improvements [\#165](https://github.com/nymtech/nym/pull/165) ([jstuczyn](https://github.com/jstuczyn)) +- Feature/packet chunking [\#158](https://github.com/nymtech/nym/pull/158) ([jstuczyn](https://github.com/jstuczyn)) + +## [v0.5.0](https://github.com/nymtech/nym/tree/v0.5.0) (2020-03-23) + +[Full Changelog](https://github.com/nymtech/nym/compare/v0.5.0-rc.1...v0.5.0) + +1. Introduced proper configuration options for mixnodes, clients and providers. Everything is initialised with the `init` command that creates a saved config.toml file. To run the binary you now use `nym- run`, for example `nym-mixnode run`. Each flag can be overwritten at any stage with the following priority: run flags, data in config.toml and finally init flags. +2. Made mixnet TCP connections persistent. When sending a Sphinx packet, it should no longer go through the lengthy process of establishing a TCP connection only to immediately tear it down after sending a single packet. This significantly boosts throughput. +3. A lot of work on code clean up and refactoring including some performance fixes. +4. Client now determines its default nym-sfw-provider at startup and should always try to connect to the same one. Note: we still can't reliably run more than a single provider on the network. +5. Logging messages now have timestamps and when running at more aggressive log mode (like debug or even trace) we should no longer be overwhelmed with messages from external crates. +6. Initial compatibility with Windows. Please let us know if you have problems. +7. More work on validator, including initial Tendermint integration in Rust, and the start of the mixmining system. + +**Closed issues:** + +- Introduce timestamps to log messages [\#124](https://github.com/nymtech/nym/issues/124) + +**Merged pull requests:** + +- removing spooky startup warning message [\#155](https://github.com/nymtech/nym/pull/155) ([futurechimp](https://github.com/futurechimp)) +- Some more startup fixes [\#154](https://github.com/nymtech/nym/pull/154) ([futurechimp](https://github.com/futurechimp)) +- Entering runtime context when creating mix traffic controller [\#153](https://github.com/nymtech/nym/pull/153) ([jstuczyn](https://github.com/jstuczyn)) +- Friendlification of startup messages [\#151](https://github.com/nymtech/nym/pull/151) ([futurechimp](https://github.com/futurechimp)) +- Entering runtime context when creating packet forwarder [\#150](https://github.com/nymtech/nym/pull/150) ([jstuczyn](https://github.com/jstuczyn)) +- Feature/add topology to validator [\#149](https://github.com/nymtech/nym/pull/149) ([futurechimp](https://github.com/futurechimp)) +- Making code work on windows machines [\#148](https://github.com/nymtech/nym/pull/148) ([jstuczyn](https://github.com/jstuczyn)) +- validator: adding HTTP interface [\#146](https://github.com/nymtech/nym/pull/146) ([futurechimp](https://github.com/futurechimp)) +- Extracting the log setup [\#145](https://github.com/nymtech/nym/pull/145) ([futurechimp](https://github.com/futurechimp)) +- Feature/optional location in configs [\#144](https://github.com/nymtech/nym/pull/144) ([jstuczyn](https://github.com/jstuczyn)) +- Feature/concurrent connection managers [\#142](https://github.com/nymtech/nym/pull/142) ([jstuczyn](https://github.com/jstuczyn)) +- Defaulting for global 'Info' logging level if not set in .env [\#140](https://github.com/nymtech/nym/pull/140) ([jstuczyn](https://github.com/jstuczyn)) +- Provider not storing loop cover messages [\#139](https://github.com/nymtech/nym/pull/139) ([jstuczyn](https://github.com/jstuczyn)) +- Using log builder to include timestamps + filters [\#138](https://github.com/nymtech/nym/pull/138) ([jstuczyn](https://github.com/jstuczyn)) +- Feature/client ws refactoring [\#134](https://github.com/nymtech/nym/pull/134) ([jstuczyn](https://github.com/jstuczyn)) +- Bugfix/metrics presence delay fix [\#133](https://github.com/nymtech/nym/pull/133) ([jstuczyn](https://github.com/jstuczyn)) +- Removed outdated and redundant sample-configs [\#131](https://github.com/nymtech/nym/pull/131) ([jstuczyn](https://github.com/jstuczyn)) +- If not overridden, 'announce-host' should default to 'host' [\#130](https://github.com/nymtech/nym/pull/130) ([jstuczyn](https://github.com/jstuczyn)) +- Nice to know who we're talking to at startup... [\#129](https://github.com/nymtech/nym/pull/129) ([futurechimp](https://github.com/futurechimp)) + +## [v0.5.0-rc.1](https://github.com/nymtech/nym/tree/v0.5.0-rc.1) (2020-03-06) + +[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.1...v0.5.0-rc.1) **Closed issues:** @@ -48,14 +123,18 @@ ## [v0.4.0](https://github.com/nymtech/nym/tree/v0.4.0) (2020-01-28) -[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0-rc.2...v0.4.0) +[Full Changelog](https://github.com/nymtech/nym/compare/0.4.0-rc.2...v0.4.0) + +Nym 0.4.0 Platform + +In this release, we're taking a lot more care with version numbers, so that we can ensure upgrade compatibility for mixnodes, providers, clients, and validators more easily. -Nym 0.4.0 Platform - -In this release, we're taking a lot more care with version numbers, so that we can ensure upgrade compatibility for mixnodes, providers, clients, and validators more easily. - This release also integrates a health-checker and network topology refresh into the Nym client, so that the client can intelligently choose paths which route around any non-functional or incompatible nodes. +## [0.4.0-rc.2](https://github.com/nymtech/nym/tree/0.4.0-rc.2) (2020-01-28) + +[Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0-rc.2...0.4.0-rc.2) + ## [v0.4.0-rc.2](https://github.com/nymtech/nym/tree/v0.4.0-rc.2) (2020-01-28) [Full Changelog](https://github.com/nymtech/nym/compare/v0.4.0-rc.1...v0.4.0-rc.2) diff --git a/Cargo.lock b/Cargo.lock index 4d44a85f31..158e00ed1a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -17,14 +17,6 @@ dependencies = [ "tokio 0.1.22", ] -[[package]] -name = "addressing" -version = "0.1.0" -dependencies = [ - "log 0.4.8", - "pretty_env_logger", -] - [[package]] name = "adler32" version = "1.0.4" @@ -931,13 +923,13 @@ dependencies = [ name = "healthcheck" version = "0.1.0" dependencies = [ - "addressing", "crypto", "directory-client", "futures 0.3.4", "itertools", "log 0.4.8", - "mix-client", + "multi-tcp-client", + "nymsphinx", "pretty_env_logger", "provider-client", "rand 0.7.3", @@ -1432,8 +1424,8 @@ dependencies = [ name = "mix-client" version = "0.1.0" dependencies = [ - "addressing", "log 0.4.8", + "nymsphinx", "pretty_env_logger", "rand 0.7.3", "rand_distr", @@ -1535,9 +1527,8 @@ dependencies = [ [[package]] name = "nym-client" -version = "0.5.0" +version = "0.6.0" dependencies = [ - "addressing", "bs58", "built", "clap", @@ -1552,6 +1543,7 @@ dependencies = [ "log 0.4.8", "mix-client", "multi-tcp-client", + "nymsphinx", "pem", "pemstore", "pretty_env_logger", @@ -1569,9 +1561,8 @@ dependencies = [ [[package]] name = "nym-mixnode" -version = "0.5.0" +version = "0.6.0" dependencies = [ - "addressing", "bs58", "built", "clap", @@ -1584,17 +1575,19 @@ dependencies = [ "futures 0.3.4", "log 0.4.8", "multi-tcp-client", + "nymsphinx", "pemstore", "pretty_env_logger", "serde", "sphinx", "tempfile", "tokio 0.2.12", + "topology", ] [[package]] name = "nym-sfw-provider" -version = "0.5.0" +version = "0.6.0" dependencies = [ "bs58", "built", @@ -1619,11 +1612,12 @@ dependencies = [ "sphinx", "tempfile", "tokio 0.2.12", + "topology", ] [[package]] name = "nym-validator" -version = "0.5.0" +version = "0.6.0" dependencies = [ "abci", "built", @@ -1647,6 +1641,15 @@ dependencies = [ "topology", ] +[[package]] +name = "nymsphinx" +version = "0.1.0" +dependencies = [ + "log 0.4.8", + "rand 0.7.3", + "sphinx", +] + [[package]] name = "opaque-debug" version = "0.2.3" @@ -2497,7 +2500,7 @@ dependencies = [ [[package]] name = "sphinx" version = "0.1.0" -source = "git+https://github.com/nymtech/sphinx?rev=23f9c89b257ee0936e70afd682e9ed6a62e89eee#23f9c89b257ee0936e70afd682e9ed6a62e89eee" +source = "git+https://github.com/nymtech/sphinx?rev=44d8f2aece5049eaa4fe84b7948758ce82b4b80d#44d8f2aece5049eaa4fe84b7948758ce82b4b80d" dependencies = [ "aes-ctr", "arrayref", @@ -2844,10 +2847,10 @@ dependencies = [ name = "topology" version = "0.1.0" dependencies = [ - "addressing", "bs58", "itertools", "log 0.4.8", + "nymsphinx", "pretty_env_logger", "rand 0.7.3", "serde", diff --git a/Cargo.toml b/Cargo.toml index 071c877bc6..3b3a61111d 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -9,10 +9,10 @@ members = [ "common/clients/multi-tcp-client", "common/clients/provider-client", "common/clients/validator-client", - "common/addressing", "common/config", "common/crypto", "common/healthcheck", + "common/nymsphinx", "common/pemstore", "common/topology", "mixnode", diff --git a/README.md b/README.md index a2a4b7e18b..3acdc006d5 100644 --- a/README.md +++ b/README.md @@ -18,3 +18,7 @@ Platform build instructions are available on [our docs site](https://nymtech.net ### Developing There's a `.env.sample-dev` file provided which you can rename to `.env` if you want convenient logging, backtrace, or other environment variables pre-set. The `.env` file is ignored so you don't need to worry about checking it in. + +### Developer chat + +You can chat to us in [Keybase](https://keybase.io). Download their chat app, then click **Teams -> Join a team**. Type **nymtech.friends** into the team name and hit **continue**. For general chat, hang out in the **#general** channel. Our development takes places in the **#dev** channel. Node operators should be in the **#node-operators** channel. \ No newline at end of file diff --git a/common/addressing/src/lib.rs b/common/addressing/src/lib.rs deleted file mode 100644 index 529d93924d..0000000000 --- a/common/addressing/src/lib.rs +++ /dev/null @@ -1,80 +0,0 @@ -use std::convert::{TryFrom, TryInto}; -use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr}; - -pub enum AddressType { - V4, - V6, -} - -impl Into for AddressType { - fn into(self) -> u8 { - use AddressType::*; - - match self { - V4 => 4, - V6 => 6, - } - } -} - -#[derive(Debug)] -pub enum AddressTypeError { - InvalidPrefixError, -} - -impl TryFrom for AddressType { - type Error = AddressTypeError; - - fn try_from(value: u8) -> Result { - use AddressType::*; - use AddressTypeError::*; - - match value { - 4 => Ok(V4), - 6 => Ok(V6), - _ => Err(InvalidPrefixError), - } - } -} - -/// FLAG || port || octets || zeropad -pub fn encoded_bytes_from_socket_address(address: SocketAddr) -> [u8; 32] { - let port_bytes = address.port().to_be_bytes(); - - let encoded_host: Vec = match address.ip() { - IpAddr::V4(ip) => std::iter::once(AddressType::V4.into()) - .chain(port_bytes.iter().cloned()) - .chain(ip.octets().iter().cloned()) - .chain(std::iter::repeat(0)) - .take(32) - .collect(), - IpAddr::V6(ip) => std::iter::once(AddressType::V6.into()) - .chain(port_bytes.iter().cloned()) - .chain(ip.octets().iter().cloned()) - .chain(std::iter::repeat(0)) - .take(32) - .collect(), - }; - - let mut address_bytes = [0u8; 32]; - address_bytes.copy_from_slice(&encoded_host[..32]); - - address_bytes -} - -pub fn socket_address_from_encoded_bytes(b: [u8; 32]) -> Result { - let address_type: AddressType = b[0].try_into()?; - - let port: u16 = u16::from_be_bytes([b[1], b[2]]); - - let ip = match address_type { - AddressType::V4 => IpAddr::V4(Ipv4Addr::new(b[3], b[4], b[5], b[6])), - AddressType::V6 => { - let mut address_octets = [0u8; 16]; - address_octets.copy_from_slice(&b[3..19]); - IpAddr::V6(Ipv6Addr::from(address_octets)) - } - }; - - Ok(SocketAddr::new(ip, port)) -} diff --git a/common/clients/directory-client/src/presence/mod.rs b/common/clients/directory-client/src/presence/mod.rs index 37d00fcb87..068470d716 100644 --- a/common/clients/directory-client/src/presence/mod.rs +++ b/common/clients/directory-client/src/presence/mod.rs @@ -86,7 +86,7 @@ mod converting_mixnode_presence_into_topology_mixnode { version: "".to_string(), }; - let result: Result = mix_presence.try_into(); + let _result: Result = mix_presence.try_into(); // assert!(result.is_err()) // This fails only for me. Why? // ¯\_(ツ)_/¯ - works on my machine (and travis) } diff --git a/common/clients/mix-client/Cargo.toml b/common/clients/mix-client/Cargo.toml index 40ddb9306d..502be0fb84 100644 --- a/common/clients/mix-client/Cargo.toml +++ b/common/clients/mix-client/Cargo.toml @@ -14,10 +14,10 @@ rand_distr = "0.2.2" tokio = { version = "0.2", features = ["full"] } ## internal -addressing = {path = "../../addressing"} +nymsphinx = {path = "../../nymsphinx"} topology = {path = "../../topology"} ## will be moved to proper dependencies once released -sphinx = { git = "https://github.com/nymtech/sphinx", rev="23f9c89b257ee0936e70afd682e9ed6a62e89eee" } +sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" } # sphinx = { path = "../../../../sphinx"} diff --git a/common/clients/mix-client/src/packet.rs b/common/clients/mix-client/src/packet.rs index 6b25afd6b1..29dee4db3d 100644 --- a/common/clients/mix-client/src/packet.rs +++ b/common/clients/mix-client/src/packet.rs @@ -1,7 +1,7 @@ -use addressing; -use addressing::AddressTypeError; +use nymsphinx::addressing::nodes::{NymNodeRoutingAddress, NymNodeRoutingAddressError}; use sphinx::route::{Destination, DestinationAddressBytes, SURBIdentifier}; use sphinx::SphinxPacket; +use std::convert::TryFrom; use std::net::SocketAddr; use std::time; use topology::{NymTopology, NymTopologyError}; @@ -32,8 +32,8 @@ impl From for SphinxPacketEncapsulationError } } -impl From for SphinxPacketEncapsulationError { - fn from(_: AddressTypeError) -> Self { +impl From for SphinxPacketEncapsulationError { + fn from(_: NymNodeRoutingAddressError) -> Self { use SphinxPacketEncapsulationError::*; InvalidFirstMixAddress } @@ -96,9 +96,9 @@ pub fn encapsulate_message( // we know the mix route must be valid otherwise we would have already returned an error let first_node_address = - addressing::socket_address_from_encoded_bytes(route.first().unwrap().address.to_bytes())?; + NymNodeRoutingAddress::try_from(route.first().unwrap().address.clone())?; - Ok((first_node_address, packet)) + Ok((first_node_address.into(), packet)) } pub fn encapsulate_message_route( @@ -113,7 +113,7 @@ pub fn encapsulate_message_route( let packet = sphinx::SphinxPacket::new(message, &route[..], &recipient, &delays)?; let first_node_address = - addressing::socket_address_from_encoded_bytes(route.first().unwrap().address.to_bytes())?; + NymNodeRoutingAddress::try_from(route.first().unwrap().address.clone())?; - Ok((first_node_address, packet)) + Ok((first_node_address.into(), packet)) } diff --git a/common/clients/multi-tcp-client/src/connection_manager/mod.rs b/common/clients/multi-tcp-client/src/connection_manager/mod.rs index 8a568e6ac9..b4f26b6c59 100644 --- a/common/clients/multi-tcp-client/src/connection_manager/mod.rs +++ b/common/clients/multi-tcp-client/src/connection_manager/mod.rs @@ -1,6 +1,7 @@ use crate::connection_manager::reconnector::ConnectionReconnector; use crate::connection_manager::writer::ConnectionWriter; use futures::channel::{mpsc, oneshot}; +use futures::future::{abortable, AbortHandle}; use futures::task::Poll; use futures::{AsyncWriteExt, StreamExt}; use log::*; @@ -34,22 +35,32 @@ pub(crate) struct ConnectionManager<'a> { state: ConnectionState<'a>, } +impl<'a> Drop for ConnectionManager<'a> { + fn drop(&mut self) { + debug!("Connection manager to {:?} is being dropped", self.address) + } +} + impl<'a> ConnectionManager<'static> { pub(crate) async fn new( address: SocketAddr, reconnection_backoff: Duration, maximum_reconnection_backoff: Duration, + connection_timeout: Duration, ) -> ConnectionManager<'a> { let (conn_tx, conn_rx) = mpsc::unbounded(); - // based on initial connection we will either have a writer or a reconnector - let state = match tokio::net::TcpStream::connect(address).await { - Ok(conn) => ConnectionState::Writing(ConnectionWriter::new(conn)), + // the blocking call here is fine as initially we want to wait the timeout interval (at most) anyway: + let tcp_stream_res = std::net::TcpStream::connect_timeout(&address, connection_timeout); + + let initial_state = match tcp_stream_res { + Ok(stream) => { + let tokio_stream = tokio::net::TcpStream::from_std(stream).unwrap(); + debug!("managed to establish initial connection to {}", address); + ConnectionState::Writing(ConnectionWriter::new(tokio_stream)) + } Err(e) => { - warn!( - "failed to establish initial connection to {} ({}). Going into reconnection mode", - address, e - ); + warn!("failed to establish initial connection to {} within {:?} ({}). Going into reconnection mode", address, connection_timeout, e); ConnectionState::Reconnecting(ConnectionReconnector::new( address, reconnection_backoff, @@ -64,28 +75,33 @@ impl<'a> ConnectionManager<'static> { address, maximum_reconnection_backoff, reconnection_backoff, - state, + state: initial_state, } } - /// consumes Self and returns channel for communication - pub(crate) fn start(mut self, handle: &Handle) -> ConnectionManagerSender { - let sender_clone = self.conn_tx.clone(); - handle.spawn(async move { - while let Some(msg) = self.conn_rx.next().await { - let (msg_content, res_ch) = msg; - let res = self.handle_new_message(msg_content).await; - if let Some(res_ch) = res_ch { - if let Err(e) = res_ch.send(res) { - error!( - "failed to send response on the channel to the caller! - {:?}", - e - ); - } + async fn run(mut self) { + while let Some(msg) = self.conn_rx.next().await { + let (msg_content, res_ch) = msg; + let res = self.handle_new_message(msg_content).await; + if let Some(res_ch) = res_ch { + if let Err(e) = res_ch.send(res) { + error!( + "failed to send response on the channel to the caller! - {:?}", + e + ); } } - }); - sender_clone + } + } + + /// consumes Self and returns channel for communication as well as an `AbortHandle` + pub(crate) fn start_abortable(self, handle: &Handle) -> (ConnectionManagerSender, AbortHandle) { + let sender_clone = self.conn_tx.clone(); + let (abort_fut, abort_handle) = abortable(self.run()); + + handle.spawn(async move { abort_fut.await }); + + (sender_clone, abort_handle) } async fn handle_new_message(&mut self, msg: Vec) -> io::Result<()> { diff --git a/common/clients/multi-tcp-client/src/connection_manager/reconnector.rs b/common/clients/multi-tcp-client/src/connection_manager/reconnector.rs index 7c87113d52..293ae32869 100644 --- a/common/clients/multi-tcp-client/src/connection_manager/reconnector.rs +++ b/common/clients/multi-tcp-client/src/connection_manager/reconnector.rs @@ -60,13 +60,25 @@ impl<'a> Future for ConnectionReconnector<'a> { self.current_retry_attempt += 1; // we failed to re-establish connection - continue exponential backoff + + // according to https://github.com/tokio-rs/tokio/issues/1953 there's an undocumented + // limit of tokio delay of about 2 years. + // let's ensure our delay is always on a sane side of being maximum 1 day. + let maximum_sane_delay = Duration::from_secs(24 * 60 * 60); + let next_delay = std::cmp::min( - self.maximum_reconnection_backoff, - 2_u32.pow(self.current_retry_attempt) * self.initial_reconnection_backoff, + maximum_sane_delay, + self.initial_reconnection_backoff + .checked_mul(2_u32.pow(self.current_retry_attempt)) + .unwrap_or_else(|| self.maximum_reconnection_backoff), ); - self.current_backoff_delay - .reset(tokio::time::Instant::now() + next_delay); + let now = self.current_backoff_delay.deadline(); + // this can't overflow now because next_delay is limited to one day... + // ... well, unless you've been running the system for couple of decades without + // any restarts.... + let next = now + next_delay; + self.current_backoff_delay.reset(next); self.connection = tokio::net::TcpStream::connect(self.address).boxed(); diff --git a/common/clients/multi-tcp-client/src/lib.rs b/common/clients/multi-tcp-client/src/lib.rs index 43e64b11f2..86013adb0f 100644 --- a/common/clients/multi-tcp-client/src/lib.rs +++ b/common/clients/multi-tcp-client/src/lib.rs @@ -1,5 +1,6 @@ use crate::connection_manager::{ConnectionManager, ConnectionManagerSender}; use futures::channel::oneshot; +use futures::future::AbortHandle; use log::*; use std::collections::HashMap; use std::io; @@ -12,25 +13,29 @@ mod connection_manager; pub struct Config { initial_reconnection_backoff: Duration, maximum_reconnection_backoff: Duration, + initial_connection_timeout: Duration, } impl Config { pub fn new( initial_reconnection_backoff: Duration, maximum_reconnection_backoff: Duration, + initial_connection_timeout: Duration, ) -> Self { Config { initial_reconnection_backoff, maximum_reconnection_backoff, + initial_connection_timeout, } } } pub struct Client { runtime_handle: Handle, - connections_managers: HashMap, + connections_managers: HashMap, maximum_reconnection_backoff: Duration, initial_reconnection_backoff: Duration, + initial_connection_timeout: Duration, } impl Client { @@ -43,17 +48,24 @@ impl Client { connections_managers: HashMap::new(), initial_reconnection_backoff: config.maximum_reconnection_backoff, maximum_reconnection_backoff: config.initial_reconnection_backoff, + initial_connection_timeout: config.initial_connection_timeout, } } - async fn start_new_connection_manager(&self, address: SocketAddr) -> ConnectionManagerSender { - ConnectionManager::new( + async fn start_new_connection_manager( + &mut self, + address: SocketAddr, + ) -> (ConnectionManagerSender, AbortHandle) { + let (sender, abort_handle) = ConnectionManager::new( address, self.initial_reconnection_backoff, self.maximum_reconnection_backoff, + self.initial_connection_timeout, ) .await - .start(&self.runtime_handle) + .start_abortable(&self.runtime_handle); + + (sender, abort_handle) } // if wait_for_response is set to true, we will get information about any possible IO errors @@ -66,29 +78,38 @@ impl Client { wait_for_response: bool, ) -> io::Result<()> { if !self.connections_managers.contains_key(&address) { - info!( + debug!( "There is no existing connection to {:?} - it will be established now", address ); - let new_manager_sender = self.start_new_connection_manager(address).await; + let (new_manager_sender, abort_handle) = + self.start_new_connection_manager(address).await; self.connections_managers - .insert(address, new_manager_sender); + .insert(address, (new_manager_sender, abort_handle)); } let manager = self.connections_managers.get_mut(&address).unwrap(); if wait_for_response { let (res_tx, res_rx) = oneshot::channel(); - manager.unbounded_send((message, Some(res_tx))).unwrap(); + manager.0.unbounded_send((message, Some(res_tx))).unwrap(); res_rx.await.unwrap() } else { - manager.unbounded_send((message, None)).unwrap(); + manager.0.unbounded_send((message, None)).unwrap(); Ok(()) } } } +impl Drop for Client { + fn drop(&mut self) { + for (_, abort_handle) in self.connections_managers.values() { + abort_handle.abort() + } + } +} + #[cfg(test)] mod tests { use super::*; @@ -150,7 +171,8 @@ mod tests { let mut rt = tokio::runtime::Runtime::new().unwrap(); let addr = "127.0.0.1:6000".parse().unwrap(); let reconnection_backoff = Duration::from_secs(1); - let client_config = Config::new(reconnection_backoff, 10 * reconnection_backoff); + let timeout = Duration::from_secs(1); + let client_config = Config::new(reconnection_backoff, 10 * reconnection_backoff, timeout); let messages_to_send = vec![[1u8; SERVER_MSG_LEN].to_vec(), [2; SERVER_MSG_LEN].to_vec()]; @@ -209,7 +231,8 @@ mod tests { let mut rt = tokio::runtime::Runtime::new().unwrap(); let addr = "127.0.0.1:6001".parse().unwrap(); let reconnection_backoff = Duration::from_secs(2); - let client_config = Config::new(reconnection_backoff, 10 * reconnection_backoff); + let timeout = Duration::from_secs(1); + let client_config = Config::new(reconnection_backoff, 10 * reconnection_backoff, timeout); let messages_to_send = vec![[1u8; SERVER_MSG_LEN].to_vec(), [2; SERVER_MSG_LEN].to_vec()]; diff --git a/common/clients/provider-client/Cargo.toml b/common/clients/provider-client/Cargo.toml index f5ecafe9cc..042e1162f0 100644 --- a/common/clients/provider-client/Cargo.toml +++ b/common/clients/provider-client/Cargo.toml @@ -16,4 +16,4 @@ tokio = { version = "0.2", features = ["full"] } sfw-provider-requests = { path = "../../../sfw-provider/sfw-provider-requests" } ## will be moved to proper dependencies once released -sphinx = { git = "https://github.com/nymtech/sphinx", rev="23f9c89b257ee0936e70afd682e9ed6a62e89eee" } +sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" } diff --git a/common/crypto/Cargo.toml b/common/crypto/Cargo.toml index f2a29d0ec2..53169abcc8 100644 --- a/common/crypto/Cargo.toml +++ b/common/crypto/Cargo.toml @@ -15,4 +15,4 @@ rand = "0.7.2" rand_os = "0.1" ## will be moved to proper dependencies once released -sphinx = { git = "https://github.com/nymtech/sphinx", rev="23f9c89b257ee0936e70afd682e9ed6a62e89eee" } \ No newline at end of file +sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" } \ No newline at end of file diff --git a/common/healthcheck/Cargo.toml b/common/healthcheck/Cargo.toml index b8f7212792..4f94b73733 100644 --- a/common/healthcheck/Cargo.toml +++ b/common/healthcheck/Cargo.toml @@ -18,16 +18,16 @@ serde_derive = "1.0.104" tokio = { version = "0.2", features = ["full"] } ## internal -addressing = {path = "../addressing" } crypto = { path = "../crypto" } directory-client = { path = "../clients/directory-client" } -mix-client = { path = "../clients/mix-client" } +multi-tcp-client = { path = "../clients/multi-tcp-client" } +nymsphinx = {path = "../nymsphinx" } provider-client = { path = "../clients/provider-client" } sfw-provider-requests = { path = "../../sfw-provider/sfw-provider-requests" } topology = {path = "../topology" } ## will be moved to proper dependencies once released -sphinx = { git = "https://github.com/nymtech/sphinx", rev="23f9c89b257ee0936e70afd682e9ed6a62e89eee" } +sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" } # sphinx = { path = "../../../sphinx"} [dev-dependencies] diff --git a/common/healthcheck/src/lib.rs b/common/healthcheck/src/lib.rs index 04196db3ed..0aee269d13 100644 --- a/common/healthcheck/src/lib.rs +++ b/common/healthcheck/src/lib.rs @@ -36,17 +36,20 @@ impl From for HealthCheckerError { pub struct HealthChecker { num_test_packets: usize, resolution_timeout: Duration, + connection_timeout: Duration, identity_keypair: MixIdentityKeyPair, } impl HealthChecker { pub fn new( resolution_timeout: Duration, + connection_timeout: Duration, num_test_packets: usize, identity_keypair: MixIdentityKeyPair, ) -> Self { HealthChecker { resolution_timeout, + connection_timeout, num_test_packets, identity_keypair, } @@ -62,6 +65,7 @@ impl HealthChecker { current_topology, self.num_test_packets, self.resolution_timeout, + self.connection_timeout, &self.identity_keypair, ) .await; diff --git a/common/healthcheck/src/path_check.rs b/common/healthcheck/src/path_check.rs index 3833dd5c8e..fb68953ee2 100644 --- a/common/healthcheck/src/path_check.rs +++ b/common/healthcheck/src/path_check.rs @@ -1,11 +1,14 @@ use crypto::identity::MixIdentityKeyPair; use itertools::Itertools; use log::{debug, error, info, trace, warn}; -use mix_client::MixClient; +use nymsphinx::addressing::nodes::NymNodeRoutingAddress; use provider_client::{ProviderClient, ProviderClientError}; use sphinx::header::delays::Delay; use sphinx::route::{Destination, Node as SphinxNode}; use std::collections::HashMap; +use std::convert::TryFrom; +use std::net::SocketAddr; +use std::time::Duration; use topology::provider; #[derive(Debug, PartialEq, Clone)] @@ -17,10 +20,7 @@ pub enum PathStatus { pub(crate) struct PathChecker { provider_clients: HashMap<[u8; 32], Option>, - // currently this is an overkill as MixClient is extremely cheap to create, - // however, once we introduce persistent connection between client and layer one mixes, - // this will be extremely helpful to have - layer_one_clients: HashMap<[u8; 32], Option>, + mixnet_client: multi_tcp_client::Client, paths_status: HashMap, PathStatus>, our_destination: Destination, check_id: [u8; 16], @@ -30,6 +30,7 @@ impl PathChecker { pub(crate) async fn new( providers: Vec, identity_keys: &MixIdentityKeyPair, + connection_timeout: Duration, check_id: [u8; 16], ) -> Self { let mut provider_clients = HashMap::new(); @@ -41,12 +42,12 @@ impl PathChecker { ProviderClient::new(provider.client_listener, address.clone(), None); let insertion_result = match provider_client.register().await { Ok(token) => { - debug!("registered at provider {}", provider.pub_key); + debug!("[Healthcheck] registered at provider {}", provider.pub_key); provider_client.update_token(token); provider_clients.insert(provider.get_pub_key_bytes(), Some(provider_client)) } Err(ProviderClientError::ClientAlreadyRegisteredError) => { - info!("We were already registered"); + info!("[Healthcheck] We were already registered"); provider_clients.insert(provider.get_pub_key_bytes(), Some(provider_client)) } Err(err) => { @@ -63,9 +64,16 @@ impl PathChecker { } } + // there's no reconnection allowed - if it fails, then it fails. + let mixnet_client_config = multi_tcp_client::Config::new( + Duration::from_secs(1_000_000_000), + Duration::from_secs(1_000_000_000), + connection_timeout, + ); + PathChecker { provider_clients, - layer_one_clients: HashMap::new(), + mixnet_client: multi_tcp_client::Client::new(mixnet_client_config), our_destination: Destination::new(address, Default::default()), paths_status: HashMap::new(), check_id, @@ -180,7 +188,7 @@ impl PathChecker { return; } - debug!("Checking path: {:?} ({})", path, iteration); + trace!("Checking path: {:?} ({})", path, iteration); let path_identifier = PathChecker::unique_path_key(path, self.check_id, iteration); // check if there is even any point in sending the packet @@ -212,32 +220,12 @@ impl PathChecker { let layer_one_mix = path .first() .expect("We checked the path to contain at least one entry"); - let first_node_key = layer_one_mix.pub_key.to_bytes(); // we generated the bytes data so unwrap is fine - let first_node_address = - addressing::socket_address_from_encoded_bytes(layer_one_mix.address.to_bytes()) - .unwrap(); - - let first_node_client = self - .layer_one_clients - .entry(first_node_key) - .or_insert_with(|| Some(mix_client::MixClient::new())); - - if first_node_client.is_none() { - debug!("we can ignore this path as layer one mix is inaccessible"); - if self - .paths_status - .insert(path_identifier, PathStatus::Unhealthy) - .is_some() - { - panic!("Overwriting path checks!") - } - return; - } - - // we already checked for 'None' case - let first_node_client = first_node_client.as_ref().unwrap(); + let first_node_address: SocketAddr = + NymNodeRoutingAddress::try_from(layer_one_mix.address.clone()) + .unwrap() + .into(); let delays: Vec<_> = path.iter().map(|_| Delay::new_from_nanos(0)).collect(); @@ -251,7 +239,12 @@ impl PathChecker { .unwrap(); debug!("sending test packet to {}", first_node_address); - match first_node_client.send(packet, first_node_address).await { + + match self + .mixnet_client + .send(first_node_address, packet.to_bytes(), true) + .await + { Err(err) => { debug!("failed to send packet to {} - {}", first_node_address, err); if self diff --git a/common/healthcheck/src/result.rs b/common/healthcheck/src/result.rs index 2db51ddc3a..12137fb55d 100644 --- a/common/healthcheck/src/result.rs +++ b/common/healthcheck/src/result.rs @@ -102,6 +102,7 @@ impl HealthCheckResult { topology: &T, iterations: usize, resolution_timeout: Duration, + connection_timeout: Duration, identity_keys: &MixIdentityKeyPair, ) -> Self { // currently healthchecker supports only up to 255 iterations - if we somehow @@ -127,7 +128,8 @@ impl HealthCheckResult { let providers = topology.providers(); - let mut path_checker = PathChecker::new(providers, identity_keys, check_id).await; + let mut path_checker = + PathChecker::new(providers, identity_keys, connection_timeout, check_id).await; for i in 0..iterations { debug!("running healthcheck iteration {} / {}", i + 1, iterations); for path in &all_paths { diff --git a/common/healthcheck/src/score.rs b/common/healthcheck/src/score.rs index 9bdbf74bce..b2e1c69588 100644 --- a/common/healthcheck/src/score.rs +++ b/common/healthcheck/src/score.rs @@ -153,7 +153,7 @@ impl std::fmt::Display for NodeScore { let stringified_key = self.pub_key.to_base58_string(); write!( f, - "({})\t{}/{}\t({}%)\t|| {}\tv{} <{}> - {}", + "({})\t\t{}/{}\t({:.2}%) \t|| {}\tv{} <{}> - {}", self.typ, self.packets_received, self.packets_sent, diff --git a/common/addressing/Cargo.toml b/common/nymsphinx/Cargo.toml similarity index 50% rename from common/addressing/Cargo.toml rename to common/nymsphinx/Cargo.toml index 304a25dbc9..1954f3ae26 100644 --- a/common/addressing/Cargo.toml +++ b/common/nymsphinx/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "addressing" +name = "nymsphinx" version = "0.1.0" authors = ["Jedrzej Stuczynski "] edition = "2018" @@ -7,5 +7,9 @@ edition = "2018" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies] -log = "0.4" -pretty_env_logger = "0.3" \ No newline at end of file +log = "0.4.8" +rand = "0.7.3" + +## will be moved to proper dependencies once released +sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" } + diff --git a/common/nymsphinx/src/addressing/mod.rs b/common/nymsphinx/src/addressing/mod.rs new file mode 100644 index 0000000000..69e3fea510 --- /dev/null +++ b/common/nymsphinx/src/addressing/mod.rs @@ -0,0 +1 @@ +pub mod nodes; diff --git a/common/nymsphinx/src/addressing/nodes.rs b/common/nymsphinx/src/addressing/nodes.rs new file mode 100644 index 0000000000..af0a16e09b --- /dev/null +++ b/common/nymsphinx/src/addressing/nodes.rs @@ -0,0 +1,206 @@ +use sphinx::constants::NODE_ADDRESS_LENGTH; +use sphinx::route::NodeAddressBytes; +use std::convert::{TryFrom, TryInto}; +use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr}; + +/// This module is responsible for encoding and decoding node routing information, so that +/// they could be later put into an appropriate field in a sphinx header. +/// Currently, that routing information is an IP address, but in principle it can be anything +/// for as long as it's going to fit in the field. + +#[derive(Debug)] +pub enum NymNodeRoutingAddressError { + InsufficientNumberOfBytesAvailableError, + InvalidIPVersion, +} + +/// Current representation of Node routing information used in Nym system. +/// At this point of time it is a simple `SocketAddr`. +#[derive(Copy, Clone, PartialEq, Eq, Hash, Debug)] +pub struct NymNodeRoutingAddress(SocketAddr); + +impl NymNodeRoutingAddress { + /// Minimum number of bytes that need to be available to represent self. + /// The value has no upper bound as when converted into bytes, it's always + /// padded with zeroes to be exactly NODE_ADDRESS_LENGTH long. + pub fn bytes_min_len(&self) -> usize { + match self.0 { + SocketAddr::V4(_) => 7, + SocketAddr::V6(_) => 19, + } + } + + /// Converts self into a vector of bytes. + /// Note, this represents a generic bytes vector, not necessarily a NodeAddressBytes + /// and hence is not zero-padded. + pub fn as_bytes(&self) -> Vec { + let port_bytes = self.0.port().to_be_bytes(); + let ip_octets_vec = match self.0.ip() { + IpAddr::V4(ip) => ip.octets().to_vec(), + IpAddr::V6(ip) => ip.octets().to_vec(), + }; + + std::iter::once(self.addr_type_as_u8()) + .chain(port_bytes.iter().cloned()) + .chain(ip_octets_vec.iter().cloned()) + .collect() + } + + /// Tries to recover `Self` from a bytes slice. + /// Does not care if it's zero-padded or not. + pub fn try_from_bytes(b: &[u8]) -> Result { + // the bare minimum to represent `Self` is 7 bytes (for the shorter V4 version) + if b.len() < 7 { + return Err(NymNodeRoutingAddressError::InsufficientNumberOfBytesAvailableError); + } + + let ip_version = b[0]; + let port: u16 = u16::from_be_bytes([b[1], b[2]]); + let ip = match ip_version { + 4 => IpAddr::V4(Ipv4Addr::new(b[3], b[4], b[5], b[6])), + 6 => { + if b.len() < 19 { + return Err( + NymNodeRoutingAddressError::InsufficientNumberOfBytesAvailableError, + ); + } + let mut address_octets = [0u8; 16]; + address_octets.copy_from_slice(&b[3..19]); + IpAddr::V6(Ipv6Addr::from(address_octets)) + } + _ => return Err(NymNodeRoutingAddressError::InvalidIPVersion), + }; + + Ok(Self(SocketAddr::new(ip, port))) + } + + /// Single byte representation of self ip version. + pub fn addr_type_as_u8(&self) -> u8 { + match self.0 { + SocketAddr::V4(_) => 4, + SocketAddr::V6(_) => 6, + } + } +} + +/// Considering `NymNodeRoutingAddress` is equivalent to a `SocketAddr` at this point, +/// it makes perfect sense to allow the bilateral transformation. +impl From for NymNodeRoutingAddress { + fn from(addr: SocketAddr) -> Self { + Self(addr) + } +} + +/// Considering `NymNodeRoutingAddress` is equivalent to a `SocketAddr` at this point, +/// it makes perfect sense to allow the bilateral transformation. +impl Into for NymNodeRoutingAddress { + fn into(self) -> SocketAddr { + self.0 + } +} + +impl TryInto for NymNodeRoutingAddress { + type Error = NymNodeRoutingAddressError; + + /// `NymNodeRoutingAddress` (as a `SocketAddr`) is represented the following way: + /// VersionFlag || port || octets || zeropad + /// VersionFlag is one byte representing whether self is ipv4 or ipv6 address, + /// port is 16bit big endian representation of port value + /// octets is bytes representation of octets making up the ip address of the socket address + /// (either 4 bytes for ipv4 or 16 bytes for ipv6) + /// zeropad is padding of 0 for the `NymNodeRoutingAddress` to be + /// exactly `NODE_ADDRESS_LENGTH` long. + fn try_into(self) -> Result { + // first check if we have enough bytes to represent `self`: + if self.bytes_min_len() > NODE_ADDRESS_LENGTH { + return Err(NymNodeRoutingAddressError::InsufficientNumberOfBytesAvailableError); + } + + let unpadded_address = self.as_bytes(); + let padded_address: Vec<_> = unpadded_address + .into_iter() + .chain(std::iter::repeat(0)) + .take(NODE_ADDRESS_LENGTH) + .collect(); + + let mut node_address_bytes = [0u8; 32]; + node_address_bytes.copy_from_slice(&padded_address); + + Ok(NodeAddressBytes::from_bytes(node_address_bytes)) + } +} + +impl TryFrom for NymNodeRoutingAddress { + type Error = NymNodeRoutingAddressError; + + fn try_from(value: NodeAddressBytes) -> Result { + Self::try_from_bytes(value.as_bytes()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn nym_node_routing_address_can_be_converted_to_and_from_bytes_for_v4_address() { + let address = NymNodeRoutingAddress(SocketAddr::new(IpAddr::from([1, 2, 3, 4]), 42)); + let address_bytes = address.as_bytes(); + assert_eq!( + address, + NymNodeRoutingAddress::try_from_bytes(&address_bytes).unwrap() + ) + } + + #[test] + fn nym_node_routing_address_can_be_converted_to_and_from_bytes_for_v6_address() { + let address = NymNodeRoutingAddress(SocketAddr::new( + IpAddr::from([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]), + 42, + )); + let address_bytes = address.as_bytes(); + assert_eq!( + address, + NymNodeRoutingAddress::try_from_bytes(&address_bytes).unwrap() + ) + } + + #[test] + fn nym_node_routing_address_can_be_converted_to_and_from_bytes_for_empty_v4_address() { + let address = NymNodeRoutingAddress(SocketAddr::new(IpAddr::from([0, 0, 0, 0]), 42)); + let address_bytes = address.as_bytes(); + assert_eq!( + address, + NymNodeRoutingAddress::try_from_bytes(&address_bytes).unwrap() + ) + } + + #[test] + fn nym_node_routing_address_can_be_converted_to_and_from_bytes_for_empty_v6_address() { + let address = NymNodeRoutingAddress(SocketAddr::new( + IpAddr::from([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), + 42, + )); + let address_bytes = address.as_bytes(); + assert_eq!( + address, + NymNodeRoutingAddress::try_from_bytes(&address_bytes).unwrap() + ) + } + + #[test] + fn nym_node_routing_address_can_be_converted_to_and_from_node_address_bytes_with_no_data_loss() + { + let address_v4 = NymNodeRoutingAddress(SocketAddr::new(IpAddr::from([1, 2, 3, 4]), 42)); + let address_v6 = NymNodeRoutingAddress(SocketAddr::new( + IpAddr::from([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]), + 42, + )); + + let node_address1: NodeAddressBytes = address_v4.try_into().unwrap(); + let node_address2: NodeAddressBytes = address_v6.try_into().unwrap(); + + assert_eq!(address_v4, node_address1.try_into().unwrap()); + assert_eq!(address_v6, node_address2.try_into().unwrap()); + } +} diff --git a/common/nymsphinx/src/chunking/fragment.rs b/common/nymsphinx/src/chunking/fragment.rs new file mode 100644 index 0000000000..1e9d175235 --- /dev/null +++ b/common/nymsphinx/src/chunking/fragment.rs @@ -0,0 +1,1143 @@ +use crate::chunking::ChunkingError; +use std::convert::TryInto; + +/// The entire marshaled `Fragment` can never be longer than the maximum length of the plaintext +/// data we can put into a Sphinx packet. +pub const MAXIMUM_FRAGMENT_LENGTH: usize = sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH; + +/// The minimum data overhead required for message fitting into a single `Fragment`. The single byte +/// used to literally indicate "this message is not fragmented". +pub const UNFRAGMENTED_HEADER_LEN: usize = 1; + +/// When the underlying message has to be split into multiple Fragments, but still manages to fit +/// into a single `FragmentSet`, each `FragmentHeader` needs to hold additional information to allow +/// for correct message reconstruction: 4 bytes for set id, 1 byte to represent total number +/// of fragments in the set, 1 byte to represent position of the current fragment in the set +/// and finally an extra byte to indicate the fragment has no links to other sets. +pub const UNLINKED_FRAGMENTED_HEADER_LEN: usize = 7; + +/// Logically almost identical to `UNLINKED_FRAGMENTED_HEADER_LEN`, however, the extra three +/// bytes are due to changing the final byte that used to indicate the `Fragment` is not linked +/// into 4 byte id of either previous or the next set. +/// Note that the linked headers can potentially be used only for very first and very last +/// `Fragment` in a `FragmentSet`. +pub const LINKED_FRAGMENTED_HEADER_LEN: usize = 10; + +/// Maximum size of payload of each fragment is always the maximum amount of plaintext data +/// we can put into a sphinx packet minus length of respective fragment header. +pub const UNFRAGMENTED_PAYLOAD_MAX_LEN: usize = MAXIMUM_FRAGMENT_LENGTH - UNFRAGMENTED_HEADER_LEN; + +/// Maximum size of payload of each fragment is always the maximum amount of plaintext data +/// we can put into a sphinx packet minus length of respective fragment header. +pub const UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN: usize = + MAXIMUM_FRAGMENT_LENGTH - UNLINKED_FRAGMENTED_HEADER_LEN; + +/// Maximum size of payload of each fragment is always the maximum amount of plaintext data +/// we can put into a sphinx packet minus length of respective fragment header. +pub const LINKED_FRAGMENTED_PAYLOAD_MAX_LEN: usize = + MAXIMUM_FRAGMENT_LENGTH - LINKED_FRAGMENTED_HEADER_LEN; + +/// The basic unit of division of underlying bytes message sent through the mix network. +/// Each `Fragment` after being marshaled is guaranteed to fit into a single sphinx packet. +/// The `Fragment` itself consists of part, or whole of, message to be sent as well as additional +/// header used to reconstruct the message after being received. +#[derive(PartialEq, Clone, Debug)] +pub(crate) struct Fragment { + header: FragmentHeader, + payload: Vec, +} + +impl Fragment { + /// Tries to encapsulate provided payload slice and metadata into a `Fragment`. + /// It can fail if payload would not fully fit in a single `Fragment` or some of the metadata + /// is malformed or self-contradictory, for example if current_fragment > total_fragments. + pub(crate) fn try_new_fragmented( + payload: &[u8], + id: i32, + total_fragments: u8, + current_fragment: u8, + previous_fragments_set_id: Option, + next_fragments_set_id: Option, + ) -> Result { + let header = FragmentHeader::try_new_fragmented( + id, + total_fragments, + current_fragment, + previous_fragments_set_id, + next_fragments_set_id, + )?; + + // check for whether payload has expected length, which depend on whether fragment is linked + // and if it's the only one or the last one in the set (then lower bound is removed) + if previous_fragments_set_id.is_some() { + if total_fragments > 1 { + if payload.len() != LINKED_FRAGMENTED_PAYLOAD_MAX_LEN { + return Err(ChunkingError::InvalidPayloadLengthError); + } + } else { + if payload.len() > LINKED_FRAGMENTED_PAYLOAD_MAX_LEN { + return Err(ChunkingError::InvalidPayloadLengthError); + } + } + } else if next_fragments_set_id.is_some() { + if payload.len() != LINKED_FRAGMENTED_PAYLOAD_MAX_LEN { + return Err(ChunkingError::InvalidPayloadLengthError); + } + } else { + if total_fragments != current_fragment { + if payload.len() != UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN { + return Err(ChunkingError::InvalidPayloadLengthError); + } + } else { + if payload.len() > UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN { + return Err(ChunkingError::InvalidPayloadLengthError); + } + } + } + + Ok(Fragment { + header, + payload: payload.to_vec(), + }) + } + + /// The most efficient way of representing underlying message incurring the least data overhead + /// for as long as the message can fit into a single, unfragmented, `Fragment`. + pub(crate) fn try_new_unfragmented(payload: &[u8]) -> Result { + if payload.len() > UNFRAGMENTED_PAYLOAD_MAX_LEN { + Err(ChunkingError::InvalidPayloadLengthError) + } else { + Ok(Fragment { + header: FragmentHeader::new_unfragmented(), + payload: payload.to_vec(), + }) + } + } + + /// Convert this `Fragment` into vector of bytes which can be put into a sphinx packet. + pub(crate) fn into_bytes(self) -> Vec { + self.header + .to_bytes() + .into_iter() + .chain(self.payload.into_iter()) + .collect() + } + + /// Extracts id of this `Fragment`. + pub(crate) fn id(&self) -> i32 { + self.header.id + } + + /// Extracts total number of fragments associated with this particular `Fragment` (belonging to + /// the same `FragmentSet`). + pub(crate) fn total_fragments(&self) -> u8 { + self.header.total_fragments + } + + /// Extracts position of this `Fragment` in a `FragmentSet`. + pub(crate) fn current_fragment(&self) -> u8 { + self.header.current_fragment + } + + /// Extracts information regarding id of pre-linked `FragmentSet` + pub(crate) fn previous_fragments_set_id(&self) -> Option { + self.header.previous_fragments_set_id + } + + /// Extracts information regarding id of post-linked `FragmentSet` + pub(crate) fn next_fragments_set_id(&self) -> Option { + self.header.next_fragments_set_id + } + + /// Consumes `Self` to obtain payload (i.e. part of original message) associated with this + /// `Fragment`. + pub(crate) fn extract_payload(self) -> Vec { + self.payload + } + + /// Tries to recover `Fragment` from slice of bytes extracted from received sphinx packet. + /// It can fail if payload would not fully fit in a single `Fragment` or some of the metadata + /// is malformed or self-contradictory, for example if current_fragment > total_fragments. + pub(crate) fn try_from_bytes(b: &[u8]) -> Result { + let (header, n) = FragmentHeader::try_from_bytes(b)?; + + // determine what's our expected payload size bound and whether the message fits in this + let is_payload_in_range = if header.is_fragmented { + if header.is_linked() { + if header.is_final() { + (b.len() - n) <= LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + } else { + (b.len() - n) == LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + } + } else { + if header.is_final() { + (b.len() - n) <= UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + } else { + (b.len() - n) == UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + } + } + } else { + (b.len() - n) <= UNFRAGMENTED_PAYLOAD_MAX_LEN + }; + + if !is_payload_in_range { + return Err(ChunkingError::MalformedFragmentData); + } + + Ok(Fragment { + header, + payload: b[n..].to_vec(), + }) + } +} + +/// In order to be able to re-assemble fragmented message sent through a mix-network, some +/// metadata is attached alongside the actual message payload. The idea is to include as little +/// of that data as possible due to computationally very costly process of sphinx encapsulation. +/// +/// The generic `FragmentHeader` is represented as follows: +/// IF flag || 31 bit ID || TotalFragments || CurrentFragment || LID flag || 31 bit Linked ID +/// note that LID is a valid flag only for first and +/// last fragment (if TotalFragments == CurrentFragment == 255) in given set. +/// +/// further note that if IF (isFragmented) is not set, +/// then the remaining bytes of the header are used as payload and also note that if LID is not set, +/// then the Linked ID bytes in the header are used as payload. +/// +/// Hence after marshaling `FragmentHeader` into bytes, +/// the following three alternatives are possible: +/// +/// Single byte representing that the `Fragment` is the only one into which the message was split. +/// '0'byte +/// +/// 7 byte long sequence representing that this `Fragment` is one of multiple ones in the set. +/// However, the set is not linked to any other sets: +/// '1'bit || 31-bit ID || 1-byte TF || 1 byte CF || '0'byte +/// +/// 10 byte sequence representing first (or last) fragment in the set, +/// where the set is linked to either preceding data (TF == 1) or proceeding data (TF == CF == 255) +/// '1'bit || 31-bit ID || 1-byte TF || 1 byte CF || '1'bit || 31-bit LID +/// +/// And hence for messages smaller than sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH, +/// there is only a single byte of overhead; +/// for messages larger than sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH but small enough +/// to avoid set division (which happens if message has to be fragmented into more than 255 fragments) +/// there is 7 bytes of overhead inside each sphinx packet sent +/// and finally for the longest messages, without upper bound, there is usually also only 7 bytes +/// of overhead apart from first and last fragments in each set that instead have 10 bytes of overhead. +#[derive(PartialEq, Clone, Debug)] +pub(crate) struct FragmentHeader { + /// Flag used to indicate whether this `Fragment` is the only one into which original + /// message was split. + /// If so, rest of the fields are meaningless as they are set to their default values + /// the whole time and serve no purpose when marshaling or unmarshaling a `FragmentHeader` + is_fragmented: bool, + + /// ID associated with `FragmentSet` to which this particular `Fragment` belongs. + /// Its value is restricted to (0, i32::max_value()]. + /// Note that it *excludes* 0, but *includes* i32::max_value(). + /// This allows the field to be represented using 31 bits. + id: i32, + + /// Total number of `Fragment`s in `FragmentSet` used to be able to determine if entire + /// set was fully received as well as to perform bound checks. + total_fragments: u8, + + /// Since message is always fragmented into payloads of constant lengths + /// (apart from possibly the last one), there's no need to use offsets like ipv4/ipv6 + /// and we can just simply enumerate the fragments to later reconstruct the message. + current_fragment: u8, + + /// Optional ID of previous `FragmentSet` into which the original message was split. + /// Note, this option is only valid of `current_fragment == 1` + previous_fragments_set_id: Option, + + /// Optional ID of next `FragmentSet` into which the original message was split. + /// Note, this option is only valid of `current_fragment == total_fragments == u8::max_value()` + next_fragments_set_id: Option, +} + +impl FragmentHeader { + /// Tries to create a new `FragmentHeader` using provided metadata. Bunch of logical + /// checks are performed to see if the data is not self-contradictory, + /// for example if current_fragment > total_fragments. + fn try_new_fragmented( + id: i32, + total_fragments: u8, + current_fragment: u8, + previous_fragments_set_id: Option, + next_fragments_set_id: Option, + ) -> Result { + if id <= 0 { + return Err(ChunkingError::MalformedHeaderError); + } + if total_fragments < current_fragment { + return Err(ChunkingError::MalformedHeaderError); + } + if total_fragments == 0 { + return Err(ChunkingError::MalformedHeaderError); + } + if current_fragment == 0 { + return Err(ChunkingError::MalformedHeaderError); + } + if let Some(pfid) = previous_fragments_set_id { + if pfid <= 0 || current_fragment != 1 || pfid == id { + return Err(ChunkingError::MalformedHeaderError); + } + } + if let Some(nfid) = next_fragments_set_id { + if nfid <= 0 || current_fragment != total_fragments || nfid == id { + return Err(ChunkingError::MalformedHeaderError); + } + } + + Ok(FragmentHeader { + is_fragmented: true, + id, + total_fragments, + current_fragment, + previous_fragments_set_id, + next_fragments_set_id, + }) + } + + /// When unfragmented header is created, no checks need to happen as there is no variability + /// in the values of its fields. All of them always have the same, default, values. + fn new_unfragmented() -> Self { + FragmentHeader { + is_fragmented: false, + id: 0, + total_fragments: 1, + current_fragment: 1, + previous_fragments_set_id: None, + next_fragments_set_id: None, + } + } + + /// Tries to recover `FragmentHeader` from slice of bytes extracted from received sphinx packet. + /// If successful, returns `Self` and number of bytes used, as those can differ based on the + /// type of header (unfragmented, unlinked, linked). + fn try_from_bytes(b: &[u8]) -> Result<(Self, usize), ChunkingError> { + if b.is_empty() { + return Err(ChunkingError::TooShortFragmentData); + } + // check if it's fragmented - if it's not - the whole first byte is set to 0 + // otherwise first bit is set to 1 + if b[0] == 0 { + Ok((Self::new_unfragmented(), 1)) + } else { + // if it's fragmented, it needs to be at least 7 bytes long + if b.len() < UNLINKED_FRAGMENTED_HEADER_LEN { + return Err(ChunkingError::TooShortFragmentData); + } + let frag_id = i32::from_be_bytes(b[0..4].try_into().unwrap()); + // sanity check for the fragmentation flag + if ((frag_id >> 31) & 1) == 0 { + return Err(ChunkingError::MalformedHeaderError); + } + + let id = frag_id & !(1 << 31); // make sure to clear the flag bit to parse id correctly + let total_fragments = b[4]; + let current_fragment = b[5]; + + if total_fragments == 0 || current_fragment == 0 || current_fragment > total_fragments { + return Err(ChunkingError::MalformedHeaderError); + } + + let mut previous_fragments_set_id = None; + let mut next_fragments_set_id = None; + + // check if the linking id flag might be set + let read_bytes = if b[6] != 0 { + // there's linking ID supposedly attached, make sure we have enough bytes to parse + if b.len() < LINKED_FRAGMENTED_HEADER_LEN { + return Err(ChunkingError::TooShortFragmentData); + } + let flagged_linked_id = i32::from_be_bytes(b[6..10].try_into().unwrap()); + + // sanity check for the linked flag + if ((flagged_linked_id >> 31) & 1) == 0 { + return Err(ChunkingError::MalformedHeaderError); + } + + let linked_id = flagged_linked_id & !(1 << 31); // make sure to clear the flag bit to parse id correctly + + if current_fragment == 1 { + previous_fragments_set_id = Some(linked_id); + } else if total_fragments == current_fragment && current_fragment == u8::max_value() + { + next_fragments_set_id = Some(linked_id); + } else { + return Err(ChunkingError::MalformedHeaderError); + } + + 10 + } else { + 7 + }; + + Ok(( + Self::try_new_fragmented( + id, + total_fragments, + current_fragment, + previous_fragments_set_id, + next_fragments_set_id, + )?, + read_bytes, + )) + } + } + + /// Helper method to determine if this `FragmentHeader` is used to represent a linked `Fragment`. + fn is_linked(&self) -> bool { + self.previous_fragments_set_id.is_some() || self.next_fragments_set_id.is_some() + } + + /// Helper method to determine if this `FragmentHeader` is used to represent a `Fragment` that + /// is a final one in some `FragmentSet`. + fn is_final(&self) -> bool { + self.total_fragments == self.current_fragment + } + + /// Marshal this `FragmentHeader` into vector of bytes which can be put into a sphinx packet. + fn to_bytes(&self) -> Vec { + if self.is_fragmented { + let frag_id = self.id | (1 << 31); + let frag_id_bytes = frag_id.to_be_bytes(); + let bytes_prefix_iter = frag_id_bytes + .iter() + .cloned() + .chain(std::iter::once(self.total_fragments)) + .chain(std::iter::once(self.current_fragment)); + + let is_linked = + self.previous_fragments_set_id.is_some() || self.next_fragments_set_id.is_some(); + if is_linked { + let linked_id = self + .previous_fragments_set_id + .unwrap_or_else(|| self.next_fragments_set_id.unwrap()); + let linked_id_entry = linked_id | (1 << 31); + let linked_id_bytes = linked_id_entry.to_be_bytes(); + bytes_prefix_iter + .chain(linked_id_bytes.iter().cloned()) + .collect() + } else { + bytes_prefix_iter.chain(std::iter::once(0)).collect() + } + } else { + [0].to_vec() + } + } +} + +// everything below are tests + +#[cfg(test)] +mod fragment { + use rand::{thread_rng, RngCore}; + + use super::*; + + #[test] + fn can_be_converted_to_and_from_bytes_for_unfragmented_payload() { + let mut rng = thread_rng(); + + let mlen = UNFRAGMENTED_PAYLOAD_MAX_LEN - 20; + let mut valid_message = vec![0u8; mlen]; + rng.fill_bytes(&mut valid_message); + + let valid_unfragmented_packet = Fragment { + header: FragmentHeader::new_unfragmented(), + payload: valid_message, + }; + let packet_bytes = valid_unfragmented_packet.clone().into_bytes(); + assert_eq!( + valid_unfragmented_packet, + Fragment::try_from_bytes(&packet_bytes).unwrap() + ); + + let empty_unfragmented_packet = Fragment { + header: FragmentHeader::new_unfragmented(), + payload: Vec::new(), + }; + let packet_bytes = empty_unfragmented_packet.clone().into_bytes(); + assert_eq!( + empty_unfragmented_packet, + Fragment::try_from_bytes(&packet_bytes).unwrap() + ); + + let mut full_message = vec![0u8; UNFRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut full_message); + + let full_unfragmented_packet = Fragment { + header: FragmentHeader::new_unfragmented(), + payload: full_message, + }; + let packet_bytes = full_unfragmented_packet.clone().into_bytes(); + assert_eq!( + full_unfragmented_packet, + Fragment::try_from_bytes(&packet_bytes).unwrap() + ); + } + + #[test] + fn conversion_from_bytes_fails_for_too_long_unfragmented_payload() { + let mut rng = thread_rng(); + + let mlen = UNFRAGMENTED_PAYLOAD_MAX_LEN + 1; + let mut message = vec![0u8; mlen]; + rng.fill_bytes(&mut message); + + let packet = Fragment { + header: FragmentHeader::new_unfragmented(), + payload: message, + }; + + let packet_bytes = packet.into_bytes(); + assert_eq!( + Fragment::try_from_bytes(&packet_bytes), + Err(ChunkingError::MalformedFragmentData) + ); + } + + #[test] + fn can_be_converted_to_and_from_bytes_for_unlinked_fragmented_payload() { + let mut rng = thread_rng(); + + let mut msg = vec![0u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut msg); + + let non_last_packet = Fragment { + header: FragmentHeader::try_new_fragmented(12345, 10, 5, None, None).unwrap(), + payload: msg, + }; + let packet_bytes = non_last_packet.clone().into_bytes(); + assert_eq!( + non_last_packet, + Fragment::try_from_bytes(&packet_bytes).unwrap() + ); + + let mut msg = vec![0u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut msg); + + let last_full_packet = Fragment { + header: FragmentHeader::try_new_fragmented(12345, 10, 10, None, None).unwrap(), + payload: msg, + }; + let packet_bytes = last_full_packet.clone().into_bytes(); + assert_eq!( + last_full_packet, + Fragment::try_from_bytes(&packet_bytes).unwrap() + ); + + let mut msg = vec![0u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 20]; + rng.fill_bytes(&mut msg); + + let last_non_full_packet = Fragment { + header: FragmentHeader::try_new_fragmented(12345, 10, 10, None, None).unwrap(), + payload: msg, + }; + let packet_bytes = last_non_full_packet.clone().into_bytes(); + assert_eq!( + last_non_full_packet, + Fragment::try_from_bytes(&packet_bytes).unwrap() + ); + } + + #[test] + fn conversion_from_bytes_fails_for_too_long_unlinked_fragmented_payload() { + let mut rng = thread_rng(); + + let mlen = UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1; + let mut message = vec![0u8; mlen]; + rng.fill_bytes(&mut message); + + let packet = Fragment { + header: FragmentHeader::try_new_fragmented(12345, 10, 5, None, None).unwrap(), + payload: message, + }; + + let packet_bytes = packet.into_bytes(); + assert_eq!( + Fragment::try_from_bytes(&packet_bytes), + Err(ChunkingError::MalformedFragmentData) + ); + } + + #[test] + fn conversion_from_bytes_fails_for_too_short_fragmented_payload_if_not_last() { + let mut rng = thread_rng(); + + let mlen = UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 1; + let mut message = vec![0u8; mlen]; + rng.fill_bytes(&mut message); + + let packet = Fragment { + header: FragmentHeader::try_new_fragmented(12345, 10, 5, None, None).unwrap(), + payload: message, + }; + + let packet_bytes = packet.into_bytes(); + assert_eq!( + Fragment::try_from_bytes(&packet_bytes), + Err(ChunkingError::MalformedFragmentData) + ); + } + + #[test] + fn can_be_converted_to_and_from_bytes_for_pre_linked_fragmented_payload() { + let mut rng = thread_rng(); + + let mut msg = vec![0u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut msg); + + let fragment = Fragment { + header: FragmentHeader::try_new_fragmented(12345, 10, 1, Some(1234), None).unwrap(), + payload: msg, + }; + let packet_bytes = fragment.clone().into_bytes(); + assert_eq!(fragment, Fragment::try_from_bytes(&packet_bytes).unwrap()); + + let mut msg = vec![0u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 20]; + rng.fill_bytes(&mut msg); + + let fragment = Fragment { + header: FragmentHeader::try_new_fragmented(12345, 1, 1, Some(1234), None).unwrap(), + payload: msg, + }; + let packet_bytes = fragment.clone().into_bytes(); + assert_eq!(fragment, Fragment::try_from_bytes(&packet_bytes).unwrap()); + } + + #[test] + fn conversion_from_bytes_fails_for_too_long_pre_linked_fragmented_payload() { + let mut rng = thread_rng(); + + let mlen = LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1; + let mut message = vec![0u8; mlen]; + rng.fill_bytes(&mut message); + + let packet = Fragment { + header: FragmentHeader::try_new_fragmented(12345, 10, 1, Some(1234), None).unwrap(), + payload: message, + }; + + let packet_bytes = packet.into_bytes(); + assert_eq!( + Fragment::try_from_bytes(&packet_bytes), + Err(ChunkingError::MalformedFragmentData) + ); + } + + #[test] + fn can_be_converted_to_and_from_bytes_for_post_linked_fragmented_payload() { + let mut rng = thread_rng(); + + let mut msg = vec![0u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut msg); + + let fragment = Fragment { + header: FragmentHeader::try_new_fragmented( + 12345, + u8::max_value(), + u8::max_value(), + None, + Some(1234), + ) + .unwrap(), + payload: msg, + }; + let packet_bytes = fragment.clone().into_bytes(); + assert_eq!(fragment, Fragment::try_from_bytes(&packet_bytes).unwrap()); + + let mut msg = vec![0u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 20]; + rng.fill_bytes(&mut msg); + + let fragment = Fragment { + header: FragmentHeader::try_new_fragmented( + 12345, + u8::max_value(), + u8::max_value(), + None, + Some(1234), + ) + .unwrap(), + payload: msg, + }; + let packet_bytes = fragment.clone().into_bytes(); + assert_eq!(fragment, Fragment::try_from_bytes(&packet_bytes).unwrap()); + } + + #[test] + fn conversion_from_bytes_fails_for_too_long_post_linked_fragmented_payload() { + let mut rng = thread_rng(); + + let mlen = LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1; + let mut message = vec![0u8; mlen]; + rng.fill_bytes(&mut message); + + let packet = Fragment { + header: FragmentHeader::try_new_fragmented( + 12345, + u8::max_value(), + u8::max_value(), + None, + Some(1234), + ) + .unwrap(), + payload: message, + }; + + let packet_bytes = packet.into_bytes(); + assert_eq!( + Fragment::try_from_bytes(&packet_bytes), + Err(ChunkingError::MalformedFragmentData) + ); + } + + #[test] + fn unfragmented_fragment_can_be_created_with_payload_of_valid_length() { + let payload = [1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN]; + assert!(Fragment::try_new_unfragmented(&payload).is_ok()); + + let payload = [1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN - 1]; + assert!(Fragment::try_new_unfragmented(&payload).is_ok()); + + let payload = [1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN - 20]; + assert!(Fragment::try_new_unfragmented(&payload).is_ok()); + } + + #[test] + fn unfragmented_fragment_returns_error_when_created_with_payload_of_invalid_length() { + let payload = [1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN + 1]; + assert!(Fragment::try_new_unfragmented(&payload).is_err()); + + let payload = [1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN + 20]; + assert!(Fragment::try_new_unfragmented(&payload).is_err()); + } + + #[test] + fn unlinked_fragment_can_be_created_with_payload_of_valid_length() { + let id = 12345; + let full_payload = [1u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + let non_full_payload = [1u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 1]; + let non_full_payload2 = [1u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 20]; + + assert!(Fragment::try_new_fragmented(&full_payload, id, 10, 1, None, None).is_ok()); + assert!(Fragment::try_new_fragmented(&full_payload, id, 10, 5, None, None).is_ok()); + assert!(Fragment::try_new_fragmented(&full_payload, id, 10, 10, None, None).is_ok()); + assert!(Fragment::try_new_fragmented(&full_payload, id, 1, 1, None, None).is_ok()); + + assert!(Fragment::try_new_fragmented(&non_full_payload, id, 10, 10, None, None).is_ok()); + assert!(Fragment::try_new_fragmented(&non_full_payload, id, 1, 1, None, None).is_ok()); + + assert!(Fragment::try_new_fragmented(&non_full_payload2, id, 10, 10, None, None).is_ok()); + assert!(Fragment::try_new_fragmented(&non_full_payload2, id, 1, 1, None, None).is_ok()); + } + + #[test] + fn unlinked_fragment_returns_error_when_created_with_payload_of_invalid_length() { + let id = 12345; + let non_full_payload = [1u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 1]; + let non_full_payload2 = [1u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 20]; + let too_much_payload = [1u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1]; + + assert!(Fragment::try_new_fragmented(&non_full_payload, id, 10, 1, None, None).is_err()); + assert!(Fragment::try_new_fragmented(&non_full_payload, id, 10, 5, None, None).is_err()); + + assert!(Fragment::try_new_fragmented(&too_much_payload, id, 10, 1, None, None).is_err()); + assert!(Fragment::try_new_fragmented(&too_much_payload, id, 10, 5, None, None).is_err()); + assert!(Fragment::try_new_fragmented(&too_much_payload, id, 1, 1, None, None).is_err()); + + assert!(Fragment::try_new_fragmented(&non_full_payload2, id, 10, 1, None, None).is_err()); + assert!(Fragment::try_new_fragmented(&non_full_payload2, id, 10, 5, None, None).is_err()); + } + + #[test] + fn linked_fragment_can_be_created_with_payload_of_valid_length() { + let id = 12345; + let link_id = 1234; + let full_payload = [1u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + let non_full_payload = [1u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 1]; + let non_full_payload2 = [1u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 20]; + + assert!( + Fragment::try_new_fragmented(&full_payload, id, 10, 1, Some(link_id), None).is_ok() + ); + assert!(Fragment::try_new_fragmented(&full_payload, id, 1, 1, Some(link_id), None).is_ok()); + assert!( + Fragment::try_new_fragmented(&non_full_payload, id, 1, 1, Some(link_id), None).is_ok() + ); + assert!( + Fragment::try_new_fragmented(&non_full_payload2, id, 1, 1, Some(link_id), None).is_ok() + ); + + assert!(Fragment::try_new_fragmented( + &full_payload, + id, + u8::max_value(), + u8::max_value(), + None, + Some(link_id) + ) + .is_ok()); + } + + #[test] + fn linked_fragment_returns_error_when_created_with_payload_of_invalid_length() { + let id = 12345; + let link_id = 1234; + let non_full_payload = [1u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 1]; + let non_full_payload2 = [1u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 20]; + let too_much_payload = [1u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1]; + + assert!( + Fragment::try_new_fragmented(&non_full_payload, id, 10, 1, Some(link_id), None) + .is_err() + ); + assert!( + Fragment::try_new_fragmented(&non_full_payload2, id, 10, 1, Some(link_id), None) + .is_err() + ); + assert!( + Fragment::try_new_fragmented(&too_much_payload, id, 10, 1, Some(link_id), None) + .is_err() + ); + assert!( + Fragment::try_new_fragmented(&too_much_payload, id, 1, 1, Some(link_id), None).is_err() + ); + + assert!(Fragment::try_new_fragmented( + &non_full_payload, + id, + u8::max_value(), + u8::max_value(), + None, + Some(link_id) + ) + .is_err()); + assert!(Fragment::try_new_fragmented( + &non_full_payload2, + id, + u8::max_value(), + u8::max_value(), + None, + Some(link_id) + ) + .is_err()); + + assert!(Fragment::try_new_fragmented( + &too_much_payload, + id, + u8::max_value(), + u8::max_value(), + None, + Some(link_id) + ) + .is_err()); + } +} + +#[cfg(test)] +mod fragment_header { + use super::*; + + #[cfg(test)] + mod unfragmented_payload { + use super::*; + + #[test] + fn can_be_converted_to_and_from_bytes_for_exact_number_of_bytes_provided() { + let unfragmented_header = FragmentHeader::new_unfragmented(); + + let header_bytes = unfragmented_header.to_bytes(); + let (recovered_header, bytes_used) = + FragmentHeader::try_from_bytes(&header_bytes).unwrap(); + assert_eq!(unfragmented_header, recovered_header); + assert_eq!(UNFRAGMENTED_HEADER_LEN, bytes_used); + } + + #[test] + fn can_be_converted_to_and_from_bytes_for_more_than_required_number_of_bytes() { + let unfragmented_header = FragmentHeader::new_unfragmented(); + + let mut header_bytes = unfragmented_header.to_bytes(); + header_bytes.append(vec![1, 2, 3, 4, 5].as_mut()); + + let (recovered_header, bytes_used) = + FragmentHeader::try_from_bytes(&header_bytes).unwrap(); + assert_eq!(unfragmented_header, recovered_header); + assert_eq!(UNFRAGMENTED_HEADER_LEN, bytes_used); + } + + #[test] + fn retrieval_from_bytes_fail_for_empty_slice() { + let empty_vec = Vec::new(); + + assert!(FragmentHeader::try_from_bytes(&empty_vec).is_err()) + } + + #[test] + fn retrieval_from_bytes_fail_for_invalid_fragmentation_flag() { + let unfragmented_header = FragmentHeader::new_unfragmented(); + + let mut header_bytes = unfragmented_header.to_bytes(); + header_bytes.append(vec![1, 2, 3, 4, 5].as_mut()); + + // set the fragmentation flag + header_bytes[0] |= 1 << 7; + + assert!(FragmentHeader::try_from_bytes(&header_bytes).is_err()); + } + } + + #[cfg(test)] + mod unlinked_fragmented_payload { + use super::*; + + #[test] + fn can_be_converted_to_and_from_bytes_for_exact_number_of_bytes_provided() { + let fragmented_header = + FragmentHeader::try_new_fragmented(12345, 10, 5, None, None).unwrap(); + + let header_bytes = fragmented_header.to_bytes(); + let (recovered_header, bytes_used) = + FragmentHeader::try_from_bytes(&header_bytes).unwrap(); + assert_eq!(fragmented_header, recovered_header); + assert_eq!(UNLINKED_FRAGMENTED_HEADER_LEN, bytes_used); + } + + #[test] + fn can_be_converted_to_and_from_bytes_for_more_than_required_number_of_bytes() { + let fragmented_header = + FragmentHeader::try_new_fragmented(12345, 10, 5, None, None).unwrap(); + + let mut header_bytes = fragmented_header.to_bytes(); + header_bytes.append(vec![1, 2, 3, 4, 5].as_mut()); + + let (recovered_header, bytes_used) = + FragmentHeader::try_from_bytes(&header_bytes).unwrap(); + assert_eq!(fragmented_header, recovered_header); + assert_eq!(UNLINKED_FRAGMENTED_HEADER_LEN, bytes_used); + } + + #[test] + fn retrieval_from_bytes_fail_for_insufficient_number_of_bytes_provided() { + let fragmented_header = + FragmentHeader::try_new_fragmented(12345, 10, 5, None, None).unwrap(); + + let header_bytes = fragmented_header.to_bytes(); + let header_bytes = &header_bytes[..header_bytes.len() - 1]; + assert!(FragmentHeader::try_from_bytes(&header_bytes).is_err()) + } + + #[test] + fn retrieval_from_bytes_fail_for_invalid_fragmentation_flag() { + let fragmented_header = + FragmentHeader::try_new_fragmented(10, 10, 5, None, None).unwrap(); + + let mut header_bytes_low = fragmented_header.to_bytes(); + + // clear the fragmentation flag + header_bytes_low[0] &= !(1 << 7); + + let mut header_bytes_high = header_bytes_low.clone(); + // make sure first byte of id is non-empty (apart from the fragmentation flag) + // note for anyone reading this test in the future: choice of '3' here is arbitrary. + header_bytes_high[0] |= 1 << 3; + + // this will cause it to be parsed as 'unfragmented' header due to whole byte being set to 0 + // there isn't a really a good way of preventing that apart from adding even data overhead + assert_eq!( + FragmentHeader::new_unfragmented(), + FragmentHeader::try_from_bytes(&header_bytes_low).unwrap().0 + ); + + // however, this will have cause an error as there will be a value in the first byte + assert!(FragmentHeader::try_from_bytes(&header_bytes_high).is_err()); + } + + #[test] + fn retrieval_from_bytes_fail_for_invalid_link_flag() { + let fragmented_header = + FragmentHeader::try_new_fragmented(12345, 10, 5, None, None).unwrap(); + + let mut header_bytes = fragmented_header.to_bytes(); + // set linked flag + header_bytes[6] |= 1 << 7; + assert!(FragmentHeader::try_from_bytes(&header_bytes).is_err()); + } + + #[test] + fn creation_of_header_fails_if_current_fragment_is_higher_than_total() { + assert!(FragmentHeader::try_new_fragmented(12345, 10, 11, None, None).is_err()); + } + + #[test] + fn creation_of_header_fails_if_current_fragment_is_zero() { + assert!(FragmentHeader::try_new_fragmented(12345, 10, 0, None, None).is_err()); + } + + #[test] + fn creation_of_header_fails_if_total_fragments_is_zero() { + assert!(FragmentHeader::try_new_fragmented(12345, 0, 0, None, None).is_err()); + } + + #[test] + fn creation_of_header_fails_if_id_is_negative() { + assert!(FragmentHeader::try_new_fragmented(-10, 10, 5, None, None).is_err()); + } + + #[test] + fn fragmented_header_cannot_be_created_with_zero_id() { + assert!(FragmentHeader::try_new_fragmented(0, 10, 5, None, None).is_err()); + assert!(FragmentHeader::try_new_fragmented(12345, 10, 5, Some(0), None).is_err()); + assert!(FragmentHeader::try_new_fragmented( + 12345, + u8::max_value(), + u8::max_value(), + None, + Some(0) + ) + .is_err()); + } + + #[test] + fn retrieval_from_bytes_fail_if_current_fragment_is_higher_than_total() { + // manually create header to overwrite any constructor checks + let header = FragmentHeader { + is_fragmented: true, + id: 1234, + total_fragments: 10, + current_fragment: 11, + previous_fragments_set_id: None, + next_fragments_set_id: None, + }; + let header_bytes = header.to_bytes(); + assert!(FragmentHeader::try_from_bytes(&header_bytes).is_err()); + } + + #[test] + fn retrieval_from_bytes_fail_if_current_or_total_fragment_is_zero() { + // manually create header to overwrite any constructor checks + let header = FragmentHeader { + is_fragmented: true, + id: 1234, + total_fragments: 0, + current_fragment: 0, + previous_fragments_set_id: None, + next_fragments_set_id: None, + }; + let header_bytes = header.to_bytes(); + assert!(FragmentHeader::try_from_bytes(&header_bytes).is_err()); + } + } + + #[cfg(test)] + mod linked_fragmented_payload { + use super::*; + + #[test] + fn cannot_be_linked_to_itself() { + assert!(FragmentHeader::try_new_fragmented(12345, 10, 1, Some(12345), None).is_err()); + assert!(FragmentHeader::try_new_fragmented(12345, 10, 10, None, Some(12345)).is_err()); + } + + #[test] + fn can_only_be_pre_linked_for_first_fragment() { + assert!(FragmentHeader::try_new_fragmented(12345, 10, 1, Some(1234), None).is_ok()); + assert!(FragmentHeader::try_new_fragmented(12345, 10, 2, Some(1234), None).is_err()); + } + + #[test] + fn can_only_be_post_linked_for_last_fragment() { + assert!(FragmentHeader::try_new_fragmented(12345, 10, 10, None, Some(1234)).is_ok()); + assert!(FragmentHeader::try_new_fragmented( + 12345, + u8::max_value(), + u8::max_value(), + None, + Some(1234), + ) + .is_ok()); + assert!(FragmentHeader::try_new_fragmented(12345, 10, 2, Some(1234), None).is_err()); + } + + #[test] + fn pre_linked_can_be_converted_to_and_from_bytes_for_exact_number_of_bytes_provided() { + let fragmented_header = + FragmentHeader::try_new_fragmented(12345, 10, 1, Some(1234), None).unwrap(); + + let header_bytes = fragmented_header.to_bytes(); + let (recovered_header, bytes_used) = + FragmentHeader::try_from_bytes(&header_bytes).unwrap(); + assert_eq!(fragmented_header, recovered_header); + assert_eq!(LINKED_FRAGMENTED_HEADER_LEN, bytes_used); + } + + #[test] + fn pre_linked_can_be_converted_to_and_from_bytes_for_more_than_required_number_of_bytes() { + let fragmented_header = + FragmentHeader::try_new_fragmented(12345, 10, 1, Some(1234), None).unwrap(); + + let mut header_bytes = fragmented_header.to_bytes(); + header_bytes.append(vec![1, 2, 3, 4, 5].as_mut()); + + let (recovered_header, bytes_used) = + FragmentHeader::try_from_bytes(&header_bytes).unwrap(); + assert_eq!(fragmented_header, recovered_header); + assert_eq!(LINKED_FRAGMENTED_HEADER_LEN, bytes_used); + } + + #[test] + fn pre_linked_is_successfully_recovered_if_its_both_first_and_final_fragment() { + let fragmented_header = + FragmentHeader::try_new_fragmented(12345, 1, 1, Some(1234), None).unwrap(); + + let header_bytes = fragmented_header.to_bytes(); + let (recovered_header, bytes_used) = + FragmentHeader::try_from_bytes(&header_bytes).unwrap(); + assert_eq!(fragmented_header, recovered_header); + assert_eq!(LINKED_FRAGMENTED_HEADER_LEN, bytes_used); + } + + #[test] + fn post_linked_can_be_converted_to_and_from_bytes_for_exact_number_of_bytes_provided() { + let fragmented_header = FragmentHeader::try_new_fragmented( + 12345, + u8::max_value(), + u8::max_value(), + None, + Some(1234), + ) + .unwrap(); + + let header_bytes = fragmented_header.to_bytes(); + let (recovered_header, bytes_used) = + FragmentHeader::try_from_bytes(&header_bytes).unwrap(); + assert_eq!(fragmented_header, recovered_header); + assert_eq!(LINKED_FRAGMENTED_HEADER_LEN, bytes_used); + } + + #[test] + fn post_linked_can_be_converted_to_and_from_bytes_for_more_than_required_number_of_bytes() { + let fragmented_header = FragmentHeader::try_new_fragmented( + 12345, + u8::max_value(), + u8::max_value(), + None, + Some(1234), + ) + .unwrap(); + + let mut header_bytes = fragmented_header.to_bytes(); + header_bytes.append(vec![1, 2, 3, 4, 5].as_mut()); + + let (recovered_header, bytes_used) = + FragmentHeader::try_from_bytes(&header_bytes).unwrap(); + assert_eq!(fragmented_header, recovered_header); + assert_eq!(LINKED_FRAGMENTED_HEADER_LEN, bytes_used); + } + } +} diff --git a/common/nymsphinx/src/chunking/mod.rs b/common/nymsphinx/src/chunking/mod.rs new file mode 100644 index 0000000000..578560b644 --- /dev/null +++ b/common/nymsphinx/src/chunking/mod.rs @@ -0,0 +1,56 @@ +use crate::chunking::set::split_into_sets; + +pub mod fragment; +pub mod reconstruction; +pub mod set; + +/// The idea behind the process of chunking is to incur as little data overhead as possible due +/// to very computationally costly sphinx encapsulation procedure. +/// +/// To achieve this, the underlying message is split into so-called "sets", which are further +/// subdivided into the base unit of "fragment" that is directly encapsulated by a Sphinx packet. +/// This allows to encapsulate messages of arbitrary length. +/// +/// Each message, regardless of its size, consists of at least a single `Set` that has at least +/// a single `Fragment`. +/// +/// Each `Fragment` can have variable, yet fully deterministic, length, +/// that depends on its position in the set as well as total number of sets. This is further +/// explained in `fragment.rs` file. +/// +/// Similarly, each `Set` can have a variable number of `Fragment`s inside. However, that +/// value is more restrictive: if it's the last set into which the message was split +/// (or implicitly the only one), it has no lower bound on the number of `Fragment`s. +/// (Apart from the restriction of containing at least a single one). If the set is located +/// somewhere in the middle, *it must be* full. Finally, regardless of its position, it must also be +/// true that it contains no more than `u8::max_value()`, i.e. 255 `Fragment`s. +/// Again, the reasoning for this is further explained in `set.rs` file. However, you might +/// also want to look at `fragment.rs` to understand the full context behind that design choice. +/// +/// Both of those concepts as well as their structures, i.e. `Set` and `Fragment` +/// are further explained in the respective files. + +#[derive(PartialEq, Debug)] +pub enum ChunkingError { + InvalidPayloadLengthError, + TooBigMessageToSplit, + MalformedHeaderError, + NoValidProvidersError, + NoValidRoutesAvailableError, + InvalidTopologyError, + TooShortFragmentData, + MalformedFragmentData, + UnexpectedFragmentCount, +} + +/// Takes the entire message and splits it into bytes chunks that will fit into sphinx packets +/// directly. After receiving they can be combined using `reconstruction::MessageReconstructor` +/// to obtain the original message back. +pub fn split_and_prepare_payloads(message: &[u8]) -> Vec> { + let fragmented_messages = split_into_sets(message); + fragmented_messages + .into_iter() + .flat_map(|fragment_set| fragment_set.into_iter()) + .map(|fragment| fragment.into_bytes()) + .collect() +} diff --git a/common/nymsphinx/src/chunking/reconstruction.rs b/common/nymsphinx/src/chunking/reconstruction.rs new file mode 100644 index 0000000000..ff1b4ba377 --- /dev/null +++ b/common/nymsphinx/src/chunking/reconstruction.rs @@ -0,0 +1,1348 @@ +use crate::chunking::fragment::Fragment; +use log::*; +use std::collections::HashMap; + +// TODO: perhaps a more sophisticated approach with writing to disk periodically in case +// we're receiving fast & furious in uncompressed 4K - we don't want to keep that in memory; +// perhaps write whole sets to the disk if there are still more of them to recover? +// Then either combine files on the disk to target destination or read everything to memory +/// `ReconstructionBuffer` is a per data set structure used to reconstruct the underlying data +/// and allows for relatively easy way of determining if the original message is split +/// into multiple buffers. +#[derive(PartialEq, Debug, Clone)] +struct ReconstructionBuffer { + /// Easier way to determine if buffer has received all fragments it expected to get. + /// This way it is not required to iterate through the entire `fragments` vector looking for + /// possible `None` elements. + is_complete: bool, + + /// Once all fragments are received, the value of `previous_fragments_set_id` is copied + /// from the first `Fragment` in the set. + previous_fragments_set_id: Option, + /// Once all fragments are received, the value of `next_fragments_set_id` is copied + /// from the last `Fragment` in the set (assuming the set is full, i.e. it contains + /// `u8::max_value()` elements). + next_fragments_set_id: Option, + + /// The actual `Fragment` data held by the `ReconstructionBuffer`. When created it is already + /// appropriately resized and all missing fragments are set to a `None`, thus keeping + /// everything in order the whole time, allowing for O(1) insertions and O(n) reconstruction. + fragments: Vec>, +} + +impl ReconstructionBuffer { + /// Initialises new instance of a `ReconstructionBuffer` with given size, i.e. + /// number of expected `Fragment`s in the set. + /// The `u8` input type of `size` argument ensures it has the `u8::max_value()` upper bound. + fn new(size: u8) -> Self { + // Note: `new` should have never been called with size 0 in the first place + // as `size` value is based on the first recovered `Fragment` in the set. + // A `Fragment` cannot be successfully recovered if it indicates that `total_fragments` + // count is 0. + debug_assert!(size > 0); + + let mut fragments_buffer = Vec::new(); + fragments_buffer.resize(size as usize, None); + + ReconstructionBuffer { + is_complete: false, + previous_fragments_set_id: None, + next_fragments_set_id: None, + fragments: fragments_buffer, + } + } + + /// After receiving all data, consumes `self` in order to recover original data + /// encapsulated in this particular set. + fn reconstruct_set_data(self) -> Vec { + // Note: `reconstruct_set_data` is never called without first explicitly checking + // if the set is complete. + debug_assert!(self.is_complete); + + self.fragments + .into_iter() + .map(|fragment| fragment.unwrap().extract_payload()) + .flat_map(|fragment_data| fragment_data.into_iter()) + .collect() + } + + // TODO: check what's the performance impact of this, and if it's too big, keep track of number + // of received fragments instead rather than checking whole vector, but then + // we might have false positives if somehow we receive a duplicate + /// Checks if `self` is done receiving `Fragment` data by checking if there are still + /// any `None` elements in the `fragments` vector. + fn is_done_receiving(&self) -> bool { + !self.fragments.contains(&None) + } + + /// Inserts new `Fragment` data into an appropriate position in the buffer. + /// + /// (Note: currently there is no defined behaviour for dealing with duplicate + /// fragments for the same position in the set. This might potentially corrupt + /// entire message until resolved) + /// + /// After new `Fragment` is inserted, it is checked whether the buffer should be + /// done receiving and if so, the auxiliary data fields, i.e. `is_complete`, + /// `previous_fragments_set_id` and `next_fragments_set_id` are set for the ease + /// of access. + fn insert_fragment(&mut self, fragment: Fragment) { + // all fragments in the buffer should always have the same id as before inserting an element, + // the correct buffer instance is looked up based on the fragment to be inserted. + debug_assert!({ + let present_fragment = self.fragments.iter().find(|frag| frag.is_some()); + if let Some(existing_present_fragment) = present_fragment { + existing_present_fragment.as_ref().unwrap().id() == fragment.id() + } else { + true + } + }); + + let fragment_index = fragment.current_fragment() as usize - 1; + if self.fragments[fragment_index].is_some() { + // TODO: what to do in that case? give up on the message? overwrite it? panic? + warn!( + "duplicate fragment received! - frag - {} (set id: {})", + fragment.current_fragment(), + fragment.id() + ); + } + self.fragments[fragment_index] = Some(fragment); + if self.is_done_receiving() { + self.is_complete = true; + self.previous_fragments_set_id = self.fragments[0] + .as_ref() + .unwrap() + .previous_fragments_set_id(); + self.next_fragments_set_id = if self.fragments.len() == u8::max_value() as usize { + self.fragments[u8::max_value() as usize - 1] + .as_ref() + .unwrap() + .next_fragments_set_id() + } else { + None + }; + } + } +} + +/// High level public structure used to buffer all received data `Fragment`s and eventually +/// returning original messages that they encapsulate. +#[derive(Default, PartialEq, Debug, Clone)] +pub struct MessageReconstructor { + // TODO: some cleaner thread/routine that if message is incomplete and + // we haven't received any fragments in X time, we assume they + // were lost and message can't be restored. + // Perhaps add 'latest_fragment_timestamp' to each buffer + // and after N fragments received globally, check all of buffer timestamps. + // otherwise we are vulnerable to heap overflow attacks -> somebody can keep on sending + // maximum sized sets but without one of required fragments. All of the received + // data will be kept on the heap indefinitely in the current implementation. + reconstructed_sets: HashMap, +} + +impl MessageReconstructor { + /// Creates an empty `MessageReconstructor`. + pub fn new() -> Self { + Default::default() + } + + /// Given fully received set of given `id`, if it has any post-linked sets, recursively + /// checks if all of them were also fully received. + fn check_front_chain(&self, id: i32) -> bool { + // we know that set with `id` was fully_received (otherwise this method wouldn't have been called) + // and hence the buffer has all of its fields properly set + debug_assert!(self.is_set_fully_received(id)); + + if let Some(previous_id) = self.previous_linked_set_id(id) { + self.is_set_fully_received(previous_id) && self.check_front_chain(previous_id) + } else { + true + } + } + + /// Given fully received set of given `id`, if it has any pre-linked sets, recursively + /// checks if all of them were also fully received. + fn check_back_chain(&self, id: i32) -> bool { + // we know that set with `id` was fully_received (otherwise this method wouldn't have been called) + // and hence the buffer has all of its fields properly set + debug_assert!(self.is_set_fully_received(id)); + + if let Some(next_id) = self.next_linked_set_id(id) { + self.is_set_fully_received(next_id) && self.check_back_chain(next_id) + } else { + true + } + } + + /// Check if set of given `id` is present in the `MessageReconstructor`, and if so, + /// whether it has received all `Fragment`s it expected to get. + fn is_set_fully_received(&self, id: i32) -> bool { + self.reconstructed_sets + .get(&id) + .map(|set_buf| set_buf.is_complete) + .unwrap_or_else(|| false) + } + + /// Check if message that was split into possibly multiple sets was received in fully using + /// `id` of any of its sets. + fn is_message_fully_received(&self, id: i32) -> bool { + self.is_set_fully_received(id) && self.check_back_chain(id) && self.check_front_chain(id) + } + + /// Given id of *any* one of the sets into which message was split, + /// try to obtain id of the set containing head of the message. + /// Might return `None` if one of the sets was not fully received. + fn find_starting_set_id(&self, id: i32) -> Option { + if self.is_set_fully_received(id) { + if let Some(previous_id) = self.previous_linked_set_id(id) { + self.find_starting_set_id(previous_id) + } else { + Some(id) + } + } else { + None + } + } + + /// Given id of a set, obtains (if applicable) id of the previous linked set. + /// Note, before you call this method, you *must* ensure set was fully received + fn previous_linked_set_id(&self, id: i32) -> Option { + debug_assert!(self.is_set_fully_received(id)); + self.reconstructed_sets + .get(&id) + .unwrap() + .previous_fragments_set_id + } + + /// Given id of a set, obtains (if applicable) id of the next linked set. + /// Note, before you call this method, you *must* ensure set was fully received + fn next_linked_set_id(&self, id: i32) -> Option { + debug_assert!(self.is_set_fully_received(id)); + self.reconstructed_sets + .get(&id) + .unwrap() + .next_fragments_set_id + } + + /// Given id of a set, consume its buffer and reconstruct the original payload. + /// Note, before you call this method, you *must* ensure set was fully received + fn extract_set_payload(&mut self, set_id: i32) -> Vec { + debug_assert!(self.is_set_fully_received(set_id)); + self.reconstructed_sets + .remove(&set_id) + .unwrap() + .reconstruct_set_data() + } + + // Future consideration: perhaps for long messages, rather than return whole data allocated + // on the heap, return file handle with the saved content? + /// Given id of *any* one of the sets into which message was divided, + /// reconstruct the entire original message. + /// Note, before you call this method, you *must* ensure all sets were fully received + fn reconstruct_message(&mut self, set_id: i32) -> Vec { + debug_assert!(self.is_message_fully_received(set_id)); + let starting_id = self.find_starting_set_id(set_id).unwrap(); + let set_id_sequence: Vec<_> = + std::iter::successors(Some(starting_id), |&id| self.next_linked_set_id(id)).collect(); + + set_id_sequence + .iter() + .map(|&id| self.extract_set_payload(id)) + .flat_map(|payload| payload.into_iter()) + .collect() + } + + /// Given raw `Fragment` data, tries to decode it and add into an appropriate `ReconstructionBuffer`. + /// If a buffer does not exist, a new instance is created. + /// If it was last remaining `Fragment` for the original message, the message is reconstructed + /// and returned. + pub fn new_fragment(&mut self, fragment_data: Vec) -> Option> { + let fragment_res = Fragment::try_from_bytes(&fragment_data); + if let Err(e) = fragment_res { + warn!("failed to recover fragment data: {:?}. The whole underlying message might be corrupted and unrecoverable!", e); + return None; + } + let fragment = fragment_res.unwrap(); + + let set_id = fragment.id(); + let set_len = fragment.total_fragments(); + + let buf = self + .reconstructed_sets + .entry(set_id) + .or_insert_with(|| ReconstructionBuffer::new(set_len)); + + buf.insert_fragment(fragment); + if self.is_message_fully_received(set_id) { + Some(self.reconstruct_message(set_id)) + } else { + None + } + } +} + +#[cfg(test)] +mod reconstruction_buffer { + use super::*; + use crate::chunking::fragment::UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN; + use crate::chunking::set::MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH; + use crate::chunking::split_and_prepare_payloads; + + #[test] + fn creating_new_instance_correctly_initialised_fragments_buffer() { + let buf = ReconstructionBuffer::new(1); + assert_eq!(1, buf.fragments.len()); + for frag in buf.fragments { + assert_eq!(None, frag); + } + + let buf = ReconstructionBuffer::new(42); + assert_eq!(42, buf.fragments.len()); + for frag in buf.fragments { + assert_eq!(None, frag); + } + + let buf = ReconstructionBuffer::new(u8::max_value()); + assert_eq!(u8::max_value() as usize, buf.fragments.len()); + for frag in buf.fragments { + assert_eq!(None, frag); + } + } + + #[test] + #[should_panic] + fn creating_new_instance_does_not_allow_for_creating_zero_sized_buffer() { + ReconstructionBuffer::new(0); + } + + #[test] + fn reconstructing_set_data_works_for_buffers_of_different_sizes() { + let mut buf = ReconstructionBuffer::new(1); + let message = [42u8; 42]; + let raw_fragments = split_and_prepare_payloads(&message); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[0]).unwrap()); + assert_eq!(message.to_vec(), buf.reconstruct_set_data()); + + let mut buf = ReconstructionBuffer::new(3); + let message = [42u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]; + let raw_fragments = split_and_prepare_payloads(&message); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[0]).unwrap()); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[1]).unwrap()); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[2]).unwrap()); + assert_eq!(message.to_vec(), buf.reconstruct_set_data()); + + let mut buf = ReconstructionBuffer::new(u8::max_value()); + let message = [42u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * u8::max_value() as usize]; + let raw_fragments = split_and_prepare_payloads(&message); + for raw_fragment in raw_fragments { + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragment).unwrap()) + } + assert_eq!(message.to_vec(), buf.reconstruct_set_data()); + } + + #[test] + #[should_panic] + fn reconstructing_set_data_is_not_allowed_for_incomplete_sets() { + let mut buf = ReconstructionBuffer::new(3); + let raw_fragments = + split_and_prepare_payloads(&[42u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[0]).unwrap()); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[1]).unwrap()); + + buf.reconstruct_set_data(); + } + + #[test] + fn inserting_new_fragment_puts_it_in_correct_location_based_on_its_ordering() { + let mut buf = ReconstructionBuffer::new(3); + let raw_fragments = + split_and_prepare_payloads(&[42u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[1]).unwrap()); + + assert!(buf.fragments[0].is_none()); + assert!(buf.fragments[1].is_some()); + assert!(buf.fragments[2].is_none()); + } + + #[test] + fn inserting_final_fragment_correctly_sets_auxiliary_flags() { + let mut buf = ReconstructionBuffer::new(3); + let message = [42u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]; + let raw_fragments = split_and_prepare_payloads(&message); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[0]).unwrap()); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[2]).unwrap()); + + assert!(!buf.is_complete); + assert!(buf.previous_fragments_set_id.is_none()); + assert!(buf.next_fragments_set_id.is_none()); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[1]).unwrap()); + assert!(buf.is_complete); + assert!(buf.previous_fragments_set_id.is_none()); + assert!(buf.next_fragments_set_id.is_none()); + + let mut buf = ReconstructionBuffer::new(255); + let message = [42u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 123]; + let raw_fragments = split_and_prepare_payloads(&message); + for i in 0..254 { + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[i]).unwrap()); + } + + assert!(!buf.is_complete); + assert!(buf.previous_fragments_set_id.is_none()); + assert!(buf.next_fragments_set_id.is_none()); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[254]).unwrap()); + assert!(buf.is_complete); + assert!(buf.previous_fragments_set_id.is_none()); + assert!(buf.next_fragments_set_id.is_some()); + + let mut buf = ReconstructionBuffer::new(1); + assert!(!buf.is_complete); + assert!(buf.previous_fragments_set_id.is_none()); + assert!(buf.next_fragments_set_id.is_none()); + let fragment = Fragment::try_from_bytes(&raw_fragments[255]); + buf.insert_fragment(fragment.unwrap()); + assert!(buf.is_complete); + assert!(buf.previous_fragments_set_id.is_some()); + assert!(buf.next_fragments_set_id.is_none()); + } + + #[test] + #[should_panic] + fn does_not_allow_for_inserting_new_fragments_with_different_ids() { + let mut buf = ReconstructionBuffer::new(3); + + // they will have different IDs + let raw_fragments1 = + split_and_prepare_payloads(&[42u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]); + let raw_fragments2 = + split_and_prepare_payloads(&[42u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]); + + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments1[0]).unwrap()); + buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments2[0]).unwrap()); + } +} + +#[cfg(test)] +mod message_reconstructor { + use super::*; + use crate::chunking::fragment::{ + UNFRAGMENTED_PAYLOAD_MAX_LEN, UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN, + }; + use crate::chunking::set::{ + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH, TWO_WAY_LINKED_SET_PAYLOAD_LENGTH, + }; + use crate::chunking::split_and_prepare_payloads; + use rand::{thread_rng, RngCore}; + + #[test] + #[should_panic] + fn checking_front_chain_is_not_allowed_for_incomplete_sets() { + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 123]; + let raw_fragments = split_and_prepare_payloads(&message); + // first set is fully inserted + for i in 0..255 { + assert!(reconstructor + .new_fragment(raw_fragments[i].clone()) + .is_none()); + } + + assert!(reconstructor + .new_fragment(raw_fragments[255].clone()) + .is_none()); + + let second_set_id = Fragment::try_from_bytes(&raw_fragments[255]).unwrap().id(); + reconstructor.check_front_chain(second_set_id); + } + + #[test] + #[should_panic] + fn checking_back_chain_is_not_allowed_for_incomplete_sets() { + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 123]; + let raw_fragments = split_and_prepare_payloads(&message); + for i in 0..254 { + assert!(reconstructor + .new_fragment(raw_fragments[i].clone()) + .is_none()); + } + // finish next set for good measure + assert!(reconstructor + .new_fragment(raw_fragments[255].clone()) + .is_none()); + assert!(reconstructor + .new_fragment(raw_fragments[256].clone()) + .is_none()); + + let first_set_id = Fragment::try_from_bytes(&raw_fragments[0]).unwrap().id(); + reconstructor.check_back_chain(first_set_id); + } + + #[test] + fn checking_front_chain_returns_false_for_complete_set_but_incomplete_message() { + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 123]; + let raw_fragments = split_and_prepare_payloads(&message); + + // note that first set is not fully inserted + for i in 0..254 { + assert!(reconstructor + .new_fragment(raw_fragments[i].clone()) + .is_none()); + } + + assert!(reconstructor + .new_fragment(raw_fragments[255].clone()) + .is_none()); + assert!(reconstructor + .new_fragment(raw_fragments[256].clone()) + .is_none()); + + let second_set_id = Fragment::try_from_bytes(&raw_fragments[255]).unwrap().id(); + assert!(!reconstructor.check_front_chain(second_set_id)); + } + + #[test] + fn checking_back_chain_returns_false_for_complete_set_but_incomplete_message() { + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 123]; + let raw_fragments = split_and_prepare_payloads(&message); + for i in 0..255 { + assert!(reconstructor + .new_fragment(raw_fragments[i].clone()) + .is_none()); + } + + // notice that entirety of second set is not inserted + assert!(reconstructor + .new_fragment(raw_fragments[255].clone()) + .is_none()); + + let first_set_id = Fragment::try_from_bytes(&raw_fragments[0]).unwrap().id(); + + assert!(!reconstructor.check_back_chain(first_set_id)); + } + + #[test] + fn checking_front_chain_returns_true_for_if_there_are_no_more_front_sets() { + // case of 2 sets: [id1 -- id2], where id1 is completed and being checked + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 123]; + let raw_fragments = split_and_prepare_payloads(&message); + for i in 0..255 { + assert!(reconstructor + .new_fragment(raw_fragments[i].clone()) + .is_none()); + } + + // notice that entirety of second set is not inserted + assert!(reconstructor + .new_fragment(raw_fragments[255].clone()) + .is_none()); + + let first_set_id = Fragment::try_from_bytes(&raw_fragments[0]).unwrap().id(); + + assert!(reconstructor.check_front_chain(first_set_id)); + } + + #[test] + fn checking_back_chain_returns_true_for_if_there_are_no_more_back_sets() { + // case of 2 sets: [id1 -- id2], where id2 is completed and being checked + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 123]; + let raw_fragments = split_and_prepare_payloads(&message); + + // note that first set is not fully inserted + for i in 0..254 { + assert!(reconstructor + .new_fragment(raw_fragments[i].clone()) + .is_none()); + } + + assert!(reconstructor + .new_fragment(raw_fragments[255].clone()) + .is_none()); + + let second_set_id = Fragment::try_from_bytes(&raw_fragments[255]).unwrap().id(); + assert!(reconstructor.check_back_chain(second_set_id)); + } + + #[test] + fn checking_front_chain_returns_true_for_complete_front_chain() { + // case of 3 sets: [id1 -- id2 -- id3], where id1 and id2 are completed and id2 is being checked + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + + TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + + 123]; + let raw_fragments = split_and_prepare_payloads(&message); + for i in 0..(u8::max_value() as usize) * 2 { + assert!(reconstructor + .new_fragment(raw_fragments[i].clone()) + .is_none()); + } + + // notice that entirety of third set is not inserted + assert!(reconstructor + .new_fragment(raw_fragments[(u8::max_value() as usize) * 2].clone()) + .is_none()); + + let second_set_id = Fragment::try_from_bytes(&raw_fragments[300]).unwrap().id(); + + assert!(reconstructor.check_front_chain(second_set_id)); + } + + #[test] + fn checking_back_chain_returns_true_for_complete_back_chain() { + // case of 3 sets: [id1 -- id2 -- id3], where id2 and id3 are completed and id2 is being checked + let mut reconstructor = MessageReconstructor::new(); + + let message = + [42u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + 123]; + let raw_fragments = split_and_prepare_payloads(&message); + // note that first set is not fully inserted + for i in 1..(u8::max_value() as usize) * 2 { + assert!(reconstructor + .new_fragment(raw_fragments[i].clone()) + .is_none()); + } + + assert!(reconstructor + .new_fragment(raw_fragments[(u8::max_value() as usize) * 2].clone()) + .is_none()); + + let second_set_id = Fragment::try_from_bytes(&raw_fragments[300]).unwrap().id(); + + assert!(reconstructor.check_back_chain(second_set_id)); + } + + #[test] + fn checking_if_set_is_fully_received_returns_false_if_no_fragments_were_ever_received() { + let reconstructor = MessageReconstructor::new(); + assert!(!reconstructor.is_set_fully_received(12345)); + } + + #[test] + fn checking_if_set_is_fully_received_if_exists_returns_whatever_is_complete_flag_is_set_to() { + let mut reconstructor = MessageReconstructor::new(); + reconstructor.reconstructed_sets.insert( + 12345, + ReconstructionBuffer { + is_complete: false, + previous_fragments_set_id: None, + next_fragments_set_id: None, + fragments: vec![], + }, + ); + + reconstructor.reconstructed_sets.insert( + 1234, + ReconstructionBuffer { + is_complete: true, + previous_fragments_set_id: None, + next_fragments_set_id: None, + fragments: vec![], + }, + ); + + assert!(!reconstructor.is_set_fully_received(12345)); + assert!(reconstructor.is_set_fully_received(1234)); + } + + #[test] + fn finding_starting_set_id_returns_none_if_message_was_not_fully_received() { + let mut reconstructor = MessageReconstructor::new(); + + let message1 = [42u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 123]; + let raw_fragments1 = split_and_prepare_payloads(&message1); + + // note that first set is not fully inserted + for i in 0..254 { + assert!(reconstructor + .new_fragment(raw_fragments1[i].clone()) + .is_none()); + } + + assert!(reconstructor + .new_fragment(raw_fragments1[255].clone()) + .is_none()); + + let second_set_id = Fragment::try_from_bytes(&raw_fragments1[255]).unwrap().id(); + assert!(reconstructor.find_starting_set_id(second_set_id).is_none()); + + let message2 = [43u8; + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 123]; + let raw_fragments2 = split_and_prepare_payloads(&message2); + + for i in 0..255 { + assert!(reconstructor + .new_fragment(raw_fragments2[i].clone()) + .is_none()); + } + + // notice that entirety of second set is not inserted + assert!(reconstructor + .new_fragment(raw_fragments2[255].clone()) + .is_none()); + + let second_set_id = Fragment::try_from_bytes(&raw_fragments2[255]).unwrap().id(); + assert!(reconstructor.find_starting_set_id(second_set_id).is_none()); + } + + #[test] + fn finding_starting_set_id_returns_expected_starting_id() { + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 123]; + let raw_fragments = split_and_prepare_payloads(&message); + for i in 0..255 { + assert!(reconstructor + .new_fragment(raw_fragments[i].clone()) + .is_none()); + } + + // notice that entirety of second set is not inserted + assert!(reconstructor + .new_fragment(raw_fragments[255].clone()) + .is_none()); + + let first_set_id = Fragment::try_from_bytes(&raw_fragments[0]).unwrap().id(); + assert_eq!( + reconstructor.find_starting_set_id(first_set_id), + Some(first_set_id) + ); + + reconstructor.reconstructed_sets.insert( + 12345, + ReconstructionBuffer { + is_complete: true, + previous_fragments_set_id: None, + next_fragments_set_id: Some(1234), + fragments: vec![], + }, + ); + + reconstructor.reconstructed_sets.insert( + 1234, + ReconstructionBuffer { + is_complete: true, + previous_fragments_set_id: Some(12345), + next_fragments_set_id: Some(123), + fragments: vec![], + }, + ); + + reconstructor.reconstructed_sets.insert( + 123, + ReconstructionBuffer { + is_complete: true, + previous_fragments_set_id: Some(1234), + next_fragments_set_id: Some(12), + fragments: vec![], + }, + ); + + reconstructor.reconstructed_sets.insert( + 12, + ReconstructionBuffer { + is_complete: true, + previous_fragments_set_id: Some(123), + next_fragments_set_id: None, + fragments: vec![], + }, + ); + + assert_eq!(reconstructor.find_starting_set_id(12), Some(12345)); + } + + #[test] + #[should_panic] + fn getting_previous_linked_set_id_is_not_allowed_for_incomplete_sets() { + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]; + let raw_fragments = split_and_prepare_payloads(&message); + assert!(reconstructor + .new_fragment(raw_fragments[0].clone()) + .is_none()); + assert!(reconstructor + .new_fragment(raw_fragments[1].clone()) + .is_none()); + + let id = Fragment::try_from_bytes(&raw_fragments[0]).unwrap().id(); + reconstructor.previous_linked_set_id(id); + } + + #[test] + fn getting_previous_linked_set_id_returns_id_of_previous_set() { + let mut reconstructor = MessageReconstructor::new(); + reconstructor.reconstructed_sets.insert( + 12345, + ReconstructionBuffer { + is_complete: true, + previous_fragments_set_id: None, + next_fragments_set_id: Some(1234), + fragments: vec![], + }, + ); + reconstructor.reconstructed_sets.insert( + 1234, + ReconstructionBuffer { + is_complete: true, + previous_fragments_set_id: Some(12345), + next_fragments_set_id: None, + fragments: vec![], + }, + ); + assert_eq!(reconstructor.previous_linked_set_id(12345), None); + assert_eq!(reconstructor.previous_linked_set_id(1234), Some(12345)); + } + + #[test] + #[should_panic] + fn getting_next_linked_set_id_is_not_allowed_for_incomplete_sets() { + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]; + let raw_fragments = split_and_prepare_payloads(&message); + assert!(reconstructor + .new_fragment(raw_fragments[0].clone()) + .is_none()); + assert!(reconstructor + .new_fragment(raw_fragments[1].clone()) + .is_none()); + + let id = Fragment::try_from_bytes(&raw_fragments[0]).unwrap().id(); + reconstructor.next_linked_set_id(id); + } + + #[test] + fn getting_next_linked_set_id_returns_id_of_next_set() { + let mut reconstructor = MessageReconstructor::new(); + reconstructor.reconstructed_sets.insert( + 12345, + ReconstructionBuffer { + is_complete: true, + previous_fragments_set_id: None, + next_fragments_set_id: Some(1234), + fragments: vec![], + }, + ); + reconstructor.reconstructed_sets.insert( + 1234, + ReconstructionBuffer { + is_complete: true, + previous_fragments_set_id: Some(12345), + next_fragments_set_id: None, + fragments: vec![], + }, + ); + assert_eq!(reconstructor.next_linked_set_id(12345), Some(1234)); + assert_eq!(reconstructor.next_linked_set_id(1234), None); + } + + #[test] + #[should_panic] + fn extracting_set_payload_is_not_allowed_for_incomplete_sets() { + let mut reconstructor = MessageReconstructor::new(); + + let message = [42u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]; + let raw_fragments = split_and_prepare_payloads(&message); + assert!(reconstructor + .new_fragment(raw_fragments[0].clone()) + .is_none()); + assert!(reconstructor + .new_fragment(raw_fragments[1].clone()) + .is_none()); + + let id = Fragment::try_from_bytes(&raw_fragments[0]).unwrap().id(); + reconstructor.extract_set_payload(id); + } + + #[test] + fn extracting_set_payload_is_returns_entire_set_data() { + let mut reconstructor = MessageReconstructor::new(); + let mut set_buf = ReconstructionBuffer::new(3); + let mut rng = thread_rng(); + + let mut message = [0u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]; + rng.fill_bytes(&mut message); + + let raw_fragments = split_and_prepare_payloads(&message); + set_buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[0]).unwrap()); + set_buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[1]).unwrap()); + set_buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[2]).unwrap()); + + let set_id = Fragment::try_from_bytes(&raw_fragments[0]).unwrap().id(); + let buf_clone = set_buf.clone(); + let another_buf_clone = set_buf.clone(); + reconstructor.reconstructed_sets.insert(set_id, set_buf); + assert_eq!( + reconstructor.extract_set_payload(set_id), + buf_clone.reconstruct_set_data() + ); + assert_eq!(another_buf_clone.reconstruct_set_data(), message.to_vec()); + } + + #[test] + fn reconstructing_message_for_single_set_is_equivalent_to_extracting_set_payload() { + // we're inserting this via the buffer approach as not to trigger immediate re-assembly + let mut reconstructor = MessageReconstructor::new(); + let mut set_buf = ReconstructionBuffer::new(3); + let mut rng = thread_rng(); + + let mut message = [0u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN * 3]; + rng.fill_bytes(&mut message); + + let raw_fragments = split_and_prepare_payloads(&message); + set_buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[0]).unwrap()); + set_buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[1]).unwrap()); + set_buf.insert_fragment(Fragment::try_from_bytes(&raw_fragments[2]).unwrap()); + + let set_id = Fragment::try_from_bytes(&raw_fragments[0]).unwrap().id(); + + reconstructor.reconstructed_sets.insert(set_id, set_buf); + let mut reconstructor_clone = reconstructor.clone(); + assert_eq!( + reconstructor.extract_set_payload(set_id), + reconstructor_clone.reconstruct_message(set_id) + ); + } + + #[test] + fn reconstructing_message_for_two_sets_is_equivalent_to_combining_results_of_extracting_set_payload( + ) { + // we're inserting this via the buffer approach as not to trigger immediate re-assembly + let mut reconstructor = MessageReconstructor::new(); + let mut set_buf1 = ReconstructionBuffer::new(u8::max_value()); + let mut set_buf2 = ReconstructionBuffer::new(1); + + let mut rng = thread_rng(); + let mut message = [42u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 123]; + rng.fill_bytes(&mut message); + + let raw_fragments = split_and_prepare_payloads(&message); + for i in 0..255 { + set_buf1.insert_fragment(Fragment::try_from_bytes(&raw_fragments[i]).unwrap()); + } + + set_buf2.insert_fragment(Fragment::try_from_bytes(&raw_fragments[255]).unwrap()); + + let set_id1 = Fragment::try_from_bytes(&raw_fragments[0]).unwrap().id(); + let set_id2 = Fragment::try_from_bytes(&raw_fragments[255]).unwrap().id(); + + reconstructor.reconstructed_sets.insert(set_id1, set_buf1); + reconstructor.reconstructed_sets.insert(set_id2, set_buf2); + let mut reconstructor_clone = reconstructor.clone(); + let mut reconstructor_clone2 = reconstructor.clone(); + + let extracted_set1 = reconstructor.extract_set_payload(set_id1); + let extracted_set2 = reconstructor.extract_set_payload(set_id2); + + let manually_combined_message = [extracted_set1, extracted_set2].concat(); + + // make sure we can use any id that is part of the message + assert_eq!( + reconstructor_clone.reconstruct_message(set_id1), + manually_combined_message + ); + assert_eq!( + reconstructor_clone2.reconstruct_message(set_id2), + manually_combined_message + ); + } + + #[test] + fn adding_invalid_fragment_does_not_change_reconstructor_state() { + let mut empty_reconstructor = MessageReconstructor::new(); + assert!(empty_reconstructor + .new_fragment([24u8; 43].to_vec()) + .is_none()); + assert_eq!(empty_reconstructor, MessageReconstructor::new()); + + let mut reconstructor_with_data = MessageReconstructor::new(); + let dummy_message = [24u8; UNFRAGMENTED_PAYLOAD_MAX_LEN + 30]; + let mut fragments = split_and_prepare_payloads(&dummy_message); + reconstructor_with_data.new_fragment(fragments.pop().unwrap()); + let reconstructor_clone = reconstructor_with_data.clone(); + + assert!(empty_reconstructor + .new_fragment([24u8; 43].to_vec()) + .is_none()); + assert_eq!(reconstructor_with_data, reconstructor_clone); + } +} + +#[cfg(test)] +mod message_reconstruction { + use super::*; + use crate::chunking::split_and_prepare_payloads; + use rand::seq::SliceRandom; + use rand::{thread_rng, RngCore}; + + #[cfg(test)] + mod single_set_split { + use super::*; + use crate::chunking::fragment::{ + UNFRAGMENTED_PAYLOAD_MAX_LEN, UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN, + }; + use crate::chunking::set::MAX_UNLINKED_SET_PAYLOAD_LENGTH; + + #[test] + fn it_reconstructs_unfragmented_message() { + let mut rng = thread_rng(); + + let mut message = vec![0u8; UNFRAGMENTED_PAYLOAD_MAX_LEN - 20]; + rng.fill_bytes(&mut message); + + let fragment = split_and_prepare_payloads(&message); + assert_eq!(fragment.len(), 1); + + let mut message_reconstructor = MessageReconstructor::new(); + assert_eq!( + message_reconstructor + .new_fragment(fragment[0].clone()) + .unwrap(), + message + ) + } + + #[test] + fn it_reconstructs_unfragmented_message_of_max_length() { + let mut rng = thread_rng(); + + let mut message = vec![0u8; UNFRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut message); + + let fragment = split_and_prepare_payloads(&message); + assert_eq!(fragment.len(), 1); + + let mut message_reconstructor = MessageReconstructor::new(); + assert_eq!( + message_reconstructor + .new_fragment(fragment[0].clone()) + .unwrap(), + message + ) + } + + #[test] + fn it_reconstructs_fragmented_message_in_order_of_2_max_lenghts() { + let mut rng = thread_rng(); + + let mut message = vec![0u8; 2 * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut message); + + let fragments = split_and_prepare_payloads(&message); + assert_eq!(fragments.len(), 2); + + let mut message_reconstructor = MessageReconstructor::new(); + assert!(message_reconstructor + .new_fragment(fragments[0].clone()) + .is_none()); + assert_eq!( + message_reconstructor + .new_fragment(fragments[1].clone()) + .unwrap(), + message + ) + } + + #[test] + fn it_reconstructs_fragmented_message_in_order_of_with_non_max_tail() { + let mut rng = thread_rng(); + + let mut message = vec![0u8; 2 * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN - 42]; + rng.fill_bytes(&mut message); + + let fragments = split_and_prepare_payloads(&message); + assert_eq!(fragments.len(), 2); + + let mut message_reconstructor = MessageReconstructor::new(); + assert!(message_reconstructor + .new_fragment(fragments[0].clone()) + .is_none()); + assert_eq!( + message_reconstructor + .new_fragment(fragments[1].clone()) + .unwrap(), + message + ) + } + + #[test] + fn it_reconstructs_fragmented_message_in_order_of_30_fragments() { + let mut rng = thread_rng(); + + let mut message = vec![0u8; 30 * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut message); + + let fragments = split_and_prepare_payloads(&message); + assert_eq!(fragments.len(), 30); + + let mut message_reconstructor = MessageReconstructor::new(); + for i in 0..29 { + assert!(message_reconstructor + .new_fragment(fragments[i].clone()) + .is_none()); + } + + assert_eq!( + message_reconstructor + .new_fragment(fragments[29].clone()) + .unwrap(), + message + ) + } + + #[test] + fn it_reconstructs_fragmented_message_not_in_order_of_30_fragments() { + let mut rng = thread_rng(); + + let mut message = vec![0u8; 30 * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut message); + + let mut fragments = split_and_prepare_payloads(&message); + assert_eq!(fragments.len(), 30); + + // shuffle the fragments + fragments.shuffle(&mut rng); + + let mut message_reconstructor = MessageReconstructor::new(); + for i in 0..29 { + assert!(message_reconstructor + .new_fragment(fragments[i].clone()) + .is_none()); + } + + assert_eq!( + message_reconstructor + .new_fragment(fragments[29].clone()) + .unwrap(), + message + ) + } + + #[test] + fn it_reconstructs_two_different_fragmented_messages_not_in_order_of_30_fragments_each() { + let mut rng = thread_rng(); + + let mut message1 = vec![0u8; 30 * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut message1); + let mut message2 = vec![0u8; 30 * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut message2); + // introduce dummy way to identify the messages + message1[0] = 1; + message2[0] = 2; + + let mut fragments1 = split_and_prepare_payloads(&message1); + assert_eq!(fragments1.len(), 30); + let mut fragments2 = split_and_prepare_payloads(&message2); + assert_eq!(fragments2.len(), 30); + + // combine and shuffle fragments + fragments1.append(fragments2.as_mut()); + fragments1.shuffle(&mut rng); + let fragments = fragments1; + assert_eq!(fragments.len(), 60); + + let mut message_reconstructor = MessageReconstructor::new(); + for fragment in fragments { + if let Some(msg) = message_reconstructor.new_fragment(fragment) { + match msg[0] { + 1 => assert_eq!(msg, message1), + 2 => assert_eq!(msg, message2), + _ => panic!("Unknown message!"), + } + } + } + } + + #[test] + fn it_reconstructs_two_different_messages_not_in_order_of_maximum_single_set_size_each() { + let mut rng = thread_rng(); + + let mut message1 = vec![0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH]; + rng.fill_bytes(&mut message1); + let mut message2 = vec![0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH]; + rng.fill_bytes(&mut message2); + // introduce dummy way to identify the messages + message1[0] = 1; + message2[0] = 2; + + let mut fragments1 = split_and_prepare_payloads(&message1); + assert_eq!(fragments1.len(), u8::max_value() as usize); + let mut fragments2 = split_and_prepare_payloads(&message2); + assert_eq!(fragments2.len(), u8::max_value() as usize); + + // combine and shuffle fragments + fragments1.append(fragments2.as_mut()); + fragments1.shuffle(&mut rng); + let fragments = fragments1; + assert_eq!(fragments.len(), (u8::max_value() as usize) * 2); + + let mut message_reconstructor = MessageReconstructor::new(); + for fragment in fragments.into_iter() { + if let Some(msg) = message_reconstructor.new_fragment(fragment) { + match msg[0] { + 1 => assert_eq!(msg, message1), + 2 => assert_eq!(msg, message2), + _ => panic!("Unknown message!"), + } + } + } + } + } + + #[cfg(test)] + mod multiple_sets_split { + use super::*; + use crate::chunking::set::{ + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH, TWO_WAY_LINKED_SET_PAYLOAD_LENGTH, + }; + + #[test] + fn it_reconstructs_fragmented_message_not_in_order_split_into_two_sets() { + let mut rng = thread_rng(); + + let mut message = vec![0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 12345]; + rng.fill_bytes(&mut message); + + let mut fragments = split_and_prepare_payloads(&message); + // shuffle the fragments + fragments.shuffle(&mut rng); + + let mut message_reconstructor = MessageReconstructor::new(); + let mut finished_reconstruction = false; + for fragment in fragments.into_iter() { + if finished_reconstruction { + panic!( + "Shouldn't have gone into another iteration if message was reconstructed!" + ) + } + if let Some(msg) = message_reconstructor.new_fragment(fragment) { + assert_eq!(msg, message); + finished_reconstruction = true; + } + } + } + + #[test] + fn it_reconstructs_fragmented_message_not_in_order_split_into_four_sets() { + let mut rng = thread_rng(); + + let mut message = vec![ + 0u8; + 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + + 12345 + ]; + rng.fill_bytes(&mut message); + + let mut fragments = split_and_prepare_payloads(&message); + // shuffle the fragments + fragments.shuffle(&mut rng); + + let mut message_reconstructor = MessageReconstructor::new(); + let mut finished_reconstruction = false; + for fragment in fragments.into_iter() { + if finished_reconstruction { + panic!( + "Shouldn't have gone into another iteration if message was reconstructed!" + ) + } + if let Some(msg) = message_reconstructor.new_fragment(fragment) { + assert_eq!(msg, message); + finished_reconstruction = true; + } + } + } + + #[test] + fn it_reconstructs_fragmented_message_not_in_order_split_into_four_full_sets() { + let mut rng = thread_rng(); + + let mut message = vec![ + 0u8; + 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + ]; + rng.fill_bytes(&mut message); + + let mut fragments = split_and_prepare_payloads(&message); + assert_eq!(fragments.len(), 4 * (u8::max_value() as usize)); + // shuffle the fragments + fragments.shuffle(&mut rng); + + let mut message_reconstructor = MessageReconstructor::new(); + let mut finished_reconstruction = false; + for fragment in fragments.into_iter() { + if finished_reconstruction { + panic!( + "Shouldn't have gone into another iteration if message was reconstructed!" + ) + } + if let Some(msg) = message_reconstructor.new_fragment(fragment) { + assert_eq!(msg, message); + finished_reconstruction = true; + } + } + } + + #[test] + fn it_reconstructs_two_fragmented_messages_not_in_order_split_into_four_sets() { + let mut rng = thread_rng(); + + let mut message1 = vec![ + 0u8; + 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + ]; + rng.fill_bytes(&mut message1); + let mut message2 = vec![ + 0u8; + 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + ]; + rng.fill_bytes(&mut message2); + // introduce dummy way to identify the messages + message1[0] = 1; + message2[0] = 2; + + let mut fragments1 = split_and_prepare_payloads(&message1); + assert_eq!(fragments1.len(), 4 * (u8::max_value() as usize)); + let mut fragments2 = split_and_prepare_payloads(&message2); + assert_eq!(fragments2.len(), 4 * (u8::max_value() as usize)); + + // combine and shuffle fragments + fragments1.append(fragments2.as_mut()); + fragments1.shuffle(&mut rng); + let fragments = fragments1; + assert_eq!(fragments.len(), (u8::max_value() as usize) * 8); + + let mut message_reconstructor = MessageReconstructor::new(); + for fragment in fragments.into_iter() { + if let Some(msg) = message_reconstructor.new_fragment(fragment) { + match msg[0] { + 1 => assert_eq!(msg, message1), + 2 => assert_eq!(msg, message2), + _ => panic!("Unknown message!"), + } + } + } + } + } +} diff --git a/common/nymsphinx/src/chunking/set.rs b/common/nymsphinx/src/chunking/set.rs new file mode 100644 index 0000000000..582b501960 --- /dev/null +++ b/common/nymsphinx/src/chunking/set.rs @@ -0,0 +1,788 @@ +use crate::chunking::fragment::{ + Fragment, LINKED_FRAGMENTED_HEADER_LEN, LINKED_FRAGMENTED_PAYLOAD_MAX_LEN, + UNFRAGMENTED_PAYLOAD_MAX_LEN, UNLINKED_FRAGMENTED_HEADER_LEN, + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN, +}; +use rand::Rng; + +/// In the simplest case of message being divided into a single set, the set has the upper bound +/// on its payload length of the maximum number of `Fragment`s multiplied by their maximum, +/// fragmented, length. +pub const MAX_UNLINKED_SET_PAYLOAD_LENGTH: usize = + u8::max_value() as usize * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN; + +/// If the set is being linked to another one, by either being the very first set, or the very last, +/// one of its `Fragment`s must be changed from "unlinked" into "linked" to compensate for a tiny +/// bit extra data overhead: id of the other set. +/// Note that the "MAX" prefix only applies to if the set is the last one as it does not have +/// a lower bound on its length. If the set is one way linked and a first one, it *must have* +/// this exact payload length instead. +pub const MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH: usize = MAX_UNLINKED_SET_PAYLOAD_LENGTH + - (LINKED_FRAGMENTED_HEADER_LEN - UNLINKED_FRAGMENTED_HEADER_LEN); + +/// If the set is being linked two others sets by being stuck in the middle of divided message, +/// two of its `Fragment`s (first and final one) must be changed from +/// "unlinked" into "linked" to compensate for data overhead. +/// Note that this constant no longer has a "MAX" prefix, this is because each set being stuck +/// between different sets, *must* have this exact payload length. +pub const TWO_WAY_LINKED_SET_PAYLOAD_LENGTH: usize = MAX_UNLINKED_SET_PAYLOAD_LENGTH + - 2 * (LINKED_FRAGMENTED_HEADER_LEN - UNLINKED_FRAGMENTED_HEADER_LEN); + +/// `FragmentSet` is an ordered collection of 1 to 255 `Fragment`s, each with the same ID +/// that can be used to produce original message, assuming no linking took place. +/// +/// Otherwise, if set linking took place, then first or last `Fragment` from the `FragmentSet` +/// is used to determine preceding or succeeding other `FragmentSet` +/// that should be used in tandem to reconstruct original message. The linking reconstruction +/// is a recursive process as a message could have been divided into an arbitrary number of +/// `FragmentSet`s with no upper bound at all. +/// +/// For example if a message was divided into 300 `Fragment`s (i.e. 2 `FragmentSet`s, +/// the structures might look as follows: +/// +/// Set1: [f1 {id = 12345}, f2 {id = 12345}, ... f255 {id = 12345, next_id = 54321}] +/// Set2: [f1 {id = 54321, previous_id = 12345}, f2 {id = 54321}, ... f45 {id = 54321}] +pub(crate) type FragmentSet = Vec; + +/// Generate a pseudo-random id for a `FragmentSet`. +/// Its value is restricted to (0, i32::max_value()]. +/// Note that it *excludes* 0, but *includes* i32::max_value(). +/// This particular range allows for the id to be represented using 31bits, rather than +/// the full length of 32 while still providing more than enough variability to +/// distinguish different `FragmentSet`s. +/// The extra bit, as explained in `Fragment` definition is used to represents additional information, +/// indicating how further bytes should be parsed. +/// This approach saves whole byte per `Fragment`, which while may seem insignificant and +/// introduces extra complexity, quickly adds up when faced with sphinx packet encapsulation for longer +/// messages. +/// Finally, the reason 0 id is not allowed is to explicitly distinguish it from unfragmented +/// `Fragment`s thus allowing for some additional optimizations by letting it skip +/// certain procedures when reconstructing. +fn generate_set_id(rng: &mut R) -> i32 { + let potential_id = rng.gen::().abs(); + // make sure id is always non-zero, as we do not want to accidentally have weird + // reconstruction cases where unfragmented payload overwrites some part of set with id0 + if potential_id == 0 { + generate_set_id(rng) + } else { + potential_id + } +} + +/// The simplest case of splitting underlying message - when it fits into a single +/// `Fragment` thus requiring no linking or even a set id. +/// For obvious reasons the most efficient approach. +fn prepare_unfragmented_set(message: &[u8]) -> FragmentSet { + vec![Fragment::try_new_unfragmented(&message).unwrap()] +} + +/// Splits underlying message into multiple `Fragment`s while all of them fit in a single +/// `Set` (number of `Fragment`s <= 255) +fn prepare_unlinked_fragmented_set(message: &[u8], id: i32) -> FragmentSet { + let pre_casted_frags = + (message.len() as f64 / UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN as f64).ceil() as usize; + + debug_assert!(pre_casted_frags <= u8::max_value() as usize); + let num_fragments = pre_casted_frags as u8; + + let mut fragments = Vec::with_capacity(num_fragments as usize); + + for i in 1..(pre_casted_frags + 1) { + // we can't use u8 directly here as upper (NON-INCLUSIVE, so it would always fit) bound could be u8::max_value() + 1 + let lb = (i as usize - 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN; + let ub = usize::min( + message.len(), + i as usize * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN, + ); + fragments.push( + Fragment::try_new_fragmented(&message[lb..ub], id, num_fragments, i as u8, None, None) + .unwrap(), + ) + } + + fragments +} + +/// Similarly to `prepare_unlinked_fragmented_set`, splits part of underlying message into +/// multiple `Fragment`s. The byte slice of the message *must* fit into a single linked set, however, +/// the whole message itself is still longer than a single `Set` (number of `Fragment`s > 255). +/// During the process of splitting message, this function is called multiple times. +fn prepare_linked_fragment_set( + message: &[u8], + id: i32, + previous_link_id: Option, + next_link_id: Option, +) -> FragmentSet { + // determine number of fragments in the set: + let num_frags_usize = if next_link_id.is_some() { + u8::max_value() as usize + } else { + // we know this set is linked, if it's not post-linked then it MUST BE pre-linked + let tail_len = if message.len() >= LINKED_FRAGMENTED_PAYLOAD_MAX_LEN { + message.len() - LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + } else { + 0 + }; + let pre_casted_frags = + 1 + (tail_len as f64 / UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN as f64).ceil() as usize; + if pre_casted_frags > u8::max_value() as usize { + panic!("message would produce too many fragments!") + }; + pre_casted_frags + }; + + // determine bounds for the first fragment which depends on whether set is pre-linked + let mut lb = 0; + let mut ub = if previous_link_id.is_some() { + usize::min(message.len(), LINKED_FRAGMENTED_PAYLOAD_MAX_LEN) + } else { + // the set might be linked, but fragment itself is not (i.e. the set is linked at the tail) + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + }; + + let mut fragments = Vec::with_capacity(num_frags_usize); + for i in 1..(num_frags_usize + 1) { + // we can't use u8 directly here as upper (NON-INCLUSIVE, so i would always fit) bound could be u8::max_value() + 1 + let fragment = Fragment::try_new_fragmented( + &message[lb..ub], + id, + num_frags_usize as u8, + i as u8, + if i == 1 { previous_link_id } else { None }, + if i == num_frags_usize { + next_link_id + } else { + None + }, + ) + .unwrap(); + + fragments.push(fragment); + // update bounds for the next fragment + lb = ub; + ub = usize::min(message.len(), ub + UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN); + } + + fragments +} + +/// Based on total message length, determines the number of sets into which it is going to be split. +fn total_number_of_sets(message_len: usize) -> usize { + if message_len <= MAX_UNLINKED_SET_PAYLOAD_LENGTH { + 1 + } else if message_len > MAX_UNLINKED_SET_PAYLOAD_LENGTH + && message_len <= 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + { + 2 + } else { + let len_without_edges = message_len - 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH; + // every set in between edges must be two way linked + (len_without_edges as f64 / TWO_WAY_LINKED_SET_PAYLOAD_LENGTH as f64).ceil() as usize + 2 + } +} + +/// Given part of the underlying message as well id of the set as well as its potential linked sets, +/// correctly delegates to appropriate set constructor. +fn prepare_fragment_set( + message: &[u8], + id: i32, + previous_link_id: Option, + next_link_id: Option, +) -> FragmentSet { + if previous_link_id.is_some() || next_link_id.is_some() { + prepare_linked_fragment_set(message, id, previous_link_id, next_link_id) + } else if message.len() > UNFRAGMENTED_PAYLOAD_MAX_LEN { + // the bounds on whether the message fits in an unlinked set should have been done by the callee + // when determining ids of other sets + prepare_unlinked_fragmented_set(message, id) + } else { + prepare_unfragmented_set(message) + } +} + +/// Entry point for splitting whole message into possibly multiple `Set`s. +pub(crate) fn split_into_sets(message: &[u8]) -> Vec { + use rand::thread_rng; + + let mut rng = thread_rng(); + let num_of_sets = total_number_of_sets(message.len()); + if num_of_sets == 1 { + let set_id = generate_set_id(&mut rng); + vec![prepare_fragment_set(message, set_id, None, None)] + } else { + let mut sets = Vec::with_capacity(num_of_sets); + // pre-generate all ids for the sets + let set_ids: Vec<_> = std::iter::repeat(()) + .map(|_| generate_set_id(&mut rng)) + .take(num_of_sets) + .collect(); + + // initial bounds for the set payloads + let mut lb = 0; + let mut ub = MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH; + + for i in 0..num_of_sets { + let fragment_set = prepare_fragment_set( + &message[lb..ub], + set_ids[i], + if i == 0 { None } else { Some(set_ids[i - 1]) }, + if i == (num_of_sets - 1) { + None + } else { + Some(set_ids[i + 1]) + }, + ); + + sets.push(fragment_set); + // update bounds for the next set + lb = ub; + ub = if i == num_of_sets - 2 { + // we're going to go into the last iteration now, hence the last set will be one-way linked + usize::min(message.len(), ub + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH) + } else { + usize::min(message.len(), ub + TWO_WAY_LINKED_SET_PAYLOAD_LENGTH) + } + } + + sets + } +} + +// reason for top level tests module is to be able to use the helper functions to verify sets payloads +#[cfg(test)] +mod tests { + use super::*; + + fn verify_unlinked_set_payload(mut set: FragmentSet, payload: &[u8]) { + for i in (0..set.len()).rev() { + assert_eq!( + set.pop().unwrap().extract_payload(), + payload[i * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + ..usize::min(payload.len(), (i + 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN)] + .to_vec() + ) + } + } + + fn verify_pre_linked_set_payload(mut set: FragmentSet, payload: &[u8]) { + for i in (0..set.len()).rev() { + let lb = if i == 0 { + 0 + } else { + (i - 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + }; + let ub = usize::min( + payload.len(), + i * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + LINKED_FRAGMENTED_PAYLOAD_MAX_LEN, + ); + + assert_eq!( + set.pop().unwrap().extract_payload(), + payload[lb..ub].to_vec() + ) + } + } + + fn verify_post_linked_set_payload(mut set: FragmentSet, payload: &[u8]) { + for i in (0..set.len()).rev() { + let lb = i * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN; + let ub = if i == (u8::max_value() as usize - 1) { + i * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + } else { + (i + 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + }; + + assert_eq!( + set.pop().unwrap().extract_payload(), + payload[lb..ub].to_vec(), + ) + } + } + + fn verify_two_way_linked_set_payload(mut set: FragmentSet, payload: &[u8]) { + for i in (0..set.len()).rev() { + let lb = if i == 0 { + 0 + } else { + (i - 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + }; + let ub = if i == (u8::max_value() as usize - 1) { + (i - 1) * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + + 2 * LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + } else { + i * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + }; + + assert_eq!( + set.pop().unwrap().extract_payload(), + payload[lb..ub].to_vec(), + ) + } + } + + fn verfiy_correct_link(left: &FragmentSet, right: &FragmentSet) { + let first_id = left[0].id(); + let post_id = left[254].next_fragments_set_id().unwrap(); + + let second_id = right[0].id(); + let pre_id = right[0].previous_fragments_set_id().unwrap(); + + assert_eq!(first_id, pre_id); + assert_eq!(second_id, post_id); + } + + #[cfg(test)] + mod preparing_unfragmented_set { + use super::*; + + #[test] + fn makes_set_with_single_unfragmented_element_for_valid_message_lengths() { + let mut set = prepare_unfragmented_set(&[1]); + assert_eq!(1, set.len()); + assert_eq!(set.pop().unwrap().extract_payload(), [1].to_vec()); + + let mut set = prepare_unfragmented_set(&[1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN]); + assert_eq!(1, set.len()); + assert_eq!( + set.pop().unwrap().extract_payload(), + [1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN].to_vec() + ); + } + + #[test] + #[should_panic] + fn panics_for_too_long_payload() { + prepare_unfragmented_set(&[1u8; UNFRAGMENTED_PAYLOAD_MAX_LEN + 1]); + } + } + + #[cfg(test)] + mod preparing_unlinked_set { + // remember this this is only called for a sole set with <= 255 fragments + use super::*; + use rand::{thread_rng, RngCore}; + + #[test] + fn makes_set_with_correctly_split_payload() { + let id = 12345; + let mut rng = thread_rng(); + + let mut two_element_set_payload = [0u8; UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1]; + rng.fill_bytes(&mut two_element_set_payload); + let two_element_set = prepare_unlinked_fragmented_set(&two_element_set_payload, id); + assert_eq!(2, two_element_set.len()); + verify_unlinked_set_payload(two_element_set, &two_element_set_payload); + + let mut forty_two_element_set_payload = + [0u8; 41 * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 42]; + rng.fill_bytes(&mut forty_two_element_set_payload); + let forty_two_element_set = + prepare_unlinked_fragmented_set(&forty_two_element_set_payload, id); + assert_eq!(42, forty_two_element_set.len()); + verify_unlinked_set_payload(forty_two_element_set, &forty_two_element_set_payload); + + let mut max_fragments_set_payload = + [0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH - UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1]; // last fragment should have a single byte of data + rng.fill_bytes(&mut max_fragments_set_payload); + let max_fragment_set = prepare_unlinked_fragmented_set(&max_fragments_set_payload, id); + assert_eq!(u8::max_value() as usize, max_fragment_set.len()); + verify_unlinked_set_payload(max_fragment_set, &max_fragments_set_payload); + + let mut full_set_payload = [0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH]; + rng.fill_bytes(&mut full_set_payload); + let full_fragment_set = prepare_unlinked_fragmented_set(&full_set_payload, id); + assert_eq!(u8::max_value() as usize, full_fragment_set.len()); + verify_unlinked_set_payload(full_fragment_set, &full_set_payload); + } + + #[test] + #[should_panic] + fn panics_for_too_long_payload() { + prepare_unlinked_fragmented_set(&[0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH + 1], 12345); + } + } + + #[cfg(test)] + mod preparing_linked_set { + use super::*; + use rand::{thread_rng, RngCore}; + + #[test] + fn makes_set_with_correctly_split_payload_for_pre_linked_set() { + let id = 12345; + let link_id = 1234; + let mut rng = thread_rng(); + + let mut two_element_set_payload = [0u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1]; + rng.fill_bytes(&mut two_element_set_payload); + let two_element_set = + prepare_linked_fragment_set(&two_element_set_payload, id, Some(link_id), None); + assert_eq!(2, two_element_set.len()); + verify_pre_linked_set_payload(two_element_set, &two_element_set_payload); + + let mut forty_two_element_set_payload = [0u8; LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + + 40 * UNLINKED_FRAGMENTED_PAYLOAD_MAX_LEN + + 42]; + rng.fill_bytes(&mut forty_two_element_set_payload); + let forty_two_element_set = prepare_linked_fragment_set( + &forty_two_element_set_payload, + id, + Some(link_id), + None, + ); + assert_eq!(42, forty_two_element_set.len()); + verify_pre_linked_set_payload(forty_two_element_set, &forty_two_element_set_payload); + + let mut max_fragments_set_payload = + [0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH - LINKED_FRAGMENTED_PAYLOAD_MAX_LEN + 1]; // last fragment should have a single byte of data + rng.fill_bytes(&mut max_fragments_set_payload); + + let max_fragment_set = + prepare_linked_fragment_set(&max_fragments_set_payload, id, Some(link_id), None); + assert_eq!(u8::max_value() as usize, max_fragment_set.len()); + verify_pre_linked_set_payload(max_fragment_set, &max_fragments_set_payload); + + let mut full_set_payload = [0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH]; + rng.fill_bytes(&mut full_set_payload); + let full_fragment_set = + prepare_linked_fragment_set(&full_set_payload, id, Some(link_id), None); + assert_eq!(u8::max_value() as usize, full_fragment_set.len()); + verify_pre_linked_set_payload(full_fragment_set, &full_set_payload); + } + + #[test] + #[should_panic] + fn panics_for_too_long_payload_for_pre_linked_set() { + prepare_linked_fragment_set( + &[0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 1], + 12345, + Some(1234), + None, + ); + } + + #[test] + fn makes_set_with_correctly_split_payload_for_post_linked_set() { + let id = 12345; + let link_id = 1234; + let mut rng = thread_rng(); + + // if set is post-linked, there is only a single valid case - full length payload + let mut full_set_payload = [0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH]; + rng.fill_bytes(&mut full_set_payload); + let full_fragment_set = + prepare_linked_fragment_set(&full_set_payload, id, None, Some(link_id)); + assert_eq!(u8::max_value() as usize, full_fragment_set.len()); + verify_post_linked_set_payload(full_fragment_set, &full_set_payload); + } + + #[test] + #[should_panic] + fn panics_for_too_long_payload_for_post_linked_set() { + prepare_linked_fragment_set( + &[0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 1], + 12345, + None, + Some(1234), + ); + } + + #[test] + #[should_panic] + fn panics_for_too_short_payload_for_post_linked_set() { + prepare_linked_fragment_set( + &[0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH - 1], + 12345, + None, + Some(1234), + ); + } + + #[test] + fn makes_set_with_correctly_split_payload_for_two_way_linked_set() { + // again, relatively simple case - + // if set is two-way-linked, there is only a single valid case - full length payload + let id = 12345; + let pre_link_id = 1234; + let post_link_id = 123456; + let mut rng = thread_rng(); + + let mut full_set_payload = [0u8; TWO_WAY_LINKED_SET_PAYLOAD_LENGTH]; + rng.fill_bytes(&mut full_set_payload); + let full_fragment_set = prepare_linked_fragment_set( + &full_set_payload, + id, + Some(pre_link_id), + Some(post_link_id), + ); + assert_eq!(u8::max_value() as usize, full_fragment_set.len()); + verify_two_way_linked_set_payload(full_fragment_set, &full_set_payload); + } + + #[test] + #[should_panic] + fn panics_for_too_long_payload_for_two_way_linked_set() { + prepare_linked_fragment_set( + &[0u8; TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + 1], + 12345, + Some(123456), + Some(1234), + ); + } + + #[test] + #[should_panic] + fn panics_for_too_short_payload_for_two_way_linked_set() { + prepare_linked_fragment_set( + &[0u8; TWO_WAY_LINKED_SET_PAYLOAD_LENGTH - 1], + 12345, + Some(123456), + Some(1234), + ); + } + } + + #[cfg(test)] + mod splitting_into_sets { + use super::*; + use rand::{thread_rng, RngCore}; + + #[test] + fn correctly_creates_set_with_single_unfragmented_element_when_expected() { + let mut rng = thread_rng(); + let tiny_message = [1, 2, 3, 4, 5]; + let mut max_unfragmented_message = [0u8; UNFRAGMENTED_PAYLOAD_MAX_LEN]; + rng.fill_bytes(&mut max_unfragmented_message); + + let mut sets = split_into_sets(&tiny_message); + assert_eq!(1, sets.len()); + assert_eq!(1, sets[0].len()); + assert_eq!( + tiny_message.to_vec(), + sets.pop().unwrap().pop().unwrap().extract_payload() + ); + + let mut sets = split_into_sets(&max_unfragmented_message); + assert_eq!(1, sets.len()); + assert_eq!(1, sets[0].len()); + assert_eq!( + max_unfragmented_message.to_vec(), + sets.pop().unwrap().pop().unwrap().extract_payload() + ); + } + + #[test] + fn correctly_creates_single_fragmented_set_when_expected() { + let mut rng = thread_rng(); + let mut message = [0u8; MAX_UNLINKED_SET_PAYLOAD_LENGTH - 2345]; + rng.fill_bytes(&mut message); + + let mut sets = split_into_sets(&message); + assert_eq!(1, sets.len()); + verify_unlinked_set_payload(sets.pop().unwrap(), &message); + } + + // a very specific test case that would have saved a lot of headache if was introduced + // earlier... + #[test] + fn correctly_creates_two_singly_linked_sets_with_second_set_containing_data_fitting_in_unfragmented_payload( + ) { + let mut rng = thread_rng(); + let mut message = [0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 123]; + rng.fill_bytes(&mut message); + let mut sets = split_into_sets(&message); + assert_eq!(2, sets.len()); + verfiy_correct_link(&sets[0], &sets[1]); + verify_pre_linked_set_payload( + sets.pop().unwrap(), + &message[MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH..], + ); + verify_post_linked_set_payload( + sets.pop().unwrap(), + &message[..MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH], + ); + } + + #[test] + fn correctly_creates_two_singly_linked_sets_when_expected() { + let mut rng = thread_rng(); + let mut message = [0u8; MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 2345]; + rng.fill_bytes(&mut message); + let mut sets = split_into_sets(&message); + assert_eq!(2, sets.len()); + verfiy_correct_link(&sets[0], &sets[1]); + verify_pre_linked_set_payload( + sets.pop().unwrap(), + &message[MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH..], + ); + verify_post_linked_set_payload( + sets.pop().unwrap(), + &message[..MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH], + ); + + let mut message = [0u8; 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH]; + rng.fill_bytes(&mut message); + + let mut sets = split_into_sets(&message); + assert_eq!(2, sets.len()); + assert_eq!(sets[0].len(), u8::max_value() as usize); + assert_eq!(sets[1].len(), u8::max_value() as usize); + verfiy_correct_link(&sets[0], &sets[1]); + verify_pre_linked_set_payload( + sets.pop().unwrap(), + &message[MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH..], + ); + + verify_post_linked_set_payload( + sets.pop().unwrap(), + &message[..MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH], + ); + } + + #[test] + fn correctly_creates_four_correctly_formed_sets_when_expected() { + let mut rng = thread_rng(); + let mut message = [0u8; 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + + 2345]; + rng.fill_bytes(&mut message); + let mut sets = split_into_sets(&message); + assert_eq!(4, sets.len()); + assert_eq!(sets[0].len(), u8::max_value() as usize); + assert_eq!(sets[1].len(), u8::max_value() as usize); + assert_eq!(sets[2].len(), u8::max_value() as usize); + + verfiy_correct_link(&sets[0], &sets[1]); + verfiy_correct_link(&sets[1], &sets[2]); + verfiy_correct_link(&sets[2], &sets[3]); + verify_pre_linked_set_payload( + sets.pop().unwrap(), + &message[2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH..], + ); + verify_two_way_linked_set_payload( + sets.pop().unwrap(), + &message[TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + ..2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH], + ); + verify_two_way_linked_set_payload( + sets.pop().unwrap(), + &message[MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + ..TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH], + ); + verify_post_linked_set_payload( + sets.pop().unwrap(), + &message[..MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH], + ); + + let mut message = [0u8; 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH]; + rng.fill_bytes(&mut message); + + let mut sets = split_into_sets(&message); + assert_eq!(4, sets.len()); + assert_eq!(sets[0].len(), u8::max_value() as usize); + assert_eq!(sets[1].len(), u8::max_value() as usize); + assert_eq!(sets[2].len(), u8::max_value() as usize); + assert_eq!(sets[3].len(), u8::max_value() as usize); + + verfiy_correct_link(&sets[0], &sets[1]); + verfiy_correct_link(&sets[1], &sets[2]); + verfiy_correct_link(&sets[2], &sets[3]); + verify_pre_linked_set_payload( + sets.pop().unwrap(), + &message[2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH..], + ); + verify_two_way_linked_set_payload( + sets.pop().unwrap(), + &message[TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + ..2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH], + ); + verify_two_way_linked_set_payload( + sets.pop().unwrap(), + &message[MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + ..TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH], + ); + verify_post_linked_set_payload( + sets.pop().unwrap(), + &message[..MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH], + ); + } + } + + #[cfg(test)] + mod helpers { + use super::*; + + #[test] + fn total_number_of_sets() { + assert_eq!( + 1, + super::total_number_of_sets(MAX_UNLINKED_SET_PAYLOAD_LENGTH - 1) + ); + assert_eq!( + 1, + super::total_number_of_sets(MAX_UNLINKED_SET_PAYLOAD_LENGTH) + ); + assert_eq!( + 2, + super::total_number_of_sets(MAX_UNLINKED_SET_PAYLOAD_LENGTH + 1) + ); + assert_eq!( + 2, + super::total_number_of_sets(2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH) + ); + assert_eq!( + 3, + super::total_number_of_sets(2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + 1) + ); + assert_eq!( + 3, + super::total_number_of_sets( + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + - 1 + ) + ); + assert_eq!( + 3, + super::total_number_of_sets( + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + ) + ); + assert_eq!( + 4, + super::total_number_of_sets( + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + + TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + 1 + ) + ); + assert_eq!( + 4, + super::total_number_of_sets( + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + + 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + - 1 + ) + ); + assert_eq!( + 4, + super::total_number_of_sets( + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + + 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + ) + ); + assert_eq!( + 5, + super::total_number_of_sets( + 2 * MAX_ONE_WAY_LINKED_SET_PAYLOAD_LENGTH + + 2 * TWO_WAY_LINKED_SET_PAYLOAD_LENGTH + + 1 + ) + ); + } + } +} diff --git a/common/nymsphinx/src/lib.rs b/common/nymsphinx/src/lib.rs new file mode 100644 index 0000000000..79bf24329f --- /dev/null +++ b/common/nymsphinx/src/lib.rs @@ -0,0 +1,9 @@ +pub mod addressing; +pub mod chunking; + +// Future consideration: currently in a lot of places, the payloads have randomised content +// which is not a perfect testing strategy as it might not detect some edge cases I never would +// have assumed could be possible. A better approach would be to research some Fuzz testing +// library like: https://github.com/rust-fuzz/afl.rs and use that instead for the inputs. + +// perhaps it might be useful down the line for interaction testing between client,mixes,etc? diff --git a/common/topology/Cargo.toml b/common/topology/Cargo.toml index b414460c82..4f64d5f429 100644 --- a/common/topology/Cargo.toml +++ b/common/topology/Cargo.toml @@ -15,9 +15,9 @@ rand = "0.7.2" serde = { version = "1.0.104", features = ["derive"] } ## internal -addressing = {path = "../addressing"} +nymsphinx = {path = "../nymsphinx"} version-checker = {path = "../version-checker" } ## will be moved to proper dependencies once released -sphinx = { git = "https://github.com/nymtech/sphinx", rev="23f9c89b257ee0936e70afd682e9ed6a62e89eee" } +sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" } # sphinx = { path = "../../../sphinx"} diff --git a/common/topology/src/lib.rs b/common/topology/src/lib.rs index eb81cdfd42..ff944eafe4 100644 --- a/common/topology/src/lib.rs +++ b/common/topology/src/lib.rs @@ -114,7 +114,9 @@ pub trait NymTopology: Sized + std::fmt::Debug + Send + Sync + Clone { } fn can_construct_path_through(&self) -> bool { - self.make_layered_topology().is_ok() + !self.mix_nodes().is_empty() + && !self.providers().is_empty() + && self.make_layered_topology().is_ok() } } diff --git a/common/topology/src/mix.rs b/common/topology/src/mix.rs index abf04e72b5..fd5ae78fee 100644 --- a/common/topology/src/mix.rs +++ b/common/topology/src/mix.rs @@ -1,6 +1,7 @@ use crate::filter; +use nymsphinx::addressing::nodes::NymNodeRoutingAddress; use sphinx::route::Node as SphinxNode; -use sphinx::route::NodeAddressBytes; +use std::convert::TryInto; use std::net::SocketAddr; #[derive(Debug, Clone)] @@ -29,10 +30,10 @@ impl filter::Versioned for Node { impl Into for Node { fn into(self) -> SphinxNode { - let address_bytes = addressing::encoded_bytes_from_socket_address(self.host); + let node_address_bytes = NymNodeRoutingAddress::from(self.host).try_into().unwrap(); let key_bytes = self.get_pub_key_bytes(); let key = sphinx::key::new(key_bytes); - SphinxNode::new(NodeAddressBytes::from_bytes(address_bytes), key) + SphinxNode::new(node_address_bytes, key) } } diff --git a/common/topology/src/provider.rs b/common/topology/src/provider.rs index 7687489e3c..d1ed27e909 100644 --- a/common/topology/src/provider.rs +++ b/common/topology/src/provider.rs @@ -1,6 +1,7 @@ use crate::filter; +use nymsphinx::addressing::nodes::NymNodeRoutingAddress; use sphinx::route::Node as SphinxNode; -use sphinx::route::NodeAddressBytes; +use std::convert::TryInto; use std::net::SocketAddr; #[derive(Debug, Clone)] @@ -35,10 +36,12 @@ impl filter::Versioned for Node { impl Into for Node { fn into(self) -> SphinxNode { - let address_bytes = addressing::encoded_bytes_from_socket_address(self.mixnet_listener); + let node_address_bytes = NymNodeRoutingAddress::from(self.mixnet_listener) + .try_into() + .unwrap(); let key_bytes = self.get_pub_key_bytes(); let key = sphinx::key::new(key_bytes); - SphinxNode::new(NodeAddressBytes::from_bytes(address_bytes), key) + SphinxNode::new(node_address_bytes, key) } } diff --git a/mixnode/Cargo.toml b/mixnode/Cargo.toml index 68c8689a1d..33f91d3afd 100644 --- a/mixnode/Cargo.toml +++ b/mixnode/Cargo.toml @@ -1,7 +1,7 @@ [package] build = "build.rs" name = "nym-mixnode" -version = "0.5.0" +version = "0.6.0" authors = ["Dave Hrycyszyn ", "Jędrzej Stuczyński "] edition = "2018" @@ -20,15 +20,16 @@ serde = { version = "1.0.104", features = ["derive"] } tokio = { version = "0.2", features = ["full"] } ## internal -addressing = {path = "../common/addressing" } config = {path = "../common/config"} crypto = {path = "../common/crypto"} directory-client = { path = "../common/clients/directory-client" } multi-tcp-client = { path = "../common/clients/multi-tcp-client" } +nymsphinx = {path = "../common/nymsphinx" } pemstore = {path = "../common/pemstore"} +topology = {path = "../common/topology"} ## will be moved to proper dependencies once released -sphinx = { git = "https://github.com/nymtech/sphinx", rev="23f9c89b257ee0936e70afd682e9ed6a62e89eee" } +sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" } [build-dependencies] built = "0.3.2" diff --git a/mixnode/src/config/mod.rs b/mixnode/src/config/mod.rs index e69c7a3431..75f206aa52 100644 --- a/mixnode/src/config/mod.rs +++ b/mixnode/src/config/mod.rs @@ -17,8 +17,10 @@ const DEFAULT_LISTENING_PORT: u16 = 1789; // where applicable, the below are defined in milliseconds const DEFAULT_PRESENCE_SENDING_DELAY: u64 = 1500; const DEFAULT_METRICS_SENDING_DELAY: u64 = 1000; +const DEFAULT_METRICS_RUNNING_STATS_LOGGING_DELAY: u64 = 60_000; // 1min const DEFAULT_PACKET_FORWARDING_INITIAL_BACKOFF: u64 = 10_000; // 10s const DEFAULT_PACKET_FORWARDING_MAXIMUM_BACKOFF: u64 = 300_000; // 5min +const DEFAULT_INITIAL_CONNECTION_TIMEOUT: u64 = 1_500; // 1.5s #[derive(Debug, Default, Deserialize, PartialEq, Serialize)] #[serde(deny_unknown_fields)] @@ -213,6 +215,10 @@ impl Config { time::Duration::from_millis(self.debug.metrics_sending_delay) } + pub fn get_metrics_running_stats_logging_delay(&self) -> time::Duration { + time::Duration::from_millis(self.debug.metrics_running_stats_logging_delay) + } + pub fn get_layer(&self) -> u64 { self.mixnode.layer } @@ -232,6 +238,10 @@ impl Config { pub fn get_packet_forwarding_maximum_backoff(&self) -> time::Duration { time::Duration::from_millis(self.debug.packet_forwarding_maximum_backoff) } + + pub fn get_initial_connection_timeout(&self) -> time::Duration { + time::Duration::from_millis(self.debug.initial_connection_timeout) + } } #[derive(Debug, Deserialize, PartialEq, Serialize)] @@ -313,7 +323,7 @@ impl Default for Logging { } #[derive(Debug, Deserialize, PartialEq, Serialize)] -#[serde(deny_unknown_fields)] +#[serde(default, deny_unknown_fields)] pub struct Debug { // The idea of additional 'directory servers' is to let mixes report their presence // and metrics to separate places @@ -331,6 +341,10 @@ pub struct Debug { /// The provided value is interpreted as milliseconds. metrics_sending_delay: u64, + /// Delay between each subsequent running metrics statistics being logged. + /// The provided value is interpreted as milliseconds. + metrics_running_stats_logging_delay: u64, + /// Initial value of an exponential backoff to reconnect to dropped TCP connection when /// forwarding sphinx packets. /// The provided value is interpreted as milliseconds. @@ -340,6 +354,10 @@ pub struct Debug { /// forwarding sphinx packets. /// The provided value is interpreted as milliseconds. packet_forwarding_maximum_backoff: u64, + + /// Timeout for establishing initial connection when trying to forward a sphinx packet. + /// The provider value is interpreted as milliseconds. + initial_connection_timeout: u64, } impl Debug { @@ -360,8 +378,10 @@ impl Default for Debug { presence_sending_delay: DEFAULT_PRESENCE_SENDING_DELAY, metrics_directory_server: Self::default_directory_server(), metrics_sending_delay: DEFAULT_METRICS_SENDING_DELAY, + metrics_running_stats_logging_delay: DEFAULT_METRICS_RUNNING_STATS_LOGGING_DELAY, packet_forwarding_initial_backoff: DEFAULT_PACKET_FORWARDING_INITIAL_BACKOFF, packet_forwarding_maximum_backoff: DEFAULT_PACKET_FORWARDING_MAXIMUM_BACKOFF, + initial_connection_timeout: DEFAULT_INITIAL_CONNECTION_TIMEOUT, } } } diff --git a/mixnode/src/config/template.rs b/mixnode/src/config/template.rs index 36accd4392..fe747516eb 100644 --- a/mixnode/src/config/template.rs +++ b/mixnode/src/config/template.rs @@ -53,35 +53,5 @@ nym_root_directory = '{{ mixnode.nym_root_directory }}' # TODO - -##### debug configuration options ##### -# The following options should not be modified unless you know EXACTLY what you are doing -# as if set incorrectly, they may impact your anonymity. - -[debug] - -# Directory server to which the server will be reporting their presence data. -presence_directory_server = '{{ debug.presence_directory_server}}' - -# Delay between each subsequent presence data being sent. -# The provided value is interpreted as milliseconds. -presence_sending_delay = {{ debug.presence_sending_delay }} - -# Directory server to which the server will be reporting their metrics data. -metrics_directory_server = '{{ debug.metrics_directory_server }}' - -# Delay between each subsequent metrics data being sent. -# The provided value is interpreted as milliseconds. -metrics_sending_delay = {{ debug.metrics_sending_delay }} - -# Initial value of an exponential backoff to reconnect to dropped TCP connection when -# forwarding sphinx packets. -# The provided value is interpreted as milliseconds. -packet_forwarding_initial_backoff = {{ debug.packet_forwarding_initial_backoff }} - -# Maximum value of an exponential backoff to reconnect to dropped TCP connection when -# forwarding sphinx packets. -# The provided value is interpreted as milliseconds. -packet_forwarding_maximum_backoff = {{ debug.packet_forwarding_maximum_backoff }} "# } diff --git a/mixnode/src/node/listener.rs b/mixnode/src/node/listener.rs index 05f7b554bb..76d6e13b2c 100644 --- a/mixnode/src/node/listener.rs +++ b/mixnode/src/node/listener.rs @@ -48,6 +48,8 @@ async fn process_socket_connection( return; } Ok(n) => { + // If I understand it correctly, this if should never be executed as if `read_exact` + // does not fill buffer, it will throw UnexpectedEof? if n != sphinx::PACKET_SIZE { warn!("read data of different length than expected sphinx packet size - {} (expected {})", n, sphinx::PACKET_SIZE); continue; @@ -63,10 +65,16 @@ async fn process_socket_connection( )) } Err(e) => { - warn!( - "failed to read from socket. Closing the connection; err = {:?}", - e - ); + if e.kind() == io::ErrorKind::UnexpectedEof { + debug!("Read buffer was not fully filled. Most likely the client ({:?}) closed the connection.\ + Also closing the connection on this end.", socket.peer_addr()) + } else { + warn!( + "failed to read from socket (source: {:?}). Closing the connection; err = {:?}", + socket.peer_addr(), + e + ); + } return; } }; diff --git a/mixnode/src/node/metrics.rs b/mixnode/src/node/metrics.rs index f62a470407..87712380db 100644 --- a/mixnode/src/node/metrics.rs +++ b/mixnode/src/node/metrics.rs @@ -4,13 +4,15 @@ use directory_client::DirectoryClient; use futures::channel::mpsc; use futures::lock::Mutex; use futures::StreamExt; -use log::{debug, error}; +use log::*; use std::collections::HashMap; use std::sync::Arc; -use std::time::Duration; +use std::time::{Duration, SystemTime}; use tokio::runtime::Handle; use tokio::task::JoinHandle; +type SentMetricsMap = HashMap; + pub(crate) enum MetricEvent { Sent(String), Received, @@ -25,7 +27,7 @@ struct MixMetrics { struct MixMetricsInner { received: u64, - sent: HashMap, + sent: SentMetricsMap, } impl MixMetrics { @@ -49,7 +51,7 @@ impl MixMetrics { *receiver_count += 1; } - async fn acquire_and_reset_metrics(&mut self) -> (u64, HashMap) { + async fn acquire_and_reset_metrics(&mut self) -> (u64, SentMetricsMap) { let mut unlocked = self.inner.lock().await; let received = unlocked.received; @@ -92,6 +94,7 @@ struct MetricsSender { directory_client: directory_client::Client, pub_key_str: String, sending_delay: Duration, + metrics_informer: MetricsInformer, } impl MetricsSender { @@ -100,6 +103,7 @@ impl MetricsSender { directory_server: String, pub_key_str: String, sending_delay: Duration, + running_logging_delay: Duration, ) -> Self { MetricsSender { metrics, @@ -108,6 +112,7 @@ impl MetricsSender { )), pub_key_str, sending_delay, + metrics_informer: MetricsInformer::new(running_logging_delay), } } @@ -118,6 +123,10 @@ impl MetricsSender { let sending_delay = tokio::time::delay_for(self.sending_delay); let (received, sent) = self.metrics.acquire_and_reset_metrics().await; + self.metrics_informer.update_running_stats(received, &sent); + self.metrics_informer.log_report_stats(received, &sent); + self.metrics_informer.try_log_running_stats(); + match self.directory_client.metrics_post.post(&MixMetric { pub_key: self.pub_key_str.clone(), received, @@ -134,6 +143,71 @@ impl MetricsSender { } } +struct MetricsInformer { + total_received: u64, + sent_map: SentMetricsMap, + + running_stats_logging_delay: Duration, + last_reported_stats: SystemTime, +} + +impl MetricsInformer { + fn new(running_stats_logging_delay: Duration) -> Self { + MetricsInformer { + total_received: 0, + sent_map: HashMap::new(), + running_stats_logging_delay, + last_reported_stats: SystemTime::now(), + } + } + + fn should_log_running_stats(&self) -> bool { + self.last_reported_stats + self.running_stats_logging_delay < SystemTime::now() + } + + fn try_log_running_stats(&mut self) { + if self.should_log_running_stats() { + self.log_running_stats() + } + } + + fn update_running_stats(&mut self, pre_reset_received: u64, pre_reset_sent: &SentMetricsMap) { + self.total_received += pre_reset_received; + + for (mix, count) in pre_reset_sent.iter() { + *self.sent_map.entry(mix.clone()).or_insert(0) += *count; + } + } + + fn log_report_stats(&self, pre_reset_received: u64, pre_reset_sent: &SentMetricsMap) { + debug!( + "Since last metrics report mixed {} packets!", + pre_reset_received + ); + debug!( + "Since last metrics report received {} packets", + pre_reset_sent.values().sum::() + ); + trace!( + "Since last metrics report sent packets to the following: \n{:#?}", + pre_reset_sent + ); + } + + fn log_running_stats(&mut self) { + info!( + "Since startup mixed {} packets!", + self.sent_map.values().sum::() + ); + debug!("Since startup received {} packets", self.total_received); + trace!( + "Since startup sent packets to the following: \n{:#?}", + self.sent_map + ); + self.last_reported_stats = SystemTime::now(); + } +} + #[derive(Clone)] pub struct MetricsReporter { metrics_tx: mpsc::UnboundedSender, @@ -173,6 +247,7 @@ impl MetricsController { directory_server: String, pub_key_str: String, sending_delay: Duration, + running_stats_logging_delay: Duration, ) -> Self { let (metrics_tx, metrics_rx) = mpsc::unbounded(); let shared_metrics = MixMetrics::new(); @@ -183,6 +258,7 @@ impl MetricsController { directory_server, pub_key_str, sending_delay, + running_stats_logging_delay, ), receiver: MetricsReceiver::new(shared_metrics, metrics_rx), reporter: MetricsReporter::new(metrics_tx), diff --git a/mixnode/src/node/mod.rs b/mixnode/src/node/mod.rs index 80d0b6a743..338734eda1 100644 --- a/mixnode/src/node/mod.rs +++ b/mixnode/src/node/mod.rs @@ -2,11 +2,13 @@ use crate::config::persistence::pathfinder::MixNodePathfinder; use crate::config::Config; use crate::node::packet_processing::PacketProcessor; use crypto::encryption; +use directory_client::presence::Topology; use futures::channel::mpsc; use log::*; use pemstore::pemstore::PemStore; use std::net::SocketAddr; use tokio::runtime::Runtime; +use topology::NymTopology; mod listener; mod metrics; @@ -62,6 +64,7 @@ impl MixNode { self.config.get_metrics_directory_server(), self.sphinx_keypair.public_key().to_base58_string(), self.config.get_metrics_sending_delay(), + self.config.get_metrics_running_stats_logging_delay(), ) .start(self.runtime.handle()) } @@ -92,17 +95,39 @@ impl MixNode { packet_forwarding::PacketForwarder::new( self.config.get_packet_forwarding_initial_backoff(), self.config.get_packet_forwarding_maximum_backoff(), + self.config.get_initial_connection_timeout(), ) }) .start(self.runtime.handle()) } + fn check_if_same_ip_node_exists(&self) -> Option { + // TODO: once we change to graph topology this here will need to be updated! + let topology = Topology::new(self.config.get_presence_directory_server()); + let existing_mixes_presence = topology.mix_nodes; + existing_mixes_presence + .iter() + .find(|node| node.host == self.config.get_announce_address()) + .map(|node| node.pub_key.clone()) + } + pub fn run(&mut self) { + info!("Starting nym mixnode"); + + if let Some(duplicate_node_key) = self.check_if_same_ip_node_exists() { + error!( + "Our announce-host is identical to one of existing nodes! (its key is {:?}", + duplicate_node_key + ); + return; + } let forwarding_channel = self.start_packet_forwarder(); let metrics_reporter = self.start_metrics_reporter(); self.start_socket_listener(metrics_reporter, forwarding_channel); self.start_presence_notifier(); + info!("Finished nym mixnode startup procedure - it should now be able to receive mix traffic!"); + if let Err(e) = self.runtime.block_on(tokio::signal::ctrl_c()) { error!( "There was an error while capturing SIGINT - {:?}. We will terminate regardless", diff --git a/mixnode/src/node/packet_forwarding.rs b/mixnode/src/node/packet_forwarding.rs index bdba773d84..58d2fce1b7 100644 --- a/mixnode/src/node/packet_forwarding.rs +++ b/mixnode/src/node/packet_forwarding.rs @@ -14,10 +14,12 @@ impl PacketForwarder { pub(crate) fn new( initial_reconnection_backoff: Duration, maximum_reconnection_backoff: Duration, + initial_connection_timeout: Duration, ) -> PacketForwarder { let tcp_client_config = multi_tcp_client::Config::new( initial_reconnection_backoff, maximum_reconnection_backoff, + initial_connection_timeout, ); let (conn_tx, conn_rx) = mpsc::unbounded(); diff --git a/mixnode/src/node/packet_processing.rs b/mixnode/src/node/packet_processing.rs index ec6187e9aa..21ad6fdce7 100644 --- a/mixnode/src/node/packet_processing.rs +++ b/mixnode/src/node/packet_processing.rs @@ -1,10 +1,11 @@ use crate::node::metrics; -use addressing::AddressTypeError; use crypto::encryption; use log::*; +use nymsphinx::addressing::nodes::{NymNodeRoutingAddress, NymNodeRoutingAddressError}; use sphinx::header::delays::Delay as SphinxDelay; use sphinx::route::NodeAddressBytes; use sphinx::{ProcessedPacket, SphinxPacket}; +use std::convert::TryFrom; use std::net::SocketAddr; use std::ops::Deref; use std::sync::Arc; @@ -32,8 +33,8 @@ impl From for MixProcessingError { } } -impl From for MixProcessingError { - fn from(_: AddressTypeError) -> Self { +impl From for MixProcessingError { + fn from(_: NymNodeRoutingAddressError) -> Self { use MixProcessingError::*; InvalidHopAddress @@ -68,8 +69,7 @@ impl PacketProcessor { forward_address: NodeAddressBytes, delay: SphinxDelay, ) -> Result { - let next_hop_address = - addressing::socket_address_from_encoded_bytes(forward_address.to_bytes())?; + let next_hop_address: SocketAddr = NymNodeRoutingAddress::try_from(forward_address)?.into(); // Delay packet for as long as required tokio::time::delay_for(delay.to_duration()).await; diff --git a/nym-client/Cargo.toml b/nym-client/Cargo.toml index 08ce1cdaeb..a75eddac67 100644 --- a/nym-client/Cargo.toml +++ b/nym-client/Cargo.toml @@ -1,7 +1,7 @@ [package] build = "build.rs" name = "nym-client" -version = "0.5.0" +version = "0.6.0" authors = ["Dave Hrycyszyn ", "Jędrzej Stuczyński "] edition = "2018" @@ -28,20 +28,20 @@ tokio = { version = "0.2", features = ["full"] } tokio-tungstenite = "0.10.1" ## internal -addressing = {path = "../common/addressing" } config = {path = "../common/config"} crypto = {path = "../common/crypto"} directory-client = { path = "../common/clients/directory-client" } healthcheck = { path = "../common/healthcheck" } mix-client = { path = "../common/clients/mix-client" } multi-tcp-client = { path = "../common/clients/multi-tcp-client" } +nymsphinx = { path = "../common/nymsphinx" } pemstore = {path = "../common/pemstore"} provider-client = { path = "../common/clients/provider-client" } sfw-provider-requests = { path = "../sfw-provider/sfw-provider-requests" } topology = {path = "../common/topology" } ## will be moved to proper dependencies once released -sphinx = { git = "https://github.com/nymtech/sphinx", rev="23f9c89b257ee0936e70afd682e9ed6a62e89eee" } +sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" } # sphinx = { path = "../../sphinx"} [build-dependencies] diff --git a/nym-client/src/client/cover_traffic_stream.rs b/nym-client/src/client/cover_traffic_stream.rs index c0e4e8dc57..4c076cbb30 100644 --- a/nym-client/src/client/cover_traffic_stream.rs +++ b/nym-client/src/client/cover_traffic_stream.rs @@ -99,6 +99,9 @@ impl LoopCoverTrafficStream { self.mix_tx .unbounded_send(MixMessage::new(cover_message.0, cover_message.1)) .unwrap(); + // JS: due to identical logical structure to OutQueueControl::on_message(), this is also + // presumably required to prevent bugs in the future. Exact reason is still unknown to me. + tokio::task::yield_now().await; } async fn run(&mut self) { diff --git a/nym-client/src/client/mix_traffic.rs b/nym-client/src/client/mix_traffic.rs index 515eeaa1f1..5e322db539 100644 --- a/nym-client/src/client/mix_traffic.rs +++ b/nym-client/src/client/mix_traffic.rs @@ -26,11 +26,13 @@ impl MixTrafficController { pub(crate) fn new( initial_reconnection_backoff: Duration, maximum_reconnection_backoff: Duration, + initial_connection_timeout: Duration, mix_rx: MixMessageReceiver, ) -> Self { let tcp_client_config = multi_tcp_client::Config::new( initial_reconnection_backoff, maximum_reconnection_backoff, + initial_connection_timeout, ); MixTrafficController { diff --git a/nym-client/src/client/mod.rs b/nym-client/src/client/mod.rs index 6446aed6f5..4a6af32df8 100644 --- a/nym-client/src/client/mod.rs +++ b/nym-client/src/client/mod.rs @@ -12,12 +12,12 @@ use crate::config::{Config, SocketType}; use crate::sockets::{tcp, websocket}; use crypto::identity::MixIdentityKeyPair; use directory_client::presence; -use futures::channel::mpsc; +use futures::channel::{mpsc, oneshot}; use log::*; +use nymsphinx::chunking::split_and_prepare_payloads; use pemstore::pemstore::PemStore; use sfw_provider_requests::AuthToken; use sphinx::route::Destination; -use std::net::SocketAddr; use tokio::runtime::Runtime; use topology::NymTopology; @@ -38,6 +38,9 @@ pub struct NymClient { // to be used by "send" function or socket, etc input_tx: Option, + + // to be used by "receive" function or socket, etc + receive_tx: Option, } #[derive(Debug)] @@ -64,6 +67,7 @@ impl NymClient { config, identity_keypair, input_tx: None, + receive_tx: None, } } @@ -75,18 +79,6 @@ impl NymClient { ) } - async fn get_provider_socket_address( - provider_id: String, - mut topology_accessor: TopologyAccessor, - ) -> SocketAddr { - topology_accessor.get_current_topology_clone().await.as_ref().expect("The current network topology is empty - are you using correct directory server?") - .providers() - .iter() - .find(|provider| provider.pub_key == provider_id) - .unwrap_or_else( || panic!("Could not find provider with id {:?} - are you sure it is still online? Perhaps try to run `nym-client init` again to obtain a new provider", provider_id)) - .client_listener - } - // future constantly pumping loop cover traffic at some specified average rate // the pumped traffic goes to the MixTrafficController fn start_cover_traffic_stream( @@ -149,16 +141,26 @@ impl NymClient { // the received messages are sent to ReceivedMessagesBuffer to be available to rest of the system fn start_provider_poller( &mut self, - topology_accessor: TopologyAccessor, + mut topology_accessor: TopologyAccessor, poller_input_tx: PolledMessagesSender, ) { info!("Starting provider poller..."); // we already have our provider written in the config let provider_id = self.config.get_provider_id(); + // TODO: a slightly more graceful termination here + if !self.runtime.block_on(topology_accessor.is_routable()) { + panic!( + "The current network topology seem to be insufficient to route any packets through\ + - check if enough nodes and a sfw-provider are online" + ); + } + + // TODO: a slightly more graceful termination here let provider_client_listener_address = self.runtime.block_on( - Self::get_provider_socket_address(provider_id, topology_accessor), - ); + topology_accessor.get_provider_socket_addr(&provider_id) + ).unwrap_or_else(|| panic!("Could not find provider with id {:?}. It does not seem to be present in the current network topology.\ + Are you sure it is still online? Perhaps try to run `nym-client init` again to obtain a new provider", provider_id)); let mut provider_poller = provider_poller::ProviderPoller::new( poller_input_tx, @@ -193,6 +195,7 @@ impl NymClient { self.config.get_topology_refresh_rate(), healthcheck_keys, self.config.get_topology_resolution_timeout(), + self.config.get_healthcheck_connection_timeout(), self.config.get_number_of_healthcheck_test_packets() as usize, self.config.get_node_score_threshold(), ); @@ -217,6 +220,7 @@ impl NymClient { MixTrafficController::new( self.config.get_packet_forwarding_initial_backoff(), self.config.get_packet_forwarding_maximum_backoff(), + self.config.get_initial_connection_timeout(), mix_rx, ) }) @@ -255,13 +259,36 @@ impl NymClient { } /// EXPERIMENTAL DIRECT RUST API - /// It's entirely untested and there are absolutely no guarantees about it - pub fn send_message(&self, destination: Destination, message: Vec) { - self.input_tx + /// It's untested and there are absolutely no guarantees about it (but seems to have worked + /// well enough in local tests) + pub fn send_message(&mut self, destination: Destination, message: Vec) { + let split_message = split_and_prepare_payloads(&message); + debug!( + "Splitting message into {:?} fragments!", + split_message.len() + ); + for message_fragment in split_message { + let input_msg = InputMessage(destination.clone(), message_fragment); + self.input_tx + .as_ref() + .expect("start method was not called before!") + .unbounded_send(input_msg) + .unwrap() + } + } + + /// EXPERIMENTAL DIRECT RUST API + /// It's untested and there are absolutely no guarantees about it (but seems to have worked + /// well enough in local tests) + pub async fn check_for_messages_async(&self) -> Vec> { + let (res_tx, res_rx) = oneshot::channel(); + self.receive_tx .as_ref() .expect("start method was not called before!") - .unbounded_send(InputMessage(destination, message)) - .unwrap() + .unbounded_send(res_tx) + .unwrap(); + + res_rx.await.unwrap() } /// blocking version of `start` method. Will run forever (or until SIGINT is sent) @@ -275,7 +302,7 @@ impl NymClient { } println!( - "Received SIGINT - the mixnode will terminate now (threads are not YET nicely stopped)" + "Received SIGINT - the client will terminate now (threads are not YET nicely stopped)" ); } @@ -316,9 +343,12 @@ impl NymClient { self.start_real_traffic_stream(shared_topology_accessor.clone(), mix_tx, input_rx); self.start_socket_listener( shared_topology_accessor, - received_messages_buffer_output_tx, + received_messages_buffer_output_tx.clone(), input_tx.clone(), ); self.input_tx = Some(input_tx); + self.receive_tx = Some(received_messages_buffer_output_tx); + + info!("Client startup finished!"); } } diff --git a/nym-client/src/client/provider_poller.rs b/nym-client/src/client/provider_poller.rs index 24d329c9ae..57f7cb55ca 100644 --- a/nym-client/src/client/provider_poller.rs +++ b/nym-client/src/client/provider_poller.rs @@ -78,7 +78,11 @@ impl ProviderPoller { Ok(messages) => messages, }; - let good_messages = messages + // todo: this will probably need to be updated at some point due to changed structure + // of nym-sphinx messages sent. However, for time being this is still compatible. + // Basically it is more of a personal note of if client keeps getting errors and is + // not getting messages, look at this filter here. + let good_messages: Vec<_> = messages .into_iter() .filter(|message| message != loop_message && message != dummy_message) .collect(); @@ -88,7 +92,9 @@ impl ProviderPoller { // - we run out of memory // - the receiver channel is closed // in either case there's no recovery and we can only panic - self.poller_tx.unbounded_send(good_messages).unwrap(); + if !good_messages.is_empty() { + self.poller_tx.unbounded_send(good_messages).unwrap(); + } tokio::time::delay_for(self.polling_rate).await; } diff --git a/nym-client/src/client/real_traffic_stream.rs b/nym-client/src/client/real_traffic_stream.rs index 2e83196610..997f709438 100644 --- a/nym-client/src/client/real_traffic_stream.rs +++ b/nym-client/src/client/real_traffic_stream.rs @@ -127,6 +127,9 @@ impl OutQueueControl { self.mix_tx .unbounded_send(MixMessage::new(next_packet.0, next_packet.1)) .unwrap(); + // JS: Not entirely sure why or how it fixes stuff, but without the yield call, + // the UnboundedReceiver [of mix_rx] will not get a chance to read anything + tokio::task::yield_now().await; } pub(crate) async fn run_out_queue_control(mut self) { diff --git a/nym-client/src/client/received_buffer.rs b/nym-client/src/client/received_buffer.rs index 123fc92882..09ef0eb16f 100644 --- a/nym-client/src/client/received_buffer.rs +++ b/nym-client/src/client/received_buffer.rs @@ -3,6 +3,7 @@ use futures::channel::{mpsc, oneshot}; use futures::lock::Mutex; use futures::StreamExt; use log::*; +use nymsphinx::chunking::reconstruction::MessageReconstructor; use std::sync::Arc; use tokio::runtime::Handle; use tokio::task::JoinHandle; @@ -13,6 +14,7 @@ pub(crate) type ReceivedBufferRequestReceiver = mpsc::UnboundedReceiver>, + message_reconstructor: MessageReconstructor, } #[derive(Debug, Clone)] @@ -27,15 +29,40 @@ impl ReceivedMessagesBuffer { ReceivedMessagesBuffer { inner: Arc::new(Mutex::new(ReceivedMessagesBufferInner { messages: Vec::new(), + message_reconstructor: MessageReconstructor::new(), })), } } - async fn add_new_messages(&mut self, msgs: Vec>) { + async fn add_reconstructed_messages(&mut self, msgs: Vec>) { + debug!("Adding {:?} new messages to the buffer!", msgs.len()); trace!("Adding new messages to the buffer! {:?}", msgs); self.inner.lock().await.messages.extend(msgs) } + async fn add_new_message_fragments(&mut self, msgs: Vec>) { + debug!( + "Adding {:?} new message fragments to the buffer!", + msgs.len() + ); + trace!("Adding new message fragments to the buffer! {:?}", msgs); + + let mut completed_messages = Vec::new(); + let mut inner_guard = self.inner.lock().await; + for msg_fragment in msgs { + if let Some(reconstructed_message) = + inner_guard.message_reconstructor.new_fragment(msg_fragment) + { + completed_messages.push(reconstructed_message); + } + } + // make sure to drop the lock to not deadlock + drop(inner_guard); + if !completed_messages.is_empty() { + self.add_reconstructed_messages(completed_messages).await; + } + } + async fn acquire_and_empty(&mut self) -> Vec> { trace!("Emptying the buffer and returning all messages"); let mut mutex_guard = self.inner.lock().await; @@ -67,7 +94,7 @@ impl RequestReceiver { error!( "Failed to send the messages to the requester. Adding them back to the buffer" ); - self.received_buffer.add_new_messages(failed_messages).await; + self.received_buffer.add_reconstructed_messages(failed_messages).await; } } }) @@ -92,7 +119,9 @@ impl MessageReceiver { fn start(mut self, handle: &Handle) -> JoinHandle<()> { handle.spawn(async move { while let Some(new_messages) = self.poller_receiver.next().await { - self.received_buffer.add_new_messages(new_messages).await; + self.received_buffer + .add_new_message_fragments(new_messages) + .await; } }) } diff --git a/nym-client/src/client/topology_control.rs b/nym-client/src/client/topology_control.rs index bf1678222a..4f4de04ef0 100644 --- a/nym-client/src/client/topology_control.rs +++ b/nym-client/src/client/topology_control.rs @@ -8,6 +8,7 @@ use std::time; use std::time::Duration; use tokio::runtime::Handle; // use tokio::sync::RwLock; +use std::net::SocketAddr; use tokio::task::JoinHandle; use topology::{provider, NymTopology}; @@ -41,6 +42,25 @@ impl TopologyAccessor { self.inner.lock().await.update(new_topology); } + pub(crate) async fn get_provider_socket_addr(&mut self, id: &str) -> Option { + match &self.inner.lock().await.0 { + None => None, + Some(ref topology) => topology + .providers() + .iter() + .find(|provider| provider.pub_key == id) + .map(|provider| provider.client_listener), + } + } + + // only used by the client at startup to get a slightly more reasonable error message + pub(crate) async fn is_routable(&self) -> bool { + match &self.inner.lock().await.0 { + None => false, + Some(ref topology) => topology.can_construct_path_through(), + } + } + // Unless you absolutely need the entire topology, use `random_route` instead pub(crate) async fn get_current_topology_clone(&mut self) -> Option { self.inner.lock().await.0.clone() @@ -90,6 +110,7 @@ pub(crate) struct TopologyRefresherConfig { refresh_rate: time::Duration, identity_keypair: MixIdentityKeyPair, resolution_timeout: time::Duration, + connection_timeout: time::Duration, number_test_packets: usize, node_score_threshold: f64, } @@ -100,6 +121,7 @@ impl TopologyRefresherConfig { refresh_rate: time::Duration, identity_keypair: MixIdentityKeyPair, resolution_timeout: time::Duration, + connection_timeout: time::Duration, number_test_packets: usize, node_score_threshold: f64, ) -> Self { @@ -108,6 +130,7 @@ impl TopologyRefresherConfig { refresh_rate, identity_keypair, resolution_timeout, + connection_timeout, number_test_packets, node_score_threshold, } @@ -130,6 +153,7 @@ impl TopologyRefresher { // this is a temporary solution as the healthcheck will eventually be moved to validators let health_checker = healthcheck::HealthChecker::new( cfg.resolution_timeout, + cfg.connection_timeout, cfg.number_test_packets, cfg.identity_keypair, ); @@ -163,6 +187,8 @@ impl TopologyRefresher { Ok(scores) => scores, }; + debug!("{}", healthcheck_scores); + let healthy_topology = healthcheck_scores .filter_topology_by_score(&version_filtered_topology, self.node_score_threshold); diff --git a/nym-client/src/config/mod.rs b/nym-client/src/config/mod.rs index 207f5b222a..d747a22eb4 100644 --- a/nym-client/src/config/mod.rs +++ b/nym-client/src/config/mod.rs @@ -20,6 +20,8 @@ const DEFAULT_TOPOLOGY_REFRESH_RATE: u64 = 10_000; const DEFAULT_TOPOLOGY_RESOLUTION_TIMEOUT: u64 = 5_000; const DEFAULT_PACKET_FORWARDING_INITIAL_BACKOFF: u64 = 10_000; // 10s const DEFAULT_PACKET_FORWARDING_MAXIMUM_BACKOFF: u64 = 300_000; // 5min +const DEFAULT_INITIAL_CONNECTION_TIMEOUT: u64 = 1_500; // 1.5s +const DEFAULT_HEALTHCHECK_CONNECTION_TIMEOUT: u64 = DEFAULT_INITIAL_CONNECTION_TIMEOUT; const DEFAULT_NUMBER_OF_HEALTHCHECK_TEST_PACKETS: u64 = 2; const DEFAULT_NODE_SCORE_THRESHOLD: f64 = 0.0; @@ -213,6 +215,14 @@ impl Config { pub fn get_packet_forwarding_maximum_backoff(&self) -> time::Duration { time::Duration::from_millis(self.debug.packet_forwarding_maximum_backoff) } + + pub fn get_initial_connection_timeout(&self) -> time::Duration { + time::Duration::from_millis(self.debug.initial_connection_timeout) + } + + pub fn get_healthcheck_connection_timeout(&self) -> time::Duration { + time::Duration::from_millis(self.debug.healthcheck_connection_timeout) + } } fn de_option_string<'de, D>(deserializer: D) -> Result, D::Error> @@ -317,7 +327,7 @@ impl Default for Logging { } #[derive(Debug, Deserialize, PartialEq, Serialize)] -#[serde(deny_unknown_fields)] +#[serde(default, deny_unknown_fields)] pub struct Debug { /// The parameter of Poisson distribution determining how long, on average, /// sent packet is going to be delayed at any given mix node. @@ -376,6 +386,15 @@ pub struct Debug { /// forwarding sphinx packets. /// The provided value is interpreted as milliseconds. packet_forwarding_maximum_backoff: u64, + + /// Timeout for establishing initial connection when trying to forward a sphinx packet. + /// The provider value is interpreted as milliseconds. + initial_connection_timeout: u64, + + /// Timeout for establishing initial connection when trying to forward a sphinx packet + /// during healthcheck. + /// The provider value is interpreted as milliseconds. + healthcheck_connection_timeout: u64, } impl Default for Debug { @@ -392,6 +411,8 @@ impl Default for Debug { node_score_threshold: DEFAULT_NODE_SCORE_THRESHOLD, packet_forwarding_initial_backoff: DEFAULT_PACKET_FORWARDING_INITIAL_BACKOFF, packet_forwarding_maximum_backoff: DEFAULT_PACKET_FORWARDING_MAXIMUM_BACKOFF, + initial_connection_timeout: DEFAULT_INITIAL_CONNECTION_TIMEOUT, + healthcheck_connection_timeout: DEFAULT_HEALTHCHECK_CONNECTION_TIMEOUT, } } } diff --git a/nym-client/src/config/template.rs b/nym-client/src/config/template.rs index ff31cd37e8..5ca0ec1650 100644 --- a/nym-client/src/config/template.rs +++ b/nym-client/src/config/template.rs @@ -62,62 +62,10 @@ listening_port = {{ socket.listening_port }} [debug] -# The parameter of Poisson distribution determining how long, on average, -# sent packet is going to be delayed at any given mix node. -# So for a packet going through three mix nodes, on average, it will take three times this value -# until the packet reaches its destination. -# The provided value is interpreted as milliseconds. average_packet_delay = {{ debug.average_packet_delay }} - -# The parameter of Poisson distribution determining how long, on average, -# it is going to take for another loop cover traffic message to be sent. -# The provided value is interpreted as milliseconds. loop_cover_traffic_average_delay = {{ debug.loop_cover_traffic_average_delay }} - -# The uniform delay every which clients are querying the providers for received packets. -# The provided value is interpreted as milliseconds. fetch_message_delay = {{ debug.fetch_message_delay }} - -# The parameter of Poisson distribution determining how long, on average, -# it is going to take another 'real traffic stream' message to be sent. -# If no real packets are available and cover traffic is enabled, -# a loop cover message is sent instead in order to preserve the rate. -# The provided value is interpreted as milliseconds. message_sending_average_delay = {{ debug.message_sending_average_delay }} -# Whether loop cover messages should be sent to respect message_sending_rate. -# In the case of it being disabled and not having enough real traffic -# waiting to be sent the actual sending rate is going be lower than the desired value -# thus decreasing the anonymity. -rate_compliant_cover_messages_disabled = {{ debug.rate_compliant_cover_messages_disabled }} - -# The uniform delay every which clients are querying the directory server -# to try to obtain a compatible network topology to send sphinx packets through. -# The provided value is interpreted as milliseconds. -topology_refresh_rate = {{ debug.topology_refresh_rate }} - -# During topology refresh, test packets are sent through every single possible network -# path. This timeout determines waiting period until it is decided that the packet -# did not reach its destination. -# The provided value is interpreted as milliseconds. -topology_resolution_timeout = {{ debug.topology_resolution_timeout }} - -# How many packets should be sent through each path during the healthcheck -number_of_healthcheck_test_packets = {{ debug.number_of_healthcheck_test_packets }} - -# In the current healthcheck implementation, threshold indicating percentage of packets -# node received during healthcheck. Node's score must be above that value to be -# considered healthy. -node_score_threshold = {{ debug.node_score_threshold }} - -# Initial value of an exponential backoff to reconnect to dropped TCP connection when -# forwarding sphinx packets. -# The provided value is interpreted as milliseconds. -packet_forwarding_initial_backoff = {{ debug.packet_forwarding_initial_backoff }} - -# Maximum value of an exponential backoff to reconnect to dropped TCP connection when -# forwarding sphinx packets. -# The provided value is interpreted as milliseconds. -packet_forwarding_maximum_backoff = {{ debug.packet_forwarding_maximum_backoff }} "# } diff --git a/nym-client/src/sockets/tcp.rs b/nym-client/src/sockets/tcp.rs index a9d1282dc0..6c9626f481 100644 --- a/nym-client/src/sockets/tcp.rs +++ b/nym-client/src/sockets/tcp.rs @@ -6,6 +6,7 @@ use futures::future::FutureExt; use futures::io::Error; use futures::SinkExt; use log::*; +use nymsphinx::chunking::split_and_prepare_payloads; use sphinx::route::{Destination, DestinationAddressBytes}; use std::convert::TryFrom; use std::io; @@ -94,21 +95,21 @@ impl ClientRequest { async fn handle_send( msg: Vec, recipient_address: DestinationAddressBytes, - mut input_tx: mpsc::UnboundedSender, + input_tx: mpsc::UnboundedSender, ) -> ServerResponse { trace!("sending to: {:?}, msg: {:?}", recipient_address, msg); - if msg.len() > sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH { - return ServerResponse::Error { - message: format!( - "too long message. Sent {} bytes while the maximum is {}", - msg.len(), - sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH - ), - }; - } let dummy_surb = [0; 16]; - let input_msg = InputMessage(Destination::new(recipient_address, dummy_surb), msg); - input_tx.send(input_msg).await.unwrap(); + + // in case the message is too long and needs to be split into multiple packets: + let split_message = split_and_prepare_payloads(&msg); + for message_fragment in split_message { + let input_msg = InputMessage( + Destination::new(recipient_address.clone(), dummy_surb), + message_fragment, + ); + input_tx.unbounded_send(input_msg).unwrap(); + } + ServerResponse::Send } diff --git a/nym-client/src/sockets/websocket/connection.rs b/nym-client/src/sockets/websocket/connection.rs index 2f88b186d4..434e51a78f 100644 --- a/nym-client/src/sockets/websocket/connection.rs +++ b/nym-client/src/sockets/websocket/connection.rs @@ -5,6 +5,7 @@ use crate::sockets::websocket::types::{ClientRequest, ServerResponse}; use futures::channel::oneshot; use futures::{SinkExt, StreamExt}; use log::*; +use nymsphinx::chunking::split_and_prepare_payloads; use sphinx::route::{Destination, DestinationAddressBytes}; use std::convert::TryFrom; use tokio_tungstenite::tungstenite::protocol::frame::coding::CloseCode; @@ -71,23 +72,21 @@ impl Connection { fn handle_text_send(&self, msg: String, recipient_address: String) -> ServerResponse { let message_bytes = msg.into_bytes(); - if message_bytes.len() > sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH { - return ServerResponse::Error { - message: format!( - "message too long. Sent {} bytes, but the maximum is {}", - message_bytes.len(), - sphinx::constants::MAXIMUM_PLAINTEXT_LENGTH - ), - }; - } // TODO: the below can panic if recipient_address is malformed, but it should be // resolved when refactoring sphinx code to make `from_base58_string` return a Result let address = DestinationAddressBytes::from_base58_string(recipient_address); let dummy_surb = [0; 16]; - let input_msg = InputMessage(Destination::new(address, dummy_surb), message_bytes); - self.msg_input.unbounded_send(input_msg).unwrap(); + // in case the message is too long and needs to be split into multiple packets: + let split_message = split_and_prepare_payloads(&message_bytes); + for message_fragment in split_message { + let input_msg = InputMessage( + Destination::new(address.clone(), dummy_surb), + message_fragment, + ); + self.msg_input.unbounded_send(input_msg).unwrap(); + } ServerResponse::Send } diff --git a/scripts/generate_changelog.sh b/scripts/generate_changelog.sh index 22097168ba..009376d1ff 100755 --- a/scripts/generate_changelog.sh +++ b/scripts/generate_changelog.sh @@ -1,3 +1,11 @@ #!/bin/bash # set CHANGELOG_GITHUB_TOKEN in your .bashrc file +# For each version, you can add a release summary with text, images, gif animations, etc, and show new features and notes clearly to the user. This is done using GitHub metadata. + +# Example: adding the release summary for v1.0.0: + +# 1. Create a new GitHub Issue +# 2. In the Issue's Description field, add your release summary content +# 3. Set the Issue Label `release-summary` and add it to the GitHub Milestone `v1.0.0` +# 4. Close the Issue and execute `github-changelog-generator` github_changelog_generator -u nymtech -p nym --exclude-tags 0.1.0 --token "$CHANGELOG_GITHUB_TOKEN" diff --git a/scripts/run_local_network.sh b/scripts/run_local_network.sh old mode 100755 new mode 100644 diff --git a/sfw-provider/Cargo.toml b/sfw-provider/Cargo.toml index ef376e6442..f38700107b 100644 --- a/sfw-provider/Cargo.toml +++ b/sfw-provider/Cargo.toml @@ -1,7 +1,7 @@ [package] build = "build.rs" name = "nym-sfw-provider" -version = "0.5.0" +version = "0.6.0" authors = ["Dave Hrycyszyn ", "Jędrzej Stuczyński "] edition = "2018" @@ -29,6 +29,7 @@ config = {path = "../common/config"} directory-client = { path = "../common/clients/directory-client" } pemstore = {path = "../common/pemstore"} sfw-provider-requests = { path = "./sfw-provider-requests" } +topology = {path = "../common/topology"} # this dependency is due to requiring to know content of loop message. however, mix-client module itself # is going to be removed or renamed at some point as it no longer servers its purpose of sending traffic to mix network @@ -37,7 +38,7 @@ mix-client = { path = "../common/clients/mix-client" } ## will be moved to proper dependencies once released -sphinx = { git = "https://github.com/nymtech/sphinx", rev="23f9c89b257ee0936e70afd682e9ed6a62e89eee" } +sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" } [build-dependencies] built = "0.3.2" diff --git a/sfw-provider/sfw-provider-requests/Cargo.toml b/sfw-provider/sfw-provider-requests/Cargo.toml index a34bae654e..4909c709a2 100644 --- a/sfw-provider/sfw-provider-requests/Cargo.toml +++ b/sfw-provider/sfw-provider-requests/Cargo.toml @@ -8,4 +8,4 @@ edition = "2018" [dependencies] bs58 = "0.3.0" -sphinx = { git = "https://github.com/nymtech/sphinx", rev="23f9c89b257ee0936e70afd682e9ed6a62e89eee" } +sphinx = { git = "https://github.com/nymtech/sphinx", rev="44d8f2aece5049eaa4fe84b7948758ce82b4b80d" } diff --git a/sfw-provider/src/config/mod.rs b/sfw-provider/src/config/mod.rs index 96ba4d8baa..9b49692a4d 100644 --- a/sfw-provider/src/config/mod.rs +++ b/sfw-provider/src/config/mod.rs @@ -468,7 +468,7 @@ impl Default for Logging { } #[derive(Debug, Deserialize, PartialEq, Serialize)] -#[serde(deny_unknown_fields)] +#[serde(default, deny_unknown_fields)] pub struct Debug { /// Directory server to which the server will be reporting their presence data. presence_directory_server: String, diff --git a/sfw-provider/src/config/template.rs b/sfw-provider/src/config/template.rs index ee31ce45cb..5b51b234a9 100644 --- a/sfw-provider/src/config/template.rs +++ b/sfw-provider/src/config/template.rs @@ -75,26 +75,5 @@ ledger_path = '{{ clients_endpoint.ledger_path }}' # TODO - -##### debug configuration options ##### -# The following options should not be modified unless you know EXACTLY what you are doing -# as if set incorrectly, they may impact your anonymity. - -[debug] - -# Directory server to which the server will be reporting their presence data. -presence_directory_server = '{{ debug.presence_directory_server}}' - -# Delay between each subsequent presence data being sent. -presence_sending_delay = {{ debug.presence_sending_delay }} - -# Length of filenames for new client messages. -stored_messages_filename_length = {{ debug.stored_messages_filename_length }} - -# number of messages client gets on each request -# if there are no real messages, dummy ones are create to always return -# `message_retrieval_limit` total messages -message_retrieval_limit = {{ debug.message_retrieval_limit }} - "# } diff --git a/sfw-provider/src/provider/client_handling/listener.rs b/sfw-provider/src/provider/client_handling/listener.rs index 260c947b07..1508efa232 100644 --- a/sfw-provider/src/provider/client_handling/listener.rs +++ b/sfw-provider/src/provider/client_handling/listener.rs @@ -65,10 +65,16 @@ async fn process_socket_connection( // plus realistically it wouldn't really introduce any speed up Ok(n) => process_request(&mut socket, &buf[0..n], &mut request_processor).await, Err(e) => { - warn!( - "failed to read from socket. Closing the connection; err = {:?}", - e - ); + if e.kind() == io::ErrorKind::UnexpectedEof { + debug!("Read buffer was not fully filled. Most likely the client ({:?}) closed the connection.\ + Also closing the connection on this end.", socket.peer_addr()) + } else { + warn!( + "failed to read from socket (source: {:?}). Closing the connection; err = {:?}", + socket.peer_addr(), + e + ); + } return; } }; diff --git a/sfw-provider/src/provider/mix_handling/listener.rs b/sfw-provider/src/provider/mix_handling/listener.rs index 8bcc13fe7e..f93eba48d9 100644 --- a/sfw-provider/src/provider/mix_handling/listener.rs +++ b/sfw-provider/src/provider/mix_handling/listener.rs @@ -36,6 +36,8 @@ async fn process_socket_connection( return; } Ok(n) => { + // If I understand it correctly, this if should never be executed as if `read_exact` + // does not fill buffer, it will throw UnexpectedEof? if n != sphinx::PACKET_SIZE { warn!("read data of different length than expected sphinx packet size - {} (expected {})", n, sphinx::PACKET_SIZE); continue; @@ -45,10 +47,16 @@ async fn process_socket_connection( tokio::spawn(process_received_packet(buf, packet_processor.clone())) } Err(e) => { - warn!( - "failed to read from socket. Closing the connection; err = {:?}", - e - ); + if e.kind() == io::ErrorKind::UnexpectedEof { + debug!("Read buffer was not fully filled. Most likely the client ({:?}) closed the connection.\ + Also closing the connection on this end.", socket.peer_addr()) + } else { + warn!( + "failed to read from socket (source: {:?}). Closing the connection; err = {:?}", + socket.peer_addr(), + e + ); + } return; } }; diff --git a/sfw-provider/src/provider/mod.rs b/sfw-provider/src/provider/mod.rs index 78affe4264..fc2aba3e20 100644 --- a/sfw-provider/src/provider/mod.rs +++ b/sfw-provider/src/provider/mod.rs @@ -3,9 +3,11 @@ use crate::config::Config; use crate::provider::client_handling::ledger::ClientLedger; use crate::provider::storage::ClientStorage; use crypto::encryption; +use directory_client::presence::Topology; use log::*; use pemstore::pemstore::PemStore; use tokio::runtime::Runtime; +use topology::NymTopology; mod client_handling; mod mix_handling; @@ -85,11 +87,33 @@ impl ServiceProvider { ); } + fn check_if_same_ip_provider_exists(&self) -> Option { + // TODO: once we change to graph topology this here will need to be updated! + let topology = Topology::new(self.config.get_presence_directory_server()); + let existing_providers_presence = topology.mix_provider_nodes; + existing_providers_presence + .iter() + .find(|node| { + node.mixnet_listener == self.config.get_mix_announce_address() + || node.client_listener == self.config.get_clients_announce_address() + }) + .map(|node| node.pub_key.clone()) + } + pub fn run(&mut self) { // A possible future optimisation, depending on bottlenecks and resource usage: // considering, presumably, there will be more mix packets received than client requests: // create 2 separate runtimes - one with bigger threadpool dedicated solely for // the mix handling and the other one for the rest of tasks + info!("Starting nym sfw-provider"); + + if let Some(duplicate_provider_key) = self.check_if_same_ip_provider_exists() { + error!( + "Our announce-host is identical to one of existing nodes! (its key is {:?}", + duplicate_provider_key + ); + return; + } let client_storage = ClientStorage::new( self.config.get_message_retrieval_limit() as usize, @@ -101,6 +125,8 @@ impl ServiceProvider { self.start_mix_socket_listener(client_storage.clone()); self.start_client_socket_listener(client_storage); + info!("Finished nym sfw-provider startup procedure - it should now be able to receive mix and client traffic!"); + if let Err(e) = self.runtime.block_on(tokio::signal::ctrl_c()) { error!( "There was an error while capturing SIGINT - {:?}. We will terminate regardless", diff --git a/validator/Cargo.toml b/validator/Cargo.toml index 4bea1343f5..70df8c0ef6 100644 --- a/validator/Cargo.toml +++ b/validator/Cargo.toml @@ -1,8 +1,8 @@ [package] build = "build.rs" name = "nym-validator" -version = "0.5.0" -authors = ["Jedrzej Stuczynski "] +version = "0.6.0" +authors = ["Dave Hrycyszyn ", "Jedrzej Stuczynski "] edition = "2018" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html diff --git a/validator/src/config/mod.rs b/validator/src/config/mod.rs index 5ee4f3e2c4..1396287272 100644 --- a/validator/src/config/mod.rs +++ b/validator/src/config/mod.rs @@ -11,6 +11,7 @@ mod template; // 'MIXMINING' const DEFAULT_MIX_MINING_DELAY: u64 = 10_000; const DEFAULT_MIX_MINING_RESOLUTION_TIMEOUT: u64 = 5_000; +const DEFAULT_MIX_MINING_CONNECTION_TIMEOUT: u64 = 1_500; const DEFAULT_NUMBER_OF_MIX_MINING_TEST_PACKETS: u64 = 2; @@ -139,6 +140,10 @@ impl Config { pub fn get_mix_mining_number_of_test_packets(&self) -> u64 { self.mix_mining.number_of_test_packets } + + pub fn get_healthcheck_connection_timeout(&self) -> time::Duration { + time::Duration::from_millis(self.mix_mining.connection_timeout) + } } #[derive(Debug, Deserialize, PartialEq, Serialize)] @@ -190,6 +195,10 @@ pub struct MixMining { /// The provided value is interpreted as milliseconds. resolution_timeout: u64, + /// Timeout for trying to establish connection to node endpoints. + /// The provided value is interpreted as milliseconds. + connection_timeout: u64, + /// How many packets should be sent through each path during the mix-mining procedure. number_of_test_packets: u64, } @@ -201,6 +210,7 @@ impl Default for MixMining { run_delay: DEFAULT_MIX_MINING_DELAY, resolution_timeout: DEFAULT_MIX_MINING_RESOLUTION_TIMEOUT, number_of_test_packets: DEFAULT_NUMBER_OF_MIX_MINING_TEST_PACKETS, + connection_timeout: DEFAULT_MIX_MINING_CONNECTION_TIMEOUT, } } } @@ -226,7 +236,7 @@ impl Default for Logging { } #[derive(Debug, Deserialize, PartialEq, Serialize)] -#[serde(deny_unknown_fields)] +#[serde(default, deny_unknown_fields)] pub struct Debug { /// Directory server to which the server will be reporting their presence data. presence_directory_server: String, diff --git a/validator/src/config/template.rs b/validator/src/config/template.rs index 5b70197ee4..17ba8d1ca8 100644 --- a/validator/src/config/template.rs +++ b/validator/src/config/template.rs @@ -43,6 +43,10 @@ run_delay = {{ mix_mining.run_delay }} # The provided value is interpreted as milliseconds. resolution_timeout = {{ mix_mining.resolution_timeout }} +# Timeout for trying to establish connection to node endpoints. +# The provided value is interpreted as milliseconds. +connection_timeout = {{ mix_mining.connection_timeout }} + # How many packets should be sent through each path during the mix-mining procedure. number_of_test_packets = {{ mix_mining.number_of_test_packets }} @@ -59,18 +63,5 @@ number_of_test_packets = {{ mix_mining.number_of_test_packets }} # TODO - -##### debug configuration options ##### -# The following options should not be modified unless you know EXACTLY what you are doing -# as if set incorrectly, they may impact your anonymity. - -[debug] - -# Directory server to which the server will be reporting their presence data. -presence_directory_server = '{{ debug.presence_directory_server }}' - -# Delay between each subsequent presence data being sent. -presence_sending_delay = {{ debug.presence_sending_delay }} - "# } diff --git a/validator/src/validator.rs b/validator/src/validator.rs index 9c12cef95a..51c78d9402 100644 --- a/validator/src/validator.rs +++ b/validator/src/validator.rs @@ -20,6 +20,7 @@ impl Validator { let dummy_healthcheck_keypair = MixIdentityKeyPair::new(); let hc = HealthChecker::new( config.get_mix_mining_resolution_timeout(), + config.get_healthcheck_connection_timeout(), config.get_mix_mining_number_of_test_packets() as usize, dummy_healthcheck_keypair, );