Commit Graph

25 Commits

Author SHA1 Message Date
Tommy Verrall 5681920092 Merge pull request #3386 from nymtech/feature/wasm-key-persistance
Feature/wasm key persistance
2023-05-12 09:39:10 +01:00
Jon Häggblad c46d04d3c4 Fix non-wasm build of wasm-client (#3399) 2023-05-10 18:45:46 +02:00
Jędrzej Stuczyński 28a965e698 persistent wasm client keys 2023-05-09 10:46:40 +01:00
Jędrzej Stuczyński 5478c23563 experimenting with storing concrete crypto types 2023-05-09 10:45:48 +01:00
Jędrzej Stuczyński 9f39e93e7b dummy indexeddb storage 2023-05-09 10:44:43 +01:00
Jędrzej Stuczyński cbbeb66b5b Feature/wasm client topology injection (#3311)
* added cargo config file to explicitly specify build target

* wip

* Config option to disable topology refreshing

* extracted common parsing code

* helper trait for working on wasm topology

* wasm topology parsing

* restored (slightly modified) old js-example

* wip

* Moved message preparation into a trait

* wip

* long-winded way of sending test packet

* standalone NymNodeTester

* finishing the test upon receiving all packets even if timeout wasnt reached

* initial round of cleanup

* sending multiple test packets in normal NymClient

* javascript-side cleanup

* starting mixnode test on btn click

* Improved NymNodeTester constructors

* improved error handling and constructors

* tester utils error handling

* further cleanup + using BTreeMap for NymTopology mixnodes

* handling missed errors

* splitting up 'test_node'

* split up and cleaned up generation of test result

* clippy + fixed example

* post rebase fixes

* another broken test

* prevent running multiple parallel tests

* cargo fmt

* Added nym- prefix to node tester utils
2023-04-24 09:56:26 +01:00
Mark Sinclair 5376c2a4ba WASM Client: use rollup to bundle web worker (#2884)
* WASM Client: simplify sending of custom messages by always setting headers and a mime-type for the content

* Use rollup to bundle the web worker script to support more downstream bundlers

The WASM bundle is embedded as a base64 encoded resource and loaded synchronously, because this is the only mechanism widely supported to load WASM inside a web worker currently. Hopefully in the future this can be changed to pure modules.

* Suppress errors in build script

* Add Parcel 2.0 example

* WASM client: fix tests

* Update SDK docs and images

* wasm-client: add method to validate a recipient's address

* Revert "Removing unused prestart"

This reverts commit cbeac10383.

* Revert "Removing pointless dependency build command from TypeScript SDK example"

This reverts commit 0e0a62938d.

* Add typing for React 18

* Improve README files

Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
2023-01-20 16:46:52 +00:00
Mark Sinclair 43ef098aad Add utilities to handle text and binary payloads for mixnet messages 2022-11-08 17:47:10 +00:00
Jędrzej Stuczyński a43d183b4f Feature/wasm client updates (#1673)
* Compiles but runtime time fails

* wip

* Beginning of clean-up - creation of config to keep things together

* Removed unused module

* Removed hardcoded constants

* Easier way of sending binary messages

* WIP cleanup before machine switch

* Upgrade wasm-bindgen to 0.2.83

* Fixed compilation warnings for wasm client

* all clients compiling without warnings

* disabling topology refresh in wasm

* Added a config option to disable loop cover traffic stream

* config changes

* Make webassembly work in a web worker
- `wasm-timer` modified to work in web worker
- add worker target to webpack
- add client to call from HTML
- update README to build WASM for bundling (this does not build ES modules)

* Restored topology refreshing

* correctly polling items in the wasm delay_queue

* Allow client to read up to 8 messages at once from gateway connection (#1669)

* Allow client to read up to 8 messages at once from gateway connection

* Importing tokio::select in wasm32 target

* Updated changelog

* missing imports

* Introduced disable_main_poisson_packet_distribution to force real_traffic_stream to disable poisson sending (#1664)

* Introduced disable_main_poisson_packet_distribution to force real_traffic_stream to disable poisson sending

* Updated changelog

* Adjusting default settings

* Introduced a client-configurable option to force it to use extended packet size

* local adjustments

* Removed warning associated with receiving extended packets

* Minimal v2-required changes

* Updated changelog

* explicitly allowing clippy drop_non_drop

Co-authored-by: Mark Sinclair <mmsinclair@gmail.com>
2022-10-10 16:27:51 +01:00
Jędrzej Stuczyński e5afd54ce0 Feature/node status api (#680)
* Basic storage stub

* New models for new node status api

* Route handling

* Mounting new routes

* Missing selective commit

* Moved network monitor related files to separate package

* Starting to see some sqlx action

* Schema updates

* Log statement upon finished migration

* Removed old diesel related imports

* Converted mixnode cache initialisation into a fairing

* Moved cache related functionalities to separate package

Also defined staging there

* Created run method for validator cache + removed unwrap

* Removed old node-status-api types and left bunch of todo placeholders in their place

* Fixed managing validatorcache

* Status reports are starting to get constructed

* Submitting some dummy results to the database

* Removing duplicate code for generating reports

* Removed statuses older than 48h

* Initial attempt at trying to obtain reports for all active nodes

* Removed duplicates from the full report

* Grabbing uptime history

* Updating historical uptimes of active nodes

* Updated sqlx-data.json

* Removed all placeholder foomp owner values

* Changed Layer serde behaviour for easier usage

* Extended validator api config

* Initial (seems working !) integration with network monitor

* Added database path configuration to config

* Using ValidatorCache in NetworkMonitor

* Flag indicating whether validator cache has been initialised

* Introduced a locla-only route for reward script to perform daily chores

* Flag to save config to a file

* Moved spawning of receiving future to run method rather than new

* Removed arguments that dont make sense to be configured via CLI

* Removed dead code from config file

* More dead code removal

* Added validator API to CI

* Corrected manifest-path arguments

* Constructing network monitor by passing config

* Combined validator API CI with the main CI file

* Using query_as for NodeStatus

* Checking if historical uptimes were already calculated on particular day

* Making id field NOT NULL

* More query_as! action

* Updated sqlx-data.json

* Removed unused chrono feature

* Renamed the migration file

* Changed default validator endpoint to point to local validator

* Removing unnecessary clone

* More appropriate naming

* Removed dead code

* Lock file updates

* Updated network monitor address in contract code

* Don't stage node status api if network monitor is disabled

* cargo fmt

* Updated all license notices to SPDX
2021-07-19 14:02:47 +01:00
Jędrzej Stuczyński 1e35eeefaa Feature/version in config (#376)
* Added version field to config files

* Removed built dependency

* Missing non-removed call to built
2020-10-12 16:35:26 +01:00
Jędrzej Stuczyński c50e9a9ed7 Feature/wasm update (#341)
* Split text and binary client apis

* Very initial attempt at new serialization

* Defined ser+de for Recipient and ReplySURB

* Response errors

* builds with changes

* Working WS API + moved to separate crate

* updated python examples

* Fixed parsing bug

* Updated go examples

* Updated rust examples

* formatting

* Removed unused imports

* dependency updates

* Further dependency changes

* nymsphinx exposingn framing only if not in wasm32

* Cargo lock changes before develop merge

* Pending work

* Actually sending and receiving websocket from rust!

* more WIP

* Initial wasm client + establishing shared key with gateway!

* Splitting and sending a message!

* WIP

* WIP

* Initial wasm-ification of the gateway client

* Passing reconstruction result to js callback!

* Initial WASM cleaning pass

* Dependency pruning

* Moved processing loop to received_processor + at least ack unwrappingn

* Post merge fix

* Kinda updated react example

* Old print statement removed

* Removed yarn.lock

* Fixed building issue for other clients

* Fixed travis test command
2020-09-16 15:18:04 +01:00
Jędrzej Stuczyński 7d2d512405 Feature/client binary api update (#306)
* Split text and binary client apis

* Very initial attempt at new serialization

* Defined ser+de for Recipient and ReplySURB

* Response errors

* builds with changes

* Working WS API + moved to separate crate

* updated python examples

* Fixed parsing bug

* Updated go examples

* Updated rust examples

* formatting

* Removed unused imports

* Removed accidentally left panic call
2020-08-17 10:44:40 +01:00
Jędrzej Stuczyński d9d549fd0f Feature/reply surbs (#299)
* Changed identity keypair to use ed25519

* Encryption key is now x25519 based + compatibiltiy with sphinx

* Pathing and import fixes

* Moved all asymmetric keys to sub-module in crypto

* Extracted aes to separate module

* kdf module in crypto

* Ability to perform diffie hellman on encryption keys

* ecdsa on identity keys

* Extremely rough and incomplete registration handshake

* Authentication primitives

* Creating new random authenticationIV

* Wrapper type for the derived shared key

* Removed AuthToken in favour of using SharedKey for authentication

* Gateway identity keys

* Registration handshake without error mapping

* Gateway address in client config

* Added extra key for gateway presence

* Updated pemstore to work on borrows instead

* Gateway client trying to perform the handshake

* Gateway changes to allow for handshake and shared key

* Debug trait on sharedkey

* native client using updated gateway client

* Slightly updated gateway API

* Minor cleanup

* Fixed pemstore to correctly save multiple keypairs

* Gateway actually deriving shared key during handshake

* Gateway sending correct mid-handshake message

* Missing quotation mark in client config template

* Fixed template for correct shared key serialization

* Fixed gateway authentication

* Fixed tests

* Using correct gateway key when converting to sphinx node

* "get_all_clients" takes them from gateways as opposed to providers now

* cargo fmt

* Renamed pemstore methods

* Unused import

* Encryption of forward requests between client and gateway

* Updated sphinx dependency to use public revision

* Sending 'error' on handshake processing error

* Removed some dead code

* Dead code I forgot to remove before

* Extracted AckAes128Key into a struct

* Slight pemstore revamp allowing for symmetric key store

* ibid.

* PemStorableKey for SharedKey

* Introduced single location responsible for key management for client

* WIP

* Sphinx version update

* Stop using NodeAddressBytes for two distinct and confusing purposes

* Abstracting away SocketAddr from sphinx forwarding

* Passing the bool for reply surbs

* Attack plan for replies + encryption

* Comment + removed variable binding

* ReplySURB usage

* Topology import in nymsphinx

* Sphinx version update

* Changed 'Recipient' to contain client's encryption key

* Message preparation taking shape!

* reply surb also containing the encryption key

* Very initial message receiver

* Sphinx version update

* A possibly working way of receiving surbs

* Fixed incorrect field name in client config template

* camel casing all request arguments

* Renamed and moved `MessageMode` to more appropriate file

* Restored reconstruction tests

* Removed dead code from chunking

* Made rust examples compilable

* reply SURB key storage

* Replies as an InputMessage

* Forgotten commented code

* No retransmission processing for cover or replies

* Received reply processing

* Renamed client pathfinder to something more appropriate

* Made HasherOutputSize public

* Added key store path to config

* Reply surb attaching key digest when used

* Changes due to previous renaming

* Removed comment

* Fixed insert_encryption_key

* Assigning initial value of key store path

* Computing key digest with correct algorithm

* Initial and presumably temporary request serialization

* hacky way of introducing 'FragmentIdentifier' for replies

* Moved responsibility of reply encryption, padding, etc, to message preparer

* Optional recipient in try_get_valid_topology_ref

* Handling new reply surbs with acks and padding

* Updated go and python examples to include replies in text and binary cases

* Updated rust examples + binaryserverresponse

* Helpers in rust examples

* And updated JS example

* Moved shared key generation function to crypto crate

* Cover traffic encryption!

* hmac computation in crypto

* Updated aes imports due to new dependencies

* hkdf made more generic

* crypto cleanup + algorithms in params

* Clippy cleanup pass

* Generating encryption+mac shared keys between client and gateway

* MACs attached to forward requests to gateway

* Gateway messages encrypted and mac'd

* Lowered logging level

* compiler warning cleanup

* Some minor cleanup

* Generic stream cipher

* Generic shared key derivation + algorithm definitions

* Project-wide AES clean-up

* Comment fix

* Removed commented imports

* Updated comments

* Fixed topology test fixture
2020-08-07 18:16:54 +01:00
Jędrzej Stuczyński a53d0a4aac Feature/topology refactor (#274)
* VersionFilterable for HashMap

* Removed NymTopology trait in favour of concrete type

* Removed providers from NymTopology

* Made gateway conversion use reference, similarly to mixes

* Using more concrete types in topology rather than b58 strings

* Allowing gateways to have DNS-resolvable mix listener address

* Error propagation for gateway key conversion
2020-07-07 10:56:50 +01:00
Jędrzej Stuczyński e849e45b12 Feature/gateway shared key generation (#272)
* Changed identity keypair to use ed25519

* Encryption key is now x25519 based + compatibiltiy with sphinx

* Pathing and import fixes

* Moved all asymmetric keys to sub-module in crypto

* Extracted aes to separate module

* kdf module in crypto

* Ability to perform diffie hellman on encryption keys

* ecdsa on identity keys

* Extremely rough and incomplete registration handshake

* Authentication primitives

* Creating new random authenticationIV

* Wrapper type for the derived shared key

* Removed AuthToken in favour of using SharedKey for authentication

* Gateway identity keys

* Registration handshake without error mapping

* Gateway address in client config

* Added extra key for gateway presence

* Updated pemstore to work on borrows instead

* Gateway client trying to perform the handshake

* Gateway changes to allow for handshake and shared key

* Debug trait on sharedkey

* native client using updated gateway client

* Slightly updated gateway API

* Minor cleanup

* Fixed pemstore to correctly save multiple keypairs

* Gateway actually deriving shared key during handshake

* Gateway sending correct mid-handshake message

* Missing quotation mark in client config template

* Fixed template for correct shared key serialization

* Fixed gateway authentication

* Fixed tests

* Using correct gateway key when converting to sphinx node

* "get_all_clients" takes them from gateways as opposed to providers now

* cargo fmt

* Renamed pemstore methods

* Unused import

* Encryption of forward requests between client and gateway

* Updated sphinx dependency to use public revision

* Sending 'error' on handshake processing error

* Removed some dead code
2020-06-30 12:01:45 +01:00
Jędrzej Stuczyński fcfe463efa Feature/wasm topology duplication (#265)
* Removed constructor from NymTopology trait + split directory_client

* Removed bunch of duplicate wasm topology code

* Post-merge changes
2020-06-16 12:12:19 +01:00
Jędrzej Stuczyński b57a548350 Feature/packet retransmission (#263)
* Ability to send sphinx packets of different sizes + more efficient decoding

* Closing connection on connection corruption

* Missing semicolons

* Missing license notices

* Default for packetsize

* Split nymsphinx

* Replaced Mutex with RwLock for TopologyAccessor + impl Deref

* Sphinx update + import cleanup

* Moved packet_sizes file

* Updated NymTopology API

* sphinx version bump

* Missing license notice

* nymsphinx-params crate

* Changes due to ibid.

* Chunking rework to allow variable size fragments

* Initial ack crate

* Version bump to new dev build

* Cargo lock changes

* random_route_to_gateway by node address

* exposing getting read permit

* Very initial draft on ack control

* Correctly dereferencing out of topology read permit

* All pending changes + compilation todo!s

* Restricted scope of deref on TopologyAccessorInner

* Type path alias for generate_key

* Derived traits for MessageChunker

* Ack control starting to take shape!

* Awaiting callbacks

* Most of work done on acks. Now to wire it all together

* Import cleanup

* rng generalization

* Connected real traffic together; only acks from gateway left

* Removed redundant things from nymsphinx::utils

* nymsphinx-cover crate

* Ack-related fields in client config

* Decreased packet store log level

* Restored forward sphinx request

* Slight adjustements to surb acks

* Changed TopologyReadPermit from type alias into a struct

* Changes due to ibid.

* Sphinx version upgrade

* Gateway being able to understand and handle acks

* Special Cover FragmentIdentifier + removal of dead code

* Initial packet router for gateway client

* Kill client if it fails to send to gateway too many times

* Cover messages with acks

* Moved out gateway client errors

* Ignoring cover traffic acks

* Changes in ack control

* Another sphinx version upgrade

* websocket handler delegating message chunking

* Using config defined ack wait additions

* Other minor changes I should have been more dilligent with splitting

* Import path fix

* sphinx_receiver => mixnet_receiver

* Missing renamed variable instance

* Updated aes-ctr to 0.4.0

* Removed concept of 'unfragmented' single fragment

* Replay fragments detection

* Long method split

* typo

* Cleaner client init

* Fixed race condition

* Fixed similar issue for retransmission

* Cargo fmt

* Minor clenaup
2020-06-16 11:22:02 +01:00
hyperfekt db5540677c add disabling feature 'offline-test' for network-dependent tests (#260) 2020-06-15 15:53:52 +01:00
Jędrzej Stuczyński 54114f03b5 Feature/explicit gateway addressing (#239)
* validator: fixing a warning, untestify this when you need it for real code

* webassembly: minor readme changes.

* README changes in wasm

* Updated wasm version

* clients/webassembly: security vuln updates

* typo fix

* WIP commit

* Significantly simplified the API

* Changed switch to have default branch

* Managed to get rid of `this` bind

* Moved 'Recipient' definition

* Examples update

* Slightly more generalised 'try_from_string' for recipient

* Updated to use client@gateway addressing

* Updated tests

Co-authored-by: Dave Hrycyszyn <futurechimp@users.noreply.github.com>
2020-05-19 12:16:04 +01:00
Dave Hrycyszyn 8745fb4230 Feature/clean up (#238)
* validator: fixing a warning, untestify this when you need it for real code

* webassembly: minor readme changes.

* README changes in wasm

* Updated wasm version

* clients/webassembly: security vuln updates

* typo fix

* WIP commit

* Significantly simplified the API

* Changed switch to have default branch

* Managed to get rid of `this` bind

* Filled in a missing word

Co-authored-by: jstuczyn <jedrzej.stuczynski@gmail.com>
2020-05-19 10:32:52 +01:00
Jędrzej Stuczyński 97a944a33e Updated wasm code to work with new gateway and updated the example (#219)
* Updated wasm code to work with new gateway and updated the example

* cargo fmt

* Fixed test by ensuring destination is present at a gateway

* Updated hardcoded default port

* ibid. to correct value
2020-05-06 16:20:43 +01:00
Jędrzej Stuczyński abbf7041a4 Feature/gateway provider merge (#208)
* Initial draft for ClientsHandler

* Created listener struct

* typo

* Stateful websocket connection handler

* Exposing modules

* Depdendencies updates

* Moved listener to correct file + made start consume listener

* Main starting new listener

* Catching sigint

* Copied client storage from provider into gateway

* Exposed websocket listener type for nicer import path

* Defined websocket message receiver concrete type

* Client ledger struct without implementation

* ClientsHandler using more concrete types

* Mixnet sender + receiver and exposed listener type

* Handling mix packets

* Ability to forward mix packets

* "starting" both listeners at main

* Depedencies updates

* Initial type definitions for client messages

* Initial "gateway-requests" with AuthToken

* ibid.

* Restored most of ledger's functionalities

* Ability to retrieve all messages regardless of rate limit

* ClientsHandler request handling logic

* Required 'new' dependencies

* Main changes required for compilation

* PacketProcessor getting private key pointer

* "moved" types into gateway requests crate

* Moved and renamed types

* ibid.

* Added required serde_json dependency

* Skeleton for websocket request handling

* helper methods on ServerResponse

* WebSocket Handler pushing received mix messages directly to client

* PacketForwarder returning JoinHandle alongside the channel

* ClientsHandler following the same pattern

* Made websocket listener start method consistent with mix listener

* Syntax error + formatting

* Websocket handler having access to mix forwarder

* Minimal binary request parsing

* Implicitly derived std::error::Error on GatewayRequestsError

* Handling of all websocket requests

* Types import cleanup

* Updated placeholder fields

* Everything put into main

* Missing license notices

* Cleaned up unused code

* Copied and did initial minor changes to commands and config

* It's actually gateway

* Gateway sending its regular presence to directory server

* Re-organisation of gateway code + Gateway struct

* Updated provider argument description

* Restored duplicate ip check without importing NymTopology trait

* Moved hardcoded values into config

* Cargo fmt

* Compilation errors in other crates due to topology adjustments

* Test fixes

* Initial gateway-client

* Seemingly working version with bunch of hardcoded and temporary values

* cargo fmt

* Removed provider poller

* Updated config with gateway values instead of provider

* Gateway address including ws

* Removed hardcoded gateway address

* Properly skipping loop cover messages

* Updated log filter with tokio tungstenite related modules

* Updated sphinx version used

* Very minor cleanup

* unused import
2020-04-30 10:14:21 +01:00
Dave Hrycyszyn 903ef44328 Feature/route from topology (#201)
* directory-client: removing explicit type definitions

* clients/webassembly: ugly commit, removes dependency on HTTP crates

...at the cost of copying quite a bit of NymTopology-related stuff into
the WebAssembly client. I'll make an issue to refactor that later.

* A more helpful hint in the error message when websocket connection fails.

* Removing superfluous JavaScript, route checking now happens in Rust
2020-04-23 12:01:25 +01:00
Dave Hrycyszyn 8baa236d81 Feature/wasm js demo (#191)
* Moved `nym-sphinx-wasm` into `wasm` folder and renamed its examples

* A proposal for some structural cleanup

* desktop client: beginning of a Rust example for chunking.

* Simplifying desktop client readme

* Tarting up webassembly README

* A more visual webassembly client example

* clients/webassembly: grouping a few things into named functions for clarity

* clients/webassembly: put URLs at top of file, prepping for drop-down action

* mixnode + provider run: changed startup error messages a bit

* clients/webassembly: scrolling from the bottom in the demo
2020-04-21 10:05:38 +01:00