Commit Graph

153 Commits

Author SHA1 Message Date
Jon Häggblad 35be8de9f1 Update task fork names to be consistent 2025-02-17 14:37:34 +01:00
Jędrzej Stuczyński 226c040a13 feature: periodically remove stale gateway messages (#5312)
* add timestamp to stored client messages

* removed dead code

* starting node task to remove old messages

* added log for number of removed messages

* debug log on task finishing
2025-01-09 09:03:19 +00:00
Jędrzej Stuczyński 7d5e3ef7d3 feature: expand nym-node prometheus metrics (#5298)
* fixed bearer auth for prometheus route

* basic prometheus metrics

* added rates on global values

* improved structure on the prometheus metrics

* added additional metrics for ingress websockets and egress mixnet connections

* some channel business metrics

* fixed metrics registration and added additional variants

* added counter for number of disk persisted packets

* counter for pending egress packets

* counter for pending egress forward packets

* clippy
2024-12-20 10:32:56 +00:00
Jędrzej Stuczyński 67976b1b30 feature: wireguard metrics (#5278)
* experimental log

* introduce wireguard metrics updates

* add wireguard traffic rates to console logger

* missing import

* changed order of displayed values

* expose bytes information via rest endpoint

* clippy
2024-12-19 10:49:56 +00:00
Jędrzej Stuczyński 5a07b73375 feature: hopefully final steps of the smoosh™️ (#5201)
* removed mnemonic from gateway config struct

scaffolding for common mixnet listener

running verloc unconditionally in a nym-node

remove filtering by mixnode

extracted verloc to separate crate

integrated nym-node-http-server more tightly with the binary

most logic for handling forward packets

running all mixnode-related tasks natively inside nymnode

removed gateway storage trait in favour of the only concrete implementation

most logic for handling final hop packets

using nym-node owned socket listener for gateways

utility for sending plain message through mixnet + gateway fix

using common packet forwarding in both modes

nifying nym-node metrics

reproduce behaviour of the console logger

cleaned up cli args

redesigned gateway tasks startup procedure

removing dead code

scaffolding for old config v6

config migration

implemented MixnetMetricsCleaner

* clippy

* require entry/exit for wireguard

* removed dead code in migration code

* updated config template

* use custom user agent for verloc queries

* fixed premature shutdown of gateway tasks

* hidden nym-api flag to allow illegal node ips

* experiment: final hop handing with wireguard

* added additional startup logs

* typo

* fixed legacy stats endpoint data

* additional logs

* apply review comments

* fixed local testnet manager
2024-12-05 17:21:36 +00:00
Jędrzej Stuczyński 889d464e98 improvement: make internal gateway clients use the same topology cache (#5191) 2024-11-29 09:45:12 +00:00
Bogdan-Ștefan Neacşu fa72f90bfa Remove peers with no allowed ip from storage 2024-11-25 16:44:23 +00:00
Jędrzej Stuczyński 69718db6d2 chore: remove standalone legacy mixnode/gateway binaries (#5135)
* remove standalone gateway overhead

* remove standalone mixnode overhead

* additional cleanup: removed unused dependencies et al.

* removed calls to 'log::'
2024-11-15 12:37:35 +00:00
Simon Wicky b09ac57597 [Product Data] Client-side stats collection (#5107)
* draft of client data collection

* refactor gateway stats collection to fit client stats collection in same common crate

* moved client stats event and reporter to common crate

* basic os reporting

* add stats reporting address in sdk

* integrate stats scaffolding changes

* remove tokio spawn to potentially accomodate wasm32

* fmt

* fix typo

* add client_stats_id

* unify stats reporting

* avoid shutdown handle drop

* add client_type to stats reporting

* better way to build statsReportingconfig

* disarm shutdown on sink

* remove sink reporter and env dev-dependency

* cherrypick from jon/send-packet-stats

* uncoditionally start controller + licensing

* improve ClientStatsReport serialization

* better time handling

* reintroduce proper local reporting

* Let task wait for shutdown when exiting

* Log tweak

---------

Co-authored-by: jmwample <jmwample@users.noreply.github.com>
Co-authored-by: Jon Häggblad <jon.haggblad@gmail.com>
2024-11-08 14:00:29 +01:00
dynco-nym e840c1fe93 Merge branch 'release/2024.13-magura' into develop-release-merge 2024-11-04 16:50:44 +01:00
Jędrzej Stuczyński b9d1fc40e7 deprecated old nym-api client methods and replaced them when possible (#5069) 2024-10-31 12:08:58 +00:00
Jędrzej Stuczyński 000f2f1c29 Merge branch 'release/2024.13-magura' into develop 2024-10-29 15:31:51 +00:00
Simon Wicky ab11508235 [Product Data] Introduce data persistence on gateways (#5022)
* add stats storage to gateways

* config fix

* add stats storage model and logic

* adapt stats collection to new storage

* stats cleanup on start

* change to linux only code

* tweaks

* modified stats cleanup + change session started

* change wrong table name

* store crashed session as 0 duration

* adapt for sqlx 0.7

* remove unused dependencies

* revert changes from gateway config, as it is broken anyway

* copyright and misc stuff

---------

Co-authored-by: Simon Wicky <simon@linode2-2.net>
2024-10-28 09:25:37 +01:00
Jędrzej Stuczyński d626e7689f bugfix: make gateways insert themselves into [local] topology (#5038)
* added explicit SP suffix to started tasks

* added 'GatewayTopologyProvider' that always injects itself into the network

* use the new topology provider to bypass described bootstrapping problem
2024-10-25 12:06:16 +01:00
Simon Wicky 355991adc9 [Product Data] Add session type based on ecash ticket received (#4974)
* add session type based on ecash ticket collection

* avoid setting session type if already set

* change duration type to duration

* add a cap for finished sessions
2024-10-17 14:11:05 +02:00
Simon Wicky 435f236812 [Product Data] First step in gateway usage data collection (#4963)
* add stats model

* add stats collection

* add stats route

* propagate stuff and run stuff

* cargo stuff

* sqlx unused what?

* add sessions started stat

* session durations in miliseconds

* apply Jon's comments

* [Product Data] Second step in gateway usage data collection  (#4964)

* turn stats collection into event based

* move events into a common crate for future use elsewhere

* apply Jon's comments
2024-10-15 09:18:02 +02:00
Bogdan-Ștefan Neacşu 95ec91daa1 Entry wireguard tickets (#4888)
* Create credential verifier in authenticator

* Add new version of peer storage with client id

* Fix v1 to what it was before

* Compact storage into ecash verifier

* Fix non-linux build

* Less overlapping conditions

* Remove moved code

* Use handler thread for each peer

* Re-spawn stored handles at startup

* Keep new function without async & Result

* Put query peer in function too

* Query bandwidth

* Fix clippy

* Replace tap with inspect_err

* Fix copyright year

* Handle version 2 on the reqeust deser

* Add protocol type in req/resp messages
2024-09-23 14:49:18 +02:00
Jędrzej Stuczyński e8e86c273e Merge branch 'release/2024.10-caramello' into feature/2024.10-caramello-merge 2024-09-11 12:43:39 +01:00
Bogdan-Ștefan Neacşu 63d0ab49e1 Move credential verification into common crate (#4853)
* Move cred verification to common crate

* Put crate in toml file manually
2024-09-09 19:13:10 +02:00
Bogdan-Ștefan Neacşu 7b4dc78f41 Remove wireguard feature flag and pass runtime enabled flag (#4839)
* Remove wireguard feature flag

* Use wg enabled runtime flag

* Fix unintended flag removal
2024-09-03 15:25:05 +02:00
Bogdan-Ștefan Neacşu 7c1fca8ce4 Persist used wireguard private IPs (#4771)
* Persist used wireguard private IPs

* Fix imports

* Remove unnecessary type specification
2024-08-21 11:26:14 +02:00
Bogdan-Ștefan Neacşu bd94dd3055 Persist wireguard peers (#4732)
* Store wireguard peers in db

* Add update to nym-node

* Move gateway-requests and gateway-storage to common

* Carry storage to PeerController field

* Double kernel modifications with storage ones

* Take storage peers at boot

* Link storage query for registration flow

* Move authenticator peer comms in peer manager

* Modify template too

* Remove unused

* Fix clippy

* Fix clippy non-linux

* Keep storage data up-to-date on every check

* Check for staleness in storage timestamps

* Remove potential for panic in unwrap

* Fix clippy

* Fmt

* Clippy after rebase

* Remove in memory test structure
2024-07-30 18:26:40 +02:00
Simon Wicky fc2eedfc66 Another Grand Ecash Squasheroo
add offline ecash library

minor changes in coconut benchmarks

add ecash smart contract

change contract traits from coconut to ecash

first wave of andrew's suggestion

first wave of andrew's suggestion

second wave of andrew's suggestion for ecash lib

andrew's suggestion for ecash contract

licensing commit

safety comments for most unwraps

more unwrap handling

change chrono crate for time

latest cargo lock

error revamp

small visibility fix

small fix

remove indexedmap from contract + some tweaks

add cw2 version in ecash contract

remove envryption key from contract

change types from coconut to ecash types

adapt api model for credential issuance

adapt issued credential storage on API

add signatures cache on API

change API routes for new blind signing

modify issued_credential table

add issuance logic client-side

credential and signature storage client side

utils for credential issuance

first wave of fix

some of andrew's suggestions

remove encryption key from deposit

freepass issuance client side

freepass issuance API side

andrew's suggested fixes

other suggested fix

adapt change from PR below

allow offline verification flag

credential spending models

credential spending models for client

credential preperation for the client

credential preperation for the client

credential storage for spending on client

bloom filter for API

spent credential storage on validators

API route for spending online and offline ecash

API routes in the client lib

credential storage on gateway

ecash verifier to replace coconut verifier

accept credentials on gateway

bandwidth expiration for gateways

client ask for more bandwidth if it runs out

credential import

adapt nym validator rewarder and sdk

fix tests api tests and add constants

cargo fmt and lock and small test fix

cargo fmt and lock and small test fix

cargo lock

move stuff where they belong in ecash and static parameters

move some constants, error handling and phase out time crate

error revamp part 2

secret key by ref instead of clone

change l in wallet and v visibility

rework payinfo

rework monster tuples

fix expiration date signature cloning

minor fixes

final bits and bobs fixes

final bits and bobs fixes

rename l accessor to tickets_spent

wave of fixes

second wave of fixes

change hash domain value

removed benchmark flag

remove useless stringification in storage

nuke Bandwidth voucher

change timestamps to offsetdatetime

key name change

post-rebase fixes

update nym-connect 'time' dep due to broken semver

upload ecash contract to the build server

make wasm zknym-lib compile

but it won't work properly just yet

make wasm zknym-lib compile

but it won't work properly just yet

fix typo in ecash contract deps

make sure to use 0.1.0 sphinx packet

optimise pairings in 'check_vk_pairing'

derive serde for ecash types

simplified g1 tuple byte conversion

further optimise the pairing

unified signature type + renamed nym-api coconut module to ecash

using bincode serialiser for more complex binary types

using multimiller loop instead of rayon for verifying coin indices signatures

batching signature verification wherever possible

feature-locked rayon

clippy

refactor ecash contract a bit + introduce deposit storage

reworked find_proposal_id

various minor fixed

add offline_zk_nyms to nym-node everywhere

add missing #query

change test value to fit new serialization

optimised deposits storage

removed duplicate decompression code

using deposit_id instead of transaction hash

removed freepasses

split up ecash handling

unified shared state

fixed deposit_id parsing

log recovered deposit id

removed online verification

add detailed build info to ecash contract

fixed deserialisation of deposit amount received from nyxd queries

changed deposit to only persist attached pubkey

first iteration of split of verification and redemption

basic tool for setting up new network

expanded the tool with the option to bypass DKG

rename + init network without DKG

setting up locally running apis

ecash key migration

more local functionalities

wip fixing sql schemas

gateway immediately submitting redemption proposal

and getting it passed if valid

most of the gateway logic for split redemption with error recovery

fixed gateway not persisting ecash signers

simplify creation of compatible client

create properly serialised ecash key from the beginning

rebuild missing tickets and proposals on startup

stop ticket issuance during DKG transition

fixing build issues

split out ecash storage on nym-api side

master-verification-key route

caching all the signatures and keys

implemented aggregated routes for nym-apis

swagger UI for ecash endpoints

added explicit annotation for index and expiration signatures

revamped client ticketbook storage

save all recovery information in the same underlying storage

wrapper for bloomfilter

being more aggressive with marking tickets as used

ensure client has correct signatures before making deposit

fix deserialisation of AggregatedExpirationDateSignatureResponse + add ticketbook table

split nym-api ecash routes handlers into multiple files

fixed deserialisation of encoded expiration date

add tt_gamma1 to challenge and change naming for paper consistency

rotating double spending bloomfilter

nym-api test fixes + make sure to insert initial BF params

fixed ecash benchmark code

updated contract schema

updated CI to not upload gateway/mixnode binaries

ticket bandwidth revocation

added default deserialisation for zk nym config

post-rebase fixes
2024-07-30 11:27:21 +01:00
Bogdan-Ștefan Neacşu fd1d437211 Add 1GB/day/user bandwidth cap (#4717)
* Add check for 1GB/day/user and remove stale check

* Use saturated_sub

* Remove from wg peers

* Use 10 seconds instead of 1

* Query bandwidth message

* Ad client query message too

* Keep stale check

* Make bandwidth cap value public

* Fix consumed vs available bug

* Don't overwrite existing registrations

* Use self pub key instead of peer's
2024-07-23 20:49:49 +02:00
Bogdan-Ștefan Neacşu 1f144690da Add upgrades to nym-node for authenticator changes (#4703) (#4710)
* Add iterative upgrades to nym-node

* Authenticator correct configuration

* Add info log

* Enable auth opts on entry gw

* Move ephemeral config from exit_gateway

* Fix fmt

* Fix clippy

* Pass custom transceiver for authenticator

* Fix non-linux build

* Feature gate wg_api

* Change naming from semver to simple incremental

* Move opts unwrap inside the mutable function

* Remove unneeded authenticator_description
2024-07-12 14:45:59 +02:00
Bogdan-Ștefan Neacşu a4eb3a7dbf Named fork for better logging 2024-07-03 13:03:32 +00:00
Bogdan-Ștefan Neacşu 2159f71888 Fix wg feature 2024-07-03 09:04:01 +00:00
Bogdan-Ștefan Neacşu a9abea3446 Fix macos build 2024-07-03 08:27:10 +00:00
Bogdan-Ștefan Neacşu 8e2713c9ba Remove unwrap 2024-07-03 08:21:58 +00:00
Bogdan-Ștefan Neacşu c465eb3efc Fix error type 2024-07-02 08:58:48 +00:00
Bogdan-Ștefan Neacşu b90136ac4e Uniformise non-linux wg function 2024-07-01 15:10:36 +00:00
Bogdan-Ștefan Neacşu 6f3942f6b7 Move implementation for final request 2024-07-01 14:38:48 +00:00
Bogdan-Ștefan Neacşu f75b4843e8 Merge remote-tracking branch 'origin/develop' into feature/authenticator 2024-07-01 09:56:34 +00:00
Jon Häggblad fec570613c Remove code that refers to removed nym-network-statistics (#4679)
* Remove nym-network-statistics from workspace

* Remove nym-network-statistics

* Cargo.lock

* Update ci workflow

* Remove code that refers to removed nym-network-statistics

* Remove more

* Remove nym-statistics-common

* Delete commented out modules

* Remove commented out code

* Remove more commented out code

* Remove more commented out stuff

* Remove unused function
2024-06-27 10:32:32 +02:00
Bogdan-Ștefan Neacşu b43844bd7a Embed into gateway 2024-06-21 15:51:02 +00:00
Jędrzej Stuczyński 0626e54b6a make embedded NR/IPR ignore performance of the gateway 2024-06-21 10:03:24 +01:00
Bogdan-Ștefan Neacşu 17553d606e Requests and responses 2024-06-20 16:58:21 +03:00
Bogdan-Ștefan Neacşu 70fa41c165 Update kernel peers on peer modification (#4622)
* Include wireguard in gw config

* Support nym node first

* Create wg keypair

* Activate wg feature in gw dep

* Move key paths to separate structure

* Use client reg

* Generate and use own private key

* Rename network to ip for wg gw

* Propagate wireguard setup error message

* Remove logs

* Bump gateway version number

* Remove upgrade code

* Init wireguard on migration

* Upgrade code for nym-node too

* Wireguard paths upgrade

* Init wg keys on upgrade

* Simplify pub key translatations

* Fix clippy

* Undo comment change

* Fix tests

* Don't bump version just yet

* Remove redundant source attr

* Remove unused wg details

* Rename wg device

* Init for mixnode migration as well

* Add upgrade for gw wireguard deleted field

* Move interface removal to Drop trait impl

* Fix clippy

* Wgapi could be included on other platforms

* Handle peer control msgs

* Send add peer msg

* Handle double registration

* Fix deadlock

* Fix clippy

* More clippy

* Use same defguard

* Fix unit test

* Fix wasm build

* Move implementations from types to wireguard crate

* Fix linux
2024-06-07 14:02:47 +03:00
Bogdan-Ștefan Neacşu 3acf521fc1 Refine wireguard on gateway side (#4615)
* Include wireguard in gw config

* Support nym node first

* Create wg keypair

* Activate wg feature in gw dep

* Move key paths to separate structure

* Use client reg

* Generate and use own private key

* Rename network to ip for wg gw

* Propagate wireguard setup error message

* Remove logs

* Bump gateway version number

* Remove upgrade code

* Init wireguard on migration

* Upgrade code for nym-node too

* Wireguard paths upgrade

* Init wg keys on upgrade

* Simplify pub key translatations

* Fix clippy

* Undo comment change

* Fix tests

* Don't bump version just yet

* Remove redundant source attr

* Remove unused wg details

* Rename wg device

* Init for mixnode migration as well

* Add upgrade for gw wireguard deleted field

* Move interface removal to Drop trait impl

* Fix clippy

* Wgapi could be included on other platforms
2024-05-28 11:52:14 +02:00
Jędrzej Stuczyński 67467ca76d renamed 'shared_state' to 'common_state' 2024-04-12 15:00:49 +01:00
Jędrzej Stuczyński bfc495ef29 expire freepass bandwidth + stagger db flushes 2024-04-12 15:00:48 +01:00
Jędrzej Stuczyński 78fb3c2293 run http api by default 2024-04-09 16:26:54 +01:00
Jędrzej Stuczyński 4c5351ba60 Feature/one binary to rule them all (#4500) 2024-04-05 19:34:31 +02:00
Jędrzej Stuczyński ebd1eeb38d allow gateways to migrate configs of embedded NR/IPR 2024-03-27 17:20:39 +00:00
Tommy Verrall bd7779ec63 Merge pull request #4468 from nymtech/bugfix/optional-env-values
Bugfix/optional env values
2024-03-25 08:52:55 +00:00
Jon Häggblad 34be5abaf3 More minor naming fixes 2024-03-18 13:00:27 +01:00
Jon Häggblad 7f3c53e196 Rename struct 2024-03-18 13:00:27 +01:00
Jon Häggblad b710fbe524 Imports 2024-03-18 13:00:27 +01:00
Jon Häggblad 7132e2dae5 Remove outdated comments 2024-03-18 13:00:27 +01:00
Jędrzej Stuczyński 5e40e480bc adjusting severity of logs for missing DKG contract in the gateway 2024-03-13 11:17:52 +00:00