Commit Graph

35 Commits

Author SHA1 Message Date
dynco-nym b7e3687757 Dz nym node stats (#5418)
* Remove blacklisted, inactive, reserve fields

* Remove gw.blacklisted

* Remove blacklisted and bonded count

* DB operations

* Improve logging

* Remove unused functions

* get_nym_nodes for scraping WIP

* Separate nym_nodes from mixnode stats
- fixes FOREIGN_KEY_CONSTRAINT error when storing
  stats for nym_nodes which aren't in mixnodes table

* Daily aggregation works

* mixnodes/stats exposes correct info

* Undo unnecessary tidbits

* Replace obsolete stats

* Add total_stake

* Bump cargo.toml version

* Rename MixingNodeKind for better clarity
2025-02-11 12:07:15 +01:00
Jon Häggblad 134a0196f8 Disable the test for checking the remaining bandwidth in nym-node-status-api (#5425)
* Disable the test for checking the remaining bandwidth in nym-node-status-api

This check fails almost every time on CI, possibly due to rate limiting?
It's not good to disable the check, but it's blocking CI as it stands
now. Given that we have the check above for locating the ip, we at least
have a little coverage.

* Remove unused
2025-02-07 11:39:37 +01:00
dependabot[bot] 23a7f01c05 build(deps): bump tokio from 1.40.0 to 1.43.0 (#5370)
* build(deps): bump tokio from 1.40.0 to 1.43.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.40.0 to 1.43.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.40.0...tokio-1.43.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* wip: test if token is set

* Try with an artifical delay between calls

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jon Häggblad <jon.haggblad@gmail.com>
2025-02-05 10:38:28 +01:00
Sachin Kamath fffec65cab NS API: add mixnet scraper (#5200)
* ns-api: add mixnode scraper

* clippy

* rebase
2025-01-15 13:12:11 +01:00
Jon Häggblad 9b02de3e75 Use expect in geodata test to give error message on failure (#5314)
* Use expect in geodata test to give error message on failure

I keep hitting this error on CI, from what I think is network hickup.
But it's hard to tell form the log since the error is swallowed.

Explicitly unwrap the result so we get a more detailed error output

* Add nym-node-status-api to ci-build
2025-01-08 10:56:26 +01:00
dynco-nym 41fb17a31b Extend swagger docs (#5235)
* WIP adding derive(ToSchema)

* Derive ToSchema for more types

* ContractBuildInformation on /nym_contracts_detailed

* rustfmt

* Add cfg_attr

* A bunch of annotations

* Compiles with utoipa 5.2

* WIP

* Post rebase fixes

* Gitattributes to ignore .sqlx diffs

* generate Sqlx schema files

* Improvements

* Move ecash schema out of ecash crate

* Move redocly config to nym-api/

* Move redocly config to nym-api/

* Remove ErrorResponse

* Move generated openapi spec to .gitignore

* Include BSL licence

* Remove utoipa from ecash toml file

* Remove placeholder annotations

* Chain-watcher rebase changes

* Update licence info

* Treat Scalar as String in OpenAPI
2024-12-20 12:18:45 +01:00
Bogdan-Ștefan Neacşu 61e872f033 Add windows to CI builds (#5269)
* Add windows to CI builds

* Fix win build for node status api

* Fix win build for sdk

* Fix win build for cred proxy
2024-12-17 15:18:11 +02:00
dynco-nym b4f51baf94 Change sqlite journal mode to WAL (#5213)
* Change sqlite journal mode to WAL

* Synchronous mode & auto vacuum

* Bump probe git ref to 1.1.0
2024-12-16 16:40:02 +01:00
Drazen Urch 84d7004cb2 Add control messages to GatewayTransciver (#5247)
* Add control messages to GatewayTransciver

* Add forget me flag to clients

* CI gate IPIINFO test

* Handle ForgetMe for client and stats db

* fmt
2024-12-16 15:18:04 +01:00
Jędrzej Stuczyński 4e2aa2c0b3 Merge branch 'release/2024.14-crunch' into merge/release/2024.14-crunch 2024-12-10 15:29:26 +00:00
benedetta davico c29fce0856 Update NS-api version in Cargo.toml 2024-12-10 11:16:16 +01:00
Jędrzej Stuczyński 675e5a0305 removed semver filtering (#5224) 2024-12-06 17:21:21 +00:00
Jędrzej Stuczyński 5a07b73375 feature: hopefully final steps of the smoosh™️ (#5201)
* removed mnemonic from gateway config struct

scaffolding for common mixnet listener

running verloc unconditionally in a nym-node

remove filtering by mixnode

extracted verloc to separate crate

integrated nym-node-http-server more tightly with the binary

most logic for handling forward packets

running all mixnode-related tasks natively inside nymnode

removed gateway storage trait in favour of the only concrete implementation

most logic for handling final hop packets

using nym-node owned socket listener for gateways

utility for sending plain message through mixnet + gateway fix

using common packet forwarding in both modes

nifying nym-node metrics

reproduce behaviour of the console logger

cleaned up cli args

redesigned gateway tasks startup procedure

removing dead code

scaffolding for old config v6

config migration

implemented MixnetMetricsCleaner

* clippy

* require entry/exit for wireguard

* removed dead code in migration code

* updated config template

* use custom user agent for verloc queries

* fixed premature shutdown of gateway tasks

* hidden nym-api flag to allow illegal node ips

* experiment: final hop handing with wireguard

* added additional startup logs

* typo

* fixed legacy stats endpoint data

* additional logs

* apply review comments

* fixed local testnet manager
2024-12-05 17:21:36 +00:00
Simon Wicky f3bf5d080b better date serilization (#5207) 2024-12-04 11:11:51 +01:00
dynco-nym e06d442e95 Restore Location fields (#5208)
* Add latitude/longitude fields to Location

* Add regression test

* Bump package version

* Load secret during workflow
2024-12-03 18:35:56 +01:00
dynco-nym fc79f739d4 Fix overflow (#5204) 2024-12-03 10:20:28 +01:00
Simon Wicky 4851614375 NS API - Gateway stats scraping (#5180)
* squashed commit before rebasing

* removed blank lines
2024-12-02 12:15:30 +01:00
dynco-nym a9e62889c3 Remove explorer dependency (#5190)
* Move monitor code to a struct
- to store state in a struct

* explorer deprecation wip

* Replace explorer with ipinfo calls

* PR feedback

* Fix clippy

* Bump package version

* Remove ipinfo crate due to openssl dep

* Add remaining bandwidth log
2024-11-29 16:45:55 +01:00
dynco-nym 96ebe3fc4f Fix overflow (#5184) 2024-11-27 17:07:01 +01:00
dynco-nym e7f806219c Move NS client to separate package under NS API (#5171)
* Move client code to NS API

* Move client to separate package

* Move things around

* Adjust run scripts

* rustfmt

* Add client to workspace
2024-11-26 15:59:42 +01:00
Dinko Zdravac 1f53da7456 Fix env var name (#5165) 2024-11-22 15:41:16 +00:00
Dinko Zdravac 3f072e4e9d NS Agent auth with NS API (#5127)
* Agents authenticate with NSAPI

* /submit with better auth
- also adjust agent run script to authenticate, even in parallel

* /request better authentication
- moved agent API calls to Client struct

* Replay protection

* Fix testrun cleanup bug
- introduce a new column last_assigned which is different than
  created_at so that stale testruns get cleaned up based on
  last_assigned
- created_at is still useful for determining the "oldest" testrun
  to be picked up

* Uniform request authentication

* Suppress ts-rs serde warnings

* Update cargo version

* All agents use the same key
- remove assigned_agent column
- remove logic which would stop agents with
  the same key to connect
- as a safety measure, add cap to total no. of agents
2024-11-21 14:32:15 +01:00
Jędrzej Stuczyński 7bafe6583a Merge branch 'release/2024.13-magura' into merge/release/2024.13-magura 2024-11-19 15:02:21 +02:00
Dinko Zdravac d03c5b3650 Graceful agent 1.1.5 (#5093)
* Bump NS agent to 0.1.5

* API improvements
- agent exits gracefully when no testrun available
- API doesn't log every error

* Bump NSAPI to 0.1.6
2024-11-05 15:36:16 +01:00
dynco-nym e840c1fe93 Merge branch 'release/2024.13-magura' into develop-release-merge 2024-11-04 16:50:44 +01:00
Fran Arbanas de4239a5dd fix: update dockerfile env vars description (#5079) 2024-11-04 10:35:49 +01:00
Dinko Zdravac 09b68a8204 Cherry pick NS API from develop (#5074)
* Revert "NS API with directory v2 (#5068)"

This reverts commit cf4fe5f875.

* Merge pull request #5050 from nymtech/dz-node-status-api

Node Status API

* Ns agent workflow (#5055)

* feat: add dockerfile

* add github workflow for node status agent

---------

Co-authored-by: Fran Arbanas <arbanasfran@gmail.com>

* NS API with directory v2 (#5058)

* Use unstable explorer client

* Clean up stale testruns & logging
- log gw identity key
- better agent testrun logging
- log responses
- change response code for agents

* Better logging on agent

* Testrun stores gw identity key instead of gw pk

* Agent 0.1.3

* Agent 0.1.4

* Sqlx offline query data + clippy

* Compatible with directory v2

* Point to internal deps + rebase + v0.1.5

* self described field not null

* Fix build.rs typo

* Fix clippy

---------

Co-authored-by: Fran Arbanas <arbanasfran@gmail.com>
2024-11-01 01:24:41 +01:00
Dinko Zdravac cf4fe5f875 NS API with directory v2 (#5068)
* Use unstable explorer client

* Clean up stale testruns & logging
- log gw identity key
- better agent testrun logging
- log responses
- change response code for agents

* Better logging on agent

* Testrun stores gw identity key instead of gw pk

* Agent 0.1.3

* Agent 0.1.4

* Sqlx offline query data + clippy

* Compatible with directory v2

* Point to internal deps + rebase + v0.1.5

* self described field not null

* Fix build.rs typo
2024-10-31 13:52:20 +01:00
Dinko Zdravac c740f84336 NS API with directory v2 (#5058)
* Use unstable explorer client

* Clean up stale testruns & logging
- log gw identity key
- better agent testrun logging
- log responses
- change response code for agents

* Better logging on agent

* Testrun stores gw identity key instead of gw pk

* Agent 0.1.3

* Agent 0.1.4

* Sqlx offline query data + clippy

* Compatible with directory v2

* Point to internal deps + rebase + v0.1.5

* self described field not null

* Fix build.rs typo
2024-10-31 04:32:41 +01:00
dynco-nym 9583a5c6c8 Fix build script 2024-10-29 00:24:18 +01:00
Dinko Zdravac cc983963d4 Fully functional network scores (#5048)
* Compile & copy wg probe

* Node status agent WIP

* Enable debug logging

* Agent submits results
- add clap to agent
- agent runs network probe
- /submit endpoint on NS API

* Build clients with timeouts

* Update logging and dev scripts

* Replace /blaclisted endpoint

* Testruns fully functional
- task that queues testruns periodically
- testruns read/write in DB

* Probe scores fully working
- testruns are assigned on API
- submit updates testruns correctly on NS API side
- agent registers with API
- agent submits results correctly

* Clippy fixes

* PR feedback

* Clippy again

* PR feedback

* Run clippy earlier in CI

* Make refresh delay configurable in server & agent
2024-10-28 17:31:43 +01:00
Fran Arbanas 40d9321aec Node status API dockerfile and env vars (#4986)
* feat: add dockerfile and env variables

* Added workflow for pushing node status api on harbor

* Misc changes to pathing and using yq instead of jq

* fix: change the way we read env vars for nyxd, nym api and explorer

* fix: docker build workflow

* Remove config in favor of clap args

* Added naming and tags

* change from value to result

---------

Co-authored-by: Lawrence Stalder <lawrence@nymtech.net>
Co-authored-by: dynco-nym <173912580+dynco-nym@users.noreply.github.com>
2024-10-28 17:07:38 +01:00
dynco-nym e5a29cc76e Work with directory pre-v2.1
Rebase + point to earlier network client code

Adjust to new Nym API types

Refer to earlier client code

Revert "Rebase + point to earlier network client code"

This reverts commit dd75e7dc0695c25b0883e2f5dd15b7d70165e9e8.

Point to earlier commit
2024-10-28 17:04:22 +01:00
Dinko Zdravac 56c55f6b95 Working HTTP server (#4941)
* Server file structure

* Create HTTP server
- graceful shutdown
- routes
- logging, CORS

* gateways WIP

* gateways API + swagger docs complete

* Mixnodes API + swagger docs complete

* Services API + swagger docs complete

* Commit summary insert

* Make troubleshooting DB easier

* Summary API + swagger docs

* Client log changes

* QOL improvements

- remove implicit panics via `as`
- safer DTO conversions
- add logging
- new config
2024-10-28 16:59:12 +01:00
Dinko Zdravac 2f051fd943 Node Status API background task (#4854)
* Setup new package

* Setup DB

* Fetch & store mixnodes/GWs
- refactor db package structure
- finally solve DATABASE_URL: absolute path works best

* Additional query functionality
- missing only daily summary, which requires type refactoring

* Replace type alias tuples with structs

* Insert summary

* Add github job to build package

* Build script for sqlx

* Remove data dir
- useless now that sqlx DB sits in OUT_DIR

* PR feedback
2024-10-28 16:54:26 +01:00