mfahampshire
cf3fd00350
* - standardise versions for all nym-sdk workspace dependencies - prepend sqlx-pool-guard with 'nym-' * Test remove nym-api from deps * Add oneliner to client_pool doc comments * Add note to commented out docs.rs link in sdk * remove nym-api from script * add publishing file * bring non-binary / contract / tools into workspace version * added more info to publishing.md * make deps workspace version * remove uploaded sphinx-types crate from script * remove erroueously included ignore-defaults * add zeroise to feature * chore: Release * add topology to batch * more cargo versioning * more cargo versioning - wasm utils * more cargo versioning - wasm utils * Add publish=false to manifest for cargo workspaces / crates.io publishing exclusion * remove script now switched to manifest based exclusion * rename import based on rename of contracts-common dep * Making workspace versions for publication + removing unnecessary crates from publication * Remove OOD info from publishing sdk guide * rename contract imports + remove package * temp commit: continuing with removal of path from cargo manifest and replacing with workspace version import for publication * continuing with cargo.toml updates * dryrun only erroring on known version problem crates * remove old published-crates file * Minor comment change * remove default features warning * Additional info on workspace dep comment re publish list * Add missing description to cargo.toml * Fix missing feature flags * Add missing descriptions * Fix remaining path import * Add workspace repo / homepage / documentation links to cargo.toml files * remove workspace version from excluded crate * Remove todo descriptions * Minor comment change * add homepage etc * move from bls git import to nym_bls_fork crate * Modify rest of imports from path to workspace import, excluding binaries * add directory/homepage info * fix cargo fmt * add notes to gitignore * better solution to contracts/ experiment * wasm -> nym_wasm crate renaming * fix fatfinger * add metadata to ecash cargo.toml * stub publishing guide * fix misrevolved netlink- version * Fixes and block publication of rebase re: LP * first pass @ workflows
Nym Data Observatory
Collects data about the Nym network including:
- Chain scraper - that parses blocks, transactions and messages on the Nyx chain
- Price scraper - to get the NYM/USD token price from CoinGecko
- Webhooks - trigger on messages or all messages to call with details
Running locally
1. Install Prerequisites
# Install sqlx-cli if not already installed
make sqlx-cli
2. Prepare PostgreSQL for Development
# This will:
# - Start PostgreSQL in Docker
# - Run migrations
# - Generate SQLx offline query cache
# - Stop the database
make prepare-pg
3. Build
make build-pg
4. Run with PostgreSQL
# Start PostgreSQL for development (keeps running)
make test-db-up
# In another terminal, run the application
NYM_DATA_OBSERVATORY_DB_URL=postgres://testuser:testpass@localhost:5433/nym_data_observatory_test \
NYM_DATA_OBSERVATORY_WEBHOOK_URL="https://webhook.site" \
NYM_DATA_OBSERVATORY_WEBHOOK_AUTH=1234 \
cargo run -- run
To start from a block add the env var: NYXD_SCRAPER_START_HEIGHT=19266184.
Deploying
Connect with psql to your local database:
CREATE USER nym_data_observatory WITH PASSWORD 'data-data-data';
CREATE DATABASE nym_data_observatory_data;
GRANT ALL ON DATABASE nym_data_observatory_data TO nym_data_observatory;
Then run:
cargo run -- init --db_url postgres://testuser:testpass@localhost:5433/nym_data_observatory_test
and then:
NYM_DATA_OBSERVATORY_DB_URL=postgres://testuser:testpass@localhost:5433/nym_data_observatory_test \
NYM_DATA_OBSERVATORY_WEBHOOK_URL="https://webhook.site" \
NYM_DATA_OBSERVATORY_WEBHOOK_AUTH=1234 \
cargo run -- run --websocket-url wss://rpc.nymtech.net/websocket --rpc-url https://rpc.nymtech.net
or just:
NYM_DATA_OBSERVATORY_DB_URL=postgres://testuser:testpass@localhost:5433/nym_data_observatory_test cargo run -- run --websocket-url wss://rpc.nymtech.net/websocket --rpc-url https://rpc.nymtech.net
If you want to watch for cosmwasm messages and send to a webhook:
NYM_DATA_OBSERVATORY_WEBHOOK_URL=https://webhook.site \
NYM_DATA_OBSERVATORY_DB_URL=postgres://testuser:testpass@localhost:5433/nym_data_observatory_test\
cargo run -- run --websocket-url wss://rpc.nymtech.net/websocket --rpc-url https://rpc.nymtech.net --start-block-height 20966360 --watch-for-chain-message-types "/cosmwasm.wasm.v1.MsgExecuteContract"
Troubleshooting
SQLx Offline Mode
If you see "no cached data for this query" errors:
- Ensure PostgreSQL is running:
make dev-db - Run:
make test-db-prepare
Also see README_SQLX.md.
Connection Refused
If you see "Connection refused" errors:
- Check Docker is running:
docker ps - Check PostgreSQL container: `docker ps | grep nym_data_observatory
- Restart database:
make test-db-down && make dev-db