Files
Mark Sinclair f2091cc9d6 Data Observatory (#6172)
* rename nyxd-scraper to sqlite

wip: made storage mostly generic minus modules

changed error types to make modules dyn compatible

implemented traits for sqlite instance

using sqlite instance for rewarder and chain watcher

psql scaffolding

initial postgres support - missing some proto -> json parsing

use postgres in chain scraper

added message registry to block processor

message content parsing in psql

involved addresses

adding null value for logs

Revert "use postgres in chain scraper"

This reverts commit 83c84bfd2d.

using SignerInfo proto definitions for db serialisation

added ibc messages to MessageRegistry

* add the data observatory

* add the data observatory

* move message parsing and change webhook

* handle wasm messages in a module

* formatting and clippy

* copy shared migrations and add comments to ignore file to explain

* update offline queries

* change to clap args and use url::Url to parse args

* tidy up README, startup info, typos

* tidy up validator rewarder

* lock file

* change webhook module from msg to tx handler

* ignore profiler output

* add missing things and make clippy happy

* updated cosmrs version used by the nym wallet

* add glob to workspace dependencies

* rename migration files

* remove copying from shared migrations

* duplicate shared migrations to keep things simple

* add check for manual migration sync that will fail on `cargo build` in CI

* build.rs checks data observatory migrations have content of all shared scraper migrations and errors on changes or new files

* update runner

* add reset target to make file

* process events and logs

* migrations - remove unnecessary columns

* update offline queries

* chore: run cargo fmt

* fix up: inpsect_err instead of map_err

---------

Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
Co-authored-by: benedettadavico <benedetta.davico@gmail.com>
2025-12-02 21:27:22 +00:00

1.8 KiB

Quick Start with PostgreSQL

1. Install Prerequisites

# Install sqlx-cli if not already installed
make sqlx-cli

2. Prepare PostgreSQL for Development

# This will:
# - Start PostgreSQL in Docker
# - Run migrations
# - Generate SQLx offline query cache
# - Stop the database
make prepare-pg

3. Build with PostgreSQL

# Build with PostgreSQL feature
make build-pg

# Or manually:
cargo build

4. Run with PostgreSQL

# Start PostgreSQL for development (keeps running)
make dev-db

# In another terminal, run the application
DATABASE_URL=postgres://testuser:testpass@localhost:5433/nym_node_status_api_test \
cargo run

Makefile Targets

make help                # Show all available targets
make prepare-pg         # Setup PostgreSQL and prepare SQLx cache
make dev-db            # Start PostgreSQL for development
make test-db           # Run tests with PostgreSQL
make build-pg          # Build with PostgreSQL
make psql              # Connect to running PostgreSQL
make clean             # Clean build artifacts
make clean-db          # Stop database and clean volumes
make dev-db-restart    # Stop database, clean volumes, rebuild test database and restart

Environment Variables

See .env.example for all configuration options. Key variable:

# For PostgreSQL:
DATABASE_URL=postgres://testuser:testpass@localhost:5433/nym_node_status_api_test

Troubleshooting

SQLx Offline Mode

If you see "no cached data for this query" errors:

  1. Ensure PostgreSQL is running: make dev-db
  2. Run: make test-db-prepare

Connection Refused

If you see "Connection refused" errors:

  1. Check Docker is running: docker ps
  2. Check PostgreSQL container: docker ps | grep nym_node_status_api_postgres_test
  3. Restart database: make test-db-down && make dev-db