Mark Sinclair
f2091cc9d6
* rename nyxd-scraper to sqlite
wip: made storage mostly generic minus modules
changed error types to make modules dyn compatible
implemented traits for sqlite instance
using sqlite instance for rewarder and chain watcher
psql scaffolding
initial postgres support - missing some proto -> json parsing
use postgres in chain scraper
added message registry to block processor
message content parsing in psql
involved addresses
adding null value for logs
Revert "use postgres in chain scraper"
This reverts commit 83c84bfd2d.
using SignerInfo proto definitions for db serialisation
added ibc messages to MessageRegistry
* add the data observatory
* add the data observatory
* move message parsing and change webhook
* handle wasm messages in a module
* formatting and clippy
* copy shared migrations and add comments to ignore file to explain
* update offline queries
* change to clap args and use url::Url to parse args
* tidy up README, startup info, typos
* tidy up validator rewarder
* lock file
* change webhook module from msg to tx handler
* ignore profiler output
* add missing things and make clippy happy
* updated cosmrs version used by the nym wallet
* add glob to workspace dependencies
* rename migration files
* remove copying from shared migrations
* duplicate shared migrations to keep things simple
* add check for manual migration sync that will fail on `cargo build` in CI
* build.rs checks data observatory migrations have content of all shared scraper migrations and errors on changes or new files
* update runner
* add reset target to make file
* process events and logs
* migrations - remove unnecessary columns
* update offline queries
* chore: run cargo fmt
* fix up: inpsect_err instead of map_err
---------
Co-authored-by: Jędrzej Stuczyński <jedrzej.stuczynski@gmail.com>
Co-authored-by: Mark Sinclair <mmsinclair@users.noreply.github.com>
Co-authored-by: benedettadavico <benedetta.davico@gmail.com>
Nyxd Scraper
Pruning
Similarly to cosmos-sdk, we incorporate pruning into our (scraped) chain data. We attempt to follow their strategies as closely as possible for convenience's sake. Therefore, the following are available:
Strategies
The strategies are configured in config.toml, with the format pruning = "<strategy>" where the options are:
default: only the last 362,880 states(approximately 3.5 weeks worth of state) are kept; pruning at 10 block intervalsnothing: all historic states will be saved, nothing will be deleted (i.e. archiving node)everything: 2 latest states will be kept; pruning at 10 block intervals.custom: allow pruning options to be manually specified throughpruning.keep_recent, andpruning.interval
Custom Pruning
These are applied if and only if the pruning strategy is custom:
pruning.keep_recent: N means to keep all of the last N blockspruning.interval: N means to delete old block data from disk every Nth block.