Commit Graph

38 Commits

Author SHA1 Message Date
Yeastplume e268993f5e [WIP] Needs more TUI (#765)
* first explorations at attempting to integrate more user-friendly status screen/ui

* rustfmt

* adding some logo and color for visual interest

* formatting

* better integration with stdout, cleaner looking startup and shutdown

* rustfmt

* update to framework, and first collection of stats from server

* rustfmt

* commit of basic stat screen, think it's in a good enough state to share

* rustfmt

* fix to automated tests

* fix to automated tests

* grin.toml setting

* grin.toml setting

* grin.toml setting

* merge from upstream

* merge from upstream

* merging

* merging

* formatting

* adding more status screens, beginning to collect peer data

* rustfmt

* beginnings of peer info

* adding tui dir

* rustfmt

* missing table

* create title bar

* Split up UI elements into separate files, create trait for update

* Added basic mining view
2018-03-13 18:10:45 +00:00
Yeastplume 1143d84238 Remove Sumtree References and disambiguate some naming (#747)
* start of renamathon

* api renaming

* Rename UTXO-Output to lessen ambiguity

* compile warning

* compile error

* readme fix

* remove file commit in error
2018-03-05 19:33:44 +00:00
Ignotus Peverell be8d9633e4 rustfmt all the things 2018-03-04 00:19:54 +00:00
Ignotus Peverell 22c521eec8 [WIP] Abridged sync (#440)
* Util to zip and unzip directories
* First pass at sumtree request/response. Add message types, implement the exchange in the protocol, zip up the sumtree directory and stream the file over, with necessary adapter hooks.
* Implement the sumtree archive receive logicGets the sumtree archive data stream from the network and write it to a file. Unzip the file, place it at the right spot and reconstruct the sumtree data structure, rewinding where to the right spot.
* Sumtree hash structure validation
* Simplify sumtree backend buffering logic. The backend for a sumtree has to implement some in-memory buffering logic to provide a commit/rollback interface. The backend itself is an aggregate of 3 underlying storages (an append only file, a remove log and a skip list). The buffering was previously implemented both by the backend and some of the underlying storages. Now pushing back all buffering logic to the storages to keep the backend simpler.
* Add kernel append only store file to sumtrees. The chain sumtrees structure now also saves all kernels to a dedicated file. As that storage is implemented by the append only file wrapper, it's also rewind-aware.
* Full state validation. Checks that:
    - MMRs are sane (hash and sum each node)
    - Tree roots match the corresponding header
    - Kernel signatures are valid
    - Sum of all kernel excesses equals the sum of UTXO commitments
minus the supply
* Fast sync handoff to body sync. Once the fast-sync state is fully setup, get bacj in body sync
mode to get the full bodies of the last blocks we're missing.
* First fully working fast sync
* Facility in p2p conn to deal with attachments (raw binary after message).
* Re-introduced sumtree send and receive message handling using the above.
* Fixed test and finished updating all required db state after sumtree validation.
* Massaged a little bit the pipeline orphan check to still work after the new sumtrees have been setup.
* Various cleanup. Consolidated fast sync and full sync into a single function as they're very similar. Proper conditions to trigger a sumtree request and some checks on receiving it.
2018-02-09 22:32:16 +00:00
Ignotus Peverell c7418cfe04 [WIP] Rewrite peer-to-peer logic to replace tokio with simple threading (#664) 2018-02-02 02:03:12 +00:00
Quentin Le Sceller 1e50d56c7e Add unit tests for API Endpoint (#653)
* First tests for API handlers

* Test chain UTXO handler

* Add test sumtrees handlers

* Fix typo

* Removed unused logger

* Update bitflags to ^1.0

* Missing test for bitflags update

* Moved test to grin directory

* Remove tests dependencies

* Add API P2P tests

* Fix hex string commitment

* Fix conflicting port

* Fix directory conflict and server port

* Wait for at least one block is mined on Travis
2018-02-01 13:14:32 -05:00
Quentin Le Sceller 2dac10a690 Display chain status with grin client status (#543)
* Add term and grin_p2p for grin client

* Setting msg as public for the client

* Specifying binary for cargo build

* Minimal client implementation

* Using server configuration for the client

* Display current chain height

* Display difficulty and last block pushed

* Remove unneeded secp256k1 dependency
2017-12-22 18:46:28 +00:00
Simon B 38741c91e2 cherry-pick "[testnet1] cpu_pool and buffered fold in read_msg (#479)" (#482) 2017-12-14 07:19:43 -05:00
Ignotus Peverell e50703d79e Externalized all peers selection and handling (#468)
Moved handling to the peer map out of the p2p server and into
its own struct. Allowed factoring code from the net adapter and
simplification of some interactions. Also removes the need for
the adapter to reference the p2p server or peers.

Fixes #430, #453 and #456
2017-12-14 01:13:05 +01:00
AntiochP bffd955c26 Revert "introduce cpu_pool to read_msg (#477)" (#478)
This reverts commit 45b9962109.
2017-12-13 16:30:59 -05:00
AntiochP 45b9962109 introduce cpu_pool to read_msg (#477)
* introduce cpu_pool to read_msg

* cleanup and less unwrap() in handler

* expose cpu_pool as part of net_adapter (reuse across all peer connections and handlers)

* fix DummyAdaptor for test usage

* add new() to DummyAdapter for convenience
2017-12-13 15:05:25 -05:00
AntiochP 442ef3b255 port across "simple_sync" changes from testnet1 branch (#409)
* port across "simple_sync" changes from testnet1 branch
Cleanup direct refs to peer map or peer store
P2P server acts as a facade, handling the list of connected peers
and the storage of their information. Everything else goes through
the p2p server instead of having a peer map reference or going
straight to the store.
Fix p2p tests

* fix "monitoring peers" log msg (use connected_peers)
2017-11-30 10:27:50 -05:00
AntiochP c6b94dd2c4 generate locator heights all the way down to 0 by powers of 2 (#378) 2017-11-22 22:59:33 -05:00
Ignotus Peverell e4ebb7c7cb Cleanup HTTP APIs, update ports to avoid gap, rustfmt
Moved the HTTP APIs away from the REST endpoint abstraction and
to simpler Hyper handlers. Re-established all routes as v1.
Changed wallet receiver port to 13415 to avoid a gap in port
numbers.

Finally, rustfmt seems to have ignored specific files arguments,
running on everything.
2017-10-31 19:42:56 -04:00
Yeastplume 8e382a7593 slog-rs logging (#171)
* added global slog instance, changed all logging macro formats to include logger instance
* adding configuration to logging, allowing for multiple log outputs
* updates to test, changes to build docs
* rustfmt
* moving logging functions into util crate
2017-10-12 16:56:44 +00:00
AntiochP 3b4a48b2fd Cleanup build warnings (#87)
* minor cleanup - unused imports
* cleanup build warnings - unused vars
* make structs pub to get rid of the private_in_public lint warning
* missing docs on RangeProof
* add missing docs to store delete function
* cleaned up deprecation warning -
tokio_core -> tokio_io
complete() -> send()
2017-08-10 00:54:10 +00:00
Jacob Payne 6a15100c96 [WIP] Rate Limiting (#57)
* Setup tests for rate limiting
* Add send and receive rate configs to P2P Config
* Setup ThrottledConnection and ThrottledWrite
* Add Tokio io
* Fix ThrottledReader + Add Bytes
* Attach Rate Limiting to Connection
2017-06-18 01:15:46 +02:00
Ignotus Peverell 40d23db6c1 Revert "[WIP] Partial Transition from Writeable/Readable to Codecs (#51)"
This reverts commit de4ebdde71.
2017-05-31 15:47:52 -07:00
Jacob Payne de4ebdde71 [WIP] Partial Transition from Writeable/Readable to Codecs (#51)
* Sample Signatures for put_enc and get_dec
* Implement put_enc and get_dec
* Implement ChainCodec in grin_chain
* Truncate src only on complete Blocks
* Truncate src only on complete Tip + Check Len
* Move BlockHeader Encoding to BlockHeaderCodec
* Define put_enc for store::Batch
* Replace BlockCodec and BlockHeaderCodec with generic BlockCodec<T>
* Implement Default for BlockCodec Manually
* Replace get_ser/put_ser with get_enc/get_dec for chain::ChainKVStore
* Remove Writeable/Readable for chain::Tip
* Add Tokio-io and Bytes to grin_p2p
* Additional Setup for Message enum + Msg{Encode,Decode} traits
* base msg ping pong encoding and test
* fill out msg-codec tests
* Implement Hand Encoding/Decoding
* msg-encode shake
* msg-encode getpeeraddr
* codec peer-addrs message, SockAddr struct wierdness
* header message codec
* msg encoding finished prelim
* Implement PeerCodec Encoding/Decoding
* Set PeerStore to use PeerCodec for Encoding/Decoding
* Add a DecIterator
* Prune PeerStore
* Replace Decoding and Encoding in handle_payload
* Prune Writeable/Readable methods in store::Store
* Remove Incomplete Frame Testing ( Not Nessesary right now )
* separate block and tx codec tests
* Refactor {Tx,Block}Codec Tests
2017-05-22 19:16:13 +02:00
Ignotus Peverell 3fe28b0f03 Server types cleanup and serde for configuration types
Moved error and configuration types from server.rs to types.rs
where they should have been from the start. Configuration types
for the server and p2p modules now derive serde Serialize and
Deserialize so we can have a simple JSON configuration file.
2017-04-24 18:49:57 -07:00
Ignotus Peverell 7c72ccec7c Cleaned up error handling in p2p module. Setting peer status on disconnect. 2017-02-26 20:08:40 -08:00
Ignotus Peverell eb024e91d2 Discovery and seeding of other peers. Relies either on a gist with IP addresses or a static list. Connects to a first list, sending peer request messages to discover more. Monitor the server to make sure there's always enough peer connections. 2017-02-18 18:42:34 -08:00
Ignotus Peverell 786da24653 A db store for peer data. 2017-02-09 20:16:34 -08:00
Ignotus Peverell 4af049a887 Added full node sync mode. Follows closely the bitcoin header-first sync. Related p2p messages and protocol. 2017-02-07 13:52:17 -08:00
Ignotus Peverell 7f029cb4c0 Added a Connection wrapper to handle timeouts when we want information from a peer in a request/response style. 2017-02-01 19:05:17 -08:00
Ignotus Peverell 572c1951e1 Introduced a higher-level connection abstraction, allowing the protocol to stay relatively simple. Deals with the lower level details of sending, receiving data, timeouts and dealing with futures. 2017-01-29 22:43:20 -08:00
Ignotus Peverell e688ff99e6 Started putting in place the adapter between the chain and p2p modules to forward blocks and transactions. Cleaned up chain store references. 2016-12-18 15:51:54 -08:00
Ignotus Peverell 4b5c010b05 Rewrote most of p2p code to use futures-rs instead of mioco. Need some cleanup and support for a few more message types. 2016-12-10 19:11:49 -08:00
Ignotus Peverell 309cb497f8 Configurable host and port for p2p server. Utility for client connection. 2016-11-29 18:49:03 -08:00
Ignotus Peverell ea425dc614 Protocol cleanup. Tests cleanup. Additional test for transaction broadcast. 2016-11-05 16:31:45 -07:00
Ignotus Peverell 098d17ee42 Added limited and counting readers for protocol to maintain its stats. Added primitive error count. Cleaned up protocol. 2016-11-02 14:19:40 -07:00
Garrick Ollivander d7dcd6d3ae nicer processing of message types 2016-10-31 13:08:48 +01:00
Ignotus Peverell ee6fcab8db Proper server and protocol event loop. Channels for shutdown and ping/pong trivial roundtrip. Working test. 2016-10-30 18:24:19 -07:00
Ignotus Peverell 42769c373c Re-implemented the whole interaction between server, peer and protocol to be more Rust-ish. Server maintains peer references and protocol is internally mutable. 2016-10-29 12:36:45 -07:00
Ignotus Peverell fdaf2ba6af First working test with related fixes demonstrating a full peer handshake. 2016-10-27 14:28:02 -07:00
Ignotus Peverell b1762cb5f4 p2p module getting close to first integration test allowing 2 peers to connect. Fleshed out handshake and protocol use. Fixed most errors, just need to make the borrow checker happy. 2016-10-25 22:06:13 -07:00
Ignotus Peverell 4657b09c4e WIP: more p2p code, mostly interaction between peer, protocol and server as well as ser/deser for first message types. 2016-10-24 21:35:10 -07:00
Ignotus Peverell c8aa8d7c18 Getting started on the p2p networking code. 2016-10-23 17:02:02 -04:00