Commit Graph

51 Commits

Author SHA1 Message Date
Antioch Peverell a362888ab9 read header_head and sync_head from the header and sync MMR respectively (#3045)
no need to maintain header_head and sync_head in the db explicitly
2019-10-29 16:47:08 +00:00
Antioch Peverell 8f4a1cba67 track opts when requesting blocks (not just if we requested them) (#3089) 2019-10-10 09:38:25 +01:00
Quentin Le Sceller eefd87aa2e More robust peer banning (#3086)
More robust peer banning
2019-10-04 18:00:49 -04:00
Quentin Le Sceller 6be6391225 Change 2018 to 2019 in copyright files (#3072) 2019-10-02 09:40:20 +01:00
Antioch Peverell 5bf813ee93 improve checking for p2p connection limits (revised) (#2993)
* Add check for p2p connection limits

* Simplify undesirable connection shutdown

* Make inbound and outbound connections more explicit

* Cleanup inbound and outbound connections

* Cleanup an outbound peers check

* Rename healthy_peers_mix to enough_outbound_peers

* be a lot less restrictive when picking some candidate peers to connect to
keep the peer address queue drained but actually attempt a healthy number of connections
as most of these attempts are going to fail due to majority of nodes not being publicly accessible
2019-08-21 19:58:43 +01:00
antiochp 17dddeeb0d Revert "Improve checking for p2p connection limits (#2985)"
This reverts commit 24f0a52437.
2019-08-21 15:02:10 +01:00
j01tz 24f0a52437 Improve checking for p2p connection limits (#2985)
* Add check for p2p connection limits

* Simplify undesirable connection shutdown

* Make inbound and outbound connections more explicit

* Cleanup inbound and outbound connections

* Cleanup an outbound peers check

* Rename healthy_peers_mix to enough_outbound_peers
2019-08-21 12:43:09 +01:00
antiochp c6cea4744c Merge branch 'master' into milestone/2.x.x 2019-07-24 15:10:26 +01:00
Quentin Le Sceller 38ddc32cf3 Move SyncStatus and SyncState to chain crate (#2964)
* Moved SyncState and SyncStatus to chain

* Rustfmt

* Fix typo in SyncError
2019-07-23 16:47:59 +02:00
Quentin Le Sceller 4966dc04f7 [2.x.x] Generate txhashset archives on 720 block intervals. (#2951)
Generate txhashset archives on 720 block intervals.
2019-07-23 10:46:29 +02:00
hashmap dbd2535f42 Increase peer's send buffer (#2931) (#2934)
When we send a txhashet archive a peer's thread is busy with sending it
and can't send other messages, eg pings. If the network connection is
slow buffer capacity 10 may be not enough,  hence the peer's drop.

Safer attempt to address #2929 in 2.0.0
2019-06-30 22:26:17 +02:00
Quentin Le Sceller fd6fe35777 Revert #2813 (#2920)
* Revert #2813

* Rustfmt
2019-06-26 10:37:34 +01:00
Cadmus Peverell 5ebe2aa397 Generate txhashset archives on 720 block intervals. (#2813)
* generate txhashset archives on 250 block intervals.

* moved txhashset_archive_interval to global and added a simple test.

* cleaning up the tests and adding license.

* increasing cleanup duration to 24 hours to prevent premature deletion of the current txhashset archive

* bug fixes and changing request_state to request height using archive_interval.

* removing stopstate from chain_test_helper to fix compile issue
2019-06-06 10:39:07 +08:00
Gary Yu e345405201 fix: fraud peer ban doesn't work in some cases (#2867) 2019-05-31 08:02:54 +08:00
hashmap 9ab23f6eef Implement graceful shutdown (#2812)
* Remove stop status mutex
* remove some deadlocks
* Rewrite stop channel handling
* fix deadlock in peers object
* add missing test fixes
2019-05-15 08:51:35 -07:00
Antioch Peverell e8c50359e4 Pull initial chain compaction out of init() and into the syncer (#2738) 2019-04-12 15:38:07 -07:00
hashmap 94732b0d58 Return Result from methods of ChainAdapter (#2722)
Most of the methods return nothing or bool which is used to decide if a
sender of a message should be banned or not. However underlying chain
implementation may fail so we need a way to reflect this fact in API.

Also it allows to reduce number of unwraps and makes the code more robust.
2019-04-08 22:13:28 +02:00
hashmap 325e32821d Reduce number of unwraps in servers crate (#2707)
It doesn't include stratum server which is sufficiently changed in 1.1
branch and adapters, which is big enough for a separate PR.
2019-03-31 23:24:11 +02:00
Gary Yu 52b5dc0e8d fix: the restart of state sync doesn't work sometimes (#2687) 2019-03-18 10:44:51 -07:00
hashmap dc59f67c7b Reduce number of unwraps in chain crate (#2679) 2019-03-17 13:32:48 +01:00
Agreene 4c9984cf11 Display the current download rate rather than the average when syncing the chain (#2633)
* When syncing the chain, calculate the displayed download speed using the current rate from the most recent iteration, rather than the average download speed from the entire syncing process.

* Replace the explicitly ignored variables in the pattern with an implicit ignore
2019-03-03 13:44:31 -08:00
Antioch Peverell 23cb9e2514 Map peers by ip only (ignoring port unless on loopback ip) (#2540)
* wip

* big refactor, regretting doing this now

* PeerAddr everywhere

* cleanup

* fixup server tests

* peers api working for GET, POST is still WIP

* we can now ban/unban peers by ip only (port optional)
2019-02-18 12:15:32 +00:00
Gary Yu 7698b28e16 fix: wait_for_min_peers shouldn't wait forever when all peers are in same work (#2340) 2019-01-12 07:38:27 +08:00
Gary Yu ea4407a919 wait_for_min_peers should continue waiting if none connected (#2326)
* wait_for_min_peers should continue waiting if none connected

* allow no peers to go in non-production mode, such as travis-ci test
2019-01-10 21:08:51 +08:00
Gary Yu c0507b2ab1 Ban a fraud peer which claims a fake height and total difficulty (#2237)
* ban a peer with highest height but can't send us the proper headers
* reset self.stalling_ts if syncing goes well
* fix: switch to peers.ban_peer() instead of peer.set_banned()
2018-12-28 14:09:09 -08:00
Antioch Peverell ea51663f9c Remember to reset sync head on first transition to header sync (#2222)
Also cleanup reset_sync_head code
2018-12-26 13:33:01 -08:00
Gary Yu ea1c3a9d4d fix get_locator which loop the whole headers (#2159) 2018-12-17 07:29:49 +08:00
Antioch Peverell 2d4538c428 Clean shutdown via stop_state (#2117)
* clean shutdown wip

* rustfmt

* introduce StopState that we can lock on

* rustfmt

* take lock on stop_state during critical processing (process_block_single etc.)

* rustfmt

* take lock on stop_state during chain::init()

* cleanup

* cleanup

* rustfmt

* docs/comments

* fixup servers tests

* cleanup p2p tests
2018-12-11 11:07:41 +00:00
hashmap aedac483f5 Convert to Rust 2018 edition (#2084)
* Convert to Rust 2018 edition

* Update gitignore
2018-12-08 00:59:40 +01:00
Antioch Peverell 919fd84b78 cleanup unnecessary reset_head calls (#2072) 2018-12-03 22:27:03 +00:00
Gary Yu 9af9ca9518 refactor the state_sync to handle the long fork (#1902)
* split horizon into two explicit values for cut through and txhashset request

* let node which has 2-7 days of history be able to handle forks larger than 2 days

* add test simulate_long_fork

* add pause/resume feature on p2p for tests

* refactor the state_sync

* ignore the test case simulate_long_fork for normal Travis-CI

* refactor function check_txhashset_needed to be shared with body_sync

* fix: state TxHashsetDone should allow header sync
2018-11-10 11:27:52 +08:00
Antioch Peverell 12be191ecd validate root of header MMR when processing headers (during sync and full blocks) (#1836)
validate root of header MMR when processing headers (during sync and full blocks) (#1836)
2018-11-01 09:51:32 +00:00
Gary Yu 5651fa7a8f add some log to help analysis root cause of #1880 (#1891) 2018-10-31 10:23:15 +00:00
Ignotus Peverell 6f14f4c998 Cleanup and simplify sync (locator, body, etc.) (#1860)
* Cleanup syncer and sync header (locator)
* Simplify body sync
* Remove duplicate head in locator, add greater case in close_enough
* Various sync small fixes and tuning after testing
* More close_enough tests and related minor fixes
2018-10-30 14:09:24 -07:00
Antioch Peverell 38cbd6eafb full node == fast sync (no full archival guarantees) (#1809)
* wip - fast sync only

* wip

* cleanup

* cleanup

* cleanup comments in default config file around capabilities

* fixup p2p tests
2018-10-23 13:01:19 +01:00
Ignotus Peverell a4a4c5610f Cleanup sync loop (#1811)
* Add a struct to encapsulate common references and avoid passing
them around on every function.
* Consolidate `skip_sync_wait` and `awaiting_peers` in an
additional sync status.
* New awaiting peer status is initial too
2018-10-22 22:50:13 -07:00
eupn 1195071f5b Replace logging backend to log4rs and add log rotation (#1789)
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
2018-10-21 13:30:56 -07:00
Gary Yu 0d06561a91 replace stdlib RwLock and Mutex with parking_lot (#1793)
* replace all stdlib RwLock with parking_lot RwLock

* replace stdlib Mutex with parking_lot Mutex

* rustfmt
2018-10-20 08:13:07 +08:00
hashmap 85433c659d Introduce a constant for peer send channel capacity (#1761)
We implicitly use it also in body_sync, so it's hard to keep it in sync.
2018-10-16 16:31:00 -07:00
hashmap 466ce986a6 Use round robin for peer selection in body sync
We select a peer to ask a block randomly. Peer's send channel has
capacity 10. If we need too many blocks we limit number of blocks to asks as a number of peers
* 10, which means that there is some probability (pretty high) that we
will overflow send buffer capacity.
This fix freezes a peer list (which gives also some performance boost)
and create a cycle iteraror to equally distribute requests among the
peers.
There is a risk that a peer may be disconnected while we are sending a
request to the chanel, but stricltly speaking it was possible in the old
code too, perhaps with a lower probability.
Fixes #1748
2018-10-16 12:14:17 +02:00
Antioch Peverell 86c1d7683b The Header MMR (One MMR To Rule Them All) (#1716) (#1747)
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync

* add the (currently unused) sync_head mmr

* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download

* support missing header MMR (rebuild as necessary) for legacy nodes

* rename to HashOnly

* cleanup backend.append()

* simplify vec_backend to match simpler append api
2018-10-15 19:24:01 +01:00
Ignotus Peverell 43f4f92730 [T4] Secondary proof of work difficulty adjustments (#1709)
* First pass at secondary proof of work difficulty adjustments
* Core and chain test fixes
* Next difficulty calc now needs a height. Scaling calculation fixes. Setting scaling on mined block.
* Change factor to u32 instead of u64.
* Cleanup structs used by next_difficulty
* Fix header size calc with u32 scaling
2018-10-13 13:57:01 -07:00
Gary Yu 9a716aea72 feature: txhashset downloading progress display on tui (#1729) (#1730)
(cherry picked from commit 5c0eb11a7d)
2018-10-13 10:12:13 +08:00
Gary Yu 4f462cdfdc fix: avoid a confusing log when fastsync start (#1720) 2018-10-11 12:38:13 +08:00
Gary Yu 009baa1b91 fix: panic at get_locator (#1711) 2018-10-10 16:48:41 +08:00
Antioch Peverell 4a1e3c27bb sleep for 10s in syncer when not syncing (#1705) 2018-10-09 21:50:55 +01:00
Antioch Peverell 8cfe9e64ac Remove RwLock around individual peer instances (#1673)
* get rid of RwLock around peer instances

* rustfmt

* rename stuff to live_info

* rustfmt

* serialize into PeerInfoDisplay
limit live_info write lock to peer_info.update()
rename to PeerLiveInfo

* simplify broadcast logic a bit more
connected peers are connected by definition

* return true/false on broadcast so we can send to more peers

* rustfmt

* fix p2p and server tests for PeerInfoDisplay

* commit
2018-10-09 08:27:34 +01:00
Ignotus Peverell ffe6682ce9 Merged upstream 2018-10-05 16:15:48 +00:00
Ignotus Peverell 53265693cf rustfmt 2018-10-05 04:35:14 +00:00
Ignotus Peverell 8925d1e48d Fix for body sync to only run when state sync is off or done 2018-10-05 04:34:47 +00:00