Commit Graph

92 Commits

Author SHA1 Message Date
yeastplume 6980278b95 merge T4 into master - rustfmt 2018-10-18 11:23:22 +01:00
Ignotus Peverell d65a9adb59 Updated seed 2018-10-17 19:59:56 +00:00
Yeastplume 404165a8fd [T4] Add sec pow info to TUI, change magic number, genesis diff to 1 (temporarily) (#1768)
* add sec scaling stats to tui

* rustfmt
2018-10-17 10:37:28 +01:00
Ignotus Peverell fffe5154d2 Secondary PoW scaling factor dampening, cleanup (#1765)
* Remove useless time median window
* Secondary PoW factor dampening
* Fix off-by-one in time window, cleanup dampening, fix tests
2018-10-16 16:55:40 -07:00
hashmap 85433c659d Introduce a constant for peer send channel capacity (#1761)
We implicitly use it also in body_sync, so it's hard to keep it in sync.
2018-10-16 16:31:00 -07:00
hashmap 466ce986a6 Use round robin for peer selection in body sync
We select a peer to ask a block randomly. Peer's send channel has
capacity 10. If we need too many blocks we limit number of blocks to asks as a number of peers
* 10, which means that there is some probability (pretty high) that we
will overflow send buffer capacity.
This fix freezes a peer list (which gives also some performance boost)
and create a cycle iteraror to equally distribute requests among the
peers.
There is a risk that a peer may be disconnected while we are sending a
request to the chanel, but stricltly speaking it was possible in the old
code too, perhaps with a lower probability.
Fixes #1748
2018-10-16 12:14:17 +02:00
John Tromp 34646ddf51 [T4] Rename all shiftsize / cuckoo_size to edge_bits and change value for T4 (#1752)
* replace all size_shift / cuckoo_size by edge_bits and change some constants for T4
* replace remaining occurrences of sizeshift
2018-10-15 16:14:23 -07:00
Antioch Peverell 86c1d7683b The Header MMR (One MMR To Rule Them All) (#1716) (#1747)
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync

* add the (currently unused) sync_head mmr

* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download

* support missing header MMR (rebuild as necessary) for legacy nodes

* rename to HashOnly

* cleanup backend.append()

* simplify vec_backend to match simpler append api
2018-10-15 19:24:01 +01:00
Gary Yu 2e6a242827 small improvement on the servers test (#1737)
* cherry-pick from master for #1736
2018-10-14 20:13:49 +08:00
Ignotus Peverell 43f4f92730 [T4] Secondary proof of work difficulty adjustments (#1709)
* First pass at secondary proof of work difficulty adjustments
* Core and chain test fixes
* Next difficulty calc now needs a height. Scaling calculation fixes. Setting scaling on mined block.
* Change factor to u32 instead of u64.
* Cleanup structs used by next_difficulty
* Fix header size calc with u32 scaling
2018-10-13 13:57:01 -07:00
Gary Yu 9a716aea72 feature: txhashset downloading progress display on tui (#1729) (#1730)
(cherry picked from commit 5c0eb11a7d)
2018-10-13 10:12:13 +08:00
Gary Yu 4f462cdfdc fix: avoid a confusing log when fastsync start (#1720) 2018-10-11 12:38:13 +08:00
Yeastplume 92f826a917 [T4 ONLY] T4 PoW changes (#1663)
* T4 PoW changes

* rustfmt

* adjust sizeshift depending on pow type during ser/deser

* update block size tests for cuckatoo sizeshift
2018-10-10 10:09:44 +01:00
Gary Yu 009baa1b91 fix: panic at get_locator (#1711) 2018-10-10 16:48:41 +08:00
Antioch Peverell 4a1e3c27bb sleep for 10s in syncer when not syncing (#1705) 2018-10-09 21:50:55 +01:00
Antioch Peverell a676eb1b39 Replace RefCell with RwLock in OneTime (and cleanup Weak usage) (#1694)
* rustfmt

* use RwLock in OneTime (and not RefCell)

* rustfmt

* put weak refs back and cleanup

* rustfmt

* revert weak in adapters
2018-10-09 16:53:57 +01:00
Antioch Peverell 8cfe9e64ac Remove RwLock around individual peer instances (#1673)
* get rid of RwLock around peer instances

* rustfmt

* rename stuff to live_info

* rustfmt

* serialize into PeerInfoDisplay
limit live_info write lock to peer_info.update()
rename to PeerLiveInfo

* simplify broadcast logic a bit more
connected peers are connected by definition

* return true/false on broadcast so we can send to more peers

* rustfmt

* fix p2p and server tests for PeerInfoDisplay

* commit
2018-10-09 08:27:34 +01:00
Antioch Peverell 3eb64af1ef Spit chain and peer lmdb envs (#1680)
separate files on disk
separate transactions (batches)
2018-10-07 10:25:12 -07:00
Gary Yu 1a6101f870 refactor: remove thread p2p-moniter, move the Ping to the connect_and_monitor (#1670) 2018-10-07 10:24:06 -07:00
Antioch Peverell 60ef7268ba cleanup mine_block difficulty_iterator usage (#1674)
* take care with diff_iter (contains an active lmdb batch)

* rustfmt
2018-10-06 21:06:02 +01:00
Ignotus Peverell ffe6682ce9 Merged upstream 2018-10-05 16:15:48 +00:00
Antioch Peverell bcf41438dc Simplify chain, remove head. (#1657)
* Go back to db for head and header_head
* Use batch consistently for db access
* Pass active txhashset and batch in the ctx
* Only update head in db if total work increases
* Only updated header_head if total work (on header chain) increases
2018-10-05 08:29:33 +01:00
Ignotus Peverell 53265693cf rustfmt 2018-10-05 04:35:14 +00:00
Ignotus Peverell 8925d1e48d Fix for body sync to only run when state sync is off or done 2018-10-05 04:34:47 +00:00
Ignotus Peverell d7d56733e5 Breaking down sync into multiple structs and files 2018-10-05 00:21:30 +00:00
Ignotus Peverell f2b4c6dc07 Resurrect a random peer on every monitor (#1646)
Picks a random defunct peer and marks it healthy again anytime
we're querying peers to find more. Over enough time, any peer will
see another as defunct, leading to peer list atrophy. This
allows us random retries.

In addition, we need to cleanup peers after a while, we quickly
accumulate hundreds of dead peers over time. This should be
tracked by a different issue however.

Related: #1632
2018-10-03 09:16:32 -07:00
Ignotus Peverell 2919a78b0f Don't stall sync when header chain regresses (#1644)
This should normally never happen but this is meant to catch unforeseen edge cases or block acceptance bugs.
2018-10-03 09:07:57 -07:00
hashmap 4a6cae0fe6 Allow TLS for Wallet APIs (#1626)
* Allow TLS for Wallet APIs

This PR adds an optional support of TLS for wallet APIs. Only PKCS12 format is supported, will address .pem support in next PR and provide some documentation.
Address #1425
2018-10-02 09:49:36 +02:00
Gary Yu d7fbeb2c62 fix: no need switching to HeaderSync in FastSync states (#1622)
* fix: no need switching to HeaderSync in FastSync states
2018-10-02 12:09:49 +08:00
Antioch Peverell 4d2cbe6596 fix: init_sync_head was not behaving as expected (#1624)
Fix #1612
2018-10-01 23:03:06 +08:00
Antioch Peverell 56f84cc2f4 pass batch to process_block and process_block_header (#1594)
* use a batch consistently when processing blocks and headers

* rustfmt
2018-09-27 09:35:25 +01:00
Quentin Le Sceller 62fd8f2124 Implement Basic Auth for API and Owner API (#1566)
* Add api_secret

* Add to base64 method

* Add basic auth in API

* Add Basic Auth to owner API

* Add flag to enable disable basic auth

* Add .api_secret file
2018-09-26 22:38:44 +02:00
Gary Yu 8d62247c99 Small cleanup of rand usage (#1576)
Clean 3 build warning of deprecated item 'rand::Rand::rand'
2018-09-24 15:15:55 -07:00
Gary Yu 32486e2f63 Always enable header sync on initial state transition from NoSync / Initial (#1582) 2018-09-24 15:09:57 -07:00
Antioch Peverell e72d8b58e4 [WIP] txpool (tx validation) using block sums for full validation (#1567)
tx validation (txpool) rework/simplify
2018-09-24 09:24:10 +01:00
Gary Yu 3adddfba76 Refactor: run_sync() ballooning, extract 3 utility functions (#1562) 2018-09-20 10:00:09 -07:00
Gary Yu e1c8dc5a3a fix: reset sync_head to header_head on initial transition to HeaderSync (#1531)
* fix for bug #1524, reset sync_head to header_head on initial transition to HeaderSync.
* using @antiochp fix solution in #1539
2018-09-20 07:29:24 +08:00
Gary Yu 2ca6ecc163 fix: early detection of peer connection lost when downloading txhashset (#1546) 2018-09-20 07:08:02 +08:00
Ignotus Peverell 9cc7aed713 Dual proof of work, first step (#1534)
* Move Proof and Difficulty types to pow module. Difficulty scaling calculation.
* Diffculty scaling and size shift for 2nd PoW
* Backport e7eb26ee
* Test compilation fixes
* Scaling only need to go one way, as @tromp pointed out
2018-09-18 15:12:57 -07:00
Gary Yu d3a5ee130f Refactoring and cleanup on seed (#1513) 2018-09-14 16:09:25 -07:00
Gary Yu 7a689d468c Remove 'WebStatic' seed_type since it was obsoleted a while ago (#1512) 2018-09-11 20:28:27 -07:00
Gary Yu 48857b7e16 Fix infinite loop in case of some peer connect errors (#1494) 2018-09-10 15:23:49 -07:00
Gary Yu e760ea2ce2 a little bit speed-up for full node syncing (#1463)
* simplified: track the head and orphans instead of every single block hash that sync requests
* change 1st block timeout from 1s to 5s
2018-09-10 12:32:20 -07:00
Gary Yu 277687c17c Add 2 more dns seeds (#1496) 2018-09-10 12:11:03 -07:00
Antioch Peverell dbe53f63c8 disable chain compaction on archive nodes (#1498)
* disable chain compaction when in archive mode

* rustfmt
2018-09-10 15:59:42 +01:00
hashmap da87f5fef4 Don't wait 30 seconds in case of a quick restart (#1488)
Currently on startup we wait for 30 secs if we don't have at least 4 peers with more work. If a node was quickly restarted it is already fully synced so thereare no peers with more work at all. This pr assumes that if we've done some work already and we have enough peers but still 0 with more work we are fully synced and good to go.
2018-09-07 13:01:54 -07:00
Gary Yu ed73db671f fix for unstable travis-ci test on servers module (#1434)
* fix for unstable travis-ci test on servers module

* skip the PeerWithSelf connection request on sending side

* logs and comments change; and remove an unnecessary unwrap() in test
2018-09-04 10:52:11 +02:00
Quentin Le Sceller 45fffd8534 Missing logic in preferred peers (#1471) 2018-09-03 16:27:40 +02:00
hashmap fa00a0ebe6 Implement exponential backoff for peer monitoring at startup (#1459)
* Implement exponential backoff for peer monitoring at startup

Fixes #1413
2018-09-01 23:25:47 +02:00
Gary Yu d719493483 HeaderSync optimization (#1372) (#1400)
* improve: HeaderSync optimization (#1372)
* remove get_locator() optimization, which should be an independent pr for security review
* refactoring: move 'headers_streaming_body()' from Message to Protocol
* move 2 headers utils functions out of Protocol, and remove 'pub'
* support reading variable size of BlockHeader, from Cuckoo30 to Cuckoo36
* fix: use global::min_sizeshift() instead of hardcoded 30, because Cuckoo10 will be used for AutomatedTesting chain
* fix: should use global::proofsize() instead of hardcoded 42 when calculate serialized_size_of_header
* replace another 42 with global::proofsize()
2018-08-30 19:50:55 -04:00