Commit Graph

100 Commits

Author SHA1 Message Date
Antioch Peverell 4050f7fccb handle re-orgs in transaction_pool (#1829)
* use reorg_cache in transaction_pool to safely handle txs during a re-org

* rustfmt

* comments
2018-10-24 17:57:31 +01:00
Antioch Peverell 38cbd6eafb full node == fast sync (no full archival guarantees) (#1809)
* wip - fast sync only

* wip

* cleanup

* cleanup

* cleanup comments in default config file around capabilities

* fixup p2p tests
2018-10-23 13:01:19 +01:00
Ignotus Peverell a4a4c5610f Cleanup sync loop (#1811)
* Add a struct to encapsulate common references and avoid passing
them around on every function.
* Consolidate `skip_sync_wait` and `awaiting_peers` in an
additional sync status.
* New awaiting peer status is initial too
2018-10-22 22:50:13 -07:00
Ignotus Peverell f4eb116fbf Minor update to seed module documentation 2018-10-22 14:03:46 -07:00
eupn 8f42f7306c feat: remove expired peers from the storage (#1794)
* Initial expired peers removal
* Stop expired peers
* Simplify peer removal and remove only Defunct peers
* Make seed to check for expired peers every hour
* Get rid of unused vector of peers to remove
* Make peer deletion predicate closure immutable
2018-10-22 13:59:40 -07:00
Quentin Le Sceller 5f2e8db092 Fix shift by 1 block in TUI (#1780) (#1810)
* Fix shift by 1 block in TUI (#1780)
* Only use graph_weight
* Fix shift by 1 in block
* Remove debug logging
2018-10-22 10:44:14 -07:00
eupn 1195071f5b Replace logging backend to log4rs and add log rotation (#1789)
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
2018-10-21 13:30:56 -07:00
Gary Yu 0d06561a91 replace stdlib RwLock and Mutex with parking_lot (#1793)
* replace all stdlib RwLock with parking_lot RwLock

* replace stdlib Mutex with parking_lot Mutex

* rustfmt
2018-10-20 08:13:07 +08:00
yeastplume 6980278b95 merge T4 into master - rustfmt 2018-10-18 11:23:22 +01:00
Ignotus Peverell d65a9adb59 Updated seed 2018-10-17 19:59:56 +00:00
Yeastplume 404165a8fd [T4] Add sec pow info to TUI, change magic number, genesis diff to 1 (temporarily) (#1768)
* add sec scaling stats to tui

* rustfmt
2018-10-17 10:37:28 +01:00
Ignotus Peverell fffe5154d2 Secondary PoW scaling factor dampening, cleanup (#1765)
* Remove useless time median window
* Secondary PoW factor dampening
* Fix off-by-one in time window, cleanup dampening, fix tests
2018-10-16 16:55:40 -07:00
hashmap 85433c659d Introduce a constant for peer send channel capacity (#1761)
We implicitly use it also in body_sync, so it's hard to keep it in sync.
2018-10-16 16:31:00 -07:00
hashmap 466ce986a6 Use round robin for peer selection in body sync
We select a peer to ask a block randomly. Peer's send channel has
capacity 10. If we need too many blocks we limit number of blocks to asks as a number of peers
* 10, which means that there is some probability (pretty high) that we
will overflow send buffer capacity.
This fix freezes a peer list (which gives also some performance boost)
and create a cycle iteraror to equally distribute requests among the
peers.
There is a risk that a peer may be disconnected while we are sending a
request to the chanel, but stricltly speaking it was possible in the old
code too, perhaps with a lower probability.
Fixes #1748
2018-10-16 12:14:17 +02:00
John Tromp 34646ddf51 [T4] Rename all shiftsize / cuckoo_size to edge_bits and change value for T4 (#1752)
* replace all size_shift / cuckoo_size by edge_bits and change some constants for T4
* replace remaining occurrences of sizeshift
2018-10-15 16:14:23 -07:00
Antioch Peverell 86c1d7683b The Header MMR (One MMR To Rule Them All) (#1716) (#1747)
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync

* add the (currently unused) sync_head mmr

* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download

* support missing header MMR (rebuild as necessary) for legacy nodes

* rename to HashOnly

* cleanup backend.append()

* simplify vec_backend to match simpler append api
2018-10-15 19:24:01 +01:00
Gary Yu 2e6a242827 small improvement on the servers test (#1737)
* cherry-pick from master for #1736
2018-10-14 20:13:49 +08:00
Ignotus Peverell 43f4f92730 [T4] Secondary proof of work difficulty adjustments (#1709)
* First pass at secondary proof of work difficulty adjustments
* Core and chain test fixes
* Next difficulty calc now needs a height. Scaling calculation fixes. Setting scaling on mined block.
* Change factor to u32 instead of u64.
* Cleanup structs used by next_difficulty
* Fix header size calc with u32 scaling
2018-10-13 13:57:01 -07:00
Gary Yu 9a716aea72 feature: txhashset downloading progress display on tui (#1729) (#1730)
(cherry picked from commit 5c0eb11a7d)
2018-10-13 10:12:13 +08:00
Gary Yu 4f462cdfdc fix: avoid a confusing log when fastsync start (#1720) 2018-10-11 12:38:13 +08:00
Yeastplume 92f826a917 [T4 ONLY] T4 PoW changes (#1663)
* T4 PoW changes

* rustfmt

* adjust sizeshift depending on pow type during ser/deser

* update block size tests for cuckatoo sizeshift
2018-10-10 10:09:44 +01:00
Gary Yu 009baa1b91 fix: panic at get_locator (#1711) 2018-10-10 16:48:41 +08:00
Antioch Peverell 4a1e3c27bb sleep for 10s in syncer when not syncing (#1705) 2018-10-09 21:50:55 +01:00
Antioch Peverell a676eb1b39 Replace RefCell with RwLock in OneTime (and cleanup Weak usage) (#1694)
* rustfmt

* use RwLock in OneTime (and not RefCell)

* rustfmt

* put weak refs back and cleanup

* rustfmt

* revert weak in adapters
2018-10-09 16:53:57 +01:00
Antioch Peverell 8cfe9e64ac Remove RwLock around individual peer instances (#1673)
* get rid of RwLock around peer instances

* rustfmt

* rename stuff to live_info

* rustfmt

* serialize into PeerInfoDisplay
limit live_info write lock to peer_info.update()
rename to PeerLiveInfo

* simplify broadcast logic a bit more
connected peers are connected by definition

* return true/false on broadcast so we can send to more peers

* rustfmt

* fix p2p and server tests for PeerInfoDisplay

* commit
2018-10-09 08:27:34 +01:00
Antioch Peverell 3eb64af1ef Spit chain and peer lmdb envs (#1680)
separate files on disk
separate transactions (batches)
2018-10-07 10:25:12 -07:00
Gary Yu 1a6101f870 refactor: remove thread p2p-moniter, move the Ping to the connect_and_monitor (#1670) 2018-10-07 10:24:06 -07:00
Antioch Peverell 60ef7268ba cleanup mine_block difficulty_iterator usage (#1674)
* take care with diff_iter (contains an active lmdb batch)

* rustfmt
2018-10-06 21:06:02 +01:00
Ignotus Peverell ffe6682ce9 Merged upstream 2018-10-05 16:15:48 +00:00
Antioch Peverell bcf41438dc Simplify chain, remove head. (#1657)
* Go back to db for head and header_head
* Use batch consistently for db access
* Pass active txhashset and batch in the ctx
* Only update head in db if total work increases
* Only updated header_head if total work (on header chain) increases
2018-10-05 08:29:33 +01:00
Ignotus Peverell 53265693cf rustfmt 2018-10-05 04:35:14 +00:00
Ignotus Peverell 8925d1e48d Fix for body sync to only run when state sync is off or done 2018-10-05 04:34:47 +00:00
Ignotus Peverell d7d56733e5 Breaking down sync into multiple structs and files 2018-10-05 00:21:30 +00:00
Ignotus Peverell f2b4c6dc07 Resurrect a random peer on every monitor (#1646)
Picks a random defunct peer and marks it healthy again anytime
we're querying peers to find more. Over enough time, any peer will
see another as defunct, leading to peer list atrophy. This
allows us random retries.

In addition, we need to cleanup peers after a while, we quickly
accumulate hundreds of dead peers over time. This should be
tracked by a different issue however.

Related: #1632
2018-10-03 09:16:32 -07:00
Ignotus Peverell 2919a78b0f Don't stall sync when header chain regresses (#1644)
This should normally never happen but this is meant to catch unforeseen edge cases or block acceptance bugs.
2018-10-03 09:07:57 -07:00
hashmap 4a6cae0fe6 Allow TLS for Wallet APIs (#1626)
* Allow TLS for Wallet APIs

This PR adds an optional support of TLS for wallet APIs. Only PKCS12 format is supported, will address .pem support in next PR and provide some documentation.
Address #1425
2018-10-02 09:49:36 +02:00
Gary Yu d7fbeb2c62 fix: no need switching to HeaderSync in FastSync states (#1622)
* fix: no need switching to HeaderSync in FastSync states
2018-10-02 12:09:49 +08:00
Antioch Peverell 4d2cbe6596 fix: init_sync_head was not behaving as expected (#1624)
Fix #1612
2018-10-01 23:03:06 +08:00
Antioch Peverell 56f84cc2f4 pass batch to process_block and process_block_header (#1594)
* use a batch consistently when processing blocks and headers

* rustfmt
2018-09-27 09:35:25 +01:00
Quentin Le Sceller 62fd8f2124 Implement Basic Auth for API and Owner API (#1566)
* Add api_secret

* Add to base64 method

* Add basic auth in API

* Add Basic Auth to owner API

* Add flag to enable disable basic auth

* Add .api_secret file
2018-09-26 22:38:44 +02:00
Gary Yu 8d62247c99 Small cleanup of rand usage (#1576)
Clean 3 build warning of deprecated item 'rand::Rand::rand'
2018-09-24 15:15:55 -07:00
Gary Yu 32486e2f63 Always enable header sync on initial state transition from NoSync / Initial (#1582) 2018-09-24 15:09:57 -07:00
Antioch Peverell e72d8b58e4 [WIP] txpool (tx validation) using block sums for full validation (#1567)
tx validation (txpool) rework/simplify
2018-09-24 09:24:10 +01:00
Gary Yu 3adddfba76 Refactor: run_sync() ballooning, extract 3 utility functions (#1562) 2018-09-20 10:00:09 -07:00
Gary Yu e1c8dc5a3a fix: reset sync_head to header_head on initial transition to HeaderSync (#1531)
* fix for bug #1524, reset sync_head to header_head on initial transition to HeaderSync.
* using @antiochp fix solution in #1539
2018-09-20 07:29:24 +08:00
Gary Yu 2ca6ecc163 fix: early detection of peer connection lost when downloading txhashset (#1546) 2018-09-20 07:08:02 +08:00
Ignotus Peverell 9cc7aed713 Dual proof of work, first step (#1534)
* Move Proof and Difficulty types to pow module. Difficulty scaling calculation.
* Diffculty scaling and size shift for 2nd PoW
* Backport e7eb26ee
* Test compilation fixes
* Scaling only need to go one way, as @tromp pointed out
2018-09-18 15:12:57 -07:00
Gary Yu d3a5ee130f Refactoring and cleanup on seed (#1513) 2018-09-14 16:09:25 -07:00
Gary Yu 7a689d468c Remove 'WebStatic' seed_type since it was obsoleted a while ago (#1512) 2018-09-11 20:28:27 -07:00
Gary Yu 48857b7e16 Fix infinite loop in case of some peer connect errors (#1494) 2018-09-10 15:23:49 -07:00