Commit Graph

78 Commits

Author SHA1 Message Date
Antioch Peverell 4a1e3c27bb sleep for 10s in syncer when not syncing (#1705) 2018-10-09 21:50:55 +01:00
Antioch Peverell a676eb1b39 Replace RefCell with RwLock in OneTime (and cleanup Weak usage) (#1694)
* rustfmt

* use RwLock in OneTime (and not RefCell)

* rustfmt

* put weak refs back and cleanup

* rustfmt

* revert weak in adapters
2018-10-09 16:53:57 +01:00
Antioch Peverell 8cfe9e64ac Remove RwLock around individual peer instances (#1673)
* get rid of RwLock around peer instances

* rustfmt

* rename stuff to live_info

* rustfmt

* serialize into PeerInfoDisplay
limit live_info write lock to peer_info.update()
rename to PeerLiveInfo

* simplify broadcast logic a bit more
connected peers are connected by definition

* return true/false on broadcast so we can send to more peers

* rustfmt

* fix p2p and server tests for PeerInfoDisplay

* commit
2018-10-09 08:27:34 +01:00
Antioch Peverell 3eb64af1ef Spit chain and peer lmdb envs (#1680)
separate files on disk
separate transactions (batches)
2018-10-07 10:25:12 -07:00
Gary Yu 1a6101f870 refactor: remove thread p2p-moniter, move the Ping to the connect_and_monitor (#1670) 2018-10-07 10:24:06 -07:00
Antioch Peverell 60ef7268ba cleanup mine_block difficulty_iterator usage (#1674)
* take care with diff_iter (contains an active lmdb batch)

* rustfmt
2018-10-06 21:06:02 +01:00
Ignotus Peverell ffe6682ce9 Merged upstream 2018-10-05 16:15:48 +00:00
Antioch Peverell bcf41438dc Simplify chain, remove head. (#1657)
* Go back to db for head and header_head
* Use batch consistently for db access
* Pass active txhashset and batch in the ctx
* Only update head in db if total work increases
* Only updated header_head if total work (on header chain) increases
2018-10-05 08:29:33 +01:00
Ignotus Peverell 53265693cf rustfmt 2018-10-05 04:35:14 +00:00
Ignotus Peverell 8925d1e48d Fix for body sync to only run when state sync is off or done 2018-10-05 04:34:47 +00:00
Ignotus Peverell d7d56733e5 Breaking down sync into multiple structs and files 2018-10-05 00:21:30 +00:00
Ignotus Peverell f2b4c6dc07 Resurrect a random peer on every monitor (#1646)
Picks a random defunct peer and marks it healthy again anytime
we're querying peers to find more. Over enough time, any peer will
see another as defunct, leading to peer list atrophy. This
allows us random retries.

In addition, we need to cleanup peers after a while, we quickly
accumulate hundreds of dead peers over time. This should be
tracked by a different issue however.

Related: #1632
2018-10-03 09:16:32 -07:00
Ignotus Peverell 2919a78b0f Don't stall sync when header chain regresses (#1644)
This should normally never happen but this is meant to catch unforeseen edge cases or block acceptance bugs.
2018-10-03 09:07:57 -07:00
hashmap 4a6cae0fe6 Allow TLS for Wallet APIs (#1626)
* Allow TLS for Wallet APIs

This PR adds an optional support of TLS for wallet APIs. Only PKCS12 format is supported, will address .pem support in next PR and provide some documentation.
Address #1425
2018-10-02 09:49:36 +02:00
Gary Yu d7fbeb2c62 fix: no need switching to HeaderSync in FastSync states (#1622)
* fix: no need switching to HeaderSync in FastSync states
2018-10-02 12:09:49 +08:00
Antioch Peverell 4d2cbe6596 fix: init_sync_head was not behaving as expected (#1624)
Fix #1612
2018-10-01 23:03:06 +08:00
Antioch Peverell 56f84cc2f4 pass batch to process_block and process_block_header (#1594)
* use a batch consistently when processing blocks and headers

* rustfmt
2018-09-27 09:35:25 +01:00
Quentin Le Sceller 62fd8f2124 Implement Basic Auth for API and Owner API (#1566)
* Add api_secret

* Add to base64 method

* Add basic auth in API

* Add Basic Auth to owner API

* Add flag to enable disable basic auth

* Add .api_secret file
2018-09-26 22:38:44 +02:00
Gary Yu 8d62247c99 Small cleanup of rand usage (#1576)
Clean 3 build warning of deprecated item 'rand::Rand::rand'
2018-09-24 15:15:55 -07:00
Gary Yu 32486e2f63 Always enable header sync on initial state transition from NoSync / Initial (#1582) 2018-09-24 15:09:57 -07:00
Antioch Peverell e72d8b58e4 [WIP] txpool (tx validation) using block sums for full validation (#1567)
tx validation (txpool) rework/simplify
2018-09-24 09:24:10 +01:00
Gary Yu 3adddfba76 Refactor: run_sync() ballooning, extract 3 utility functions (#1562) 2018-09-20 10:00:09 -07:00
Gary Yu e1c8dc5a3a fix: reset sync_head to header_head on initial transition to HeaderSync (#1531)
* fix for bug #1524, reset sync_head to header_head on initial transition to HeaderSync.
* using @antiochp fix solution in #1539
2018-09-20 07:29:24 +08:00
Gary Yu 2ca6ecc163 fix: early detection of peer connection lost when downloading txhashset (#1546) 2018-09-20 07:08:02 +08:00
Ignotus Peverell 9cc7aed713 Dual proof of work, first step (#1534)
* Move Proof and Difficulty types to pow module. Difficulty scaling calculation.
* Diffculty scaling and size shift for 2nd PoW
* Backport e7eb26ee
* Test compilation fixes
* Scaling only need to go one way, as @tromp pointed out
2018-09-18 15:12:57 -07:00
Gary Yu d3a5ee130f Refactoring and cleanup on seed (#1513) 2018-09-14 16:09:25 -07:00
Gary Yu 7a689d468c Remove 'WebStatic' seed_type since it was obsoleted a while ago (#1512) 2018-09-11 20:28:27 -07:00
Gary Yu 48857b7e16 Fix infinite loop in case of some peer connect errors (#1494) 2018-09-10 15:23:49 -07:00
Gary Yu e760ea2ce2 a little bit speed-up for full node syncing (#1463)
* simplified: track the head and orphans instead of every single block hash that sync requests
* change 1st block timeout from 1s to 5s
2018-09-10 12:32:20 -07:00
Gary Yu 277687c17c Add 2 more dns seeds (#1496) 2018-09-10 12:11:03 -07:00
Antioch Peverell dbe53f63c8 disable chain compaction on archive nodes (#1498)
* disable chain compaction when in archive mode

* rustfmt
2018-09-10 15:59:42 +01:00
hashmap da87f5fef4 Don't wait 30 seconds in case of a quick restart (#1488)
Currently on startup we wait for 30 secs if we don't have at least 4 peers with more work. If a node was quickly restarted it is already fully synced so thereare no peers with more work at all. This pr assumes that if we've done some work already and we have enough peers but still 0 with more work we are fully synced and good to go.
2018-09-07 13:01:54 -07:00
Gary Yu ed73db671f fix for unstable travis-ci test on servers module (#1434)
* fix for unstable travis-ci test on servers module

* skip the PeerWithSelf connection request on sending side

* logs and comments change; and remove an unnecessary unwrap() in test
2018-09-04 10:52:11 +02:00
Quentin Le Sceller 45fffd8534 Missing logic in preferred peers (#1471) 2018-09-03 16:27:40 +02:00
hashmap fa00a0ebe6 Implement exponential backoff for peer monitoring at startup (#1459)
* Implement exponential backoff for peer monitoring at startup

Fixes #1413
2018-09-01 23:25:47 +02:00
Gary Yu d719493483 HeaderSync optimization (#1372) (#1400)
* improve: HeaderSync optimization (#1372)
* remove get_locator() optimization, which should be an independent pr for security review
* refactoring: move 'headers_streaming_body()' from Message to Protocol
* move 2 headers utils functions out of Protocol, and remove 'pub'
* support reading variable size of BlockHeader, from Cuckoo30 to Cuckoo36
* fix: use global::min_sizeshift() instead of hardcoded 30, because Cuckoo10 will be used for AutomatedTesting chain
* fix: should use global::proofsize() instead of hardcoded 42 when calculate serialized_size_of_header
* replace another 42 with global::proofsize()
2018-08-30 19:50:55 -04:00
Antioch Peverell 939d391e0e cache rangeproof and kernel signature verification results (#1419)
* wip - cache rangeproof verification

* rustfmt

* rustfmt

* rename to ok_verifier

* rustfmt

* wire in caching verifier in more places
skip rangeproof and kernel signature verification during deserialization

* rustfmt

* actually cache verification results via ok verifier

* rustfmt

* cleanup

* pass a cache around, not a verifier

* rustfmt

* cleanup trait shenanigans

* rustfmt

* core tests passing

* rustfmt

* rustfmt

* tests passing

* rustfmt

* verifier cache now takes vecs of data in and out

* rustfmt

* logging + rustfmt

* add docs and comments
2018-08-30 15:44:34 +01:00
Yeastplume 1ded3f3972 Config + Default directories (#1433)
* config file can now be generated by executable

* rustfmt

* remove now-unnecessary config defaults test

* set up paths and config file creation in user's home directory

* rustfmt

* remove default grin.toml

* add grin configuration command to spit out config file

* Split configuration into wallet and server

* rustfmt

* Restore logging to wallet configurations

* rustfmt
2018-08-30 10:10:40 +01:00
Antioch Peverell c334c557aa Simplify tx.validate() and transaction::aggregate() (#1436)
* simplify tx validation and aggregation
we *only* need to account for reward when building a block from txs

* rustfmt

* cleanup and tests passing

* rustfmt

* better comments in with_reward()

* fix wallet tests
2018-08-28 20:00:25 +01:00
Gary Yu a58558b38e improve: get_locator optimization (#1372) (#1411)
* refactoring get_locator(), to ensure the gap between neighbours >= 2^n
* security enhancement for get_locator
* in case of sync head and the header head are different, clear history locators
2018-08-27 17:42:57 -04:00
Antioch Peverell a6bd81fdb3 Cleanup generics (where not needed) (#1429)
BlockChain does not need to be generic
2018-08-27 17:22:48 -04:00
Quentin Le Sceller 8cd1b23f7a Add a peers_preferred option in Grin (#1401)
* Preferred peers option
* Move P2P configuration into proper config part
* Fix tests
2018-08-23 15:16:04 -04:00
hashmap 7dd2888b71 Get peers as quick as possible (#1398)
* Get peers as quick as possible. Currently we get 2 peers and then wait 20 seconds before trying to get more
* Second pass
* Fix peer_count
2018-08-22 21:31:06 -04:00
Ignotus Peverell 9554f93e1d Run header sync only when stalled or complete (#1393)
Header sync used to run when complete (right count of blocks
received) or every 10 sec regardless. Sometimes 10 secs is not
enough, especially when a big batch of blocks bodies is arriving
at the same time.

This tweaks the time-based heuristic just a bit to only ask for
headers when stalled, meaning that the header head has not
*progressed* for 10 seconds.

* Fix bad header sync timeout logic

* Improve stability of fast sync test
Adds regular pings so that the 2 test servers know each others'
heights (necessary to find most work peer). Stop mining on the
first server once the 2nd kicks in. Avoids chasing a fast moving
target on Cuckoo10.
2018-08-22 19:51:25 -04:00
Antioch Peverell 82a467ac3c Fix txpool race condition (#1385)
* fix txpool race condition

* rustfmt

* fixup pool tests

* rustfmt

* rebase against master, pool tests passing
2018-08-20 14:48:05 +01:00
Ignotus Peverell e9c987c075 Consolidate and cleanup tx aggregation (#1332)
* Include commitments non-duplicate checks in aggregate
* Remove said check from the pool
* Block building now uses tx aggregation to reduce duplication
2018-08-12 19:08:08 -07:00
Gary Yu 63a4d95df1 fix: in case txhashset validation fail, 'TxHashsetDownload' should retry (#1293) (#1322)
* fix: in case txhashset validation fail, 'TxHashsetDownload' should retry (#1293)

And add 2 more fixes:
1. handle sending request failure of 'send_txhashset_request';
2. use a 10-mins timeout to handle noresponse failure, in case a peer ignore our txhashset request, or in case a peer fail to send a file for some reason.
2018-08-08 17:14:54 +02:00
e-max 25e3d9e7d3 Remove Iron dependency and update hyper to version 0.12 (#1241)
* Remove Iron dependecy and update hyper to version 0.12 #876

* REMOVE ME

* Revert "REMOVE ME"

This reverts commit e9a976eee98a2d5a4dfae5d9e1e4f5ed640c05d3.

* Rebase and start updating libwallet

Libwallet doesn't compile yet.

* Wallet compiles

* Grin compiles

* No compilation errors in tests

* All tests pass

* Reeturn future from handler

* Refactoring

* Fix lifetime issue one more time

I have to force push to rollback all the work done in last 2 days

* Fix wallet send issue

* Clean up
2018-08-01 10:44:07 +01:00
Gary Yu 76f4915bf9 improve: time crate flagged as deprecated, switch to actively maintained chrono crate (#1297)
* improve: time crate flagged as deprecated, switch to actively maintained chrono crate

* improve: complete the switching from deprecated time crate to chrono crate, for all the tests part

* improve: complete switching to chrono crate, for the left tests in 'chain' and 'wallet'
2018-07-30 09:33:28 +01:00
Gary Yu 3b6b85ec57 Fix fast sync mode failure when 'sync_state' was switched to 'HeaderSync' (#1286) 2018-07-23 10:37:35 -07:00