Commit Graph

77 Commits

Author SHA1 Message Date
Antioch Peverell 616dad43fd Cleanup redundant AsFixedBytes and FixedLength traits (#3131)
* no need for AsFixedBytes we can just use AsRef<[u8]>

* cleanup FixedLength trait

* revert this change for now

* fix store tests

* cleanup and fix tests after rebase

* fix tests

* update based on PR review
less hard-coded values now

* cleanup
2020-01-29 13:41:50 +00:00
Quentin Le Sceller 6e5afe496b Update License to 2020 (#3196) 2020-01-20 11:40:58 +00:00
Antioch Peverell bde19777f0 drop msg if sync_sender buffer is full (do not close peer connection) (#3164) 2019-12-10 20:06:17 +00:00
Antioch Peverell 8b8f0a0abd pass stopped into consume so we can halt txhashset mid download (#3157) 2019-12-05 12:14:41 +00:00
hashmap cbc17ff5f7 Set longer timeout for msg body read/write operations (#3145)
* Set longer timeout for msg body read operations

* Introduce a channel timeout
2019-12-02 12:00:30 +00:00
hashmap 3f528a8e66 Captute "type" byte for unknow message type (#3146)
Found it useful for debugging
2019-11-29 13:04:55 +01:00
mmgen 928097ad62 Increase IO_TIMEOUT to allow nodes on high-latency connections to sync (#3109)
Commit d3dbafa80b "Use blocking IO in P2P to reduce CPU load" (merged
into v2.1.0) introduced the constant IO_TIMEOUT, setting it to 1 second.

On nodes with high-latency connections, this short timeout causes the
txhashset archive download during step 2 of the IBD process to
invariably fail before it completes.  Since there's no mechanism for
resuming a failed download, this means the node gets stuck at this stage
and never syncs.

Increasing IO_TIMEOUT to 10 seconds solves the issue on my node; others
might suggest a more optimal value for the constant.
2019-11-13 21:12:38 +01:00
Antioch Peverell a3f3fc25dc use sender thread consistently to send msgs to a peer (#3067) 2019-10-07 16:22:05 +01:00
Quentin Le Sceller 6be6391225 Change 2018 to 2019 in copyright files (#3072) 2019-10-02 09:40:20 +01:00
Gary Yu 4faac470d4 fix: the cpuload 100% in case the tcpstream WouldBlock (#3029)
* fix: the cpuload 100% in case the tcpstream WouldBlock

* extending the sleep on WouldBlock to all read/write

* rustfmt
2019-09-10 07:55:57 +08:00
antiochp c6cea4744c Merge branch 'master' into milestone/2.x.x 2019-07-24 15:10:26 +01:00
hashmap d3dbafa80b [2.x.x] Use blocking IO in P2P to reduce CPU load (#2855)
* Use blocking IO in P2P to reduce CPU load
2019-07-12 23:17:38 +02:00
Antioch Peverell d284d8f6de [2.x.x] Writeable protocol version aware (#2856)
* introduce protocol version to deserialize and read

* thread protocol version through our reader

* cleanup

* cleanup

* streaming_reader cleanup

* Pass protocol version into BinWriter to allow for version specific serialization rules.

* rustfmt

* read and write now protocol version specific
2019-07-06 15:51:03 +01:00
hashmap dbd2535f42 Increase peer's send buffer (#2931) (#2934)
When we send a txhashet archive a peer's thread is busy with sending it
and can't send other messages, eg pings. If the network connection is
slow buffer capacity 10 may be not enough,  hence the peer's drop.

Safer attempt to address #2929 in 2.0.0
2019-06-30 22:26:17 +02:00
Antioch Peverell 5aaf2d058d [2.x.x] deserialization now protocol version aware (#2824)
* introduce protocol version to deserialize and read

* thread protocol version through our reader

* example protocol version access in kernel read

* fix our StreamingReader impl (WouldBlock woes)

* debug log progress of txhashset download
2019-06-27 17:19:41 +01:00
hashmap 2863ed67fd Speedup shutdown (#2862)
I made an suboptimal (aka stupid) decision to stop and wait for peers
one by one which makes shutdown very slow - O(n). This PR decouples sending
stop signal from waiting a thread to exit. On top of it in Peers we
first send stop signal to all peers and only after that start waiting
for them to exit. It gives us a constant time of shutdown in most of the
cases.
2019-05-30 08:03:12 +08:00
hashmap 9ab23f6eef Implement graceful shutdown (#2812)
* Remove stop status mutex
* remove some deadlocks
* Rewrite stop channel handling
* fix deadlock in peers object
* add missing test fixes
2019-05-15 08:51:35 -07:00
Antioch Peverell ff1c55193f Wrap MsgHeader in MsgHeaderWrapper for Known/Unknown msg type support (#2791)
* wrap MsgHeader in MsgHeaderWrapper for Known/Unknown msg type support.
* cleanup based on feedback
2019-05-14 09:07:51 -07:00
Antioch Peverell 4ef4212f1f Simplify peer connection handling (#2801)
* connection no longer wrapped in an Option in peer

* introduce peer.send()

* remove some Arc indirection

* self.send() cleanup

* extract Peer:new() from connect and accept

* fixup

* cleanup
2019-05-03 15:56:25 +01:00
Antioch Peverell 6c54c90101 remove the error_channel and simplify how we close peer connections (#2796) 2019-05-03 15:35:43 +01:00
hashmap 304ae444ab Fix peer dropping (#2780)
It turns out that we drop connection if we fail to process a message
because of chain/store/internal error, eg we have a header already, so
we refuse it and drop the peer.
This pr doesn't forward this error to the peer error channel so the
connection will not be dropped.
2019-04-26 20:22:07 +02:00
David Burkett 0e6a249126 Increasing thread sleeps for socket listener thread and per-connection threads. (#2773) 2019-04-23 08:29:24 +02:00
hashmap fe9fa51f32 Reduce usage of unwrap in p2p crate (#2627)
Also change store crate a bit
2019-02-25 19:48:54 +01:00
Gary Yu 0b73e87f8c fix: connection close should stay outside the poll loop (#2308) 2019-01-07 14:44:07 +08:00
Gary Yu 6165c0873c fix: shutdown the tcpstream when we initiate to close a peer connection (#2288) 2019-01-04 09:08:44 -08:00
hashmap aedac483f5 Convert to Rust 2018 edition (#2084)
* Convert to Rust 2018 edition

* Update gitignore
2018-12-08 00:59:40 +01:00
antiochp f7608f5b9c rustfmt 2018-11-16 14:34:05 +00:00
antiochp 4b48eebdfe refactor some p2p to use Read/Write over TcpStream 2018-11-16 14:33:35 +00:00
Antioch Peverell f0fa410273 Streaming headers (#1989)
* headers msg is now "streamed" off the tcp stream

* rustfmt

* cleanup

* move StreamingReader into ser.rs
extract read_exact out into util crate

* rustfmt
2018-11-16 11:00:39 +00:00
Antioch Peverell cb652e7ef1 do not treat txhashset.zip download as abusive behavior (#1973)
* do not treat txhashset.zip download as abusive behavior

* count times, not bytes so we exclude quiet increments

* use inc_quiet when tracking sent bytes via an attachment

* add comment

* fixup "quiet" counter entries

* rustfmt
2018-11-13 21:34:45 +00:00
Antioch Peverell a3c1c6d603 Do not use mem::size_of() when calculating serialized size of data (#1967)
* use FixedLength to define serialized size in bytes of various structs
replace usages of mem::size_of() with ::LEN so we correctly calculate serialized sizes in bytes

* rustfmt
2018-11-13 09:30:02 +00:00
Gary Yu 7ff1ee5fde Peer send receive bytes statistics missed response messages and attachment (#1953)
* fix: peer send receive bytes statistics missed response message and attachment

* rustfmt
2018-11-11 07:30:57 +08:00
Ignotus Peverell 8b546632fe Peer rate limiting (#1933)
* Rate counter in peer conn to monitor traffic 
* Ban peers that trigger is_abusive
2018-11-06 17:51:22 -08:00
eupn 711fad6c24 Remove TODO for counting of bytes that was already done (#1838) 2018-10-25 15:57:53 +02:00
eupn 1195071f5b Replace logging backend to log4rs and add log rotation (#1789)
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
2018-10-21 13:30:56 -07:00
Gary Yu 0d06561a91 replace stdlib RwLock and Mutex with parking_lot (#1793)
* replace all stdlib RwLock with parking_lot RwLock

* replace stdlib Mutex with parking_lot Mutex

* rustfmt
2018-10-20 08:13:07 +08:00
yeastplume f94ede9af3 merge T4 into master 2018-10-18 11:23:04 +01:00
eupn b22fb55245 feat: add peers used bandwidth calculation and display in TUI (#1770)
* Add peers used bandwidth calculation and display in TUI
* Fix formatting
* Change Mutex to RwLock from peer's used bandwidth statistics in Tracker
* Make used bandwidth column in TUI peers list sort by sum of bytes
2018-10-17 10:01:42 -07:00
hashmap 85433c659d Introduce a constant for peer send channel capacity (#1761)
We implicitly use it also in body_sync, so it's hard to keep it in sync.
2018-10-16 16:31:00 -07:00
Gary Yu 9a716aea72 feature: txhashset downloading progress display on tui (#1729) (#1730)
(cherry picked from commit 5c0eb11a7d)
2018-10-13 10:12:13 +08:00
Gary Yu 5c0eb11a7d feature: txhashset downloading progress display on tui (#1729) 2018-10-12 15:53:50 -07:00
Gary Yu 8a39f1fc75 Use 10us sleep instead of 1ms sleep (#1453) 2018-08-31 13:20:59 -04:00
Gary Yu d719493483 HeaderSync optimization (#1372) (#1400)
* improve: HeaderSync optimization (#1372)
* remove get_locator() optimization, which should be an independent pr for security review
* refactoring: move 'headers_streaming_body()' from Message to Protocol
* move 2 headers utils functions out of Protocol, and remove 'pub'
* support reading variable size of BlockHeader, from Cuckoo30 to Cuckoo36
* fix: use global::min_sizeshift() instead of hardcoded 30, because Cuckoo10 will be used for AutomatedTesting chain
* fix: should use global::proofsize() instead of hardcoded 42 when calculate serialized_size_of_header
* replace another 42 with global::proofsize()
2018-08-30 19:50:55 -04:00
Gary Yu 471e80e69e fix for peer connect fail on 'WouldBlock' Error (#1440)
* fix for fail of peer connect: 'WouldBlock' Error.

* reuse the blocking variation of read_exact instead of re-implementing the loop
2018-08-30 08:21:13 +01:00
Quentin Le Sceller a30ee88236 Cleanup imports (#1161)
Reorganize imports
2018-06-14 08:16:14 -04:00
Quentin Le Sceller 8f4dbfa540 Many typo fixes (#1158)
* Typo fix
* Parameterise
2018-06-13 17:03:34 +01:00
Ignotus Peverell 5ba0dbf38d Fix and cleanup of fast sync triggering logic (#916)
* Fix and cleanup of fast sync triggering logic
* New txhashset on fast sync has to be applied, not rolled back
* Do not block if peer send buffer is full, fixes #912
2018-03-30 06:02:40 +00:00
Ignotus Peverell c3cec63a95 Use a sync channel for peer sending. Fixes #904 2018-03-29 00:00:34 +01:00
Simon B a8fc82d8c1 Console debug fixes (#861)
* move some debug! to trace!

* more informative debugs

* standardising on always showing chain tips as "cumulative difficulty @ height [hash]"

* make 2 debug outputs into a single

* "no peers" as warning (not info) to let it stand out more clearly

* move fn param (used only in this one debug line)

* clarify difficulty "units"
2018-03-25 12:41:12 -04:00
Yeastplume 1143d84238 Remove Sumtree References and disambiguate some naming (#747)
* start of renamathon

* api renaming

* Rename UTXO-Output to lessen ambiguity

* compile warning

* compile error

* readme fix

* remove file commit in error
2018-03-05 19:33:44 +00:00