Commit Graph

246 Commits

Author SHA1 Message Date
Yeastplume dcdf654bc9 [WIP] Core PMMR and API updates to support wallet restore (#950)
* update pmmr to get batch of elements by insertion position

* update pmmr to get batch of elements by insertion position

* add api + chain calls to get traversed outputs back out

* add api + chain calls to get traversed outputs back out

* first pass getting wallet restore to work again with updated utxo-walking api
2018-04-11 10:02:07 +01:00
Antioch Peverell 6ceed79c03 Use hashable_ord on short_ids (#932) 2018-04-05 15:48:30 +00:00
Riccardo Casatta 40212942da use std lib function in pmmr for all_ones() and most_significant_pos() (#924)
* use std lib function in pmmr for all_ones() and most_significant_pos()

* fix all_ones() edge case with 0 and add comparison tests with old functions
2018-04-04 13:25:03 +01:00
John Tromp c3939107a5 Avoid repeated truncation in difficulty adjust + minor refactoring (#915)
* avoid repeated truncation in difficulty adjust + minor refactoring
* reinstate needed low-diffculty exception on dampening
* leave consensus breaking change for later one line change; obsolete lower testnet1 max target
2018-03-30 18:21:06 +00:00
Simon B 3e3fe6cae6 Testnet2 small fixes (#899)
* add hint for gcc-5 for building CUDA plugin
* add N_BLOCks comment for >=6GB GPU cards
* minor logging cleanup
* mark to be kept + reformat as "@ height [hash]"
* show CuckooNN + better wording
2018-03-29 15:56:46 +00:00
Yeastplume 19bedff664 Difficulty scenario fix (#892)
* fix difficulty scenario to match testnet2

* fix difficulty scenario to match testnet2
2018-03-27 22:11:52 +01:00
Yeastplume 6a6e600e48 tests for difficulty adjustment scenarios (#887)
* tests for difficulty adjustment scenarios

* ignore test
2018-03-27 19:15:42 +01:00
Ignotus Peverell 16beeca8c9 Correct chain initialization at genesis 2018-03-26 17:38:27 +01:00
Ignotus Peverell 74674c2307 Hopefully last genesis update... 2018-03-26 17:36:54 +01:00
Ignotus Peverell aa0e488e07 Genesis update with correct nonce... 2018-03-26 17:32:10 +01:00
Ignotus Peverell e0594d81fc Genesis update with correct month 2018-03-26 17:31:13 +01:00
Ignotus Peverell f3fc26c5c9 Genesis update with 172 difficulty 2018-03-26 16:12:18 +01:00
Ignotus Peverell 327293d28e Genesis, version changes and doc for testnet2 (#865)
* Bump up crates versions

* Finally add a Cargo.lock to avoid dependency breakages

* Build doc update for testnet2

* Fix test framework not really using its mining config

* Testnet2 genesis, best so far at 128 difficulty (a nice number)

* Minor build doc update
2018-03-26 10:48:46 +01:00
John Tromp a327427178 0-based positions, bag peaks right-to-left with mmr size as index (#863)
* 0-based positions, bag peaks right-to-left with mmr size as index
* fix remaining instances of hash_with_index
* fix pmmr test
2018-03-26 02:26:11 +00:00
Simon B 9285de2c24 Here is a case where ~ is needed, to allow cargo update to only touch the patch level version (third number). Anything else than ~ means that both first numbers can be raised, so "2.1" updaates to "2.2" (#866) 2018-03-25 20:08:38 +00:00
Ignotus Peverell 2bf656646c Force slog version to 2.1 (2.2 broke some traits) 2018-03-25 19:54:41 +01:00
Simon B f9b87164e7 Idiomatic cargo toml (#862)
* Specify versions SemVer-style
 - slog and serde
* Comments for not update-able pre-1.0 crates:
 - urlencoded at "0.5" since 0.6+ lacks trait `plugin::Plugin<iron::Request<'_, '_>>`
* alpha-sort deps
* Specify versions SemVer-style
 - backtrace, bitflags, blake2-rfc, bodyparser, built, byteorder
 - chrono, clap, demonize, enum_primitive, env_logger
 - failure, failure_derive, futures
 - hyper, iron, itertools, lazy_static, libc
 - memmap, mount, net2, num_bigint, prettytable-rs
 - rand, regex, rocksdb, router, siphasher
 - term, time, tokio-core, tokio-retry, urlencoded, uuid, walkdir, zip
2018-03-25 17:44:27 +00:00
Alexey Miroshkin bb134758e7 Rename Hash::zero() to Hash::default() (#858)
It is more idiomatic, also `zero` is misleading in context of `+` operator.
2018-03-23 23:35:10 +00:00
Antioch Peverell 388beafb65 Standardize and cleanup SerializationMode (#856) 2018-03-23 23:34:50 +00:00
Antioch Peverell 7a8d6147f4 Split pmmr.get() into get_hash() and get_data() (#855) 2018-03-23 23:33:59 +00:00
Yeastplume 80887196a8 Restore wallet restore (#843)
* adding appropriate message to bulletproofs to allow for restore

* rustfmt

* should work, now test

* rustfmt

* fix to wallet restore, works now

* fix pool tests

* fix pool tests

* rustfmt
2018-03-23 10:13:57 +00:00
Alexey Miroshkin eb226beea8 Introduce fuzz tests for core (#844)
This PR introduces fuzz tests and address #592
2018-03-22 16:53:47 +00:00
Ignotus Peverell 9d366fe492 Fix difficulty calc for short chains 2018-03-22 16:38:42 +00:00
Ignotus Peverell 054114027f Index first when hashing nodes in MMR. Fixes #814 2018-03-22 04:58:25 +00:00
Ignotus Peverell 1300d7a745 Dummy change to testnet2 genesis 2018-03-22 04:55:34 +00:00
Ignotus Peverell ca8447f3bd Removed all switch commitment usages, including restore (#841)
* Removed all switch commitment usages, including restore
* Fixed pool tests
* Fix keychain tests
* Get rid of the switch key in keychain
2018-03-22 00:10:11 +00:00
Antioch Peverell 5a47a17101 remove redundant verify_proof call (#835)
* we verify rangeproofs in sum_commitments (in verify_kernels)
remove this redundant (and expensive) rangeproof verification

* rustfmt
2018-03-21 14:55:41 -04:00
Simon B 4fa9ccc4f7 Cargo.toml (#818)
* Make sure no grin crates get published by mistake: https://doc.rust-lang.org/cargo/reference/manifest.html#the-publish--field-optional
* add missing workspace = '..', [workspace docs for interested readers](https://doc.rust-lang.org/cargo/reference/manifest.html#the-workspace-section)
* dev- or dev_ works equally well, but dev- is what's typically used
* authors = Grin Developers <maillist>
* [dependencies] keep grin-* at the end + readability
2018-03-20 18:21:29 +00:00
Yeastplume 4886fa08b2 TUI Difficulty stats for past few blocks (#805)
* added mining subview, changed main menu selection

* collecting difficulty stats from miner

* add diff calc view, separate server stats

* rustfmt

* block difficulty data output

* rustfmt

* ensure diff data is always shown

* don't write to stderr when tui running
2018-03-19 19:23:58 +00:00
Roman Zeyde d4eb24d16a Fix a small comment typo (#810) 2018-03-19 16:55:17 +00:00
Ignotus Peverell 2fb97eb87f Minor increase to max tx input count 2018-03-16 22:26:36 +00:00
Alexey Miroshkin 1b3541245a Fix limits for Transaction (#800)
In prevPRr I set them too high, also typos in comments were fixed.
2018-03-16 22:23:10 +00:00
Alexey Miroshkin c96838d54d Improve bounding block timestamp (#793)
Fixes #783.
* 2^60 still fails on OSX, experimentally found safe max 2^55
* Handle negative values

I tried to use .abs(), unfortuantely fuzz test kills it with
`panicked at 'attempt to negate with overflow', /Users/travis/build/rust-lang/rust/src/libcore/num/mod.rs:1146:17`
2018-03-16 22:20:40 +00:00
Yeastplume b53b84b0c6 Move nonce/pow to end of block header, change post-pow hash (#794)
* move nonce/pow to end of block header, update call to cuckoo-miner
* change header hash to be pow only
* fix genesis block miner and tests
2018-03-16 19:04:31 +00:00
Alexey Miroshkin 1bad7188b7 Fix Readable impl for Transaction (#795)
Fuzz test demonstrated that the current implemenation panics on reading from the network when serialized tx. Contains for example huge number of input or outputs. It allows a mailicious sender
to kill the server.
2018-03-16 19:02:22 +00:00
Antioch Peverell cb71386097 hash with the pos for non-leaf nodes (#788)
* hash_with_index on non-leaf nodes
rework the Merkle proof to include the pos of each sibling in the path

* rustfmt

* cleanup

* cleanup

* use get_from_file in validate (children may have been "removed")

* rustfmt

* fixup store tests
2018-03-16 10:45:58 -04:00
Antioch Peverell 2552b34118 remove difficulty field from block_header (derive from total_difficulty) (#777)
* remove difficulty field from block_header (derive from total_difficulty)

* address feedback on PR and general cleanup

* rustfmt

* rework pow/difficulty validation in pipe::validate_header
now that we only have total_difficulty available

* cleanup various todos

* rustfmt

* rework DifficultyIterator to track header and prev_header state

* rustfmt caught some garbage syntax

* cleanup
2018-03-15 15:16:34 -04:00
Ignotus Peverell e4871065b0 Bounding block timestamp to avoid surprises. Fixes #783 2018-03-15 18:37:17 +00:00
Yeastplume 244fc8d32e PMMR data compaction (#784)
* fix and re-introduce data file compaction

* rustfmt

* remove commented line

* test cleanup

* remove println

* fix to core test
2018-03-15 17:12:30 +00:00
Antioch Peverell 00bfc4ec38 add dump_stats so we can see pmmr sizes after compaction (#785)
* add dump_stats so we can see pmmr sizes after compaction

* rustfmt

* fix tests with dump_stats()
2018-03-15 12:53:40 -04:00
Antioch Peverell 6a1a45e21f store "total kernel offset" on block header (#769)
* move from storing per block kernel_offset in the block header
to storing the total accumulated kernel offset on each block header

* rustfmt

* cleanup
2018-03-14 11:22:09 -04:00
Antioch Peverell 65633c7611 check_compact retains leaves and roots until parents are pruned (#753)
* wip

* failing test for being too eager when pruning a sibling

* commit

* rustfmt

* [WIP] modified get_shift and get_leaf_shift to account for leaving "pruned but not compacted" leaves in place
Note: this currently breaks check_compact as nothing else is aware of the modified behavior

* rustfmt

* commit

* rustfmt

* basic prune/compact/shift working

* rustfmt

* commit

* rustfmt

* next_pruned_idx working (I think)

* commit

* horizon test uncovered some subtle issues - wip

* rustfmt

* cleanup

* rustfmt

* commit

* cleanup

* cleanup

* commit

* rustfmt

* contains -> binary_search

* rustfmt

* no need for height==0 special case

* wip - works for single compact, 2nd one breaks the mmr hashes

* commit

* rustfmt

* fixed it (needs a lot of cleanup)
we were not traversing all the way up to the peak if we pruned an entire tree
so rm_log and prune list were inconsistent

* multiple compact steps are working
data file not being copmacted currently (still to investigate)

* cleanup store tests

* cleanup

* cleanup up debug

* rustfmt

* take kernel offsets into account when summing kernels and outputs for full txhashset validation
validate chain state pre and post compaction

* rustfmt

* fix wallet refresh (we need block height to be refreshed on non-coinbase outputs)
otherwise we cannot spend them...

* rustfmt
2018-03-13 14:22:34 -04:00
Ignotus Peverell 65c0a4b7b0 Fee burning no more. Fixes #767 2018-03-13 02:39:22 +00:00
Ignotus Peverell a9d1b76414 Compaction of pruned data for chain data structures (#746)
* Implementation of compaction for the chain. Single entry point on the chain triggers compaction of all MMRs as well as the cleanup of the positional index and full blocks.
* API endpoint, additional tests and more fixes for compaction
* Also prune PMMR metadata, minor bug fix
* PMMR store tests fix
2018-03-06 17:58:33 +00:00
Antioch Peverell 449fabf24f cannot use write_mode in rustfmt.toml in stable (get rid of the warnings) (#752) 2018-03-05 15:23:52 -05:00
Yeastplume 1143d84238 Remove Sumtree References and disambiguate some naming (#747)
* start of renamathon

* api renaming

* Rename UTXO-Output to lessen ambiguity

* compile warning

* compile error

* readme fix

* remove file commit in error
2018-03-05 19:33:44 +00:00
Yeastplume 90f844d382 Include index as part of pmmr element hash (#748)
* storing index as part of pmmr element hash

* factor out hash_with_index into generic trait impl

* tighten up trait bounds for PMMRable
2018-03-05 15:05:42 +00:00
Ignotus Peverell be8d9633e4 rustfmt all the things 2018-03-04 00:19:54 +00:00
Antioch Peverell cc12798d7a Merkle Proofs (#716)
* family_branch() to recursively call family() up the branch
todo
  - we hit a peak, then we need to get to the root somehow
  - actually get the hashes to build the proof

* wip

* some additional testing around merkle tree branches

* track left/right branch for each sibling as we build the merkle path up

* MerkleProof and basic (incomplete) verify fn

* I think a MerkleProof verifies correctly now
need to test on test case with multiple peaks

* basic pmmr merkle proof working

* MerkleProof now serializable/deserializable

* coinbase maturity via merkle proof basically working

* ser/deser merkle proof into hex in api and wallet.dat

* cleanup

* wip - temporarily saving merkle proofs to the commit index

* assert merkle proof in store matches the rewound version
there are cases where it does not...

* commit

* commit

* can successfully rewind the output PMMR and generate a Merkle proof
need to fix the tests up now
and cleanup the code
and add docs for functions etc.

* core tests passing

* fixup chain tests using merkle proofs

* pool tests working with merkle proofs

* api tests working with merkle proof

* fix the broken comapct block hashing behavior
made nonce for short_ids explicit to help with this

* cleanup and comment as necessary

* cleanup variety of TODOs
2018-03-02 15:47:27 -05:00
Antioch Peverell a20ffc700b fix the broken comapct block hashing behavior (#738)
made nonce for short_ids explicit to help with this
2018-03-01 13:25:33 -05:00