check_compact retains leaves and roots until parents are pruned (#753)

* wip

* failing test for being too eager when pruning a sibling

* commit

* rustfmt

* [WIP] modified get_shift and get_leaf_shift to account for leaving "pruned but not compacted" leaves in place
Note: this currently breaks check_compact as nothing else is aware of the modified behavior

* rustfmt

* commit

* rustfmt

* basic prune/compact/shift working

* rustfmt

* commit

* rustfmt

* next_pruned_idx working (I think)

* commit

* horizon test uncovered some subtle issues - wip

* rustfmt

* cleanup

* rustfmt

* commit

* cleanup

* cleanup

* commit

* rustfmt

* contains -> binary_search

* rustfmt

* no need for height==0 special case

* wip - works for single compact, 2nd one breaks the mmr hashes

* commit

* rustfmt

* fixed it (needs a lot of cleanup)
we were not traversing all the way up to the peak if we pruned an entire tree
so rm_log and prune list were inconsistent

* multiple compact steps are working
data file not being copmacted currently (still to investigate)

* cleanup store tests

* cleanup

* cleanup up debug

* rustfmt

* take kernel offsets into account when summing kernels and outputs for full txhashset validation
validate chain state pre and post compaction

* rustfmt

* fix wallet refresh (we need block height to be refreshed on non-coinbase outputs)
otherwise we cannot spend them...

* rustfmt

keychain/src/keychain.rs

@@ -245,7 +245,7 @@ impl Keychain {
 		&self,
 		amount: u64,
 		key_id: &Identifier,
-		commit: Commitment,
+		_commit: Commitment,
 		extra_data: Option<Vec<u8>>,
 		msg: ProofMessage,
 	) -> Result<RangeProof, Error> {