22c521eec8
* Util to zip and unzip directories
* First pass at sumtree request/response. Add message types, implement the exchange in the protocol, zip up the sumtree directory and stream the file over, with necessary adapter hooks.
* Implement the sumtree archive receive logicGets the sumtree archive data stream from the network and write it to a file. Unzip the file, place it at the right spot and reconstruct the sumtree data structure, rewinding where to the right spot.
* Sumtree hash structure validation
* Simplify sumtree backend buffering logic. The backend for a sumtree has to implement some in-memory buffering logic to provide a commit/rollback interface. The backend itself is an aggregate of 3 underlying storages (an append only file, a remove log and a skip list). The buffering was previously implemented both by the backend and some of the underlying storages. Now pushing back all buffering logic to the storages to keep the backend simpler.
* Add kernel append only store file to sumtrees. The chain sumtrees structure now also saves all kernels to a dedicated file. As that storage is implemented by the append only file wrapper, it's also rewind-aware.
* Full state validation. Checks that:
- MMRs are sane (hash and sum each node)
- Tree roots match the corresponding header
- Kernel signatures are valid
- Sum of all kernel excesses equals the sum of UTXO commitments
minus the supply
* Fast sync handoff to body sync. Once the fast-sync state is fully setup, get bacj in body sync
mode to get the full bodies of the last blocks we're missing.
* First fully working fast sync
* Facility in p2p conn to deal with attachments (raw binary after message).
* Re-introduced sumtree send and receive message handling using the above.
* Fixed test and finished updating all required db state after sumtree validation.
* Massaged a little bit the pipeline orphan check to still work after the new sumtrees have been setup.
* Various cleanup. Consolidated fast sync and full sync into a single function as they're very similar. Proper conditions to trigger a sumtree request and some checks on receiving it.
228 lines
6.3 KiB
Rust
228 lines
6.3 KiB
Rust
// Copyright 2016 The Grin Developers
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
//! Grin server implementation, glues the different parts of the system (mostly
|
|
//! the peer-to-peer server, the blockchain and the transaction pool) and acts
|
|
//! as a facade.
|
|
|
|
use std::net::SocketAddr;
|
|
use std::sync::{Arc, RwLock};
|
|
use std::sync::atomic::{AtomicBool, Ordering};
|
|
use std::thread;
|
|
use std::time;
|
|
|
|
use adapters::*;
|
|
use api;
|
|
use chain;
|
|
use core::{global, genesis};
|
|
use miner;
|
|
use p2p;
|
|
use pool;
|
|
use seed;
|
|
use sync;
|
|
use types::*;
|
|
use pow;
|
|
use util::LOGGER;
|
|
|
|
/// Grin server holding internal structures.
|
|
pub struct Server {
|
|
/// server config
|
|
pub config: ServerConfig,
|
|
/// handle to our network server
|
|
p2p: Arc<p2p::Server>,
|
|
/// data store access
|
|
chain: Arc<chain::Chain>,
|
|
/// in-memory transaction pool
|
|
tx_pool: Arc<RwLock<pool::TransactionPool<PoolToChainAdapter>>>,
|
|
currently_syncing: Arc<AtomicBool>,
|
|
}
|
|
|
|
impl Server {
|
|
/// Instantiates and starts a new server.
|
|
pub fn start(config: ServerConfig) -> Result<Server, Error> {
|
|
let mut mining_config = config.mining_config.clone();
|
|
let serv = Server::new(config)?;
|
|
if mining_config.as_mut().unwrap().enable_mining {
|
|
serv.start_miner(mining_config.unwrap());
|
|
}
|
|
|
|
loop {
|
|
thread::sleep(time::Duration::from_secs(10));
|
|
}
|
|
}
|
|
|
|
/// Instantiates a new server associated with the provided future reactor.
|
|
pub fn new(mut config: ServerConfig) -> Result<Server, Error> {
|
|
let pool_adapter = Arc::new(PoolToChainAdapter::new());
|
|
let pool_net_adapter = Arc::new(PoolToNetAdapter::new());
|
|
let tx_pool = Arc::new(RwLock::new(pool::TransactionPool::new(
|
|
config.pool_config.clone(),
|
|
pool_adapter.clone(),
|
|
pool_net_adapter.clone(),
|
|
)));
|
|
|
|
let chain_adapter = Arc::new(ChainToPoolAndNetAdapter::new(tx_pool.clone()));
|
|
|
|
let genesis = match config.chain_type {
|
|
global::ChainTypes::Testnet1 => genesis::genesis_testnet1(),
|
|
//global::ChainTypes::Testnet2 => genesis::genesis_testnet2(),
|
|
_ => pow::mine_genesis_block(config.mining_config.clone())?,
|
|
};
|
|
info!(
|
|
LOGGER,
|
|
"Starting server, genesis block: {}",
|
|
genesis.hash(),
|
|
);
|
|
|
|
let shared_chain = Arc::new(chain::Chain::init(
|
|
config.db_root.clone(),
|
|
chain_adapter.clone(),
|
|
genesis.clone(),
|
|
pow::verify_size,
|
|
)?);
|
|
|
|
pool_adapter.set_chain(shared_chain.clone());
|
|
|
|
let currently_syncing = Arc::new(AtomicBool::new(true));
|
|
|
|
let net_adapter = Arc::new(NetToChainAdapter::new(
|
|
currently_syncing.clone(),
|
|
shared_chain.clone(),
|
|
tx_pool.clone(),
|
|
));
|
|
|
|
let p2p_config = config.p2p_config.clone();
|
|
let p2p_server = Arc::new(p2p::Server::new(
|
|
config.db_root.clone(),
|
|
config.capabilities,
|
|
p2p_config,
|
|
net_adapter.clone(),
|
|
genesis.hash(),
|
|
)?);
|
|
chain_adapter.init(p2p_server.peers.clone());
|
|
pool_net_adapter.init(p2p_server.peers.clone());
|
|
net_adapter.init(p2p_server.peers.clone());
|
|
|
|
if config.seeding_type.clone() != Seeding::Programmatic {
|
|
|
|
let seeder = match config.seeding_type.clone() {
|
|
Seeding::None => {
|
|
warn!(LOGGER, "No seed configured, will stay solo until connected to");
|
|
seed::predefined_seeds(vec![])
|
|
}
|
|
Seeding::List => {
|
|
seed::predefined_seeds(config.seeds.as_mut().unwrap().clone())
|
|
}
|
|
Seeding::WebStatic => {
|
|
seed::web_seeds()
|
|
}
|
|
_ => unreachable!(),
|
|
};
|
|
seed::connect_and_monitor(p2p_server.clone(), config.capabilities, seeder);
|
|
}
|
|
|
|
let skip_sync_wait = match config.skip_sync_wait {
|
|
None => false,
|
|
Some(b) => b,
|
|
};
|
|
|
|
sync::run_sync(
|
|
currently_syncing.clone(),
|
|
p2p_server.peers.clone(),
|
|
shared_chain.clone(),
|
|
skip_sync_wait,
|
|
!config.archive_mode,
|
|
);
|
|
|
|
let p2p_inner = p2p_server.clone();
|
|
let _ = thread::Builder::new().name("p2p-server".to_string()).spawn(move || {
|
|
p2p_inner.listen()
|
|
});
|
|
|
|
info!(LOGGER, "Starting rest apis at: {}", &config.api_http_addr);
|
|
|
|
api::start_rest_apis(
|
|
config.api_http_addr.clone(),
|
|
shared_chain.clone(),
|
|
tx_pool.clone(),
|
|
p2p_server.peers.clone(),
|
|
);
|
|
|
|
warn!(LOGGER, "Grin server started.");
|
|
Ok(Server {
|
|
config: config,
|
|
p2p: p2p_server,
|
|
chain: shared_chain,
|
|
tx_pool: tx_pool,
|
|
currently_syncing: currently_syncing,
|
|
})
|
|
}
|
|
|
|
/// Asks the server to connect to a peer at the provided network address.
|
|
pub fn connect_peer(&self, addr: SocketAddr) -> Result<(), Error> {
|
|
self.p2p.connect(&addr)?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Number of peers
|
|
pub fn peer_count(&self) -> u32 {
|
|
self.p2p.peers.peer_count()
|
|
}
|
|
|
|
/// Start mining for blocks on a separate thread. Uses toy miner by default,
|
|
/// mostly for testing, but can also load a plugin from cuckoo-miner
|
|
pub fn start_miner(&self, config: pow::types::MinerConfig) {
|
|
let cuckoo_size = global::sizeshift();
|
|
let proof_size = global::proofsize();
|
|
let currently_syncing = self.currently_syncing.clone();
|
|
|
|
let mut miner = miner::Miner::new(config.clone(), self.chain.clone(), self.tx_pool.clone());
|
|
miner.set_debug_output_id(format!("Port {}", self.config.p2p_config.port));
|
|
let _ = thread::Builder::new()
|
|
.name("miner".to_string())
|
|
.spawn(move || {
|
|
// TODO push this down in the run loop so miner gets paused anytime we
|
|
// decide to sync again
|
|
let secs_5 = time::Duration::from_secs(5);
|
|
while currently_syncing.load(Ordering::Relaxed) {
|
|
thread::sleep(secs_5);
|
|
}
|
|
miner.run_loop(config.clone(), cuckoo_size as u32, proof_size);
|
|
});
|
|
}
|
|
|
|
/// The chain head
|
|
pub fn head(&self) -> chain::Tip {
|
|
self.chain.head().unwrap()
|
|
}
|
|
|
|
/// The head of the block header chain
|
|
pub fn header_head(&self) -> chain::Tip {
|
|
self.chain.get_header_head().unwrap()
|
|
}
|
|
|
|
/// Returns a set of stats about this server. This and the ServerStats
|
|
/// structure
|
|
/// can be updated over time to include any information needed by tests or
|
|
/// other
|
|
/// consumers
|
|
|
|
pub fn get_server_stats(&self) -> Result<ServerStats, Error> {
|
|
Ok(ServerStats {
|
|
peer_count: self.peer_count(),
|
|
head: self.head(),
|
|
})
|
|
}
|
|
}
|