The Rise of Rust and Blockchain
The Rust programming language is redefining the blockchain space very rapidly. It is my goal to learn this language through an evaluation of blockchain protocols, peer-to-peer networking layers and to understand how the concepts of ownership and modules will drive forward the industry as a whole.
This is a very, very good programming language for safety and performance.
At a high level I understand there are crates and cargo, analogous to npm and node for running packages of software written in Rust. The blockchain industry is increasingly demanding development in the Rust programming language. Here is my overview of Substrate and Libra and how to code is similar or dissimilar in a particular fashion: In learning more about Rust and WASM, WASI and Blockchains I have come to understand that the systems level programming or high performance blockchain will be running in the browser. The Rust WASM Blockchain combination enables powerful applications in the browser. Applications that today I don’t think one would want to have in the browser due to performance; but it will be a combination of having the React and Javascript interface with a WASM based blob that is binary from a program written in Rust. In learning more about Rust and specifically the cargo build system it becomes clear that the development experience will be to leverage the set of libraries across these different projects to effectively create a new ultra-performant enterprise platform that is written in rust and can run in the browser. This could be a number of enterprise applications that are actually using the browser to do the high performant computation and taking the most extensive components of the enterprise application and rewriting them so they can be a WASM blob and run in the browser on desktop and mobile
For an enterprise platform this becomes very important in that it is not just for system of record but for computational tasks such a a blockchain network or ledger across multiple network participants in the browser or for more computational advanced processes that need extra guarantees around security and scalability.
The Rust Language
Rust is a systems level programming language. It is intended for fast, safe and powerful applications. It is verbose in its compiler and strict. Debugging is enabled by default, before the code even complies.
Rust tests are staightforward.
Add a feature and add a test in the same contract. It is control and safety for a full stack application.
Ownership
Repos
I am looking into these repos that have been written in Rust and comparing what I see across them:
Substrate Parity Ethereum Client Lighthouse Ethereum 2.0 Client Libp2p Oasis Libra Starting with Substrate and looking into the pool of the blockchain. The pool is the pool of incoming extrinsics; that are things such as transactions and inherents such as timestamps that are put into a mempool and propagated on the network.
The Rise of Substrate: A multi-blockchain universe Substrate is the antithesis of a one distributed actor framework smart contract world computer. There will be thousands of application specific blockchains built on this framework. In this framework the blockchain is more of a living organism that is in constant change through an ongoing set of state transitions driven through extrinsics.
Substrate Specifications
Runtime architecture WebAssembly Implementation language Rust Component technologies provided with Substrate Here are some of the technologies bundled with Substrate.
You can swap out any of these components for your own alternative:
Networking LibP2P Consensus algorithm Hybrid PBFT/Aurand Randomness beacon Collective coin flipping Authentication algorithm Edwards-Curve Ed25519 Hashing function Blake2b Address format Versioned Base-58 + Blake2b checksum
Here is a link to the Substrate Developer Hub
A few commands to get going:
substrate-node-new
substrate-module-new
Author
Hash, BlockHash
Substrate authoring RPC API
submit_extrinsic, pending_extrinsic, watch_extrinsic, unwatch_extrinisic new client: pool: subscriptions)
Chain
Substrate blockchain API
Hash, Header, Number, SignedBlock Relay Chain and the Canonical Chain
Subscribe and Unsubscribe -> Block heads
header, block, block_hash, finalised_head, subscribe_new_head, unsubscribe_new_head, subscribe_finalised_heads, unsubscribe_finalised_heads.
State
Substrate state API
Hash,
call
storage
storage_hash
storage_size
runtime_version
query_storage State Subscribe and Unsubscribe -> Runtime Version
Subscribe and Unsubscribe -> Storage
call(method: String, data: Bytes, block:), storage(key: StorageKey, block:)
storage_hash(key: StorageKey, block:),
storage_size(key: StorageKey, block:), metadata(block:), query_storage(key: StorageKey, from: Block::Hash, to:),
subscribe_storage, unsubscribe_storage, runtime_version, subscribe_runtime_version)
System
Substrate system API
system_name, system_version, system_chain, system_properties, system_health
Rust provides a Keystore and the runtime modules. Abstract block format crypto database agnostic
For interacting with the chain using the RPCs go to https://polkadot.js.org/apps/#/toolbox
– base-16 modified merkle trie (aka Ethereum) Patricia Merkle Tree – Sparse Merkle Trees – Binary Merkle Trie
Wasm “execute_block” function Extensible Networking, CLI, RPC Roll your own Consensus Blockchain PBFT Probablistic Finality Consensus API
What do I get with Substrate?
Shared ancestry finality tool grandpa Hot-swapple, pluggable Consensus Hot-upgradeable, pluggable STFLight client Chain synchronisation Pub/Sub WebSocket JSON-RPC Transaction queue Pervasive, secure networking JS implementation Modular SRML if you want Interchain connectivity via Polkadot execute_block function
networking, block authoring and transaction queue CORE SUBSTRATE
RPCs sync, databases, crypto, networking, storage Telemetry Light client Change tracking Pluggable consensus Address formats Low level JS utils SRML SUBSTRATE
High level JS helpers Front-end GUI infrastrucuture Block authoring and transaction queue JSON config Chain explorer event tracking
SOLO CHAIN | SOLO CHAIN + BRIDGE | PARACHAIN
Architected on industry-standard WebAssembly Highly extensible Libp2p networking Rust-based primary implementation for speed and reliability Javascript secondary implementation for developability Wasm WebAssembly interpreter, written in Rust
Substrate is a blockchain platform with a completely generic State Transition Function ( STF ) and modular components for consensus, networking and configuration. Despite being “completely generic”, it comes with both standards and conventions (particularly with the Substrate Runtime Module Library ( SRML ) regarding the underlying data-structures which power the STF, thereby making rapid blockchain development a reality.
Each of these datatypes corresponds to a Rust trait. They are:
-
Hash, a type which encodes a cryptographic digest of some data. Typically just a 256-bit quantity.
-
BlockNumber, a type which encodes the total number of ancestors any valid block has. Typically a 32-bit quantity.
-
Digest, basically just a series of DigestItems, this encodes all information that is relevant for a light-client to have at hand within the block.
-
DigestItem, a type which must be able to encode one of a number of “hard-wired” alternatives relevant to consensus and change-tracking as well as any number of “soft-coded” variants, relevant to specific modules within the runtime.
-
Header, a type which is representative (cryptographically or otherwise) of all information relevant to a block. It includes the parent hash, the storage root and the extrinsics trie root, the digest and a block number.
-
Extrinsic, a type to represent a single piece of data external to the blockchain that is recognised by the blockchain. This typically involves one or more signatures, and some sort of encoded instruction (e.g. for transferring ownership of funds or calling into a smart contract).
-
Block, essentially just a combination of Header and a series of Extrinsics, together with a specification of the hashing algorithm to be used
impl system::Trait for Runtime {
/// The identifier used to distinguish between accounts.
type AccountId = AccountId;
/// The index type for storing how many extrinsics an account has signed.
type Index = Nonce;
/// The index type for blocks.
type BlockNumber = BlockNumber;
/// The type for hashing blocks and tries.
type Hash = Hash;
/// The hashing algorithm used.
type Hashing = BlakeTwo256;
/// The header digest type.
type Digest = generic::Digest
Substrate Runtime Architecture 2019-08-26 18_50_37-Architecture of a Runtime · Substrate Developer Hub
Runtime Proxy Functions
Have to implement these three:
what version what client how to execute a block // Implement our runtime API endpoints. This is just a bunch of proxying.
impl runtime_api::Core<Block> for Runtime {
fn version() -> RuntimeVersion {
VERSION
}
fn execute_block(block: Block) {
Executive::execute_block(block)
}
fn initialize_block(header: &<Block as BlockT>::Header) {
Executive::initialize_block(header)
}
fn authorities() -> Vec<AuthorityId> {
panic!("Deprecated, please use `AuthoritiesApi`.")
}
}
impl runtime_api::Metadata<Block> for Runtime {
fn metadata() -> OpaqueMetadata {
Runtime::metadata().into()
}
}
impl block_builder_api::BlockBuilder<Block> for Runtime {
fn apply_extrinsic(extrinsic: <Block as BlockT>::Extrinsic) -> ApplyResult {
Executive::apply_extrinsic(extrinsic)
}
fn finalize_block() -> <Block as BlockT>::Header {
Executive::finalize_block()
}
fn inherent_extrinsics(data: InherentData) -> Vec<<Block as BlockT>::Extrinsic> {
data.create_extrinsics()
}
fn check_inherents(block: Block, data: InherentData) -> CheckInherentsResult {
data.check_extrinsics(&block)
}
fn random_seed() -> <Block as BlockT>::Hash {
System::random_seed()
}
}
impl runtime_api::TaggedTransactionQueue<Block> for Runtime {
fn validate_transaction(tx: <Block as BlockT>::Extrinsic) -> TransactionValidity {
Executive::validate_transaction(tx)
}
}
impl consensus_aura::AuraApi<Block> for Runtime {
fn slot_duration() -> u64 {
Aura::slot_duration()
}
}
impl offchain_primitives::OffchainWorkerApi<Block> for Runtime {
fn offchain_worker(n: NumberFor<Block>) {
Executive::offchain_worker(n)
}
}
impl consensus_authorities::AuthoritiesApi<Block> for Runtime {
fn authorities() -> Vec<AuthorityId> {
Consensus::authorities()
}
}
}
Runtime Module Template Module’s configutation trait, storage, declartion and event.
/// A runtime module template with necessary imports
/// Feel free to remove or edit this file as needed.
/// If you change the name of this file, make sure to update its references in runtime/src/lib.rs
/// If you remove this file, you can remove those references
/// For more guidance on Substrate modules, see the example module
/// https://github.com/paritytech/substrate/blob/master/srml/example/src/lib.rs
use support::{decl_module, decl_storage, decl_event, StorageValue, dispatch::Result};
use system::ensure_signed;
/// The module's configuration trait.
pub trait Trait: system::Trait {
// TODO: Add other types and constants required configure this module.
/// The overarching event type.
type Event: From<Event<Self>> + Into<<Self as system::Trait>::Event>;
}
/// This module's storage items.
decl_storage! {
trait Store for Module<T: Trait> as TemplateModule {
// Just a dummy storage item.
// Here we are declaring a StorageValue, `Something` as a Option<u32>
// `get(something)` is the default getter which returns either the stored `u32` or `None` if nothing stored
Something get(something): Option<u32>;
}
}
decl_module! {
/// The module declaration.
pub struct Module<T: Trait> for enum Call where origin: T::Origin {
// Initializing events
// this is needed only if you are using events in your module
fn deposit_event<T>() = default;
// Just a dummy entry point.
// function that can be called by the external world as an extrinsics call
// takes a parameter of the type `AccountId`, stores it and emits an event
pub fn do_something(origin, something: u32) -> Result {
// TODO: You only need this if you want to check it was signed.
let who = ensure_signed(origin)?;
// TODO: Code to execute when something calls this.
// For example: the following line stores the passed in u32 in the storage
<Something<T>>::put(something);
// here we are raising the Something event
Self::deposit_event(RawEvent::SomethingStored(something, who));
Ok(())
}
}
}
decl_event!(
pub enum Event<T> where AccountId = <T as system::Trait>::AccountId {
// Just a dummy event.
// Event `Something` is declared with a parameter of the type `u32` and `AccountId`
// To emit this event, we call the deposit function, from our runtime funtions
SomethingStored(u32, AccountId),
}
); Runtime Module Key Value Example This is simple runtime module that stores a key value map.
use srml_support::{StorageMap, dispatch::Result};
pub trait Trait: system::Trait {}
decl_module! {
pub struct Module<T: Trait> for enum Call where origin: T::Origin {
fn set_mapping(_origin, key: u32, value: u32) -> Result {
<Value<T>>::insert(key, value);
Ok(())
}
}
}
decl_storage! {
trait Store for Module<T: Trait> as RuntimeExampleStorage {
Value: map u32 => u32;
}
}
Libra
Networking level and substrate based on libp2p core.
Substrate based on:
use libp2p::core:: Libra based on:
parity_multiaddr::Multiaddr; fn listen_on(&self, addr: Multiaddr) -> Result<(Self::Listener, Multiaddr) – chain level
I was able to get Libra working on my Linux Subsystem using the following commands:
git clone https://github.com/libra/libra
cd libra
./scripts/dev_setup.sh
source /home/
./scripts/cli/start_cli_testnet.sh Libra Blockchain API account_state_sets
fn put_account_state_set( store: &StateStore, account_state_set: Vec<(AccountAddress, AccountStateBlob)>, version: Version, root_hash: HashValue, expected_nodes_created: usize, expected_nodes_retired: usize, expected_blobs_retired: usize, ) -> HashValue use rocksdb
pub type Read Options = rocksdb::ReadOptions;
A simple tx script in Move IR that transfers a coin from address to another:
public main(payee: address, amount: u64) { let coin: 0x0.Currency.Coin = 0x0 .Currency .withdraw_from_sender(copy(amount));
0x0.Currency.deposit(copy(payee), move(coin)); } Oasis: Blockchain and WASM By leveraging compiler support and tools built for Wasm and WASI, the blockchain becomes a powerful tool for high-integrity — and even confidential — general-purpose “cloud” computation.
use oasis_std::Context; use statesets_std::Context;
#[derive(oasis_std::Service)] struct Quickstart; impl Quickstart { pub fn new(_ctx: &Context) -> Self {
Self
}
pub fn say_hello(&mut self, ctx: &Context) -> String {
format!("Hello, {}!", ctx.sender())
}
}
fn main() {
oasis_std::service!(Quickstart);
}
#[cfg(test)]
mod tests {
extern crate oasis_test;
use super::*;
#[test]
fn test() {
let sender = oasis_test::create_account(1);
let ctx = Context::default().with_sender(sender);
let mut client = Quickstart::new(&ctx);
println!("{}", client.say_hello(&ctx));
}
} From https://medium.com/oasislabs/blockchain-flavored-wasi-50e3612b8eba:
Notes on struct, impl, and pub fn.
Becomes one main.rs file as opposed to separated. The compiler checks and the gives wasm blog which is deployed on chain with rpc endpoints getting, changing and setting states.
define and implement Oasis service RPCs in Rust.
fn main() { oasis_std::service!(Ballot); } Compiles to WASM. Deploy to the platform and setup client to call it.
use oasis_std::Context;
use map_vec::Map; use oasis_std::{Address, Context};
[derive(oasis_std::Service)]
derive [Serialize, Deserialize, Service]
pub struct X {
fields:
''
}
pub fn new(ctx: &Context, description: String, candidates: Vec
/// Returns the candidates being voted upon.
pub fn candidates(&self, _ctx: &Context) -> Vec<&str> { self.candidates.iter().map(String::as_ref).collect() }
/// Returns the description of this ballot.
pub fn description(&self, _ctx: &Context) -> &str { &self.description }
/// Returns whether voting is still open. pub fn voting_open(&self, _ctx: &Context) -> bool { self.accepting_votes } we have access to the state of the service, as provided by a reference to self.
For state changes:
you’ll see that &self has changed to &mut self, but this is just Rust’s way to know that you want a mutable reference.
Then define Getter Fns to Get State from the Stores
similar to the hyperledger composer model
define a data model in cto file declare assets, transactions and events
define javascript functions for logic with refernece to namespace is same as impl with struct (service state object) RPCs that are defined as pub fn’s.
instead of seperate REST API to make calls to on the model.
A service is created on chain with RPC endpoints.
Clients can call the service endpoint and add listeners for events from the service.
Blockchain WASI While cloud computing has long brought cost and ease of use, switching from an on-prem solution to cloud has traditionally come with its own inherent risks including a degradation in security and a lack of auditability. These are areas that have the potential to be solved with new emerging technologies including Web Assembly, the Web Assembly System Interface, and blockchain.
We propose a mechanism for trustworthy, uncensorable, and autonomous cloud computation based on the combination of three emerging technologies: Web Assembly, the Web Assembly System Interface, and blockchain.
Oasis Network The Oasis Network uses 3 main protocols for communication:
Tendermint grpc libp2p Confidentiality is achieved in the Oasis Network by relying on trusted execution environments (TEEs) to secure the execution of any given smart contract. Initially, the Oasis Network will utilize Intel SGX. As more TEE technologies mature, we expect to support more than TEEs than Intel SGX.
from https://docs.oasis.dev/operators/architectural-overview.html#modular-architecture
libp2p
lib-p2p is a modularized and extensible network stack to overcome the networking challenges faced when doing peer-to-peer applications. It is very well defined, composable, swappable; a modular system of protocols, specifications, and libraries that enable the development of peer-to-peer network applications.
It ultimately is a collection of peer-to-peer protocols for finding peers, connecting to them for finding content, and transferring it to them.
Here is a sample ping example from rust-libp2p:
use libp2p::{ identity, PeerId, ping::{Ping, PingConfig}, Swarm };
use std::env;
fn main() {
env_logger::init();
// Create a random PeerId.
let id_keys = identity::Keypair::generate_ed25519();
let peer_id = PeerId::from(id_keys.public());
println!("Local peer id: {:?}", peer_id);
// Create a transport.
let transport = libp2p::build_development_transport(id_keys);
// Create a ping network behaviour.
//
// For illustrative purposes, the ping protocol is configured to
// keep the connection alive, so a continuous sequence of pings
// can be observed.
let behaviour = Ping::new(PingConfig::new().with_keep_alive(true));
// Create a Swarm that establishes connections through the given transport
// and applies the ping behaviour on each connection.
let mut swarm = Swarm::new(transport, behaviour, peer_id);
// Dial the peer identified by the multi-address given as the second
// command-line argument, if any.
if let Some(addr) = env::args().nth(1) {
let remote_addr = addr.clone();
match addr.parse() {
Ok(remote) => {
match Swarm::dial_addr(&mut swarm, remote) {
Ok(()) => println!("Dialed {:?}", remote_addr),
Err(e) => println!("Dialing {:?} failed with: {:?}", remote_addr, e)
}
},
Err(err) => println!("Failed to parse address to dial: {:?}", err),
}
}
// Tell the swarm to listen on all interfaces and a random, OS-assigned port.
Swarm::listen_on(&mut swarm, "/ip4/0.0.0.0/tcp/0".parse().unwrap()).unwrap();
// Use tokio to drive the `Swarm`.
let mut listening = false;
tokio::run(future::poll_fn(move || -> Result<_, ()> {
loop {
match swarm.poll().expect("Error while polling swarm") {
Async::Ready(Some(e)) => println!("{:?}", e),
Async::Ready(None) | Async::NotReady => {
if !listening {
if let Some(a) = Swarm::listeners(&swarm).next() {
println!("Listening on {:?}", a);
listening = true;
}
}
return Ok(Async::NotReady)
}
}
}
}));
}