Data

Tools

Indexes

Applications

Experiments

Open Source Science

cole

Open source code of FAST' 24 paper, COLE: A Column-based Learned Storage for Blockchain Systems

https://github.com/hkbudb/cole

Science Score: 52.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ✓
    Institutional organization owner
    Organization hkbudb has institutional domain (www.comp.hkbu.edu.hk)
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.4%) to scientific vocabulary

Keywords

ai-for-storage blockchain
Last synced: 6 months ago · JSON representation ·

Repository

Open source code of FAST' 24 paper, COLE: A Column-based Learned Storage for Blockchain Systems

Basic Info
Statistics
  • Stars: 12
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Topics
ai-for-storage blockchain
Created almost 3 years ago · Last pushed over 1 year ago
Metadata Files
Readme License Citation

README.md

COLE: A Column-based Learned Storage for Blockchain Systems

Components

  • cole-index and cole-star are COLE and COLE with asynchronous merge
  • patricia-trie is the implementation of the MPT
  • lipp is the the updatable learned index with node persistence
  • non-learn-cmi is the column-based Merkle index (CMI) that uses non-learned index
  • exp is the evaluation backend of all systems including the throughput and the provenance queries

Install Dependencies

  • Install Rust. sudo apt install -y curl curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.bashrc source ~/.bashrc
  • Run sudo apt update and sudo apt -y install git make clang pkg-config libssl-dev libsqlite3-dev llvm m4 build-essential

Download Repository to $HOME directory

cd ~ git clone https://github.com/hkbudb/cole

Build

  • Build the latency testing binary and provenance testing binary cd ~/cole cargo build --release --bin latency cargo build --release --bin prov
  • If the program is built successfully, you should find two executable programs latency and prov in the directory ~/cole/target/release/

Prepare YCSB Dataset

  • Download the latest release of YCSB to the $HOME directory: cd ~ curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.17.0/ycsb-0.17.0.tar.gz tar xfvz ycsb-0.17.0.tar.gz
  • Install Java sudo apt -y install default-jdk sudo apt -y install default-jre

  • Use script build_ycsb.sh to generate readonly, writeonly, and readwriteeven datasets cd ~/cole/exp ./build_ycsb.sh

  • After the build process finishes, three txt files will be generate:

    • cole/exp/readonly/readonly-data.txt
    • cole/exp/writeonly/writeonly-data.txt
    • cole/exp/readwriteeven/readwriteeven-data.txt

  • Next, prepare the dataset for provenance queries: cd ~/cole/exp/ ./build_prov_ycsb.sh

  • After the build process finishes, a file named cole/exp/prov/prov-data.txt will be generated.

Run Script

cd ~/cole/exp/ python3 run.py

  • Use functions like test_overall_kvstore(), test_overall_smallbank(), and test_prov() in cole/exp/run.py to evaluate the workload of KVStore, SmallBank, and provenance query performance.
  • You may select different scales scale = [1000, 10000, 100000, 1000000, 10000000] or different indexes indexes = ["mpt", "cole", "cole_star", "non_learn_cmi"]

Check the Result

The result json files can be found in each workload directory (e.g., smallbank, writeonly, prov)

  • *-storage.json stores the storage information
  • *-ts.json stores the block timestamp information including start timestamp, end timestamp, and block latency, which can be used to compute the system throughput and latency

Owner

  • Name: HKBU DB
  • Login: hkbudb
  • Kind: organization

Citation (CITATION.cff) 

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
  - family-names: Zhang
    given-names: Ce
    affiliation: "Hong Kong Baptist University"
  - family-names: Xu
    given-names: Cheng
    affiliation: "Hong Kong Baptist University"
  - family-names: Hu
    given-names: Haibo
    affiliation: "Hong Kong Polytechnic University"
  - family-names: Xu
    given-names: Jianliang
    affiliation: "Hong Kong Baptist University"
title: "COLE: A Column-based Learned Storage for Blockchain Systems"
url: https://github.com/hkbudb/cole
preferred-citation:
  type: conference-paper
  authors:
    - family-names: Zhang
      given-names: Ce
      affiliation: "Hong Kong Baptist University"
    - family-names: Xu
      given-names: Cheng
      affiliation: "Hong Kong Baptist University"
    - family-names: Hu
      given-names: Haibo
      affiliation: "Hong Kong Polytechnic University"
    - family-names: Xu
      given-names: Jianliang
      affiliation: "Hong Kong Baptist University"
  title: "COLE: A Column-based Learned Storage for Blockchain Systems"
  collection-title: Proceedings of the 22nd USENIX Conference on File and Storage Technologies
  year: 2024
  month: 2

GitHub Events

Total
  • Issues event: 1
  • Watch event: 6
  • Issue comment event: 2
  • Push event: 1
  • Fork event: 2
Last Year
  • Issues event: 1
  • Watch event: 6
  • Issue comment event: 2
  • Push event: 1
  • Fork event: 2

Dependencies

utils/Cargo.toml cargo
Cargo.lock cargo
  • 199 dependencies
Cargo.toml cargo
cole-index/Cargo.toml cargo
cole-star/Cargo.toml cargo
eth-execution-engine/Cargo.toml cargo
exp/Cargo.toml cargo
growable-bitmap/Cargo.toml cargo
growable-bloom-filters/Cargo.toml cargo
  • serde_json 1.0 development
  • serde 1.0.125
  • serde_bytes 0.11
  • serde_derive 1.0.125
  • xxhash-rust 0.8.2
lipp/Cargo.toml cargo
merkle-btree-storage/Cargo.toml cargo
non-learn-cmi/Cargo.toml cargo
non-persist-trie/Cargo.toml cargo
patricia-trie/Cargo.toml cargo