Recent Releases of flock
flock - 0.4.0
Flock 0.4.0 (2022-01-27)
New Features
- Supported distributed query processing on AWS Lambda with a crazy ShuffleHashJoin implementation
- Supported AWS Graviton2 processors (Arm64 SIMD)
- Supported state management (S3)
- Optimized payload serde
Next Step
...Paper...
- Rust
Published by gangliao about 4 years ago
flock - 0.3.0
Flock 0.3.0 (2022-01-01)
New Features
- Full support for NEXMark Benchmark (Query 0 - 13) on AWS Lambda
- The original benchmark was adopted and extended by the Apache Foundation for their use in Beam, a system intended to provide a general API for a variety of streaming systems. To make things a bit more dynamic, they changed the size of the windows to be merely ten seconds, rather than the minutes and hours the original specification sets. They also added more queries, for example, q1 - q8 are from original NEXMark queries, q0 and q9 - q13 are from Apache Beam. We follow the Beam implementation, as it is the most widely adopted one.
- Full support for YSB Benchmark on AWS Lambda
- A new command-line tool:
flock-cli```bash $ flock-cli -h
Flock 0.2.0 UMD Database Group Command Line Interactive Contoller for Flock
USAGE:
flock-cli [OPTIONS]
OPTIONS:
-c, --config
SUBCOMMANDS: fsql The terminal-based front-end to Flock help Print this message or the help of the given subcommand(s) lambda The AWS Lambda Tool for Flock nexmark The NEXMark Benchmark Tool upload Uploads a function code to AWS S3 ysb The Yahoo! Streaming Benchmarks Tool ```
```ignore $ flock-cli nexmark run -h
flock-cli-nexmark-run Runs the NEXMark Benchmark
USAGE: flock-cli nexmark run [OPTIONS]
OPTIONS: -a, --async-type Runs the NEXMark benchmark with async function invocations
-e, --events-per-second <events per second>
Runs the NEXMark benchmark with a number of events per second [default: 1000]
-g, --generators <data generators>
Runs the NEXMark benchmark with a number of data generators [default: 1]
-h, --help
Print help information
--log-level <log-level>
Log level [default: info] [possible values: error, warn, info, debug, trace, off]
-m, --memory-size <memory size>
Sets the memory size (MB) for the worker function [default: 128]
-q, --query <query number>
Sets the NEXMark benchmark query number [default: 3] [possible values: 0, 1, 2, 3, 4, 5,
6, 7, 8, 9, 10, 11, 12, 13]
-s, --seconds <duration>
Runs the NEXMark benchmark for a number of seconds [default: 20]
--silent
Suppress all output
-t, --data-sink-type <data sink type>
Runs the NEXMark benchmark with a data sink type [default: blackhole] [possible values:
sqs, s3, dynamodb, efs, blackhole]
--trace
Log ultra-verbose (trace level) information
```
Codebase
```ignore
sudo docker run --rm -v $PWD:/tmp aldanial/cloc /tmp/benchmarks /tmp/flock /tmp/flock-function /tmp/playground /tmp/scripts /tmp/flock_bench.sh
263 text files.
260 unique files.
106 files ignored.
github.com/AlDanial/cloc v 1.89 T=0.16 s (998.2 files/s, 274774.4 lines/s)
Language files blank comment code
CSV 2 0 0 21314 Rust 86 1864 3401 12344 JSON 8 0 0 2696 SQL 51 28 126 748 TOML 6 46 20 233 Bourne Shell 2 16 39 115 Python 1 24 24 95
Markdown 1 22 0 64
SUM: 157 2000 3610 37609
````
- Rust
Published by gangliao about 4 years ago
flock - 0.2.0
Flock 0.2.0 (2021-12-09)
New Features
- Add a script to compile, deploy and run the NexMark Benchmark.
- Support NexMark Benchmark Query 1, 2, 3, 4, 5, 6, 7, 8, 9 on cloud function services.
- Redesign a new document webpage using Rackets: https://flock-lab.github.io/flock/index.html
Contributors
```bash $ git shortlog -sn 0.1.0..
116 Gang Liao
2 Zhenghang Hu
```
Codebase
```bash
docker run --rm -v $PWD:/tmp aldanial/cloc /tmp/src /tmp/bench /tmp/scripts /tmp/nexmark.sh
232 text files.
230 unique files.
108 files ignored.
github.com/AlDanial/cloc v 1.88 T=0.11 s (1100.6 files/s, 133843.4 lines/s)
Language files blank comment code
Rust 63 1400 2492 8942 JSON 6 0 0 696 SQL 43 19 97 654 TOML 8 42 14 264 Bourne Shell 2 20 47 162 Python 1 24 24 95
Markdown 1 24 0 64
SUM: 124 1529 2674 10877
````
- Rust
Published by gangliao about 4 years ago