0%

leopard-bft

Hu K, Guo K, Tang Q, et al. Leopard: Towards High Throughput-Preserving BFT for Large-scale Systems[C]//2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS). IEEE, 2022: 157-167.

实验

环境配置

安装go,将awesomeProject DKG cp到GOROOT

1
2
alleysira@LAPTOP-M4HO8L6S:~/leopard-bft$ sudo cp -r go/src/awesomeProject /usr/local/go/src
alleysira@LAPTOP-M4HO8L6S:~/leopard-bft$ sudo cp -r go/src/DKG /usr/local/go/src

经典编不过去

1
2
3
4
5
6
alleysira@LAPTOP-M4HO8L6S:~/leopard-bft/go/src/awesomeProject$ go build main.go
# DKG
/usr/local/go/src/DKG/bls.go:6:10: fatal error: bls/bls.h: No such file or directory
6 | #include <bls/bls.h>
| ^~~~~~~~~~~
compilation terminated.

bn

论文中bls采用了 https://github.com/dfinity-side-projects/bn,安装

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
alleysira@LAPTOP-M4HO8L6S:~/bn-master$ make
make -C mcl UPDATE_ASM=1 lib/libmcl.so CFLAGS_USER=-std=c++11 MCL_MAX_BIT_SIZE=384
make[1]: Entering directory '/home/alleysira/bn-master/mcl'
g++ -o src/gen src/gen.cpp -g3 -Wall -Wextra -Wformat=2 -Wcast-qual -Wcast-align -Wwrite-strings -Wfloat-equal -Wpointer-arith -m64 -I include -I test -I ../xbyak -I ../cybozulib/include -std=c++11 -fomit-frame-pointer -DNDEBUG -Ofast -march=native -fPIC -DMCL_MAX_BIT_SIZE=384 -DMCL_USE_LLVM=1 -O0
src/gen -old -f src/func.list > src/base64.ll
opt -O3 -o - src/base64.ll -march=x86-64 | llc -O3 -o src/asm/x86-64.s -march=x86-64 -relocation-model=pic -pre-RA-sched=list-ilp -max-sched-reorder=128
opt: src/base64.ll:16:26: error: expected comma after getelementptr's type
%r5 = getelementptr i64* %r2, i64 %r4
^
g++ -c src/asm/x86-64.s -o obj/x86-64.o -g3 -Wall -Wextra -Wformat=2 -Wcast-qual -Wcast-align -Wwrite-strings -Wfloat-equal -Wpointer-arith -m64 -I include -I test -I ../xbyak -I ../cybozulib/include -std=c++11 -fomit-frame-pointer -DNDEBUG -Ofast -march=native -fPIC -DMCL_MAX_BIT_SIZE=384 -DMCL_USE_LLVM=1
src/gen -old -f src/func.list -s bmi2 > src/base64.bmi2.ll
opt -O3 -o - src/base64.bmi2.ll -march=x86-64 | llc -O3 -o src/asm/x86-64.bmi2.s -march=x86-64 -relocation-model=pic -pre-RA-sched=list-ilp -max-sched-reorder=128 -mattr=bmi2
opt: src/base64.bmi2.ll:16:26: error: expected comma after getelementptr's type
%r5 = getelementptr i64* %r2, i64 %r4
^

make clean之后sudo make依然出错,目测是llvm的版本的问题

github 上相关的issue

opt是llvm的优化器,问题原因是采用的llvm是10.0版本,.ll文件的语法不相同,但是能编译出libbls的两个库和一个libmcl

1
2
3
4
5
6
7
8
9
10
11
g++ -o src/gen src/gen.cpp -g3 -Wall -Wextra -Wformat=2 -Wcast-qual -Wcast-align -Wwrite-strings -Wfloat-equal -Wpointer-arith -m64 -I include -I test -I ../xbyak -I ../cybozulib/include -std=c++11 -fomit-frame-pointer -DNDEBUG -Ofast -march=native -fPIC -DMCL_MAX_BIT_SIZE=384 -DMCL_USE_LLVM=1 -O0
src/gen -old -f src/func.list > src/base64.ll
opt -O3 -o - src/base64.ll -march=x86-64 | llc -O3 -o src/asm/x86-64.s -march=x86-64 -relocation-model=pic -pre-RA-sched=list-ilp -max-sched-reorder=128
opt: src/base64.ll:16:26: error: expected comma after getelementptr's type
%r5 = getelementptr i64* %r2, i64 %r4
^
g++ -c src/asm/x86-64.s -o obj/x86-64.o -g3 -Wall -Wextra -Wformat=2 -Wcast-qual -Wcast-align -Wwrite-strings -Wfloat-equal -Wpointer-arith -m64 -I include -I test -I ../xbyak -I ../cybozulib/include -std=c++11 -fomit-frame-pointer -DNDEBUG -Ofast -march=native -fPIC -DMCL_MAX_BIT_SIZE=384 -DMCL_USE_LLVM=1
src/gen -old -f src/func.list -s bmi2 > src/base64.bmi2.ll
opt -O3 -o - src/base64.bmi2.ll -march=x86-64 | llc -O3 -o src/asm/x86-64.bmi2.s -march=x86-64 -relocation-model=pic -pre-RA-sched=list-ilp -max-sched-reorder=128 -mattr=bmi2
opt: src/base64.bmi2.ll:16:26: error: expected comma after getelementptr's type
%r5 = getelementptr i64* %r2, i64 %r4

应该是缺一个mcl库,因为报错编不出来

1
2
3
4
/tmp/go-build/cgo-gcc-prolog:1354: undefined reference to `mclBn_precomputedMillerLoop'
/usr/bin/ld: $WORK/b002/_x004.o: in function `_cgo_7ee38b8fbf18_Cfunc_mclBn_precomputedMillerLoop2':
/tmp/go-build/cgo-gcc-prolog:1370: undefined reference to `mclBn_precomputedMillerLoop2'
collect2: error: ld returned 1 exit status

尝试把16.04的库直接拿过来用,不行,有一些函数的接口变了,卒

尝试修改mcl的Makefile文件,跳过对.ll的报错,失败

image-20231031180355543

一个逆天bug,/*cgo xxx*/后面不能有空行,必须直接跟着import “C”

将存放4个库文件的/usr/local/lib加入到LD路径中

1
export LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib

屈服了,下载3.9版本的llvm用cmake build

如何build的教程 https://releases.llvm.org/3.9.0/docs/CMake.html

image-20231031221526742

make bn没有报错

库都正确编译了,但是依然有问题,是fp.cpp文件的6个函数存在undefined reference(不应该啊)

1
2
3
4
5
6
7
8
9
alleysira@LAPTOP-M4HO8L6S:~/leopard-bft/go/src/awesomeProject$ go build main.go
# DKG
/usr/bin/ld: /usr/local/lib/libbls384.so: undefined reference to `mcl::fp::copyUnitToByteAsLE(unsigned char*, unsigned long const*, unsigned long)'
/usr/bin/ld: /usr/local/lib/libbls384.so: undefined reference to `mcl::fp::copyAndMask(unsigned long*, void const*, unsigned long, mcl::fp::Op const&, bool)'
/usr/bin/ld: /usr/local/lib/libbls384.so: undefined reference to `mcl::fp::arrayToStr(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, unsigned long const*, unsigned long, int)'
/usr/bin/ld: /usr/local/lib/libbls384.so: undefined reference to `mcl::fp::hash[abi:cxx11](unsigned long, void const*, unsigned long)'
/usr/bin/ld: /usr/local/lib/libbls384.so: undefined reference to `mcl::fp::streamToArray(bool*, unsigned long*, unsigned long, std::istream&, int)'
/usr/bin/ld: /usr/local/lib/libbls384.so: undefined reference to `mcl::fp::Op::init(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, mcl::fp::Mode, unsigned long)'
collect2: error: ld returned 1 exit status

猜测是bn库的版本不对(并不是)

是因为忘记删除之前\usr\lib\下尝试的新版的mcl编出来的libbslxx.so

以下是最终能通过build的bls.go

image-20231031231943209

simulateUnderNode\network文件夹cp到GOROOT下即可运行

运行结果

4个terminal对应4个节点

image-20231031233119579

image-20231031233148849

image-20231031233209582

image-20231031233217854

image-20231031233204879

image-20231031233312013

报告笔记

胡可欣老师详细介绍了工作

Introduction

Hotstuff是diem链底层的共识

半同步网络模型下,已经被证明支持的恶意敌手上界是1/3

传统的BFT类共识,当节点规模扩大,吞吐量急剧下降

  • PBFT >100

image-20231102141032382

提升可扩展性的方法

  • 横向扩展 scale out:共识节点分组,各组对交易并行,分片属于横向方案
    • 安全性要求组的大小为600 1000,和想要得到大规模节点的需求之间存在矛盾关系
  • 纵向扩展scale up:给每个节点增加物理资源,存在瓶颈

发现瓶颈存在于领导者

  • 错误概率的问题
    • 组合
    • 项目指标安全性要求是单片$10^{-18}$
  • 网络模型
    • 半同步 影响活性,一致性与网络模型无关
    • Dumbo系列共识是异步

Hotstuff降低了通信复杂度,存在轮数增加的问题;实验结果显示Hotstuff确实提高了性能和规模

协议设计

  • 数据传输与共识解耦:先分摊交易,由领导者决定哪些交易当前轮打包

    • 怎么保证 交易分散没有交集 每个节点都拥有全集,发送时可能 mod 4
    • 发 Hash(datablock) ,通过调整尺寸来保证复杂度不是n^2
    • 复杂度是摊销的结果,实际没有降低复杂度
  • 纠删码

    • 法定人数2f+1,一定能收到f+1个好的纠删码片段
    • 如何判断收到纠删码的完整性:带线性长度的证明
    • 设计没有考虑proposal可能是交易,实际场景可能存在双花问题

协议实现

  • 实现

    • 信道 reliable and auth:不是工业界的实现
    • execution engine
      • bls需要额外进行DKG
      • rate-limiter : 限制数据块分散交易的频率,利用到了半同步
      • 双通道 go channel(共识优先级高于数据)
  • 实验

    • 增加节点数量:600个节点,吞吐量仍然在10w
    • 增加节点资源:斜率更高
    • 杭州:一个机器 一个核,1个核能否带多个节点(docker)
    • 软件所资源目前能使用的资源200个CPU
    • 具体系统实现用的签名:
      • 猎豹采用的是bls门限签名
      • Schnorr(复旦大学李强老师)
      • ecdsa(pipelined Hotstuff)
    • 规模运行到600时,一笔交易确认已经非常困难了,由于databa
  • 安全性

    • 目前$n=600,Pr_{error}<10^{-6},n=1000,Pr_{error}<10^{-9}$,指标要求的是$10^{-18}$
  • 后续工作

    • 尽快将分片共识的框架搭起来,缺少经验可以学习一下刘老师团队的工作
    • 明确能否在1个核上运行2个节点
    • 实现安全性可以考虑gearbox,目前leopard BFT代码里是有一条底层链的实现的
    • leopard BFT有后续引用工作:Hu Z, Guan S, Xu W, et al. A Data Flow Framework with High Throughput and Low Latency for Permissioned Blockchains[C]//2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS). IEEE, 2023: 1-12.