Skip to content

Latest commit

 

History

History
415 lines (383 loc) · 10.8 KB

psi_v2_benchmark.md

File metadata and controls

415 lines (383 loc) · 10.8 KB
Raw

PSI V2 Benchmark

This document will introduce the PSI V2 Benchmark. It uses the PSI V2 version of the interface.

Building from source

git clone https://github.com/secretflow/psi.git
cd psi
bazel build //... -c opt

If the building is successful, you will find an executable file named main in the 'bazel-bin/psi' directory. We will use ./main file, combined with config file to run different PSI protocols. Such as:

./main --config configs/sender.config
./main --config configs/receiver.config

Generate Data

In order to measure the performance of different PSI protocols under different data scales, we need to generate dummy data through generate_psi.py

python3 generate_psi.py 10000000 10000000

python3 generate_psi.py 100000000 100000000

Prepare config file

We use the config file to specify different PSI protocols and input data.

sender.config

{
  "psi_config": {
    "protocol_config": {
      "protocol": "PROTOCOL_KKRT",
      "role": "ROLE_SENDER",
      "broadcast_result": false
    },
    "input_config": {
      "type": "IO_TYPE_FILE_CSV",
      "path": "/home/zuoxian/v2psi/datas/psi_1_1kw.csv"
    },
    "output_config": {
      "type": "IO_TYPE_FILE_CSV",
      "path": "/home/zuoxian/v2psi/sender/sender_output.csv"
    },
    "keys": [
      "id",
    ],
    "debug_options": {
      "trace_path": "/home/zuoxian/v2psi/sender/sender.trace"
    },
    "skip_duplicates_check": false,
    "disable_alignment": false,
    "advanced_join_type": "ADVANCED_JOIN_TYPE_UNSPECIFIED",
    "left_side": "ROLE_RECEIVER",
    "check_hash_digest": false,
    "recovery_config": {
      "enabled": false
    }
  },
  "link_config": {
    "parties": [
      {
        "id": "receiver",
        "host": "127.0.0.1:5300"
      },
      {
        "id": "sender",
        "host": "127.0.0.1:5400"
      }
    ]
  },
  "self_link_party": "sender"
}

receiver.config

{
  "psi_config": {
    "protocol_config": {
      "protocol": "PROTOCOL_KKRT",
      "role": "ROLE_RECEIVER",
      "broadcast_result": false
    },
    "input_config": {
      "type": "IO_TYPE_FILE_CSV",
      "path": "/home/zuoxian/v2psi/datas/psi_2_1kw.csv"
    },
    "output_config": {
      "type": "IO_TYPE_FILE_CSV",
      "path": "/home/zuoxian/v2psi/receiver/receiver_output.csv"
    },
    "keys": [
      "id",
    ],
    "debug_options": {
      "trace_path": "/home/zuoxian/v2psi/receiver/receiver.trace"
    },
    "skip_duplicates_check": false,
    "disable_alignment": false,
    "advanced_join_type": "ADVANCED_JOIN_TYPE_UNSPECIFIED",
    "left_side": "ROLE_RECEIVER",
    "check_hash_digest": false,
    "recovery_config": {
      "enabled": false
    }
  },
  "link_config": {
    "parties": [
      {
        "id": "receiver",
        "host": "127.0.0.1:5300"
      },
      {
        "id": "sender",
        "host": "127.0.0.1:5400"
      }
    ]
  },
  "self_link_party": "receiver"
}

Run PSI with docker

In order to measure the PSI V2 benchmark under different machine configurations and network configurations, we use two dockers to act as sender and receiver respectively.

alice

docker run -d -it --name "alice" \
         --mount type=bind,source="$(pwd)",target=/home/admin/dev/ \
         -w /home/admin/dev \
         --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
         --cap-add=NET_ADMIN \
         --cpuset-cpus 0-31 \
         --privileged=true \
         --memory="64g" \
         --entrypoint="bash" \
         secretflow/ubuntu-base-ci:latest

docker start alice
docker exec -it alice bash

bob

docker run -d -it --name "bob" \
         --mount type=bind,source="$(pwd)",target=/home/admin/dev/ \
         -w /home/admin/dev \
         --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
         --cap-add=NET_ADMIN \
         --cpuset-cpus 32-63 \
         --privileged=true \
         --memory="64g" \
         --entrypoint="bash" \
         secretflow/ubuntu-base-ci:latest

docker start bob
docker exec -it bob bash

Limit bandwidth and latency

BAND_WIDTH=100
burst=128
DELAY=10
netem_limit=8000

tc qdisc add dev eth0 root handle 1: tbf rate ${BAND_WIDTH}mbit burst ${burst}kb latency 800ms
tc qdisc add dev eth0 parent 1:1 handle 10: netem delay ${DELAY}msec limit ${netem_limit}

Benchmark

Here we show the PSI V2 Benchmark measured as above.

The default time unit is seconds, m represents minutes, and h represents hours.

Machine Configuration Algorithm parameters Protocol Network Configuration 10 million~10 million 100 million to 100 million 1 billion to 1 billion
32C64G receiver='alice',
protocol='ECDH_PSI_2PC',
curve_type='CURVE_FOURQ',
precheck_input=False,
sort=False,
broadcast_result=False,		 ECDH-PSI-2PC
（FourQ）		 LAN 42 459 1.45 h
100Mbps/10ms 63 559 1.72 h
receiver='alice',
protocol='ECDH_PSI_2PC',
curve_type='CURVE_25519',
precheck_input=False,
sort=False,
broadcast_result=False,		 ECDH-PSI-2PC
（CURVE_25519）		 LAN 67 697 2.09 h
100Mbps/10ms 69 669 2.08 h
receiver='alice',
protocol='KKRT_PSI_2PC',
precheck_input=False,
sort=False,
broadcast_result=False,		 KKRT_PSI_2PC
（BUCKET SIZE IS ONE MILLION）		 LAN 38 357 1.13 h
100Mbps/10ms 125 1258 3.53 h
receiver='alice',
protocol='RR22_FAST_PSI_2PC',
precheck_input=False,
sort=False,
broadcast_result=False,		 RR22_FAST_PSI
_2PC
（BUCKET SIZE IS ONE MILLION）		 LAN 17 225 0.62 h
100Mbps/10ms 44 452 1.43 h
receiver='alice',
protocol='RR22_LOWCOMM_PSI_2PC',
precheck_input=False,
sort=False,
broadcast_result=False,		 RR22_LOWCOMM
_PSI_2PC
（BUCKET SIZE IS ONE MILLION）		 LAN 21 223 0.75 h
100Mbps/10ms 38 388 1.21 h
16C32G receiver='alice',
protocol='ECDH_PSI_2PC',
curve_type='CURVE_FOURQ',
precheck_input=False,
sort=False,
broadcast_result=False,		 ECDH-PSI-2PC
（FourQ）		 LAN 46 497 1.66 h
100Mbps/10ms 62 574 2.00 h
receiver='alice',
protocol='ECDH_PSI_2PC',
curve_type='CURVE_25519',
precheck_input=False,
sort=False,
broadcast_result=False,		 ECDH-PSI-2PC
（CURVE_25519）		 LAN 69 701 2.23 h
100Mbps/10ms 74 744 2.49 h
receiver='alice',
protocol='KKRT_PSI_2PC',
precheck_input=False,
sort=False,
broadcast_result=False,		 KKRT_PSI_2PC
（BUCKET SIZE IS ONE MILLION）		 LAN 37 352 1.18 h
100Mbps/10ms 124 1228 3.69 h
receiver='alice',
protocol='RR22_FAST_PSI_2PC',
precheck_input=False,
sort=False,
broadcast_result=False,		 RR22_FAST_PSI
_2PC
（BUCKET SIZE IS ONE MILLION）		 LAN 17 178 0.72 h
100Mbps/10ms 43 452 1.70 h
receiver='alice',
protocol='RR22_LOWCOMM_PSI_2PC',
precheck_input=False,
sort=False,
broadcast_result=False,		 RR22_LOWCOMM
_PSI_2PC
（BUCKET SIZE IS ONE MILLION）		 LAN 23 216 0.92 h
100Mbps/10ms 40 384 1.52 h
8C16G receiver='alice',
protocol='ECDH_PSI_2PC',
curve_type='CURVE_FOURQ',
precheck_input=False,
sort=False,
broadcast_result=False,		 ECDH-PSI-2PC
（FourQ）		 LAN 66 669 2.6 h
100Mbps/10ms 66 666 2.6 h
receiver='alice',
protocol='ECDH_PSI_2PC',
curve_type='CURVE_25519',
precheck_input=False,
sort=False,
broadcast_result=False,		 ECDH-PSI-2PC
（CURVE_25519）		 LAN 114 1225 3.8 h
100Mbps/10ms 112 1124 3.8 h
receiver='alice',
protocol='KKRT_PSI_2PC',
precheck_input=False,
sort=False,
broadcast_result=False,		 KKRT_PSI_2PC
（BUCKET SIZE IS ONE MILLION）		 LAN 37 351 1.6 h
100Mbps/10ms 124 1211 4.0 h
receiver='alice',
protocol='RR22_FAST_PSI_2PC',
precheck_input=False,
sort=False,
broadcast_result=False,		 RR22_FAST_PSI
_2PC
（BUCKET SIZE IS ONE MILLION）		 LAN 19 166 1.08 h
100Mbps/10ms 44 440 1.91 h
receiver='alice',
protocol='RR22_LOWCOMM_PSI_2PC',
precheck_input=False,
sort=False,
broadcast_result=False,		 RR22_LOWCOMM
_PSI_2PC
（BUCKET SIZE IS ONE MILLION）		 LAN 22 204 1.17 h
100Mbps/10ms 37 374 1.72 h