Preform performance and stability test on different client implementation

[mzitnik]

In clickhouse-http-client we support different Http Connection providers

• APACHE_HTTP_CLIENT
• HTTP_CLIENT
• HTTP_URL_CONNECTION

We would like to test the performance and stability of those http connection providers

Verticals that we need to test

• inserts
• simple selects select

[zhicwu]

Just some random ideas:

• consider both small and large queries
• I used to test in LAN but we try WAN as well(like the socket timeout issue)
• make sure both Java client and JDBC are tested - JDBC is synchronous and has more overheads
• not sure if okhttp is faster or not but we may want to check that later too

[mzitnik]

@zhicwu is working to optimize it.

We are inserting 50K rows.

using DEFAULT_BUFFER_SIZE = 4096

Http Client - over wifi	Query	Insert - (20 byte)	Insert - (64 byte)	Insert - (124 byte)
Apache Http Client	70ms	271ms	564ms	960ms
HttpUrlConnection	211ms	770ms	1058ms	1310ms
Ratio	3X	2.8X	1.8X	1.3X

I will also run a test over LAN.

[mzitnik]

We are inserting 50K rows.

using BUFFER_SIZE = 8129

Http Client - over wifi	Query	Insert - (124 byte)	Insert - (256 byte)	Insert - (512 byte)
Apache Http Client	70ms	899ms	1639ms	3480
HttpUrlConnection	218ms	1429ms	2527ms	3862
Ratio	3X	1.5X	1.5X	1.1X

[zhicwu]

Is it single thread or multiple? And what's table structure and queries for the testing?

If the size here is bytes of an uncompressed row, then we're testing really small insert. It's just my opinion, since we usually create table with tens or hundreds of columns on ClickHouse, it might be better to try 1KB - 10KB row for small insert, and 100KB or more for large ones. Alternatively, we can test single-column table as in clickhouse-benchmark, because data type matters.

[mzitnik]

Planning to increase the row size in the test, we are inserting a batch of 50K rows (I will take a look on the benchmarking project )
@zhicwu what is the batch size in insert testing

[zhicwu]

what is the batch size in insert testing

It defaults to 5k but can be changed as needed.

clickhouse-java/clickhouse-benchmark/src/main/java/com/clickhouse/benchmark/jdbc/DriverBenchmark.java

Line 32 in f2ea0e1

private final int BATCH_SIZE = Integer.parseInt(System.getProperty("batchSize", "5000"));

[mzitnik]

Settings
prop.setProperty("buffer_size", "8192");
prop.setProperty("socket_rcvbuf", "804800");
prop.setProperty("socket_sndbuf", "804800");

Http Client - LAN	Insert - (256 byte)	Insert - (512 byte)	Insert - (1024 byte)	Insert - (2048 byte)
Apache Http Client	1853	2964	5323	10543
HttpUrlConnection	3153	5580	9634	14830
Ratio	1.7X	1.8X	1.8X	1.4X

[mzitnik]

Settings
prop.setProperty("buffer_size", "8192");

Http Client - LAN	Insert - (256 byte)	Insert - (512 byte)	Insert - (1024 byte)	Insert - (2048 byte)
Apache Http Client	1873	2773	5126	8497
HttpUrlConnection	2704	4843	8161	14397
Ratio	1.5X	1.7X	1.6X	1.7X