[#1607] docs: Performance report with partial TPC-DS(SF=40000) queries

[rickyma]

What changes were proposed in this pull request?

Add a performance report using TPC-DS.

Why are the changes needed?

For #1607.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

No need to be tested.

[rickyma]

We can see from here: https://github.com/rickyma/incubator-uniffle/blob/issue-1607/docs/benchmark_netty_case_report.md

[rickyma]

@zuston @jerqi PTAL.

[github-actions]

Test Results

 2 363 files  ±0   2 363 suites  ±0   4h 31m 1s ⏱️ -8s
   912 tests ±0     911 ✅ ±0   1 💤 ±0  0 ❌ ±0 
10 585 runs  ±0  10 571 ✅ ±0  14 💤 ±0  0 ❌ ±0 

Results for commit c492db1. ± Comparison against base commit b4c92b8.

♻️ This comment has been updated with latest results.

[zuston]

Spark Native -> vanilla spark

[rickyma]

Done.

[advancedxy]

Hmmm, I would call it Total task time reduction.
BTW, for performance improvement, we usually use speedup to indicate that. The definition of speedup is s = Time of old / Time of new, you can refer https://en.wikipedia.org/wiki/Speedup#Using_execution_times for how this is defined.

[rickyma]

I add a new column Netty(SSD) Speedup. Also I keep the original column and rename it to Netty(SSD) Total Task Time Reduction.

[advancedxy]

Nit: Vanilla Spark is already incapable of successfully completing tasks,

[rickyma]

Done.

[advancedxy]

BTW, thank you for the case report. I think it's quite compelling for users that Uniffle is capable of handling ten TBs of shuffle data and improves job stability and performance overall.

[zuston]

Great work! I have a question that the peek write speed is too slow from my side under the so high concurrency. Because I do the similar test about rust based server.

So is client backpressed?

[rickyma]

Great work! I have a question that the peek write speed is too slow from my side under the so high concurrency. Because I do the similar test about rust based server.

So is client backpressed?

You mean your peak write speed is too slow when you test your rust based servers? I think the number of concurrent tasks is not enough in your case maybe?

[advancedxy]

Generally LGTM from my side. @zuston Please take another look.

[zuston]

Great work! I have a question that the peek write speed is too slow from my side under the so high concurrency. Because I do the similar test about rust based server.
So is client backpressed?

You mean your peak write speed is too slow when you test your rust based servers? I think the number of concurrent tasks is not enough in your case maybe?

Misunderstand me, I think write speed in your test report is slow.

[rickyma]

Misunderstand me, I think write speed in your test report is slow.

There could be many reasons, such as differences in data scale, SQL, hardware, shuffle server's configuration, etc. For example, if the memory is larger, the block size will be larger when flushing, resulting in less random IO, which may also have an impact even on SSDs. You can try testing again following the same method described in my document, as this comparison might help clarify the issue. Currently, I cannot tell the reason either. However, the current result is indeed as such.

[zuston]

Misunderstand me, I think write speed in your test report is slow.

There could be many reasons, such as differences in data scale, SQL, hardware, shuffle server's configuration, etc. For example, if the memory is larger, the block size will be larger when flushing, resulting in less random IO, which may also have an impact even on SSDs. You can try testing again following the same method described in my document, as this comparison might help clarify the issue. Currently, I cannot tell the reason either. However, the current result is indeed as such.

Yes. Just my a little question. Anyway, this report is good for community!