remove lru partid as hint.

[xuguruogu]

Use partId as hint of LRU cache is a bad idea, as a machine only has some of the partitions. This will cause lock conflict a big performance issue.

PS:
Hash a limited num stored in the L1 cache of CPU, has little impact on the performance, as the access to the memory cost hundreds of CPU cycle.

[dangleptr]

Use partId as hint of LRU cache is a bad idea, as a machine only has some of the partitions. This will cause lock conflict a big performance issue.

Em.. How many parts on your one host?

[xuguruogu]

Use partId as hint of LRU cache is a bad idea, as a machine only has some of the partitions. This will cause lock conflict a big performance issue.

Em.. How many parts on your one host?

About 20. But more threads than parts number.

[xuguruogu]

The best practice is to use buckets of spinlock, which can fully fit into L1 cache. Like CuckooHash, use 4K spin bit lock as concurrency control.

[dangleptr]

Use partId as hint of LRU cache is a bad idea, as a machine only has some of the partitions. This will cause lock conflict a big performance issue.

Em.. How many parts on your one host?

About 20. But more threads than parts number.

Typically, we have default 100 parts on one host. The hint here is to avoid hash_combine for tuple, it has about 2x performance improvement in my benchmark.

[dangleptr]

The best practice is to use buckets of spinlock, which can fully fit into L1 cache. Like CuckooHash, use 4K spin bit lock as concurrency control.

You mean compact all spinLocks into one area?

[xuguruogu]

The best practice is to use buckets of spinlock, which can fully fit into L1 cache. Like CuckooHash, use 4K spin bit lock as concurrency control.

You mean compact all spinLocks into one area?

Yes. Spinlock is easy to implement using atomic operations. As long as lock buckets far larger than the threads, the lock collision can be ignored.

[xuguruogu]

Use partId as hint of LRU cache is a bad idea, as a machine only has some of the partitions. This will cause lock conflict a big performance issue.

Em.. How many parts on your one host?

About 20. But more threads than parts number.

Typically, we have default 100 parts on one host. The hint here is to avoid hash_combine for tuple, it has about 2x performance improvement in my benchmark.

benchmark of pure hash may mean nothing. Access the memory out of the L1/L2/L3 CPU cache overwhelm consumption.

[dangleptr]

The best practice is to use buckets of spinlock, which can fully fit into L1 cache. Like CuckooHash, use 4K spin bit lock as concurrency control.

You mean compact all spinLocks into one area?

Yes. Spinlock is easy to implement using atomic operations. As long as lock buckets far larger than the threads, the lock collision can be ignored.

Make sense. Let me think a while how to optimize it.

[dangleptr]

Accept it now.
Let's optimize it with more buckets and compacted spinLocks to protect the cache line.

[xuguruogu]

Use partId as hint of LRU cache is a bad idea, as a machine only has some of the partitions. This will cause lock conflict a big performance issue.

Em.. How many parts on your one host?

About 20. But more threads than parts number.

Typically, we have default 100 parts on one host. The hint here is to avoid hash_combine for tuple, it has about 2x performance improvement in my benchmark.

I need to correct it. We have default 100 parts on the whole cluster.

[dangleptr]

Please recheck the code style.