Add sharded rwlock map

README.md

@@ -1,71 +1,19 @@
-# Heap structure, using go generics
-[![Go Report Card](https://goreportcard.com/badge/github.com/lispad/go-generics-tools)](https://goreportcard.com/report/github.com/lispad/go-generics-tools) 
+# GoLang Generics tools: Heap structure, sharded rw-locked map.
+[![Go Report Card](https://goreportcard.com/badge/github.com/lispad/go-generics-tools)](https://goreportcard.com/report/github.com/lispad/go-generics-tools)
 [![MIT License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 
 Introduction
 ------------
 
-The Heap package contains simple [binary heap](https://en.wikipedia.org/wiki/Binary_heap) implementation, using Golang
+The [Heap](binheap/README.md) package contains simple [binary heap](https://en.wikipedia.org/wiki/Binary_heap) implementation, using Golang
 generics. There are several heap implementations
+[Details](binheap/README.md).
 
-- generic Heap implementation, that could be used for `any` type,
-- `ComparableHeap` for [comparable](https://go.dev/ref/spec#Comparison_operators) types. Additional `Search`
-  and `Delete` are implemented,
-- for [`constraints.Ordered`](https://pkg.go.dev/golang.org/x/exp/constraints#Ordered) there are
-  constructors for min, max heaps;
 
-Also use-cases provided:
-
-- `TopN` that allows getting N top elements from slice.
-  `TopN` swaps top N elements to first N elements of slice, no additional allocations are done. All slice elements are
-  kept, only order is changed.
-- `TopNHeap` allows to get N top, pushing elements from stream without allocation slice for all elements. Only O(N)
-  memory is used.
-- `TopNImmutable` allocated new slice for heap, input slice is not mutated.
-
-Both TopN and TopNImmutable has methods for creating min and max tops for `constraints.Ordered`.
-
-Usage Example
------------------
-
-    package main
-
-    import (
-        "fmt"
-
-        "github.com/lispad/go-generics-tools/binheap"
-    )
-
-    func main() {
-        someData := []float64{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0}
-        mins := binheap.MinN[float64](someData, 3)
-        fmt.Printf("--- top 3 min elements: %v\n", mins)
-        maxs := binheap.MaxN[float64](someData, 3)
-        fmt.Printf("--- top 3 max elements: %v\n\n", maxs)
-
-        heap := binheap.EmptyMaxHeap[string]()
-        heap.Push("foo")
-        heap.Push("zzz")
-        heap.Push("bar")
-        heap.Push("baz")
-        heap.Push("foobar")
-        heap.Push("foobaz")
-        fmt.Printf("--- heap has %d elements, max element:\n%s\n\n", heap.Len(), heap.Peak())
-    }
-
-A bit more examples could be found in `examples` directory
-
-Benchmark
------------------
-Theoretical complexity for getting TopN from slice with size M, N <= M: O(N*ln(M)). When N << M, the heap-based TopN
-could be much faster than sorting slice and getting top. E.g. For top-3 from 10k elements approach is ln(10^5)/ln(3) ~=
-8.38 times faster.
-
-#### Benchmark
-
-    BenchmarkSortedMaxN-8      	10303648	   136.0 ns/op	   0 B/op	   0 allocs/op
-    BenchmarkMaxNImmutable-8   	398996316	   3.029 ns/op	   0 B/op	   0 allocs/op
-    BenchmarkMaxN-8            	804041455	   1.819 ns/op	   0 B/op	   0 allocs/op
+The [ShardedLockMap](smap/README.md) package contains implementation of sharded lock map.
+Interface is similar to sync.map, but sharded lock map is faster on scenarios with huge read load with rare updates,
+and uses less memory, doing less allocations.
+[Details](smap/README.md)
 
 Compatibility
 -------------
@@ -77,6 +25,10 @@ Installation
 To install package, run:
 
     go get github.com/lispad/go-generics-tools/binheap
+or
+
+    go get github.com/lispad/go-generics-tools/smap
+
 
 License
 -------

binheap/README.md

@@ -0,0 +1,83 @@
+# Heap structure, using go generics
+[![MIT License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+
+Introduction
+------------
+
+The Heap package contains simple [binary heap](https://en.wikipedia.org/wiki/Binary_heap) implementation, using Golang
+generics. There are several heap implementations
+
+- generic Heap implementation, that could be used for `any` type,
+- `ComparableHeap` for [comparable](https://go.dev/ref/spec#Comparison_operators) types. Additional `Search`
+  and `Delete` are implemented,
+- for [`constraints.Ordered`](https://pkg.go.dev/golang.org/x/exp/constraints#Ordered) there are
+  constructors for min, max heaps;
+
+Also use-cases provided:
+
+- `TopN` that allows getting N top elements from slice.
+  `TopN` swaps top N elements to first N elements of slice, no additional allocations are done. All slice elements are
+  kept, only order is changed.
+- `TopNHeap` allows to get N top, pushing elements from stream without allocation slice for all elements. Only O(N)
+  memory is used.
+- `TopNImmutable` allocated new slice for heap, input slice is not mutated.
+
+Both TopN and TopNImmutable has methods for creating min and max tops for `constraints.Ordered`.
+
+Usage Example
+-----------------
+
+    package main
+
+    import (
+        "fmt"
+
+        "github.com/lispad/go-generics-tools/binheap"
+    )
+
+    func main() {
+        someData := []float64{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0}
+        mins := binheap.MinN[float64](someData, 3)
+        fmt.Printf("--- top 3 min elements: %v\n", mins)
+        maxs := binheap.MaxN[float64](someData, 3)
+        fmt.Printf("--- top 3 max elements: %v\n\n", maxs)
+
+        heap := binheap.EmptyMaxHeap[string]()
+        heap.Push("foo")
+        heap.Push("zzz")
+        heap.Push("bar")
+        heap.Push("baz")
+        heap.Push("foobar")
+        heap.Push("foobaz")
+        fmt.Printf("--- heap has %d elements, max element:\n%s\n\n", heap.Len(), heap.Peak())
+    }
+
+A bit more examples could be found in `examples` directory
+
+Benchmark
+-----------------
+Theoretical complexity for getting TopN from slice with size M, N <= M: O(N*ln(M)). When N << M, the heap-based TopN
+could be much faster than sorting slice and getting top. E.g. For top-3 from 10k elements approach is ln(10^5)/ln(3) ~=
+8.38 times faster.
+
+#### Benchmark
+
+    BenchmarkSortedMaxN-8      	10303648	   136.0 ns/op	   0 B/op	   0 allocs/op
+    BenchmarkMaxNImmutable-8   	398996316	   3.029 ns/op	   0 B/op	   0 allocs/op
+    BenchmarkMaxN-8            	804041455	   1.819 ns/op	   0 B/op	   0 allocs/op
+
+Compatibility
+-------------
+Minimal Golang version is 1.18. Generics and fuzz testing are used.
+
+Installation
+----------------------
+
+To install package, run:
+
+    go get github.com/lispad/go-generics-tools/binheap
+
+License
+-------
+
+The binheap package is licensed under the MIT license. Please see the LICENSE file for details.

go.mod

@@ -4,7 +4,7 @@ go 1.18
 
 require (
 	github.com/stretchr/testify v1.7.1
-	golang.org/x/exp v0.0.0-20220328175248-053ad81199eb
+	golang.org/x/exp v0.0.0-20220609121020-a51bd0440498
 )
 
 require (

go.sum

@@ -7,6 +7,8 @@ github.com/stretchr/testify v1.7.1 h1:5TQK59W5E3v0r2duFAb7P95B6hEeOyEnHRa8MjYSMT
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 golang.org/x/exp v0.0.0-20220328175248-053ad81199eb h1:pC9Okm6BVmxEw76PUu0XUbOTQ92JX11hfvqTjAV3qxM=
 golang.org/x/exp v0.0.0-20220328175248-053ad81199eb/go.mod h1:lgLbSvA5ygNOMpwM/9anMpWVlVJ7Z+cHWq/eFuinpGE=
+golang.org/x/exp v0.0.0-20220609121020-a51bd0440498 h1:TF0FvLUGEq/8wOt/9AV1nj6D4ViZGUIGCMQfCv7VRXY=
+golang.org/x/exp v0.0.0-20220609121020-a51bd0440498/go.mod h1:yh0Ynu2b5ZUe3MQfp2nM0ecK7wsgouWTDN0FNeJuIys=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=

smap/README.md

@@ -0,0 +1,81 @@
+# Sharded RWLocked Map, using go generics
+[![MIT License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+
+Introduction
+------------
+
+In some scenarios sync.Map could allocate large space for dirty-read copies, or even heavily use locks (when getting 
+missing values, that should be rechecked in map with lock). In such scenarios go internal map with rw-mutex, divided to 
+several shards could perform much better, with cost of memory for additional locks storage. But this amount could be
+much less, than sync.map uses.
+
+Interface incompatibility
+------------
+
+- `LoadOrStore` method changed to `LoadOrCreate`, with callback that generates value. Could be used to avoid 
+unnecessary creating huge values, in case if key already exists. 
+
+Usage Example
+-----------------
+
+    package main
+
+    import (
+        "fmt"
+
+        "github.com/lispad/go-generics-tools/smap"
+    )
+
+    func main() {
+    	m := NewIntegerComparable[int, int](8, 128)
+        m.Store(123, 456)
+
+        value, ok := m.Load(123)
+        fmt.Printf("%d, %t", value, ok)
+    }
+
+A bit more examples could be found in tests.
+
+Benchmark
+-----------------
+
+
+#### Benchmark
+
+Benchmark performed on Lenovo Ideapad laptop with AMD Ryzen 7 4700U, Linux Mint 20.3 with 5.13.0 kernel
+
+    BenchmarkIntegerSMap_ConcurrentGet-8              	82540485	     12.95 ns/op	    0 B/op	    0 allocs/op
+    BenchmarkSyncMap_ConcurrentGet-8                  	73431339	     18.35 ns/op	    0 B/op	    0 allocs/op
+    BenchmarkLockMap_ConcurrentGet-8                  	19327282	     54.03 ns/op	    0 B/op	    0 allocs/op
+    BenchmarkIntegerShardedMap_ConcurrentSet-8        	25605380	     42.33 ns/op	    0 B/op	    0 allocs/op
+    BenchmarkSyncMap_ConcurrentSet-8                  	 2138496	     536.1 ns/op	   36 B/op	    3 allocs/op
+    BenchmarkLockMap_ConcurrentSet-8                  	 3827476	     302.9 ns/op	    0 B/op	    0 allocs/op
+    BenchmarkIntegerShardedMap_ConcurrentGetSet5-8    	 3377473	     357.1 ns/op	    0 B/op	    0 allocs/op
+    BenchmarkSyncMap_ConcurrentGetSet5-8              	  323318	      3540 ns/op	  266 B/op	    3 allocs/op
+    BenchmarkLockMap_ConcurrentGetSet5-8              	  204633	      6176 ns/op	    0 B/op	    0 allocs/op
+    BenchmarkIntegerShardedMap_ConcurrentGetSet50-8   	18236305	     65.03 ns/op	    0 B/op	    0 allocs/op
+    BenchmarkSyncMap_ConcurrentGetSet50-8             	18337423	     56.55 ns/op	   32 B/op	    2 allocs/op
+    BenchmarkLockMap_ConcurrentGetSet50-8             	 2697315	     431.2 ns/op	    0 B/op	    0 allocs/op
+    BenchmarkIntegerShardedMap_ConcurrentGetSet1-8    	 1003506	      1076 ns/op	    0 B/op	    0 allocs/op
+    BenchmarkSyncMap_ConcurrentGetSet1-8              	   32668	     44423 ns/op	 3508 B/op	    5 allocs/op
+    BenchmarkLockMap_ConcurrentGetSet1-8              	   78486	     16019 ns/op	    0 B/op	    0 allocs/op
+
+Sharded Lock map is approximately equal to sync.Map on 50% read + 50% concurrent writes, and is much faster on 
+5% writes+95% reads, and 1% writes+99% reads. 
+Also sharded map allocated about 40x less memory in 5% writes+95% reads scenario, than sync.Map does.
+
+Compatibility
+-------------
+Minimal Golang version is 1.18. Generics are used.
+
+Installation
+----------------------
+
+To install package, run:
+
+    go get github.com/lispad/go-generics-tools/smap
+
+License
+-------
+
+The smap package is licensed under the MIT license. Please see the LICENSE file for details.

smap/comparable.go

@@ -0,0 +1,32 @@
+package smap
+
+// GenericComparable stores data in N shards, with rw mutex for each.
+// Additional CompareAndSwap method added for comparable values.
+type GenericComparable[K comparable, V comparable] struct {
+	Generic[K, V]
+}
+
+// NewGenericComparable creates generic RWLocked Sharded map for comparable values.
+// shardDetector should be idempotent function.
+func NewGenericComparable[K comparable, V comparable](shardsCount, defaultSize int, shardDetector func(key K) int) GenericComparable[K, V] {
+	return GenericComparable[K, V]{
+		Generic: NewGeneric[K, V](shardsCount, defaultSize, shardDetector),
+	}
+}
+
+// CompareAndSwap executes the compare-and-swap operation for the Key & Value pair.
+// If and only if key exists, and value for key equals old, value will be changed to new.
+// Otherwise, returns current value.
+// The ok result indicates whether value was changed to new in the map.
+func (sm GenericComparable[K, V]) CompareAndSwap(key K, old, new V) (V, bool) {
+	shardID := sm.shardDetector(key)
+	sm.locks[shardID].Lock()
+	if current, ok := sm.shards[shardID][key]; ok && current == old {
+		sm.shards[shardID][key] = new
+		sm.locks[shardID].Unlock()
+		return new, true
+	} else {
+		sm.locks[shardID].Unlock()
+		return current, false
+	}
+}