I like fp style and I haven’t found a lib with these features:
- streamingly, I can handle infinite data source such as go channel or a socket reader
- lazy evaluation, well, huge list processing wouldn’t make me oom
- generic, the interface{} type ocurrs in a map function sucks
- chain calls, functions should be compositional
- clean, I hope the core of the lib would be clean
- performance, good performance would be a bonus
And when I decide to build a new fp lib, the theory of lisp come to my mind immediately.
If I can bring cons,car,cdr into golang, that would be cool and attractive for me.
So I spend couple of days make this, and I hope you like it. Any feedback is welcome.
Own to the poor performance of golang's closure and small objects gc, the lisp like version runs a little slow. So I have to refact the whole project with iterator pattern, for now it runs 2xtimes than before and faster than go-linq at least, enjoy it. goos: darwin goarch: amd64 pkg: demo/fpdemo cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz BenchmarkFP-12 274879 3711 ns/op 1184 B/op 42 allocs/op BenchmarkGoLinq-12 246768 4545 ns/op 1632 B/op 69 allocs/op
Stream is created from a source, source is a slice, a channel, or even a reader.
e.g. create stream from slice
StreamOf([]int{1, 2, 3})
StreamOf([]string{"a", "b", "c"})
e.g. create stream from channel
ch := make(chan string, 1)
StreamOf(ch)
e.g. create stream from iterator function
var i int
fn := func() (int, bool) {
i++
return i, i < 5
}
StreamOf(fn)
e.g. create stream from custom source
type Source interface {
// source element type
ElemType() reflect.Type
// Next element
Next() (reflect.Value, bool)
}
StreamOfSource(mySource)
// create a file source, read text line by line
file, _ := os.Open("example.txt")
defer file.Close()
source := NewLineSource(file)
StreamOfSource(source)
slice := []string{"a", "b", "c"}
var out []string
StreamOf(slice).Map(strings.ToUpper).ToSlice(&out)
suite.ElementsMatch(out, []string{"A", "B", "C"})
// map with selector
slice := []string{"a", "b", "c"}
var out []string
StreamOf(slice).Map(func(e string) (string, bool) {
return strings.ToUpper(e), e == "b"
}).ToSlice(&out)
suite.ElementsMatch(out, []string{"B"})
// map with error
slice := []string{"a", "b", "c"}
var out []string
err := StreamOf(slice).Map(func(e string) (string, error) {
return strings.ToUpper(e), genErr(e == "a" || e == "c")
}).ToSlice(&out)
suite.Len(out, 0)
suite.Error(err)
// flatmap sub collection
slice := []string{"abc", "de", "f"}
out := StreamOf(slice).FlatMap(func(s string) []byte {
return []byte(s)
}).Bytes()
suite.Equal("abcdef", string(out))
// flatmap sub stream
databases := []string{"db1", "db2"}
tables := []string{"table1", "table2"}
fullnames := StreamOf(databases).FlatMap(func(db string) Stream {
return StreamOf(tables).Map(func(table string) TupleString {
return TupleStringOf(db, table)
})
}).Map(func(t TupleString) string {
return t.E1 + "." + t.E2
}).Strings()
suite.Equal([]string{"db1.table1", "db1.table2", "db2.table1", "db2.table2"}, fullnames)
slice := []string{"a", "b", "c"}
out := StreamOf(slice).Filter(func(s string) bool {
return s == "b"
}).Strings()
suite.Equal([]string{"b"}, out)
// there're some helper partial functions
slice := []string{"a", "b", "c"}
out := StreamOf(slice).Filter(Equal("b")).Strings()
suite.Equal([]string{"b"}, out)
out := StreamOf(slice).Filter(EqualIgnoreCase("B")).Strings()
suite.Equal([]string{"b"}, out)
out := StreamOf([]string{"a",""}).Reject(EmptyString()).Strings()
suite.Equal([]string{"a"}, out)
slice := []string{"a", "b", "c"}
out := StreamOf(slice).Reject(func(s string) bool {
return s == "b"
}).Strings()
suite.Equal([]string{"a", "c"}, out)
var out string
slice := []string{"abc", "de", "f"}
out1 := StreamOf(slice).Foreach(func(s string) {
out += s
}).Strings()
suite.Equal("abcdef", out)
suite.ElementsMatch(slice, out1)
slice := []string{"abc", "de", "f"}
out := StreamOf(slice).Map(func(s string) []byte {
return []byte(s)
}).Flatten().Bytes()
suite.Equal("abcdef", string(out))
deep flatten
databases := []string{"db1", "db2"}
tables := []string{"table1", "table2"}
fullnames := StreamOf(databases).FlatMap(func(db string) Stream {
return StreamOf(tables).Map(func(table string) TupleString {
return TupleStringOf(db, table)
})
}).Map(func(t TupleString) string {
return t.E1 + "." + t.E2
}).Strings()
suite.Equal([]string{"db1.table1", "db1.table2", "db2.table1", "db2.table2"}, fullnames)
slice := [][]string{
{"abc", "de", "f"},
{"g", "hi"},
}
var out [][]byte
StreamOf(slice).Map(func(s []string) [][]byte {
return StreamOf(s).Map(func(st string) []byte {
return []byte(st)
}).ToSlice(&out)
}).Flatten().Flatten().Bytes()
suite.Equal("abcdefghi", string(out))
source := []string{"a", "b", "c", "d"}
out := StreamOf(source).Partition(3).StringsList()
suite.Equal([][]string{
{"a", "b", "c"},
{"d"},
}, out)
slice := []string{"a", "b", "c", "d", "e", "c", "c"}
out := StreamOf(slice).PartitionBy(func(s string) bool {
return s == "c"
}, true).StringsList()
suite.Equal([][]string{
{"a", "b", "c"},
{"d", "e", "c"},
{"c"},
}, out)
slice := []string{"a", "b", "c", "d", "e", "c", "c"}
out := StreamOf(slice).LPartitionBy(func(s string) bool {
return s == "c"
}, true).StringsList()
suite.Equal([][]string{
{"a", "b"},
{"c", "d", "e"},
{"c"},
{"c"},
}, out)
source := []string{"a", "b", "c", "d", "a", "c"}
var out map[string]int
StreamOf(source).Reduce(map[string]int{}, func(memo map[string]int, s string) map[string]int {
memo[s] += 1
return memo
}).To(&out)
suite.Equal(map[string]int{
"a": 2,
"b": 1,
"c": 2,
"d": 1,
}, out)
max := func(i, j int) int {
if i > j {
return i
}
return j
}
min := func(i, j int) int {
if i < j {
return i
}
return j
}
sum := func(i, j int) int { return i + j }
source := []int{1, 2, 3, 4, 5, 6, 7}
ret := StreamOf(source).Reduce0(max).Int()
suite.Equal(int(7), ret)
ret = StreamOf(source).Reduce0(min).Int()
suite.Equal(int(1), ret)
ret = StreamOf(source).Reduce0(sum).Int()
suite.Equal(int(28), ret)
slice := []string{"abc", "de", "f"}
q := StreamOf(slice)
out := q.First()
suite.Equal("abc", out.String())
slice := []string{"abc", "de", "f"}
q := StreamOf(slice)
suite.False(q.IsEmpty())
out := q.First()
suite.Equal("abc", out.String())
slice := []string{"abc", "de", "f"}
out := strings.Join(StreamOf(slice).Take(2).Strings(), "")
suite.Equal("abcde", out)
slice := []string{"a", "b", "c"}
out := StreamOf(slice).TakeWhile(func(v string) bool {
return v < "c"
}).Strings()
suite.Equal([]string{"a", "b"}, out)
slice := []string{"abc", "de", "f"}
out := strings.Join(StreamOf(slice).Skip(2).Strings(), "")
suite.Equal("f", out)
slice := []string{"a", "b", "c"}
out := StreamOf(slice).SkipWhile(func(v string) bool {
return v < "c"
}).Strings()
suite.Equal([]string{"c"}, out)
slice := []int{1, 3, 2}
out := StreamOf(slice).Sort().Ints()
suite.Equal([]int{1, 2, 3}, out)
slice := []string{"abc", "de", "f"}
out := StreamOf(slice).SortBy(func(a, b string) bool {
return len(a) < len(b)
}).Strings()
suite.Equal([]string{"f", "de", "abc"}, out)
slice := []int{1, 3, 2, 1, 2, 1, 3}
out := StreamOf(slice).Uniq().Ints()
suite.ElementsMatch([]int{1, 2, 3}, out)
slice := []int{1, 3, 2, 1, 2, 1, 3}
out := StreamOf(slice).UniqBy(func(i int) bool {
return i%2 == 0
}).Ints()
suite.ElementsMatch([]int{1, 2}, out)
out := StreamOf(slice).Size()
suite.Equal(2, out)
slice := []string{"abc", "de", "f"}
q := StreamOf(slice)
suite.True(q.Contains("de"))
slice := []string{"abc", "de", "f"}
q := StreamOf(slice)
suite.True(q.ContainsBy(func(s string) bool { return strings.ToUpper(s) == "F" }))
slice1 := []string{"abc", "de", "f", "gh"}
var q map[int][]string
StreamOf(slice1).Map(strings.ToUpper).GroupBy(func(s string) int {
return len(s)
}).To(&q)
suite.Equal(map[int][]string{
1: {"F"},
2: {"DE", "GH"},
3: {"ABC"},
}, q)
slice := []string{"abc", "de"}
out := StreamOf(slice).Append("A").Strings()
suite.Equal([]string{"abc", "de", "A"}, out)
slice := []string{"abc", "de"}
out := StreamOf(slice).Prepend("A").Strings()
suite.Equal([]string{"A", "abc", "de"}, out)
slice1 := []string{"abc", "de", "f"}
slice2 := []string{"g", "hi"}
q1 := StreamOf(slice1).Map(strings.ToUpper)
q2 := StreamOf(slice2).Map(strings.ToUpper)
out := q2.Union(q1).Strings()
suite.Equal([]string{"ABC", "DE", "F", "G", "HI"}, out)
slice1 := []int{1, 2, 3, 4}
slice2 := []int{2, 1}
out := StreamOf(slice1).Sub(StreamOf(slice2)).Ints()
suite.Equal([]int{3, 4}, out)
slice1 := []int{1, 2, 3, 4}
slice2 := []int{2, 1}
out := StreamOf(slice1).Interact(StreamOf(slice2)).Ints()
suite.ElementsMatch([]int{1, 2}, out)
slice1 := []int{1, 2, 3}
slice2 := []int{4, 5, 6, 7}
out := StreamOf(slice1).Zip(StreamOf(slice2), func(i, j int) string {
return strconv.FormatInt(int64(i+j), 10)
}).Strings()
suite.ElementsMatch([]string{"5", "7", "9"}, out)
slice1 := []int{1, 2, 3}
slice2 := []int{4, 5, 6, 7}
slice3 := []int{2, 3}
out := StreamOf(slice1).ZipN(func(i, j, k int) string {
return strconv.FormatInt(int64(i+j+k), 10)
}, StreamOf(slice2), StreamOf(slice3)).Strings()
suite.ElementsMatch([]string{"7", "10"}, out)
stream transform would not work unless Run/ToSlice is invoked.
use Run if you just want stream flows but do not care about the result
// the numbers would not print without Run
StreamOf(source).Foreach(func(i int) {
fmt.Println(i)
}).Run()
slice := []string{"a", "b", "c"}
var out []string
StreamOf(slice).Map(strings.ToUpper).ToSlice(&out)
suite.ElementsMatch(out, []string{"A", "B", "C"})
You can map value with error
var v int64
err := M("a").Map(func(s string) (int64, error) {
return strconv.ParseInt(s, 10, 64)
}).To(&v)
suite.Zero(v)
suite.Error(err)
You can ExpectPass/ExpectNoError on an Maybe monand
var v int64
err := M("2").Map(func(s string) (int64, error) {
return strconv.ParseInt(s, 10, 64)
}).ExpectPass(func(i int64) bool {
return i > 0
}).To(&v)
suite.Equal(int64(2), v)
suite.NoError(err)
var v int64
err := M("2").Map(func(s string) (int64, error) {
return strconv.ParseInt(s, 10, 64)
}).ExpectNoError(func(i int64) error {
return errors.New("xerr")
}).To(&v)
suite.Equal(int64(0), v)
suite.Error(err)
var out []int
err := M("2").Map(func(s string) (int64, error) {
return strconv.ParseInt(s, 10, 64)
}).StreamOf(func(i int64) []int {
return Times(int(i)).Ints()
}).ToSlice(&out)
suite.NoError(err)
suite.Equal([]int{0, 1}, out)
m1 := M("20").Map(func(s string) (int64, error) {
return strconv.ParseInt(s, 10, 64)
})
var score int64
err := M("10").Map(func(s string) (int64, error) {
return strconv.ParseInt(s, 10, 64)
}).Zip(func(a, b int64) int64 {
return a + b
}, m1).To(&score)
suite.NoError(err)
suite.Equal(int64(30), score)
var cnt int
var score int64
m := M("10").Map(func(s string) (int64, error) {
cnt++
return strconv.ParseInt(s, 10, 64)
})
err := m.To(&score)
suite.NoError(err)
suite.Equal(int64(10), score)
suite.Equal(1, cnt)
err = m.To(&score)
suite.NoError(err)
suite.Equal(int64(10), score)
suite.Equal(2, cnt)
var v int64
err := M("a").Map(func(s string) (int64, error) {
return strconv.ParseInt(s, 10, 64)
}).To(&v)
suite.Zero(v)
suite.Error(err)
If you just want the error
err := M("21a").Map(func(s string) (int64, error) {
return strconv.ParseInt(s, 10, 64)
}).Error()
suite.Error(err)
kvstream is merely a simple wrapper for golang’s map.
iterater a map
m := map[string]int{
"a": 1,
"b": 2,
}
var keys []string
var vals []int
KVStreamOf(m).Foreach(func(key string, val int) {
keys = append(keys, key)
vals = append(vals, val)
}).Run()
suite.ElementsMatch([]string{"a", "b"}, keys)
suite.ElementsMatch([]int{1, 2}, vals)
tranform a map to another map
m := map[string]int{
"a": 1,
"b": 2,
}
var vk map[int]string
KVStreamOf(m).Map(func(k string, v int) (int, string) {
return v, k
}).To(&vk)
suite.Equal("a", vk[1])
suite.Equal("b", vk[2])
filter a map
m := map[string]int{
"a": 1,
"b": 2,
}
var b []int
KVStreamOf(m).Filter(func(k string, v int) bool {
return v == 1
}).Values().ToSlice(&b)
suite.ElementsMatch(
[]int{1},
b,
)
KVStreamOf(m).Reject(func(k string, v int) bool {
return v == 2
}).Values().ToSlice(&b)
suite.ElementsMatch(
[]int{1},
b,
)
predict key exist
m := map[string]int{
"a": 1,
"b": 2,
}
var b []int
KVStreamOf(m).Contains("a") // true
get map keys/values stream
slice := []int{1, 2, 3, 2, 1}
out := StreamOf(slice).ToSet().Keys().Ints()
suite.ElementsMatch([]int{1, 2, 3}, out)
get map size
slice := []int{1, 2, 3, 2, 1}
out := StreamOf(slice).ToSet().Size()
suite.Equal(3, out)
kvstream is also lazy evaluation, get will get the result until Run/To invoked
m := map[string]int{
"a": 1,
"b": 2,
}
var vk map[int]string
KVStreamOf(m).Map(func(k string, v int) (int, string) {
return v, k
}).To(&vk)
suite.Equal("a", vk[1])
suite.Equal("b", vk[2])